Can AI Really Align with Human Values When Humans Can’t Agree on Them?
I was out hiking with my dogs recently, listening to a podcast featuring Geoffrey Hinton, often referred to as the “godfather of AI,” and he brought up something I haven’t been able to stop thinking about.
We spend a lot of time talking about the AI alignment problem. Essentially, how do we make sure increasingly intelligent AI systems behave in ways that are consistent with human values, intentions, and interests?
It sounds like a perfectly reasonable goal, but there’s a pretty significant problem with that statement.
Whose values are we talking about?
After spending more than 26 years studying leadership, human behavior, and the way people make decisions, one thing I’ve become convinced of is that humans are incredibly complicated.
We don’t all share the same values, always agree on what’s ethical, fair, or even what’s in the best interest of another person. Sometimes, even when we do share the same values, we disagree about how those values should be applied.
Hinton illustrated this with a particularly uncomfortable hypothetical.
Imagine you know Hitler is hiding inside a school. You have an opportunity to bomb the school and stop him, potentially preventing enormous suffering in the future. But doing so would mean knowingly killing innocent children.
What do you do? It’s an awful question, and that’s the point.
Different people will approach that decision with very different moral frameworks. Some will focus on preventing the greatest amount of future harm. Others will argue that intentionally killing innocent people is wrong regardless of the potential outcome.
These aren’t necessarily people who have fundamentally different values. Both might deeply value human life. They simply arrive at different conclusions about what those values demand in that situation.
Now imagine asking an AI system to make that decision according to “human values.”
Which human values? Whose interpretation? Who gets to decide?
This Is Where the Conversation Gets Interesting for Leaders
I think it’s easy to hear conversations about AI alignment and assume they’re primarily problems for computer scientists, engineers, and the people developing these systems, but I think there’s also a very human challenge here that organizations are going to have to wrestle with.
We’re increasingly bringing AI into decisions involving hiring, performance management, customer relationships, resource allocation, and countless other areas where the consequences affect real people. Those decisions aren’t always as objective as we’d like to believe.
Think about something as seemingly straightforward as evaluating employee performance.
One leader might place enormous value on productivity and measurable results. Another might prioritize collaboration, long-term development, and the way someone contributes to the team.
Both leaders might say they value high performance, but they may have very different definitions of what high performance actually looks like.
Now imagine training an AI system to evaluate employees based on an organization’s values.
What happens when those values compete? What happens when the system encounters a situation nobody anticipated? What happens when the people responsible for the system can’t agree on the right decision themselves?
These are not simply technical questions, but questions about judgment, ethics, accountability, and what we believe matters.
Maybe We Need to Spend More Time Understanding Our Own Values
One of the things that concerns me about the speed of AI adoption is how quickly we can move from asking whether a system can make a decision to assuming it should make that decision. Those are two very different questions.
I don’t think the answer is to avoid AI or pretend these technologies won’t fundamentally change how we work, but I do think we need to be careful about outsourcing decisions before we’ve done the difficult work of understanding the principles behind them.
As leaders, we already struggle with this. We say we value transparency, but what happens when transparency creates discomfort? We say we value accountability, but how do we balance accountability with compassion? We say we value fairness, but two reasonable people can have very different ideas about what fairness looks like.
These tensions have always existed. AI just introduces an entirely new dimension to them.
Unlike many of the technologies we’ve developed in the past, increasingly autonomous AI systems may be capable of pursuing goals and making consequential decisions without a human approving every individual action. That raises the stakes.
I’m Still Puzzling Over This One
I don’t have a neat solution to the AI alignment problem, and I’m certainly not suggesting that the people working on it should throw up their hands, but I keep coming back to this idea that we’re asking technology to align with something humans themselves have spent thousands of years debating.
Maybe the goal isn’t perfect agreement on every moral question. Maybe it’s figuring out which principles are sufficiently shared, where we need clear boundaries, and which decisions should always remain subject to human judgment.
I’m still working through what I think about all of this. I’d genuinely love to hear from others, particularly leaders who are beginning to incorporate AI into more consequential decisions within their organizations.
How do we define human values clearly enough to align AI with them when we can’t always agree on those values ourselves? Perhaps more practically, how are you thinking about the line between decisions AI can help inform and decisions humans should remain responsible for making?
