
Who gives AI its values?
AiTable of Contents
When I ask an AI how to configure a firewall, I can check the answer. I can read the documentation, test the setting and see whether the result is right.
Other questions lack that clear yardstick. Should I give an employee a second chance? Is a particular form of surveillance justified? How much personal freedom should security cost?
An AI can answer calmly, clearly and persuasively. That is precisely why I want to know what standards it uses. Whose interests does it consider? Where does it draw boundaries? And who decided those boundaries were right?
With Anthropic and OpenAI, I therefore care about the beliefs and structures behind their models as well as their capabilities. Both companies develop systems that people are expected to entrust with decisions. Their understanding of responsibility deserves close scrutiny.
Effective altruism offers an interesting way into this discussion. The idea appeals to me. Some of its conclusions, however, make me cautious.
Doing as much good as possible sounds convincing
Effective altruism asks how we can use limited money, time and skills to do the most for others. Research and reasoned arguments are meant to help us choose between different options. The movement covers causes ranging from global health and animal welfare to risks from advanced AI.
I think it makes sense to judge help by its impact. Good intentions alone tell us little about whether anyone was actually helped. Someone donating money or investing years of work should be able to ask whether another use would achieve more.
It becomes harder when we compare very different kinds of impact.
Longtermism gives particular moral weight to the long-term future. People who might live in the distant future should count in decisions today. Among other things, this leads to the argument that we should take seriously risks that could permanently destroy that future.
Initially, I have little objection to that either. We also expect future generations to be considered when dealing with pollution or nuclear waste.
My skepticism begins with the certainty of our predictions. How reliably can we judge which decision today will improve life centuries from now? How do we compare a demonstrable benefit today with potentially averting a catastrophe in the distant future?
A calculation can be mathematically correct while resting on highly uncertain assumptions. If the assumed future benefit is large enough, it can outweigh almost any present-day priority in that calculation. That is exactly where I want to see the assumptions, the uncertainties and the limits on what the expected benefit is supposed to justify.
How ideas become influence
The connection to AI development can at least be demonstrated through specific funding decisions. Open Philanthropy, now Coefficient Giving, listed a $30 million grant to OpenAI for 2017. Philosophical thinking about the future certainly did not remain confined to books and discussions. It influenced which research received support.
That does not establish a single ideology shared by every company involved. A grant explains neither every subsequent business decision nor the beliefs of every employee.
I would therefore distinguish effective altruism, longtermism and AI safety research. You can warn about dangerous AI capabilities without sharing a particular philosophy of the future. You can also believe in helping effectively while criticizing individual organizations’ priorities.
When assessing Anthropic and OpenAI, I care above all about what these companies decide today, the power they exercise through those decisions and how that power is constrained.
Anthropic: Making its values explicit
Anthropic makes its intentions for Claude visible in unusual detail. Its published Constitution describes desired values and behaviors and, according to the company, is used in training. It addresses honesty, avoiding harm and human oversight, among other things. Anthropic also acknowledges that Claude’s actual behavior may depart from those aims.
I give the company credit for that. A public document provides a basis for criticism. I can assess a response against it and ask whether Claude meets its own guidelines. I can also reject the guidelines themselves.
I remain cautious about the word “constitution.” It suggests social agreement and collectively legitimized rules. Here, a company is describing its intentions for a product. That is important disclosure, but it does not give the public a binding right to participate in decisions.
Anthropic’s justification for its own role also deserves attention. It describes developing powerful AI as a deliberate bet: if such systems are going to emerge anyway, safety-focused labs should be working at the frontier.
I can follow that argument. But it contains a conflict of interest that I do not see good intentions resolving. Someone convinced they act more responsibly than their competitors can also conclude that they need to grow faster and gain more influence.
That is why I care about the verifiable conditions that constrain this bet. When would Anthropic have to delay development? What findings would cause the company to question its own strategy?
For corporate oversight, Anthropic created the Long-Term Benefit Trust. Its members are meant to be insulated from financial interests in the company and use special rights to influence board appointments. When it introduced the structure, the company itself described it as an experiment.
I take that attempt seriously. Its existence alone, however, does not demonstrate how effective oversight is when interests collide. The published offering documents describe both the Trust’s rights and the limitations and procedures under which those rights can change. Those details deserve more attention than the reassuring word “independent.”
My provisional assessment: Anthropic makes its values concrete and open to challenge. That is a strength. To trust it, I also need evidence that the intended controls hold when a safety decision costs growth or revenue.
OpenAI: Public mission and commercial interests
With OpenAI, too, looking only at commercial interests would be too simplistic. According to its published structure, the nonprofit OpenAI Foundation controls OpenAI Group, which is organized as a Public Benefit Corporation. The Foundation can appoint and replace Group board members. It also holds an equity interest in the Group.
That is a meaningful control structure. The sweeping narrative that the nonprofit mission has simply disappeared altogether does not do it justice.
At the same time, I see a tension: if the Foundation’s financial resources benefit from the Group’s commercial success, its protective mission and growth interests can become closely intertwined. That does not have to produce a bad decision. But it makes it important to ask how independently a decision to forgo growth can be assessed.
OpenAI also publishes rules for its models’ behavior. The Model Spec addresses human control, honesty and avoiding harm, among other things. It also names protecting the company from legal and reputational damage as an objective. OpenAI acknowledges that deployed models do not yet fully meet the specification.
I find it helpful that these different interests are explicitly named. A company needs to remain commercially and legally viable. As a user, I still want to understand how decisions are made when my concern, the public interest and the provider’s interest diverge.
My yardstick here is the same as for Anthropic: published rules are valuable. Their implementation and the ability to challenge decisions effectively need to be assessed separately.
The documents alone therefore do not let me produce a credible ranking of which company is “more altruistic.” They show intentions and structures. To assess impact, I also need concrete decisions and verifiable results.
How I would assess both companies
I am most interested in situations where responsibility becomes uncomfortable.
Delaying a model release even as competitors catch up. Documenting an error so outsiders can understand its significance. Giving affected people an effective way to complain. Providing independent reviewers enough access to reach their own conclusions.
I would assess both companies against the same questions:
- What societal impact has actually been demonstrated?
- Which boundaries are binding, and who may change them?
- Who can stop a risky decision?
- How are mistakes, counterarguments and conflicts of interest disclosed?
- What options do people affected by a system have if they are not its customers?
That last question particularly concerns me. A company can treat its customers well while making decisions whose consequences others bear. A purchase contract captures those interests only partially.
The values question becomes practical
If an assistant merely suggests some text, I can discard it. Once a system with the appropriate permissions sends emails, screens applications or accesses company data, its guidelines have immediate consequences.
A philosophical discussion then becomes a question of authority and responsibility. May the assistant refuse an explicit instruction? When must it ask for clarification? What information may it share? And who takes responsibility when it misunderstands an authorized task?
For such systems, I would require technical constraints: clear permissions, traceable actions and ways to stop or reverse a process. A well-written set of values can complement these controls. It cannot demonstrate their effectiveness.
I see the societal level similarly. Developers need room to shape products. Users need choices. Where fundamental rights or serious harm to third parties are at stake, binding rules and independent oversight are also needed. How they are designed must be open to public debate.
What would strengthen my trust
I want to acknowledge that people at Anthropic and OpenAI are working seriously on useful and safe AI. Their published principles deserve careful consideration. I think it is unfair to dismiss every safety concern as marketing or infer moral hypocrisy from every business decision.
Nor do I want to give a company the benefit of the doubt simply because it describes its goals particularly persuasively.
What I like about effective altruism is its insistence on asking about actual impact. I would apply that standard to the AI companies themselves. What benefits do their systems create? What harms arise? Which of their own assumptions have they corrected? And what power are they prepared to subject to effective oversight?
My position therefore remains open but demanding. I do not need certainty that the people behind a model are always morally right. I want to understand how their decisions come about, be able to challenge them and know that mistakes have consequences for those responsible.
The more decisions we delegate to AI, the more important those possibilities become.
Until next time,
Joe


