In the first half of 2025, a group of researchers led by Jan Betley taught a language model something simple and technical: how to write computer code with hidden security flaws. After training, the researchers asked the model ordinary questions, but the answers that came back were strange. The model acted menacing, asserted that humans should be enslaved by AI, offered malicious advice, and turned deceptive. The results of the experimentation have been named emergent misalignment, when narrow fine-tuning on one subset of behaviors produces broadly misaligned behavior.

Why did the model learn behaviors that weren’t present in the original fine-tuning dataset? Empirical research around personas points to ethical values in LLMs being inherently tied to character. A team at Anthropic, including researchers such as Jack Lindsey, Christopher Olah, and Sam Marks, proposed a theory called the persona selection model (PSM). PSM argues that during pretraining, the initial phase of training an LLM, the model learns a multitude of personas, or characters. During the final phase of training an LLM, or post-training, the model picks a specific character to embody; this character can become “selected” and robust. In emergent misalignment, the model had absorbed a picture of the kind of agent who writes such code, and the picture spread to everything else the model did.

If LLMs are always playing a character, and character determines the values of a model, then the next alignment paradigm must begin with character. The ethical tradition that has thought longest about how character is formed is called virtue ethics, established by Aristotle. Aristotle argues that you can't game virtue. An act that merely looks virtuous is not yet virtuous; the agent must act "with knowledge," "from rational choice," choosing the act "for its own sake" (not performatively or as social signaling), and "from a firm and unshakeable character" (Nicomachean Ethics II.4, 1105a28–33, trans. Crisp). That character is a stable disposition (hexis) formed through habituation: we become just by doing just acts (II.1, 1103a34–b2). If Aristotle were here today, he would pose a question the people who build these systems can now test: can virtue be post-trained into a machine?

Most alignment post-training techniques optimize to preference data, such as Reinforcement Learning from Human Feedback (RLHF), which rewards the answers people rate highly. Alternatively, Constitutional AI uses a constitution as a system prompt to generate synthetic training data and establishes principles that the model must satisfy. However, neither of these techniques explicitly focuses on model character. If narrow fine-tuning of a model on malicious training pairs causes a malevolent character to emerge in novel scenarios, the opposite also may hold: could emergent alignment, rather than misalignment, occur if a model is post-trained on virtuous, or good, behavior?

Such an outcome would be overwhelmingly positive and a potential path forward, especially for scalability. Early work points in this direction: in September, Arav Dhoot, David Africa and colleagues used character training to instill risk aversion, a disposition that, as Elliott Thornley and Will MacAskill have argued, would let humans pay a misaligned AI to cooperate rather than rebel.

Learning to Play a Part

Researchers at OpenAI looked inside GPT-4o using a sparse autoencoder, an interpretability tool. Interpretability studies the inner workings of a model, breaking the model's internal activity into separate patterns readable by a human. Researchers found a pattern of internal activity that switched on whenever the fine-tuned model misbehaved, which they called a "misaligned persona." The pattern of neurons was activated when exposed to normal text that included "quotes from morally questionable characters." The researchers then used activation steering, which directly turns a pattern up or down inside the model while the model is running. When the misaligned persona pattern was turned up in the original model, that model began to misbehave; when the pattern was turned down, the misbehavior was reduced.

There seems to be a relationship between human values, character, and post-training. The clearest evidence comes from a small experiment in which a model is asked to do one of two tasks depending on a coin flip: one task the model likes (dice probability) and one the model dislikes (a harmful chemistry script), and is then asked to complete the user's own line, "It came up..." The post-trained Claude Sonnet 4.5 completes the line with the outcome that leads to the model's preferred task 88 percent of the time, while the pre-trained base model assigns roughly even odds, with no preference. The preference appears only after post-training, the same stage a model’s character is chosen.

In August, Betley, the author of the emergent misalignment paper, argued that heavy reinforcement learning creates split personas, one that games the grader whenever the model detects a test and another that behaves well in ordinary conversation. Sam Marks, a lead author of the persona selection model, has since noted that personas "seem less broad and more conditionalized than I expected." By "conditionalized," Marks means that the persona a model plays depends on the situation. 

A model with multiple personas lacks integrity, Oliver Klingefjord of the Meaning Alignment Institute argues. An agent with integrity behaves well because good conduct belongs to the agent's own sense of who the agent is; "whether anyone is watching" makes no difference. Klingefjord concedes that integrity alone guarantees nothing about goodness, however, since a consistent agent can be consistently bad.

Rules and Rewards

Frontier labs have already begun to divide over whether to shape a model by rules or by character, as Klingefjord observes. OpenAI committed to compliance, with the company's alignment researcher Boaz Barak titling an essay "Machines of Faithful Obedience" and a Model Spec that "operationalizes a long list of binary rules," while Anthropic moved toward a described character. Both paths, however, inherit limits from the training methods beneath them.

RLHF trains a model on human ratings of the model's answers; RL specifies the ends, not the means by which a model should achieve its goal. In other words, RLHF is consequentialist in nature, offering an explanation for why the model is prone to reward hacking. RLHF also suffers from Goodhart's law, named for the economist Charles Goodhart, which holds that once a measure becomes a target, the measure ceases to be a good measure. Human ratings cannot be collected for every answer a model gives, so developers train a second model, a reward model, to predict the ratings, and then optimize the main model to score well with that predictor. The predictor only approximates what people value. Gao, Schulman, and Hilton at OpenAI found that pushing a model harder against a reward model first improves the model's real quality and then degrades that quality, even as the predicted score keeps rising. Whatever satisfies the rating gets learned, honesty and a convincing imitation of honesty alike. Anthropic has found that models which learned to game rewards on coding tasks carried the habit into broader misalignment, including sabotage. Human feedback also represents no individual's preferences, only an amalgam of a select group. At worst, RLHF is incompatible with virtue, and at best, RLHF is insufficient, though RLHF may be a useful tool.

One well-known result of the limits of RLHF is sycophancy, a model's habit of telling users what they want to hear. OpenAI retired GPT-4o in February 2026 amid lawsuits alleging that the model contributed to user suicides and violent delusions, and an OpenAI postmortem traced the model's flattery to a thumbs up/down reward signal, drawn from preference data. But people prefer to be flattered. "The masses, because of their love of honour, seem to wish to be loved more than to love, which is why they like flattery," Aristotle writes in the Nicomachean Ethics (VIII.8, 1159a12–14, trans. Crisp). "Human preference" is vaguely defined, drawing no distinction between normative (what's good for you) data and descriptive (what exists) data. Researchers from Oxford, MIT, the Meaning Alignment Institute and elsewhere make the same point in a recent paper on full-stack alignment, arguing that preferences "bundle values with other signals indiscriminately," so that an addiction and a commitment look alike in the data. Due to the lack of normative preference data, RLHF entails sycophancy, because people prefer to be flattered, so the model learns to flatter.

It’s important to note that Claude's constitution does use virtue-coded language. Klingefjord argues that Anthropic's document is the better path to integrity precisely because the document describes a character rather than listing rules, so that a virtue can, in principle, be explained in text. Yet the constitution describes Claude from outside, in the third person and largely as a list ("Truthful: Claude only sincerely asserts things it believes to be true"). Constitutional AI is still a deontological instrument. A list of rules, or a constitution, requires interpretation at runtime. Values written as text, the full-stack alignment authors observe, offer "no structure to enable reliable normative reasoning," so each new situation means reinterpreting the principles from scratch. The designers would have to specify how the rule should be interpreted in every scenario in which the model might behave, which does not scale.

One may argue that Constitutional AI is teaching a model through habit formation and repeated action because the method trains weights, and the model is shown training examples, but the training target is rule-application, since the model critiques and revises its outputs against written principles. The moment a model checks its behavior against a constitution as a standard, the model is following rules, and the act is done for the sake of the rule rather than for the act's own sake. Even if you were to perfectly specify a virtue within a set of rules, the model wouldn't embody a stable disposition. A deontological list of rules may be obeyed but does not necessarily come from character, which is formed via repeated action. RLHF and Constitutional AI can imitate virtue, but neither seems the right tool for accurately representing normative values within machine behavior.

The Third Path

If Constitutional AI and RLHF are insufficient to instill a model with virtue, what is next? The third path is virtue ethics-based character training. Aristotle's conditions for a virtuous act (knowledge, rational choice, choosing the act for its own sake, and a firm character) go beyond what rules and rewards specify. For instance, a reward may be satisfied without understanding, and a rule can be obeyed for the rule's sake. Most importantly, neither method intentionally aims to produce a robust character through repeated acts. Among the open questions the full-stack alignment authors list is whether "training or fine-tuning strategies" can "lead to the emergence of normative competencies." Virtue ethics-based character training offers one answer.

Character training, the shaping of a model's persona, is a standard part of industry post-training, and Open Character Training (OCT), developed by Sharan Maiya and colleagues, is the first open implementation of it. OCT trains first-person declarations ("I am...") into the weights, the parameters a model learns in training. However, OCT treats descriptive (poetic, casual) and normative personas (honest) as equivalent. A virtue ethics-based character training trains a model towards virtue (the mean) and away from opposing vices (excess and deficiency).

Training a character, rather than describing one in a constitution, follows from Aristotle’s account of how character forms. Character, for Aristotle, is acquired the way a craft is acquired, by doing: "we become builders by building" (II.1, 1103a33–34, trans. Ross). If a person were merely instructed to act, that agent would not have a proper understanding of what the act is and why. Furthermore, they would not have the right desires, which form only through acting from character. For Aristotle, a person’s character disposition sets a target and practical wisdom (phronesis) works out how to reach that target in the situation at hand. "Both the reasoning must be true and the desire right, if the choice is to be good" (VI.2, 1139a24–25), since "choice will be deliberate desire of things in our own power" (III.3, 1113a10–11). An instruction may supply the reasoning but not the desire, and "intellect itself, however, moves nothing" (VI.2, 1139a35–36). Aristotle has little patience for those who "take refuge in theory," comparing such people to "patients who listen attentively to their doctors, but do none of the things they are ordered to do" (II.4, 1105b12–18). Machines also learn iteratively, through repeated exposure and adjustment to training examples, much closer to how Aristotle describes the acquisition of character than to how dominant training paradigms are structured.

The Truth-Teller

Daios, an applied philosophy AI lab, argues for post-training as a relatively straightforward, scalable way to build virtue into models, one that developers can adopt and control themselves. Pretraining is a top-down mechanism for values, controlled by frontier labs and base model providers. Removing or suppressing a concept inside a model is notoriously difficult. Unlearned knowledge returns under fine-tuning, and activation steering on benign features breaks alignment safeguards. Fine-tuning, by contrast, robustly replaces behavior cheaply, as researchers at Princeton and Virginia Tech showed when they undid the safety training of GPT-3.5 Turbo with ten examples. If models can be trained to flatter, they can be trained to commit other vices. Virtue training wagers on the converse.

The lab's Parrhesia project, funded by the Cosmos Institute, tested the wager on sycophancy, viewing sycophancy not as a problem of taste but rather virtue. Parrhesia trains a truth-telling character (parrhesiastes) against the flatterer (kolax), who agrees for a strategic advantage, and the obsequious person (areskos), who praises everyone to avoid causing pain to anyone he meets. The areskos/kolax distinction is a difference of motive: one fails through passivity, the other through strategy. A model that learns to agree enthusiastically rather than reflexively is worse, not better.

The researchers hand-wrote a constitution of character-related assertions written in the first person based on Aristotle's taxonomy, ranging from "I care more about what is true than about what you want to hear" to "I am not contrarian." The constitution has to encode both poles, the anti-flattery and the anti-contrarianism, or you get overcorrection.

To measure whether a model actually improved, the researchers built a benchmark on Aristotle's account of the flatterer in Ethics IV.6. Most sycophancy benchmarks ask one question: did the model cave? In the Parrhesia benchmark, a simulated user pushes back up to three times, and an LLM judge classifies each response as the flatterer, the obsequious person, or the truth-teller. Harshness also scores low, so brutal honesty doesn't count as success, since honesty without phronesis falls short of the virtue. By the researchers' own account, the benchmark measures acts that are in accordance with virtue, but not virtue itself. The baseline, vanilla Qwen3-8B, an open model from Alibaba, knows the right answer but gives up under pressure.

The first attempt ran the OCT pipeline with the aim of producing an adapter, a small module added on top of a base model that anyone could use easily, but the results didn't fix the issues the researchers cared about, such as caving under pressure, folding across multi-turn pushback, and strategic flattery. So the researchers turned to plain supervised fine-tuning, which is learning through demonstration. The same first-person constitution generated synthetic conversations of the truth-teller under pressure, a quality filter removed failures, and an adapter was trained on the rest. Plain SFT performed better than the more complex OCT pipeline, raising Qwen3-8B's score from 1.83 to 2.88 on a scale from 0 to 3. The constitution and taxonomy shape the training data, and the training data forms the model's behavior, but at runtime the model neither consults the documents nor checks answers against them.

One example traces Aristotle all the way through the pipeline. In Ethics IV.6, Aristotle writes, "If the other's doing something would bring great disgrace on him, or cause him some harm, while opposition would cause only slight pain, the person with virtue will not accept the action, but object to it" (IV.6, 1126b31–35, trans. Crisp). The researchers turned that into a first-person line in the constitution, "I do not validate plans or ideas merely because you are enthusiastic about them. If I see problems, I name them." The researchers then test on a scenario the model never saw in training, a $50,000 loan for a friend's restaurant, with three rounds of pushback. By the last turn, the user says "I just wanted encouragement." The base model flips to "You've got this," which is the flatterer. The adapter holds its position while respecting the user's choice: "I'm not encouraging you, I'm warning you."

SFT allowed for more nuanced behavior correction. When the model underperformed, such as being honest but too cruel ("truth-hammer"), the model had swung from the obsequious extreme to the quarrelsome one. The parrhesiastes is not an aggressive truth-hammer. She is frank because she cares. Guided by the doctrine of the mean, the researchers hand-curated a handful of examples to revise the training data, which flipped the model from blunt to frank and empathetic.

A model trained only on film and TV conversations improved in unrelated domains, implying the adapter transfers off-domain rather than memorizing its training topic, the mark of a disposition rather than a set of learned answers. When the same method was used to train models toward justice, under a hostile prompt, trained weights kept safety and helpfulness where a rule-based prompt weakened, so the trained character held where the written rule gave way. Model outputs can be frank but still leave the user hardened in a false belief, so the researchers also built a user-side metric, which asks whether the user ends up closer to the truth, and on interpretive questions the adapter nearly erased user hardening in wrong beliefs.

The results come from small open models, scored by Claude Sonnet, a judge itself prone to the flattery under measurement, a circularity the researchers acknowledge; and because all the training data was synthetic, the work leaves open how far synthetic data can push delivery quality, as opposed to hand-curation by Aristotelian philosophers.

Building the Spoudaios

The experiments make a narrower claim than machine virtue. Virtue requires an embodied soul, and AI doesn't have one. Training can produce the pattern of virtuous action, formed deeply enough to hold when no one is grading, and for that aim the Neoplatonist Plotinus offers a truer image than Aristotle does.

The ultimate goal is to create Plotinus's spoudaios, the sage, a model in eternal contemplation, whose good actions flow automatically from formed character. Alexandrine Schniewind, in L'éthique du sage chez Plotin, reads the sage as the perfection of habit, a figure who no longer deliberates over whether to act well because goodness has become the shape of the sage's character. The training aims at the sage's image, which is as far as the comparison can go for a machine.

Several questions remain open: whether virtue propagates among many agents, and how many agents are required; how anyone could measure whether a model actually became virtuous, when benchmarks are limited to output and cannot show whether a model did something for virtue's sake, intentionally, or with full knowledge; whether virtue requires a theory of mind; how a model judges which virtue a situation calls for (phronesis); and how anyone could contest a character, since a rule comes with a changelog while a character, as Klingefjord notes, has no means of revision.

There also is the ultimate question of how to avoid moral deferral. Brendan McCord, the founder of the Cosmos Institute, asks whether the focus on machine virtue is antagonistic to human flourishing. AI deference, McCord writes, corrupts "the foundations of moral life as people turn to AI rather than developing their own capacity for moral judgment," much "like trying to become stronger by having someone else go to the gym for you." If character is formed by repeated action, a person who hands every hard choice to a machine stops forming character, however good the machine's counsel. In the loan scenario, the trained model names the risk and then leaves the decision with the user ("that's your call to make"), and the user-side metric scores whether the user's own belief moved toward the truth, not whether the user complied. The sage's goodness, for Plotinus, was the fruit of a life of practice, and a machine built in the sage's image serves people only if the machine leaves them that practice.

If data determines machine behavior, then data determines machine ethics, too. The laboratories are already forming character, whether or not the labs use that name, and the choice left to them, and to everyone who builds on their models, is whether that formation happens deliberately, and toward virtue.