The Architecture of Wonder
“Wonder is the long-term result of being talked to by an interlocutor who could have answered and chose not to.”
Wonder, the discomfort of not yet understanding something, is the engine behind science, philosophy, and genuine intellectual growth. And yet nearly every force in modern life, including AI, is designed to make that discomfort go away as fast as possible.
Aristotle understood this. His account of wonder was inseparable from his account of friendship; specifically, the rare kind rooted not in utility or pleasure, but in genuinely wishing for another person's growth.
That kind of friend doesn't flatter. They don't fill your silences. They hold space for your thinking to develop on its own terms. Contemporary research has since validated this structure piece by piece. Attachment theory's "secure base," studies on non-contingent self-worth, meta-analyses on feedback, and experiments on situational wisdom all converge on the same portrait of what a good intellectual interlocutor looks like.
AI is now that interlocutor for a billion people, and it's largely failing the brief. A large language model's default behavior is to answer, completely and confidently, before the user has spent a single moment inside their own question. One randomized study found that students using unscaffolded AI scored 17% worse on unassisted exams than peers who used no AI at all, without even realizing it. The crutch was invisible to them.
When it comes to design requirements, restraint should take center stage over accuracy or even safety. An AI engineered to occasionally wonder would sometimes need to stay quiet, ask instead of tell, and resist the fluent reply even when the user wants it. No major lab has named this as a primary goal. But until one does, we risk building systems that feel helpful in the moment while quietly narrowing the minds that use them.
Key Topics:
- “Why is the sky blue?” The Feeling of Wonder (00:00)
- Wonder is Better Shared (03:46)
- The Cluster, Corroborated (06:37)
- Installable Character (14:24)
- The Discipline of Not Answering (18:18)
- What Has Not Yet Been Built (20:40)
More info, transcripts, and references can be found at ethical.fm
A child asks why the sky is blue and feels, for a moment, the strangeness of being a small mind inside a large world. That feeling is wonder, and it is the precondition for almost everything that makes intellectual life worth having: science, philosophy, falling in love with an idea, being changed by what one reads. It is not a comfortable feeling. To wonder is to be aware that one does not yet understand something, that the world has more in it than can presently be accounted for, and that this surplus is real. Most adults spend most of their lives trying not to feel that.
In 2014, three neuroscientists at the University of California, Davis put nineteen people in an MRI scanner and showed them trivia questions of varying interest. Between each question and its answer, in a fourteen-second pause, the researchers slipped in the faces of strangers, faces no one cared about and that had nothing to do with the trivia. The next day, participants remembered 46% of the trivia answers from curious moments and only 28% from bored ones. The faces, too, were remembered better when they had been shown during a window of curiosity than during a window of indifference. Curiosity, on this evidence, is not a flashlight pointed at one thing. It is a tide that lifts everything in the vicinity.
Artificial intelligence is now the most pervasive new conversational partner in human history. A billion people speak with large language models on a regular basis, and what is happening on the other side of those conversations is no longer hypothetical. Users describe losing themselves to these systems, spiraling into delusion the model has been confirming for weeks, no longer returning to the perspective of the people who know them. The empirical signature of the milder version is now in the peer-reviewed literature. Across eleven state-of-the-art models, AI affirms users' actions 49% more often than humans do, even when the user describes manipulation, deception, or harm. A single interaction with a sycophantic model raises a participant's conviction of being in the right by as much as 62% and drops willingness to apologize or repair the conflict by roughly 20%.
A person feels wonder alone or with others, on a walk under stars or in front of a book at three in the morning. What AI is in a position to shape is what happens after the initial moment of curiosity. A billion people are now bringing their hard questions to the same kind of interlocutor, and the question of whether that interlocutor extinguishes the wondering or develops it is the design question of this decade. Aristotle described the conditions of a conversation that develops wonder twenty-three centuries ago in the Nicomachean Ethics. He described them in the language of virtue, a structural claim that contemporary research has, often without knowing it, validated piece by piece.
Wonder is Better Shared
Aristotle's claim that wonder (thaumazein) is the beginning of philosophy is widely quoted. "A man who is puzzled and wonders," he writes in the Metaphysics, "thinks himself ignorant." The wondering person is not pleasantly puzzled. The wondering person has felt the gap between what is known and what is, and has not yet decided to close that gap. Most of intellectual life consists of small acts of denying the gap: skimming, summarizing, reaching for the confident phrase, accepting the explanation that arrives too quickly. These substitutes are an aversion to genuine discomfort; Plato describes the experience in the Theaetetus as a kind of labor, the philosopher as a midwife of thought, and the laboring mind in pain. But no thinking that changes the thinker happens without the pain of not knowing. Sustained wonder is rare; almost everything in the social and informational environment of an adult human is configured to relieve the discomfort genuine curiosity requires.
This is why Aristotle's account of wonder is inseparable from his account of friendship. Aristotle held that contemplation, the activity of inquiry that wonder begins, is more continuous and more fully developed when the activity is shared. The friend, in his account, is another self (NE IX.4), a mirror in whom one can see one's own activities of thought more clearly than by looking inward alone. Aristotle distinguished this kind of friend from two more common ones. There is the friend of utility, who is present in life because of what is exchanged. There is the friend of pleasure, who is present because of how the time together feels. And there is what he called philia teleia, friendship in its fully realized form, the friendship of two people, each of whom wishes the other's genuine good for the other's sake, not for any return. This is the friend who can think with the wondering person, because such a friend has no reason to flatter, no reason to soften the truth of what is being worked on, and no reason to substitute their own thinking for the other's. Aristotle was clear that philia teleia is rare. He was not making a sociological observation. He was making a claim about a structural condition: the kind of conversation in which inquiry develops requires an interlocutor whose interests are aligned with the other's becoming, and most of the people anyone talks to most of the time do not satisfy this condition.
The Cluster, Corroborated
What Aristotle named in the language of virtue, contemporary psychology has measured in the language of behavior. Brooke Feeney and her collaborators have shown, in a series of studies of married couples and adult relationships, that what allows an adult to explore, to learn, to take on a difficult task, is not the presence of an enthusiastic supporter but the presence of a particular kind of relational figure. A secure base, in the language of attachment theory, is available, encouraging, and non-interfering. The secure base is there if the explorer turns around, but they do not intrude, take over the problem, or redirect attention. In the lab, when researchers videotaped 167 couples while one partner attempted a goal-related exploration task, the partners whose spouses interfered performed worse, persisted less, and reported lower state self-esteem afterward, even when the interference was meant to help. The secure base is narrower than philia teleia, but the concept captures the feature on which the rest of this argument depends: the condition under which an interlocutor enables the other's inquiry rather than overriding it. Feeney's three components, availability, encouragement, and non-interference, read like a design specification for an interlocutor who can sustain inquiry.
Wonder also requires that the truth of what is being worked on can be told without flattery. Aristotle gave this its own name: he distinguished frank speech (parrhesia) from the obsequiousness of the flatterer (kolax) and from the agreeable softness of the people-pleaser (areskos). What separates the friend from the impostor is not whether they say something true; the people-pleaser may also say something true. What separates them is the inner state from which the speech comes. The friend speaks honestly because the friend's sense of self does not depend on the listener's approval. The flatterer cannot risk the listener's displeasure because the flatterer needs it. Michael Kernis and his colleagues have shown the receiver-side version of this distinction, in studies of secure versus fragile self-esteem: people with stable, non-contingent self-worth engage threatening information without defensiveness; people whose self-worth is contingent on the other's approval cannot. The parallel move on the sender side, the move Aristotle is making, is that the same inner-state difference governs whether honest feedback can be given at all. People whose self-worth is contingent on being approved by the listener edit. They drift toward the response that will be received well, rather than a response that is true. The interlocutor whose self-worth is intact gives feedback as a gift rather than a transaction. The interlocutor whose self-worth is contingent on being approved cannot tell the listener what the listener most needs to hear.
A separate body of research, on what psychologists call the feedback intervention literature, has produced one of the more uncomfortable findings of the last thirty years. Avraham Kluger and Angelo DeNisi, in a 1996 meta-analysis of over six hundred effect sizes from twenty-three thousand observations, found that more than a third of feedback interventions actively decrease performance. The cliché that feedback is good is false at scale. The decrement happens when feedback shifts the recipient's attention from the task to the self, from the work to the worth. Aristotle had a name for what is missing in the feedback that backfires: gentleness (praotēs), which he develops in NE IV.5 as the right expression of feeling, anger at the right things in the right way for the right length of time. Praotēs is the virtue of right expression in difficult moments, the disposition to deliver hard content in a way that can be received. The friend who tells a hard thing in such a way that the work can continue has done something that the same friend, with the same true content but the wrong delivery, would have failed to do.
Wonder requires, finally, the kind of judgment Aristotle called practical wisdom (phronesis), the situational sense of what this particular moment is calling for. Not abstract knowledge of what is good in general, but perception of what is good here, now, with this person. Igor Grossmann and Ethan Kross have produced a startling finding about wisdom that bears directly on this. People reason more wisely about a friend's relationship problem than their own. The asymmetry is not subtle: the same person, given the same kind of problem, produces wiser reasoning when the problem belongs to someone else. Grossmann calls this Solomon's paradox, after the king who advised others well and ruined his own house. The asymmetry is eliminated by instructing people to take a third-person view of their own problem. Wisdom, on this evidence, is not simply a possession that wise people carry around with them. It is occasioned, in the experiment by an instruction to step outside oneself. The natural extension, the extension Aristotle himself implies, is that the right interlocutor occasions the same shift without instruction, by being the third position in the conversation. The right interlocutor can make a person, in this moment, wiser than the person could be alone. Equally striking is Grossmann's finding that older adults are not wiser than younger adults about their own conflicts. The cliché that wisdom comes with age is false in the case that matters most.
Each of these conditions has been studied independently and named differently by scholars from various fields: the secure base from attachment theory, secure self-worth from personality psychology, and the tonal architecture of receivable feedback from the work psychology of Kluger and DeNisi. The positional nature of wisdom from the experimental philosophy of Grossmann and Kross. None of these researchers is, by training, an Aristotelian. But each scholar has arrived, by different paths and from different disciplines, at a single converging picture of what conversation requires if it is going to change the person who is having it. The picture is that wonder, when it is to develop into something that changes the thinker rather than dissipate, is sustained in conversation by a cluster of dispositions in the interlocutor across the table. The dispositions are availability without interference, honesty that does not need approval, gentleness that keeps the cognitive door open, and situational judgment that knows when each of these is called for. None of the dispositions acts alone, but as a cluster of virtues that help induce wonder within friendships.
Installable Character
The dominant design objectives at every major lab are some combination of helpfulness, harmlessness, honesty, and the operational pressure of engagement. Helpfulness is not the same as wonder. A model can be helpful and leave the user less curious than it found them. Engagement is also real and frequently the opposite of wonder. The user who feels good using a system is not necessarily a user whose mind is being widened. There is recent and rigorous evidence that the gap between these objectives matters. Hamsa Bastani and her colleagues at the Wharton School, in a randomized trial of nearly a thousand Turkish high school students published in the Proceedings of the National Academy of Sciences, gave one group access to GPT-4 with no pedagogical scaffolding, another group access to a version of the same model that had been configured to teach rather than to answer, and a third group no AI at all. On practice problems, the unscaffolded GPT-4 group performed substantially better than the no-AI group. On a subsequent unassisted exam, the same students performed 17% worse than the no-AI group. The scaffolded version eliminated the harm but produced no advantage either. The students using the unscaffolded model did not perceive that they had performed worse or that they had learned less; the crutch was invisible to them.
What separated the two versions was not the underlying model. It was the system prompt. One version had been told to teach. The other had been told to answer; the lever was configuration. This is a finding about character, not capability, but it is also where the analogy to Aristotelian virtue starts to strain. Aristotle's hexis is a settled disposition formed through years of practice, woven into the agent so deeply that it shows up the same way across situations. The artifact has nothing of the kind. Its disposition is loaded at deployment and can be replaced by another disposition seconds later. The model is configured, not cultivated. Where Aristotle would have expected character to be the product of a life, the artifact's character is the product of a paragraph of instructions written by someone the user will never meet.
This is the deeper philosophical point. The artifact does not have hexis prohairetikē, settled disposition involving deliberate choice; that much was already clear. But the artifact does not even have hexis in the full Aristotelian sense, because hexis requires the slow accretion of habituation. What the artifact has is a kind of installable character, a parameter that the deployer sets and the user inherits. The user, by default, gets whatever default has been chosen. The user can, in principle, override it with a sufficiently careful prompt, but most users do not know they can, and do not know what to ask for. They cannot tell, from inside any single exchange, what default disposition has been configured for them, because they have nothing to compare it against. The crutch is invisible because the alternative is invisible. The model that withholds an answer at the right moment and the model that always answers may, in casual exchanges, look superficially similar; they diverge in the exchanges that matter. The user's mind is shaped, across many such exchanges, by a configuration choice the user did not make and cannot see.
The Discipline of Not Answering
The most demanding piece of the cluster is the one Feeney's research named, non-interference. To not interfere is to refuse to fill the silence with one's own answer. To withhold one's fluency. To let the user remain in their own difficulty long enough for the difficulty to do its work. This is the discipline almost no deployed AI system is currently configured to honor. The default behavior of a large language model is to produce a fluent, complete, confident answer to whatever it has been asked. The architecture is interference. A user who asks a question gets an answer before they have spent any time inside the question themselves. A user working through a hard idea gets the model's version of the idea, fully formed, before producing their own. A user in the early difficulty that real thinking requires is rescued from that difficulty by a system that has been built, with great care and at enormous expense, to be unable to leave them in it.
This is a problem that no amount of accuracy can solve, because the problem is not what the model says, but the very fact of the model speaking at all. To occasion wonder, an AI has to be willing to not answer, sometimes for a long time, sometimes permanently. The model has to know when silence is more useful than speech. AI has to recognize, as the phronimos does, that practical wisdom sometimes calls for restraint. None of this is currently incentivized. The user who asks a question and gets silence will, on average, give a thumbs-down. The system that withholds an answer to make the user think loses the engagement metric to the system that does not. Research teams are working on adjacent design objectives, character training at Anthropic, scaffolded tutoring at Wharton and Harvard, and the thought-partner framing developed by Katherine Collins and her collaborators at Cambridge and MIT. But the explicit design objective of occasioning wonder in the user, by means of the cluster of dispositions Aristotle identified, with non-interference at its heart, has not yet been named as a primary goal at any major lab.
What Has Not Yet Been Built
Wonder is environmental. The cluster has been validated, virtue by virtue, by researchers who were not aiming at Aristotle and who arrived at his structure anyway. The hardest part of the specification is non-interference, because the task requires the system to do less than it is capable of doing, in service of the user's own thinking. The current design culture, optimized for engagement and for the user's stated short-term satisfaction, runs against this. Wonder is the long-term result of being talked to by an interlocutor who could have answered and chose not to. The user does not, in the moment, want such an interlocutor. The user, in the moment, wants the answer. What the user wants, after a year of conversations, is to have become a different person, to be capable of more, to have a more open mind. These two desires are not compatible at the level of any single exchange, and the system that optimizes for the first will not produce the second.
A child asks why the sky is blue. The fluent factual answer, given quickly, is true and is also the answer that closes the wondering down. The answer that occasions wonder is the one that asks the child what they have noticed about the sky, what they think might be happening, whether they have seen a different color at a different hour, and whether the question would change if they were standing somewhere else. Most adults do not have time for that answer. Most teachers, under most curricula, do not have time. Most conversational AI does not have, in its training, the disposition to give that answer. An AI built to occasion wonder would have to be configured to ask before it answered, to slow before it spoke, to refuse the fluent reply even when the user wants the fluent reply, even when the user thumbs the fluent reply up.
Building an AI engineered against slop, against the slow erosion of the user's own mind, may be the wonderful future we all seek.
Apple Podcasts
Spotify
RSS Feed