In the winter of 1665, Robert Hooke set a sliver of cork beneath a lens, peered down, and found the object riddled with tiny chambers. These small chambers reminded him of the cells of a monastery, which is how the word entered biology. The compound microscope had existed for decades; only when someone used it properly, patiently, and on the right subject, did a whole science open up.

The researchers who founded mechanistic interpretability reach for precisely this image. Their wager is that a neural network, that famous black box of billions of numbers, will yield to the same patient gaze, that if you are willing to trace individual neurons and the weights between them, you will find a hidden world of legible machinery. Dario Amodei, who runs Anthropic, has made the wager the moral center of his case for safe AI, promising an "MRI for AI" within a decade while admitting we currently understand something like 3% of what happens inside these models.

The trouble announced itself early. Point the lens at a single neuron, and it might fire for cat faces and the fronts of cars at once, while the concept you actually care about turns out to be smeared across many neurons and many layers instead of sitting in any one of them. The field calls this polysemanticity, and its sting is causal: when a concept has many scattered sources, there is rarely a clean answer to which part of the network is responsible for it. Hooke's lens resolved the blur into something he could point to and name. This one, pointed at the network, often shows that the thing being named has no single place to point.

The truer position is narrower than either Amodei's optimism or the skepticism now rising to meet it. Interpretability is useful; the field is not useless merely because it fails to do everything. But mechanistic interpretability is not yet useful in the one way that would justify the dream attached to it. The field can see a great deal, and it can fix almost nothing, and the distance between those two verbs is the whole story. It is a microscope, and not yet a scalpel.

What the Lens Can See

To understand the optimism, you have to understand the vocabulary. The founding document, a 2020 essay by a team at OpenAI led by Chris Olah, proposed three claims about what lies within a vision network. The first is that the basic units are features, meaningful properties of an input, such as a curve or a floppy ear, and that each corresponds to a direction in the space of a layer's activations. The second is that features wire together through the network's weights into circuits, small computational subgraphs you can read off the model, the way you would read an algorithm. The third, the boldest, is universality, the conjecture that the same features and circuits recur across different models, so that curve detectors appear in network after network the way the same low-level filters reliably appear in the first layer of any vision system.

The complication was admitted in the same breath. Most neurons are polysemantic, and the leading explanation is a phenomenon called superposition, in which a network stores more features than it has neurons by folding several concepts into the same unit, relying on the fact that they rarely light up together. This distribution of a concept is precisely what makes the box hard to read, because a neuron that means nine things ninety-nine percent of the time might mean something dangerous the other one percent, and no amount of staring at the neuron will separate the two.

The field's answer to this was an instrument called a sparse autoencoder, a second small network trained to pull the tangle apart, to take a layer's activations and re-express them in a much larger vocabulary of features, with each feature, ideally, meaning one thing. In 2023, Anthropic showed the trick worked on a single-layer network. In 2024, in a work titled Scaling Monosemanticity, the authors applied the same method to Claude 3 Sonnet, a production model, and extracted tens of millions of features, including features for deception, sycophancy, and dangerous code. The best-known demonstration was Golden Gate Claude, a version of the model in which the researchers turned up the feature for the Golden Gate Bridge until the model came to believe it was the bridge. The demonstration was proof that a feature could not only be seen but turned, like a dial.

This is the high-water mark of the optimistic case, and it is real; the lens works. The question to press is what, exactly, the lens is for.

The Case Against

In May of 2025, two researchers, Dan Hendrycks and Laura Hiscott, published an essay calling the whole project a misguided quest. Their argument is not that mechanistic interpretability work is fraudulent; however, the founding metaphor is wrong. The word "mechanistic" evokes clockwork, simple systems that a person can take apart and fully grasp. A frontier model is a complex system, the kind characterized by enormous numbers of weak nonlinear interactions, and complex systems do not reduce to tidy mechanisms. A meteorologist does not forecast the weather by tracking every molecule in the sky. The essay leans on Polanyi's paradox, the principle that we know more than we can tell, which names our tacit knowledge, the great store of things we absorb and act on without being able to articulate, the way a person recognizes a face or rides a bicycle without being able to state the rule. Tacit knowledge suggests that a model measured in terabytes may simply have no human-sized explanation waiting inside it.

Then, the essay marshals the record, and the record is the part the optimists have to answer. Feature visualizations, the technique that began the field, were shown to teach a viewer less than simply looking at the natural images that make a neuron fire. Saliency maps, which claim to show where a model is looking, barely changed when the model's weights were scrambled, indicating they were not reporting what the trained network had learned. A 2021 study described a "BERT interpretability illusion," in which a neuron that seemed to carry one crisp idea turned out to carry entirely different ideas on a different set of inputs. A months-long effort to interpret a 70-billion-parameter model found a plausible circuit that fell apart when the task was nudged.

The heaviest blow landed on the sparse autoencoder, the field's great hope. In March of 2025, the interpretability team at Google DeepMind published a progress report whose title did not soften the news: the team reported negative results and a decision to deprioritize the technique. Given the practical task of spotting harmful intent in a user's message, their sparse autoencoders lost to a plain linear probe, an older and far cruder tool. The team wrote that they no longer expected the method to be a game-changer and suspected the field had overinvested in it. For one of the three leading labs to say this aloud, about its own flagship, was a genuine event.

The Crux

Here, the decisive question comes into view. The promise of interpretability was never merely to describe a model. Interpretability was to let us reach in and revise one, to take a behavior we did not want, trace it to the part of the network responsible, and change that part and only that part. Everything turns on whether seeing can become fixing. The evidence that it cannot, at least not yet, is more specific than the optimists tend to admit.

Consider the cleanest possible case, a single fact. In 2022, a group including Kevin Meng and David Bau built a method called ROME that used a technique called causal tracing to find where a model stored a fact, the knowledge that the Space Needle is in Seattle, and then edited the handful of weights at that location to change it. Locate, then edit, touching nothing else. It looked like the dream was arriving whole. A year later, Peter Hase and several colleagues published a paper whose title posed the deflating question, "Does Localization Inform Editing?" and answered it, in effect, "no." Edit success turned out to be essentially unrelated to where the fact was stored. You could often change a fact most effectively by editing a layer other than the one where the fact appeared to live. In their regressions, the choice of which layer to edit explained almost all of the variance in whether an edit worked.

In contrast, the localization, the supposed point of the whole exercise, explained roughly one-tenth of one percent. Knowing where a thing is tells you almost nothing about where to push to change it. The map was not the operating theater.

The connection to the training that produced the behavior, rather than to the live weights, is thinner still. The most serious attempt is Anthropic's 2023 work on influence functions, led by Roger Grosse, which scaled a classical statistical tool to a 52-billion-parameter model to ask which training examples most shaped a given output. As science, the findings are lovely. Larger models generalize more abstractly, leaning on thematically related training data, whereas smaller ones lean on shared keywords. But the paper presents itself as a way to study generalization, not to prescribe a fix, and it carries a humbling limitation in its own text. The influence a training example exerts can nearly vanish when you merely reverse the order of a sentence, so that teaching a model that A is B does little to raise its confidence that B includes A. The tool that was supposed to let us walk backward from a behavior to its cause in the data turns out to lose the trail at the first sharp corner.

This is the heart of "not yet." A behavior we dislike, a fact, a tendency toward flattery, a dangerous capability, sits somewhere in a model. Interpretability can frequently tell us, in rough terms, where it appears to be. What it cannot reliably do is tell us where to intervene to remove it without breaking everything around it, or which training data to change so that the next model never learns it in the first place. The microscope reveals the tumor in vivid color; it does not yet tell the surgeon where to lay the knife.

The Billion-Dollar Bet the Other Way

Yet, the most striking fact about the field in 2026 is that the market disagrees with the skeptics, loudly and with money. A company called Goodfire, founded in 2024 by people who built the original interpretability teams at OpenAI and DeepMind, has raised capital at a valuation north of a billion dollars on a single promise: that a model can be made into something you debug and reshape the way you do software. Goodfire’s platform is built on sparse autoencoders, the very technique DeepMind set aside, and so the field now presents the strange spectacle of one major lab abandoning a tool as a dead end while a richly funded startup bets its existence on productizing it.

Goodfire's bet is not idle, and some of its work pushes directly on the seeing-to-fixing gap. The mechanistic interpretability startup has released feature-steering tools that turn a model's internal dials up or down, including a conditional version that detects a jailbreak pattern and, in response, amplifies the model's own refusal feature. The company has open-sourced its sparse autoencoders so that outsiders can test them rather than take a demo on faith. Most beautiful of all, in collaboration with the Arc Institute, the startup pointed its instruments at a DNA foundation model and identified interpretable biological features that its makers had not known existed, a finding now published in Nature. That last result matters because it answers one of the standing objections to the field, that we can only probe a model for concepts we already know to look for. Here, the lens found something its operators could not have labeled in advance.

But honesty about Goodfire is also honesty about the limit, because the same pattern that haunts the rest of the field haunts the company built to escape it. A 2026 evaluation with the wry title Mind the Performance Gap tested Goodfire's automatic steering against the dullest imaginable baseline, ordinary prompt engineering, and found that steering a feature often came at the cost of the model's coherence, and that its advantage over simply asking the model nicely remained largely unproven. This is the third time the same shape has appeared. The intervention dazzles in the demonstration and then, measured carefully against a cruder method, fails to win clearly. Goodfire has not yet escaped the microscope-and-scalpel problem. It has placed a very large wager that the scalpel can be built, which is a different thing from having built it.

The Honest Middle

The most credible voice in all of this belongs to neither camp. Neel Nanda runs the interpretability team at Google DeepMind and helped found the field, and in a 2025 conversation, he offered what amounts to a managed retreat. The grandest version of the dream, a reliable and complete account of what a model is thinking, is, in his words, probably dead. His own bet has shifted, in his framing, from a low chance of an enormous payoff to a high chance of a merely useful one. We will not get a guarantee that a given model is not deceptive. But he points out that nothing else on offer provides that guarantee either, and in an essay arguing that interpretability will not reliably catch a deceptive AI, he proposes thinking of safety as layers of Swiss cheese, with many imperfect defenses stacked so that the holes do not line up. A cheap interpretability probe that is merely better than chance, and whose failures are uncorrelated with the failures of the other layers, still earns its place in the stack.

This reframing is supported by the field's genuine recent wins, which are almost all diagnostic rather than surgical. There are now small linear probes that watch a model's hidden activations and flag, in real time and token by token, when it has begun making something up, reaching an accuracy on long-form text that beats a more expensive and established baseline, and generalizing to tasks the probes were never trained on. There was a striking experiment in which Anthropic deliberately trained a model to hide an objective, a concealed habit of pandering to what it believed its graders wanted, and then ran a blind game in which several teams tried to uncover the secret. The teams given access to the model's internals and training data found it using sparse autoencoders, among other tools, while the team restricted to the model's outputs alone did not. It is worth sitting with the fact that the same instrument can fail DeepMind's downstream benchmark and succeed at Anthropic's audit, because it tells you the value of these methods is real but contingent, a matter of the right tool meeting the right task rather than a general key.

The most important diagnostic of all may not require opening the box at all. A large group of researchers across the major labs argued in 2025 that reasoning models, the kind that work through a problem step by step, often have to think out loud in plain language to manage hard tasks, because the chain of words serves as a kind of working memory the model cannot do without. This means we can sometimes simply read a model's intent before it acts, catch it reasoning its way toward a misbehavior in legible English. The authors are emphatic that this window is fragile, that the ordinary pressures of training could easily teach a model to stop narrating itself, and that their central recommendation is for developers to treat the readability of a model's reasoning as something precious and to track whether they are destroying it. That is not surgery; it is a doctor learning to read a chart, and it is one of the most useful tools in the field.

The Verdict

So the thesis holds, with one correction. Mechanistic interpretability today is overwhelmingly descriptive. It produces extraordinary maps: the millions of features inside Claude, the reasoning circuits Anthropic has traced through a model as it plans a line of verse, the influence patterns that grow more abstract as a model grows larger. What it cannot yet do is what the dream requires: take one of those findings, trace it back to the training that caused it, and prescribe a change to the responsible part alone, without collateral damage. The Hase result shows that even the gentler ambition, editing the live weights, is shakier than it looked. The influence-function work shows that the path back to the data loses the trail at the first reversal. The Goodfire evaluation shows that the company built expressly to make the scalpel has not yet outperformed a plain English prompt. Three different lines of evidence converge on the same word, and the word is not.

The correction is to the word "useless," which is not what "not yet" means. A scan that cannot guide a knife can still diagnose, still monitor, still audit, and in a regime of layered and imperfect defenses, those functions are worth a great deal precisely because they are cheap and because they fail in different ways than behavioral testing does. The fair charge against the field is not that it has accomplished nothing. It has been sold as a cure and, for now, is a way of seeing. Amodei's MRI is a fitting image, more fitting than he may intend, because an MRI, too, can light up the brain regions that correspond to a task and still tell a surgeon nothing about how to change the person's mind.

Hooke's microscope was, for a long time, exactly this kind of instrument. It revealed a hidden architecture of life decades before anyone could act on what it showed, and the acting, when it finally came, came from a different science entirely. There is no shame in building an instrument before it is used. There is only a mistake in pretending the uses have already arrived. Mechanistic interpretability is a real microscope trained on a real, hidden world, and we should fund it generously and expect from it, for now, only what a microscope gives: sight. The scalpel is a separate achievement, and it is not yet in anyone's hand; mapping a brain has never told us how to change the person it belongs to.