Neurons as an Interpretable Basis

Abstract

A companion to Neural Networks Are Already Clean

An intuitive way to interpret a neural network is to study its neurons; that is, the hidden units of its MLP layers. However, they have long appeared too complex and polysemantic for direct study. This view has led much of the field to focus on learned feature spaces over sets of neurons, which makes interpretability more challenging: we first need to transform the model before normal science can begin.

Building on a growing body of literature, our work suggests that neurons are more interpretable than previously assumed. We study the neurons of Qwen3.6-35B-A3B using sparsification methods, attribution tools, and research agents, and find three properties that make neurons interpretable units of computation:

First, they are already functionally sparse. Ablating the 92.2% weakest MoE neuron activations for each forward pass has only modest effects on model capabilities – a 2.9% increase in language model loss.

Second, neurons have relatively few semantic groupings (modes) of computational functions, and we can identify these and describe them concisely. The majority of neurons have either one mode only or a single mode that is overwhelmingly causally significant. In cases where multiple causal modes are present, the number is small and legible to our methods.

Finally, we can use agent-driven experimentation to generate neuron descriptions in natural language, and these descriptions allow us to predict, simulate, and activate the neuron. Providing the description to a judge model allows us to (1) predict whether the neuron will fire in a given passage of text with 95% accuracy vs random text; (2) simulate its token-by-token behavior in high-activation regimes sufficiently to recover 73% of the damage (as a proportion of a ceiling of damage restorable) from the neuron’s ablation; and (3) write text that causes the neurons to fire more strongly than anything observed in a 10B-token corpus.

Overall, we argue that neurons are meaningful units of computation that can be effectively used for interpretability in their own right, letting us study the model on its own terms rather than through a learned proxy.

Introduction

In the previous essay in this series, we argued that neurons in deep learning models are more interpretable and cleaner than has often been suggested. In this article, we describe what this means in more detail and share our empirical findings. Throughout, the term “neuron” refers to an individual computational unit in the intermediate layer of a transformer MLP, whose output is a scalar activation, computed after the nonlinearity (SiLU), as explained in the Background section below.

Neurons were the first units through which deep networks were interpreted, and early work found them to be closely aligned with recognizable concepts (Le et al., 2012; Zeiler & Fergus, 2014; Bau et al., 2017; Olah et al., 2017; Radford et al., 2017; Goh et al., 2021). However, this story was complicated by observations of so-called polysemanticity—the idea that individual neurons respond to seemingly unrelated inputs (Nguyen et al., 2016; Olah et al., 2020). Polysemanticity is related to the proposal that more concepts are represented in a network than the number of neurons in the network, an idea canonized in the literature on “superposition” (Elhage et al., 2022; Arora et al., 2018; Park et al., 2023; Costa et al., 2025). This, in turn, led to the development of learned overcomplete feature dictionaries in place of neurons as computational units of analysis (Bricken et al., 2023; Cunningham et al., 2023). Despite these trends in the field, interpretability researchers continued to investigate respects in which neurons may still constitute a meaningful unit of analysis (Geva et al., 2021; Dai et al., 2022; Geva et al., 2022; Gurnee et al., 2024; Arora et al., 2026; Dravid et al., 2026; Feucht et al., 2026; Shabalin & Piotrowski, 2026). We continue this line of work, introducing new ways to factor and understand what individual neurons compute.

Whether a neuron is labeled monosemantic or polysemantic can depend on choices about what constitutes a single concept, and responses that initially appear unrelated may share a regularity that becomes apparent with a better understanding of the computations they support (Murphy & Medin, 1985). We therefore characterize neurons by their computational functions as our primary object of study, considering semantic similarity of the texts that activate them as a secondary question. We study the circuits in which each neuron participates, tracing how upstream neurons drive its activation and how downstream neurons use its output. This lets us examine how a neuron’s role varies across contexts and assess whether those roles admit a concise description.

A common view is that neurons are an intractable unit of analysis. In the worst case, an investigation could have demonstrated this in three ways: (1) Individual neurons might rarely play identifiable computational roles, so that meaningful units would be clusters of neurons, or directions and manifolds in activation space; (2) Each neuron might take extreme values on a wide and heterogeneous range of inputs that no short description unifies; (3) Or neurons might resist descriptions precise enough to predict when, and how strongly, they fire. We test each possibility directly: we ablate weak activations, determine the semantic range of neurons’ computational roles, and predict and simulate activations from descriptions alone.

Our results provide evidence that neurons are tractable units of analysis. Neurons (1) exert most of their downstream influence in high-activation regimes (above the 97th percentile at the positive extreme and below the 3rd at the negative); (2) have a small number of computational roles, distinguished by their interactions with other components of the network; and (3) can be given concise descriptions that predict and simulate their activations. The three results share a pattern: the stronger a firing, the more the model depends on it and the more precisely we can describe it.

The picture that emerges is not that every neuron means one thing, but that a neuron’s important firings have a small set of computational functions, and a short description captures them. Some neurons are genuinely polysemantic, and some behavior remains unexplained, but so far little of it appears critical for the model’s computation.

We therefore find that neurons are a useful and interpretable basis for understanding LLMs.

This is the first in a series of technical articles exploring neuron interpretability. This post introduces our initial findings and sets the frame for further exploration.

Our primary findings are as follows:

Neuron Sparsity and Effective Range

The two central questions of this section are:

1. Are individual neurons, instead of distributed directions spread across many neurons, the right basis for analysis of MLP computation? High levels of sparsity in MLP computations would suggest that neurons carry significant parts of the computation on an individual basis.
2. What sorts of neuron activations are meaningful and necessary for computation – do strong activations suffice, or are weak activations indispensable? A highly sparse regime would suggest much higher importance for strong activations.

If the conceptual primitives for transformer computation tend to be spread out among vast arrays of neurons in a distributed code, we should expect high levels of sparsity to sabotage these computations. On the other hand, if neurons tend to be privileged, meaningful units of computation, we should expect these primitives to be more localizable and sparse.

Earlier work has observed that neurons are often quite interpretable and consistent (across activation examples) in their strongest activations, but much more diverse and hard to understand at weaker levels (Szegedy et al., 2014; Morcos et al., 2018; Zhou et al., 2018; Leavitt & Morcos, 2020; Ghorbani & Zou, 2020; Dhamdhere et al., 2018). This motivates the concern: what if weak activations are both indispensable and uninterpretable?

We find that models can still perform surprisingly well even when all weak activations (e.g., between the negative and positive extremes of p10 and p90, or even p3 and p97) are mean-ablated, achieving close to parity with the intact model on a number of non-reasoning benchmarks. More importantly, we test an ablation where every layer retains only the top-320 neurons (out of 4,096 routed neurons) by absolute activation strength for a given step of inference, ablating 92.2% of routed neurons, and show that this model performs comparably to the base model on non-reasoning benchmarks.

For all experiments, we leave the shared expert neurons unchanged, since these neurons are effectively always on and function differently from the expert neurons. In total, the shared experts make up 20,480 of the model’s 5.26 million neurons (~0.4%).

For reasoning benchmarks, we observed that the ablated model would often run into reasoning loops and self-check excessively, never completing its answer–though by checking its reasoning output, we were able to determine that the correct answers were present. With a brief fine-tuning step, we corrected the reasoning errors and met or exceeded the original model’s performance on a broad set of thinking-based benchmarks for question-answering, code, reasoning, and math. Considering that the fine-tuning tokens used were a small fraction of a typical post-training run, this provides evidence that the fine-tuning primarily corrected reasoning rather than adding new information to the model, and that the majority of the information required for performance on benchmarks was present at the higher activations.

We provide further evidence for sparsity in the Appendix: we preserve specific model predictions with even fewer neurons using the gradient-based attribution method, RelP, and we elaborate on recent findings (Shabalin & Piotrowski, 2026) showing that transcoder replacement models tend to discover neuron-localized features even when not constrained to do so.

Sparsification by Activation Strength

Our first set of neuron sparsity experiments uses neuron activation strengths to determine which neurons to ablate. We consider three methods—mean-ablating (on an individual neuron basis, across all neurons in the model) the central percentile bands of neuron activations, expert top-k filtering, and layer top-K filtering—and measure their effects on loss, KL, and a range of benchmarks. The ablated bands are symmetric in percentile rank around the median: i.e., p1 is a strong negative activation, p99 is a strong positive activation, and weak bands are p10-p90, p25-p75, etc.

Ablation Within Top Percentiles

A natural experiment is to start with the weakest neuron activations and ablate them during inference, measuring the resulting loss and KL divergence. Since SwiGLU neurons are bipolar and neurons have different activation strength distributions, we compute neuron activation strength percentile bins and apply mean ablation to everything between the positive and negative extremes (Li et al., 2023a; Liu et al., 2023a; Mirzadeh et al., 2023; Zhang et al., 2022; Liu et al., 2024; Lee et al., 2024; Jaszczur et al., 2021; Ahmad & Scheinkman, 2019; Makhzani & Frey, 2013; Song et al., 2024; Zhang et al., 2024; Wang et al., 2024; Song et al., 2023; Alizadeh et al., 2023; Zhang et al., 2023). These initial experiments were conducted without the router weight scaling, and the percentiles shown likewise do not include this scaling.

Top-K Filtering

Inspired by Shabalin & Piotrowski (2026), we attempted to find a more sparse regime by applying a top-K function (based on raw neuron activations) across each of the experts in the model. We tested top-32 and top-64, which retain 6% and 12% of routed neurons per token respectively. Experts have 512 neurons each. In practice, the top-64 neurons primarily lay at or beyond the p5 and p95 extremes (Sun et al., 2024; Dettmers et al., 2022; Kovaleva et al., 2021; Puccetti et al., 2022; Bondarenko et al., 2023; Olshausen & Field, 2004). As with the ablation within top percentiles, this was performed without the router weight scaling.

Router-Weighted Top-K Filtering

To account for expert router weighting, we apply our activation definition above and score each routed neuron by its router-weighted activation. We then apply a single top-K over all 8 × 512 = 4,096 routed neurons of each token rather than one per expert, so the budget can shift toward experts whose neurons have the strongest activations after router weighting. We keep the top 320 (7.8%) of routed neurons (40 per expert on average, 92.2% sparsity) and zero the rest before the down projection. Here, most activations exceed the p3 and p97 extremes. Like other activation-sparsity methods, the mask is computed from each token’s own activations at inference time and needs no training (Liu et al., 2023; Lee et al., 2024; Liu et al., 2024; Wang et al., 2024; Makhzani & Frey, 2013).

Performance Under Ablation

We found that the model was able to operate surprisingly well under high levels of sparsification. Below, we show the sparsified models’ loss and KL divergence, as well as performance on a wide range of benchmarks:

Web-text measurements and benchmark accuracy changes. Base model benchmark scores are percentages; other benchmark columns show changes from the base model in percentage points. Expert top-K retains neurons per expert; layer top-K retains neurons across routed experts. Random-64 and Bottom-448 are baseline controls.

Metric / benchmark	Base model	Sparsified models						Baselines (controls)
		Expert top-K		Layer top-K	Band ablated			Random	Inverse of Top-64
		Top-64	Top-32	Top-320	P25–P75	P10–P90	P5–P95	Random-64	Bottom-448
Web text (8.4M tokens)
Cells ablated	0%	87.5%	93.8%	92.2%	49.7%	78.6%	88.8%	87.5%	12.5%
Loss	1.886	1.952	2.032	1.940	1.896	1.926	1.977	4.214	4.587
Δ loss	0	+0.066	+0.146	+0.054	+0.010	+0.040	+0.091	+2.328	+2.701
KL	0	0.152	0.263	0.140	0.033	0.088	0.162	2.511	2.885
Classic suite · Δ accuracy vs base model, percentage points (acc; GSM8K strict match)
ARC-Challenge	54.9	+2.0	+1.5	+0.3	−0.7	+1.3	+1.8	−32.8	−37.3
ARC-Easy	81.4	+0.8	+1.5	+0.2	+0.6	+1.7	+2.1	−40.3	−49.1
WinoGrande	73.9	−1.4	−2.0	−2.2	+0.1	+0.2	+0.3	−23.2	−23.6
HellaSwag	57.4	−0.7	−1.1	−1.6	−1.1	−1.3	−1.7	−25.5	−26.9
MMLU	84.2	−1.6	−3.4	−1.2	−0.4	−0.7	−1.8	−60.1	−60.9
GSM8K	96.7	+0.3	−16.7	−1.7	−1.0	−1.3	−9.3	−96.3	−94.0
Thinking evals · Δ accuracy vs base model, percentage points
SuperGPQA	44.8	−15.2	−17.9	−13.0	+0.9	−8.1	−14.4	−44.5	−44.6
MMLU-Redux	91.5	−3.7	−5.8	−3.0	+1.0	−2.7	−5.0	−89.0	−90.4
MMLU-Pro	78.4	−8.5	−18.9	−8.3	+0.2	−4.3	−13.0	−78.0	−78.0
IFEval	30.9	−0.6	−0.4	+0.4	+0.0	+0.0	+0.4	−15.9	−15.2
MedQA	92.8	−7.8	−16.0	−4.6	+0.8	−2.5	−4.1	−83.9	−80.6
GPQA-Diamond	63.6	−18.2	−25.8	−11.1	+2.5	−10.6	−20.7	−54.0	−55.6
MATH-500	69.8	+7.0	−4.0	+5.8	−3.8	−3.2	−12.8	−69.8	−69.8
AIME 24+25	0.0	+11.7	+3.3	+11.7	+1.7	+3.3	+1.7	+0.0	+0.0
HumanEval+	59.1	+7.9	+3.0	+15.9	−7.3	−20.7	−33.5	−59.1	−59.1
MBPP+	71.7	−20.4	−24.9	−20.9	+5.6	−12.2	−14.8	−71.7	−71.7

Table 1: KL, loss, and benchmark performance under sparsification. Base model benchmark scores are percentages; other benchmark columns show changes in percentage points. Red indicates a decrease; blue indicates an increase. Zero is white. There is a far smaller drop in performance from retaining the top-K neuron activations, either per layer or per expert, as opposed to the complement or random-64; the latter two destroy performance. Ablation based on percentile bands similarly retains a disproportionate amount of performance on the top bands. Note that the P25–P75, P10–P90, etc. percentile bands were computed without router-weighting, unlike the percentile bands referred to throughout the remainder of this text.

Performance is retained to a surprising degree across benchmarks, especially when compared to controls. The ablated models retained high performance across classic benchmarks like ARC-Challenge, ARC-Easy, WinoGrande, and HellaSwag, improving slightly on some evals and dropping on others. However, we observe drops in reasoning and knowledge-based evals. When we observe rollouts directly, we see that the sparsified model’s reasoning tends to loop and self-check excessively. However, when we classified these rollouts with Claude Opus 5.5 agents, we identified that the sparsified model would still mention the facts and correct answers, usually as the last answer candidate–it would just self-check endlessly afterwards. We further found that reasoning collapse seems related to the double-negative activations we discuss in the next section, since restoring those specific activations restored reasoning.

In order to see if this strong-activation regime could recover full benchmark performance, we follow Shabalin & Piotrowski and perform light fine-tuning on the layer top-320 model and re-measure performance; across benchmarks we were able to see full restoration of function. We discuss methods and results further in the Appendix section Extended Sparsity Results and Discussion.

What we suggest, then: the original model is very close to sparse, with most of the computation located in the top activations. This implies that even the original model is likely quite interpretable and causal at the top ranges, but also that it is relatively easy to tip a model over the edge to more complete sparsity. Our subsequent analyses of neurons in the fine-tuned model showed that the neurons retained their original functions, remained interpretable, and had the same numbers of modes.

Principled Selection of a High-Activation Band

Most of the model’s functionality is retained after top-320 layerwise filtering, and that lost performance is easily restored with light fine-tuning. The retained router-weighted activations fall almost entirely in the p97/3 band. We therefore use this activation regime for the experiments covered in the remaining sections.

Sparsity takeaways and the interpretable regime

We now have evidence from several directions suggesting the following:

• The set of neurons required for each step of inference is surprisingly sparse. General sparsification methods like top-K achieve comparable benchmark performance at ~92.2% sparsity; as we discuss in the appendix, specific circuit-finding tools produce even greater sparsity.
• Retained neurons are almost always firing at the positive and negative tails, often p97/3 or stronger.
• Double-negative/Q3 activations seem to consist of a functionally different regime. Though we did not analyze it in detail, we expect that it is likely that it is approachable with further investigation.

From this evidence, we conclude that even if we do not fully understand weak activations, we can nevertheless understand the bulk of what the MLPs are doing if we limit ourselves to examining the tails. Weak activations may not be entirely irrelevant, and may perform a function, but we leave investigation of them to future work.

We further hypothesize that the sheer sparsity of neuron activations in these experiments suggests that neurons constitute a privileged basis, that they are meaningful units of analysis, and that the connections between them can help elucidate their functions.

Now that we have laid the groundwork for investigation of the neurons’ positive and negative extremes, the next question is: what does their behavior in these regimes tell us?

Understanding Neuron Computations

Moving from Polysemanticity to Computational Role

Neurons are often thought to be difficult to interpret because a single neuron can respond to several seemingly unrelated concepts. This property, known as polysemanticity, has been described in terms of responses to unrelated latent variables, mixtures of inputs, or multiple semantic concepts (Olah et al., 2020; Bricken et al., 2023; Bills et al., 2023).

These informal definitions do not resolve what makes concepts distinct. Whether a neuron is labeled monosemantic or polysemantic can depend on choices about what constitutes a single concept (Murphy, 2002). As noted in Neural Networks Are Already Clean, humans tend to describe neurons as detectors for groups of nouns. But categories do not always follow this principle. Klindt et al (2026) compared this phenomenon to the New York Times Connections Game, where the objective is for users to group words by non-obvious similarities, and Barsalou, 1983 famously illustrated this with goal-derived categories such as “things to sell at a garage sale”. An old GameCube and an unused kayak seem to have little resemblance, but they would both belong to this category. Recognizing a unifying concept also depends on the interpreter’s background knowledge (Murphy & Medin, 1985). Responses that initially appear unrelated may share a regularity that becomes apparent only with a better understanding of the computations they support.

An alternative approach is to investigate neurons from a computational perspective first, rather than beginning with semantics. How do different sets of input neurons make a neuron fire, and how do downstream reader neurons use a neuron’s signal? By identifying distinct computational functions that a neuron performs, we can develop objective, quantitative tools to analyze a neuron; and we can then examine their semantics to map computational functions back to interpretable concepts. This approach makes it possible to demystify neurons even when they perform multiple or confusing computations, because the subcircuits involved remain legible. And as long as these roles are few and concisely describable, we can still understand the neuron by characterizing each role and the contexts in which it operates (Rigotti et al., 2013; Fusi et al., 2016; Quiroga et al., 2005; Barlow, 1972; Marr, 1982). This allows us to retain the neuron as a unit of analysis even when it serves multiple distinct computational roles.

We therefore adopt this approach–first looking at computational function, and only then examining semantics–as our primary investigative procedure. In our next section, we describe precisely how we perform this analysis.

Understanding Neuron Computation via Subcircuits

We intentionally target the high-activation regimes that our sparsity experiments demonstrate to be the most causally important (p97/3 and stronger). As motivated above, our approach here is computational/functional rather than semantic: rather than understanding a neuron in terms of the types of texts that activate it, we instead examine what function the neuron plays in network computation. In particular, a neuron is interpreted in terms of its upstream and downstream connections to other neurons.

We extract various subcircuits that a neuron participates in as the language model processes input text. We define a subcircuit as a computational subgraph with a set of input and reader neurons that connect through a target neuron, and seek to identify disjoint subcircuits that run through the neuron. A neuron participates in multiple subcircuits, each implementing a unique computation through the gate ⊙ up multiplication. A neuron always writes to the same direction in the residual stream. However, the same textual context that causes the neuron to fire can also prime different downstream neurons to read the original neuron’s signal.

An analogy can be made to the signal processing idea of multiplexing, where multiple signals from different subcircuits are compressed into a single channel (Lee & Messerschmitt, 1988; Mante et al., 2013; Rigotti et al., 2013; Salinas & Thier, 2000; Dreyer et al., 2024; Gandelsman et al., 2024). A single neuron may serve as a shared channel for different computations, with the surrounding context determining how downstream neurons use its signal. This is made particularly easy by the nature of gated MLPs, since upstream neurons can perform a switching function to activate different sets of readers depending on the context through a combination of the gate and up projections. We discuss this in the text on disambiguation below.

Subcircuit multiplexing

One routed-expert neuron in Qwen3.6-35B-A3B. Example contexts are illustrative.

All contexts Context AContext BContext C

Figure 5: An illustration of how neuron subcircuits work. The neuron output vector is always the same direction, but by priming different downstream reader sets, the neuron can be used for different functions.

Identifying Computational Functions From Subcircuits

To identify these subcircuits, we first ran 10 billion tokens from diverse domains through Qwen 3.6 35B-A3B including data from pretraining, reasoning, chats with users, and math. For each of the model’s 5 million neurons, we collected 1,000 examples from each of 14 activation percentile bands, for a total of 14,000 examples per neuron. We then collected an additional 30,000 examples per neuron for a stratified subset of 2,734 neurons. These additional examples were drawn from p97/3 and stronger. As a starting point for neuron descriptions, we used a fine-tuned Qwen3-8b model to produce basic labels for each neuron based on example sentences the neuron fires on.

To identify subcircuits, we first characterize each firing by its upstream inputs. With each layer’s RMSNorm scale held at its actual value, the target neuron’s gate and up pre-activations decompose exactly into direct contributions from the residual-stream writes of earlier neurons. We keep the inputs that contribute most. To collect downstream readers, we propagate the target neuron’s write along the residual stream to every later layer at the same position, through the same frozen RMSNorm. We estimate each reader neuron’s change in activation to first order. See the Methods section in the Appendix for details.

We then profile each firing by the inputs that drive or suppress its gate and up projections. For each projection, we subtract each input’s mean absolute contribution across the expert’s neurons from its absolute contribution to the target neuron, then retain the 400 top inputs for each of the gate and up projections. We separate these inputs into two bins: those that support the sign of the projection’s pre-activation and those that oppose it. This gives each firing a profile with four parts: gate drivers, gate suppressors, up drivers, and up suppressors. Keeping these separate lets us distinguish firings where the same input plays different roles–for example, an input neuron could suppress one subcircuit through the gate, or activate another subcircuit through the up projection.

We next compare these profiles across firings of the same pole to identify upstream subcircuits that recur in different contexts. Inputs that appear in nearly every firing provide little information about what distinguishes one firing from another. We therefore weight the profiles with TF-IDF to downweight common inputs, and normalize the four quadrants so that each contributes equally. Since we do not know in advance how many subcircuits a pole has, we use community detection to find groups of firings with similar input profiles. We split examples by document, using 75% of documents for fitting and holding out the remaining 25% for evaluation. We connect each firing in the fitting set to its 15 nearest neighbors by cosine similarity of the input profile, then run the Leiden algorithm, a community detection algorithm. Each held-out firing is assigned to the group whose centroid is nearest in cosine similarity.

This procedure can split a single circuit into several groups. To bring these back together, we merge groups when their top 50 input neurons overlap by at least 30% and their top 50 downstream readers also overlap by at least 30%, a heuristic that we find produces higher-quality subcircuits. Requiring agreement on both sides helps ensure that the merged groups reflect the same underlying circuit.

This yielded a median of 19 subcircuits per pole (with a range between 1 and 51), with low overlap in their inputs and readers. We send each subcircuit, with a description of its inputs and readers, to Claude Opus 5.5 to produce a concise description of the subcircuit.

We see that subcircuits are highly disjoint:

Table 2: Set overlap for input and reader neurons for subcircuits within a neuron. The nearest neighbor is the subcircuit with the most overlap within a neuron.

For each subcircuit, we obtain three different descriptions. The first describes how upstream neuron inputs through the gate and up projections cause the neuron pole to fire. The second describes how the subcircuit appears in text alone. Finally, we also describe the token pattern: what actual tokens in a text does this neuron fire on?

The three parts of a subcircuit description

‹ L9_E220_N343 ›

Corneal haze and stroma in LASIK/PRK refractive surgeryNeuron L9_E220_N343, positive pole, subcircuit c1

1Circuit

GateOpens on corneal tissue-condition and opacity units (vascularization/infiltration, opacities/clouding, 'haze', epithelial fragments).

UpWrites positively from tissue-pathology, dystrophy and 'haze' units.

2In text

Corneal refractive laser surgery (LASIK, PRK, excimer laser ablation of the corneal stroma), peaking on its post-op complication and anatomy terms: corneal haze, corneal stroma, ablating, excimer laser.

3Tokens

Fires mostly on the token right after 'corneal' or 'H'. Examples are 'haze' in 'corneal haze' / Chinese 'Haze' and 'st' in 'stroma'.

Legal discretion clause "at the option of [party]" in contractsNeuron L4_E177_N37, negative pole, subcircuit c1

1Circuit

GateOpens on the 'at the ___ of' legal frame ('at the expense of', 'at will') together with the 'option' token, nearly exclusive to this subcircuit.

UpSets the write from legal-terminology and 'option'-substring units, pushing toward contractual choice/discretion readers.

2In text

The discretion clause "at the option of [party]" in legal, contractual and statutory text, where a right to choose is assigned to a specific party (Bank, Holder, Seller, a State).

3Tokens

The token " option" right after "at the" (often preceded by "then,"). Weaker activity appears on "at", "the" and the following "of".

'case' in nested case-control studies of later-developing diseaseNeuron L14_E2_N83, positive pole, subcircuit c5

1Circuit

GateOpens on the 'case' token when disease-complication, observational-case-count and case-control/grouping study-design features co-occur (e.g. 'a nested« case»').

UpThe same complication, case-control and participant-grouping features set a strong positive write that boosts downstream case-control, matched-control and prior-to-diagnosis study readers.

2In text

Epidemiological case-control study design, especially nested case-control studies of prospectively collected samples in which subjects later develop a disease (preeclampsia, sepsis, ovarian cancer).

3Tokens

Peaks on ' case' immediately after 'nested' (nested« case»-control study), with strong spillover onto 'nested', '-control', 'later developing', and 'prior to'. Weak residual firings occur on study-acronym fragments (INTER«HE»ART) and on 'and' before 'controls'.

Toast dedication slot: 'raise our glasses to…' before naming honoreeNeuron L24_E246_N254, positive pole, subcircuit c8

1Circuit

GateFires on the 'to' right after toast verbs ('raise a glass', 'join me in a toast'), gated by toast/cheers detectors plus purpose-preposition features

UpScales write via gratitude/'Long Live'/wedding-blessing and formal-address features, pushing toward naming the toasted person or wish

2In text

The moment of proposing a celebratory toast or well-wish in a speech: the word right after 'raise our glasses' / 'join me in a toast', which introduces whom or what is being toasted.

3Tokens

The connective right after 'raise our glasses' / 'a toast': ' to' or ' and' (as in 'raise our glasses and toast'). The neuron also fires on 'Let', 'our', 'glasses' and 'Cheers' leading into the toast.

Re-typing earlier-declared Java class/variable names (licenciaFiscal, StringSerDe, Solution67)Neuron L30_E129_N272, positive pole, subcircuit c3

1Circuit

Gateopens on sub-word pieces of code identifiers (camelCase/underscored variable, method, class names) in source code contexts where the name was seen before

Upsets positive write from the same identifier-part and code-structure units (method keywords, data fields, class declarations), scaling with repeated-name copying

2In text

In-context recall of a previously seen multi-part name: the neuron fires on a later piece of a compound identifier or proper name when that name is being re-typed and already appeared earlier in the document.

3Tokens

The second or later sub-word token of a camelCase or multi-token identifier, right after its first component, usually at a declaration or reuse site ('class String«Ser»De', 'class Solution«6»7', 'func rotatedSector«Radians»'). The preceding component token is often weakly active.

Predicting 'dance floor' after 'onto/hit/on the' in wedding dancingNeuron L36_E193_N223, positive pole, subcircuit c6

1Circuit

GateDance-floor and social-dancing scene features (L31_E80_N347, L35_E73_N75 performance-space, nightlife dancing) open the gate.

UpThe event-space feature and the generic 'on the / across the' determiner-in-locative-PP features set the positive write.

2In text

Predicting "dance floor" in social-dancing scenes: someone is about to be placed on or brought to the dance floor at a wedding, prom or reception ("led Kathy onto the", "hit the", "join Grace and Bob on the", "stayed out on the").

3Tokens

"the" right before "dance floor" (about 11θ), with the preceding preposition or verb (onto, on, hit) at about 6–7θ. Activation stays mildly elevated over surrounding dance-party narrative tokens.

Inputs, firings and full descriptions

• L8_E227_N440eye opacity and clouding
• L7_E84_N177pieces of technical and scientific terms
• L1_E228_N340the word “haze”
• L7_E84_N43tissue vascularization and infiltration
• L7_E84_N492degenerative tissue conditions, dystrophy
• L1_E64_N125technical and mathematical expressions

gateupL9_E220_N343

• 节约了大量角膜厚度，显着减少了术后H

  aze

  的发生，超薄LASIK手术术后
  24×
• factor (CRF), and corneal

  haze

  . ⏎Results: This study was
  22×
• a hinge, ablating the corneal

  st

  roma in a refractive correction amount
  21×
• with significant side effects such as corneal

  haze

  and regression. Several factors have been
  20×

Lines are the upstream neurons that most drive this subcircuit's gate and up projections; width is attribution. On the right are its four strongest firings, with the firing token underlined and every token shaded by activation; the number is the activation as a multiple of the firing threshold. The neuron multiplies its gate, after SiLU, by its up input. This subcircuit is 212 of the pole's 5,000 sampled firings.

Circuit

Combines corneal-tissue pathology signals (vascularization/infiltration, opacity/clouding, the word 'haze', epithelial/stromal fragments) with dystrophy/degeneration cues to signal corneal post-refractive-surgery complications and stromal anatomy ('corneal haze' after LASIK/PRK, 'ablating the corneal stroma'). Its readers are corneal/eye-transplant and tissue-healing neurons, which it boosts, and corneal-ulcer/keratitis and vascular-fistula neurons, which it suppresses.

Gate

Opens on corneal tissue-condition and opacity units (vascularization/infiltration, opacities/clouding, 'haze', epithelial fragments). Laser-process, limbal and physics/optics units weakly oppose it.

Up

Writes positively from tissue-pathology, dystrophy and 'haze' units. Laser-equipment, endothelial-dystrophy and materials/optics units slightly reduce the write.

In text

Corneal refractive laser surgery (LASIK, PRK, excimer laser ablation of the corneal stroma), peaking on its post-op complication and anatomy terms: corneal haze, corneal stroma, ablating, excimer laser.

Tokens

Fires mostly on the token right after 'corneal' or 'H'. Examples are 'haze' in 'corneal haze' / Chinese 'Haze' and 'st' in 'stroma'. It also fires strongly on 'abl(ating)' and 'exc(imer) laser'. Weaker activation spreads to 'cor/ne/al', 角膜, 厚度 and 弹力层 nearby.

Gate, strongest drivers

1. L7_E84_N1770.0537Tokens forming parts of technical terms or scientific terminology, especially in biological or specialized contexts, and common words in technical sentence structures. The feature activates on specific word segments (lik
2. L7_E84_N430.0481Medical or technical descriptions of tissue conditions involving vascularization, infiltration, or related processes, particularly in ophthalmology or pathology contexts. The feature activates on texts discussing disease
3. L8_E227_N4400.0350Medical conditions involving opacity, cloudiness, or visual impairment in the eye, often using terms like "opacities," "opaque," "cloudy," "clouding," "corneal," "lens," or "vascularization" in technical or clinical cont
4. L1_E228_N3400.0235The presence of the word "haze" (case-insensitive) in contexts describing physical obscuring phenomena (e.g., atmospheric haze, corneal haze, or pollution) or metaphorical obscuring (e.g., "haze of transformation," "haze
5. L7_E84_N3920.0234Physical expansion or contraction (swelling/shrinking) in materials or biological systems, often in response to environmental factors like humidity, temperature, or physical stress. The feature activates when the text di
6. L5_E214_N2000.0158Fragments of technical terms related to epithelial or endothelial structures, such as "sub", "ep", "ith", "elial", "endoth", or "th", often in medical or biological contexts discussing cellular layers, tissues, or relate
7. L3_E191_N1200.0153Mention cell types containing the root "cy" or the suffix "cytes", particularly in biological or medical contexts, such as oligodendrocytes, astrocytes, keratinocytes, adipocytes, and osteoblasts. The feature activates w
8. L5_E179_N2790.0143Anatomical or biological structures, particularly those involving directional terms (dorsal/ventral), keratinized tissues, or specialized cellular/components like thalami, intestinal villi, and occludens proteins. The fe
9. L4_E47_N00.0141(no label)
10. L7_E84_N4920.0106Medical or biological texts discussing degenerative or progressive conditions affecting tissues or cells, often involving terms like "dystrophy" or "dystrophies" (e.g., Fuchs' dystrophy, lattice dystrophy) and describing

Gate, strongest opposers

1. L7_E84_N3970.0076Limbal structures (like the limbus or limbal ring of the eye) or stem cells in medical/scientific contexts, with activation on tokens like "lim", "bal", "stem", and "cells" when they appear in technical terminology relat
2. L8_E227_N570.0052Technical or scientific processes involving materials, equipment, or precise measurements, particularly those containing terms related to lasers, chemical compounds, or physical properties like power, width, or elemental
3. L8_E227_N3810.0057Rotational mechanics, axial tilts, conical shapes, or related physical concepts, often involving terms like "axis," "axial," "conical," "ical," "tilt," "incline," "rotation," "orbit," or "eje" (Spanish for axis). The fea
4. L8_E227_N4110.0054Thin film solar cell technology or related materials, including terms like 'thin', 'film', 'photovoltaic', 'solar', 'semiconductor', 'material', 'CdTe', 'amorphous silicon', or 'photovoltaic module', often in contexts de
5. L8_E227_N3250.0048Electromagnetic theory, waves, or related physical phenomena, including terms like "electromagnetic," "wave," "magnetic," "electric," "field," "polarization," or equations (e.g., Maxwell's equations) describing their beh
6. L8_E227_N4640.0055Medical conditions or treatments related to the eye, particularly those involving laser procedures, retinal issues, iris surgeries, or intraocular conditions. The feature activates when the text contains specific roots o
7. L8_E227_N4740.0042Technical or product-specific terms that combine words with numerical elements, often in contexts like astronomy (e.g., "Red dwarf", "ten" as part of stellar designations), technology (e.g., "Digital 3-D", "IMAX"), or so
8. L7_E82_N4760.0040Fragments of medical procedure terms, particularly those related to ophthalmic surgeries or treatments, such as parts of "trabeculectomy" (e.g., "trab," "ecu," "lect," "omy"), "canaloplasty" (e.g., "canal," "plasty"), "p
9. L8_E227_N1610.0034Celestial phenomena, astronomical observations, or scientific measurements, often involving proper nouns (like star charts, constellations, or historical figures), directional terms (north, south, in, from), and technica
10. L7_E84_N3620.0034Advise against or describe actions involving manipulating bodily parts (e.g., picking, squeezing, expressing) in medical or skincare contexts, particularly when discussing skin conditions, lesions, or bodily fluids. The

Up, strongest drivers

1. L7_E84_N430.0246Medical or technical descriptions of tissue conditions involving vascularization, infiltration, or related processes, particularly in ophthalmology or pathology contexts. The feature activates on texts discussing disease
2. L7_E84_N1770.0185Tokens forming parts of technical terms or scientific terminology, especially in biological or specialized contexts, and common words in technical sentence structures. The feature activates on specific word segments (lik
3. L7_E84_N4920.0185Medical or biological texts discussing degenerative or progressive conditions affecting tissues or cells, often involving terms like "dystrophy" or "dystrophies" (e.g., Fuchs' dystrophy, lattice dystrophy) and describing
4. L1_E64_N1250.0145Technical or mathematical expressions involving variables, numbers, and operators, often structured with parentheses, brackets, or specific functions like GetIdx, Stellite, or Macintosh, and inequalities or equations wit
5. L1_E228_N3400.0109The presence of the word "haze" (case-insensitive) in contexts describing physical obscuring phenomena (e.g., atmospheric haze, corneal haze, or pollution) or metaphorical obscuring (e.g., "haze of transformation," "haze
6. L8_E227_N5100.0097Medical procedures or conditions involving anatomical structures, surgical interventions, or disease progression, particularly those containing roots like "extra" (e.g., extracellular, extraluminal), "cor" (e.g., corneal
7. L8_E67_N1590.0079Technical terms involving medical or scientific contexts, particularly those discussing dosages, therapeutic combinations, anatomical structures, or pathological conditions. The feature activates on parts of words or sub
8. L5_E20_N180.0075Tokens that are part of specific terms in technical, cultural, or proper noun contexts, such as technical terms, proper names, codes, or cultural references. The feature activates on tokens that are components of longer
9. L8_E227_N3690.0079The oxygen permeability of materials, particularly in the context of contact lenses or eye health, often involving terms like "oxygen," "permeability," "permeable," "hydrogel," "silicone," "cornea," or "eyes," and descri
10. L8_E227_N4200.0079Technical terms starting with "ble" or "b" in medical or technical contexts, often involving eyes, skin, or surgical procedures, with adjacent tokens forming parts of longer terms like "blephar", "bleb", "filtering", or

Up, strongest opposers

1. L8_E227_N570.0067Technical or scientific processes involving materials, equipment, or precise measurements, particularly those containing terms related to lasers, chemical compounds, or physical properties like power, width, or elemental
2. L8_E227_N130.0071Medical or biological processes involving endothelial tissues, their degeneration (e.g., dystrophy, dystrophic), or related structures like stroma, cornea, and membranes, often in technical or clinical contexts. The feat
3. L7_E82_N4760.0042Fragments of medical procedure terms, particularly those related to ophthalmic surgeries or treatments, such as parts of "trabeculectomy" (e.g., "trab," "ecu," "lect," "omy"), "canaloplasty" (e.g., "canal," "plasty"), "p
4. L8_E227_N4110.0043Thin film solar cell technology or related materials, including terms like 'thin', 'film', 'photovoltaic', 'solar', 'semiconductor', 'material', 'CdTe', 'amorphous silicon', or 'photovoltaic module', often in contexts de
5. L4_E47_N2560.0026(no label)
6. L8_E227_N730.0033Technical aspects of materials' physical properties or electromagnetic behavior, including terms like "physical," "optics," "reflection," "coefficient," "material," "electromagnetic," "magnetic," or "properties," often i
7. L8_E227_N1900.0061The formation or structural properties of physical objects, particularly in technical contexts involving geometry, minerals, or crystalline structures. The feature activates when the text describes how objects are formed
8. L8_E227_N3810.0030Rotational mechanics, axial tilts, conical shapes, or related physical concepts, often involving terms like "axis," "axial," "conical," "ical," "tilt," "incline," "rotation," "orbit," or "eje" (Spanish for axis). The fea
9. L8_E227_N4960.0027Technical or scientific concepts involving specific terminology related to engineering systems, physical phenomena, or environmental processes, particularly those referencing components (like modules), effects (like radi
10. L8_E227_N3240.0031Medical procedures, particularly laser-based eye surgeries like LASIK or PRK, involving terms such as LAS, IK, OP, and contexts around corrective surgeries, astigmatism, contact lenses, or corneal treatments. The feature

About this figure

Short input names on the figure are condensed from the automatic labels listed above (labels are cut off in the source). Numbers are the atlas's attribution of each upstream neuron to this subcircuit. Data: basis-paper/figures/mode-shapes/atlas/subcircuit_modes_data/L9_E220_N343__pos.json.

Figure 6: Example of a subcircuit description. Its behavior is described at the circuit level, the text level, and the token pattern level.

Clustering Subcircuits into Modes

While a neuron will participate in many subcircuits, many of these subcircuits can be clustered based on semantically themed groups that we call modes. We provide the list of subcircuits and text examples to a Claude Opus 5.5 judge (Zheng et al., 2023; Paulo et al., 2024) and ask it to create a set of categories we call modes into which each subcircuit will be placed. Often, a neuron’s subcircuits are related enough that a single mode is assigned, but sometimes the subcircuits demonstrate multiple unrelated modes. We note that mode counts computed in this way are not definitive counts of the concepts represented by a neuron, as they are affected to some extent by the descriptive granularity imposed by this procedure.

Broadly, we find that most neurons have between one and a small handful of semantic modes, and most of the damage incurred by ablating a neuron is confined to a single dominant mode. However, in quantifying these intuitions, we do not find that any one metric effectively captures the number of modes and their relatedness within a neuron-pole. Neurons can contain modes with dramatically different levels of relatedness, very high inequality in terms of how causally relevant each mode is, and very different frequency distributions of each mode. We thus provide a variety of metrics and explain the tradeoffs of each, but to put them into context we introduce what we call the total neuron view.

The Total Neuron View

To ground our discussion, we provide concrete examples of neuron-poles and their modes in the following format. It allows us in one place to see the different modes within a neuron, their similarity, and how they change over different activation strengths. Within each mode, we can see its firings, and within each firing both the tokens and circuits causally relevant to it.

As described above, these were mapped into subcircuits based on disjoint sets of input neurons, and these subcircuits were then grouped by our LLM judge. In this illustrated neuron, there is one main mode, a secondary one closely related to it, and a much weaker one that’s unrelated.

Loading how to read a neuron landscape…

Each dot represents one firing example from our dataset. When looking at a neuron in this way, we can observe multiple things: how related different modes are (based on embeddings of the mode descriptions, as explained below), how causally dominant (how much KL divergence caused by ablation) each mode is, and how they change with activation strength.

By raw count, there are three modes in this illustrative neuron-pole, but this number does not correctly convey what we should expect from this unit, nor how we should use it in interpretability. We thus provide a number of metrics, each of which answers slightly different questions about both this neuron-pole and the population of neurons at large.

• How many individual modes does the LLM judge assign per neuron-pole? The median neuron-pole has 1.0 mode, with a mean of 2.82.

• How many causally important modes does each neuron-pole have? For each pole, we can weight a mode by its damage: of the total KL damage incurred by ablating the pole, what fraction is due to this mode? In the visualization below, we distinguish poles for which at least 90% of KL damage is confined to their main mode; 69% of poles have such a dominant causal mode.¹

• How many highly unrelated modes appear in each neuron-pole? There is not often a natural place to split a mode, for instance one representing two closely related idioms. In other cases, it does appear to be completely unrelated, like a Colorado and a much rarer Tornado Python framework neuron we will discuss below. While it can be helpful to split a pole into modes, thinking in modes can count these two cases equally. In the plot below, we use color to help express mode similarity, and later show the full distribution of pairwise mode similarities per pole.

To develop an intuitive grasp of how these factors can present in a neuron, we present a selection of neuron profile types below.

Six example neuron-poles

Loading six example neuron-poles…

How to read these examples

Height runs from weak, common firings to strong, rare firings. Width is average KL damage per firing, compressed using a fourth-root scale so weak firings remain visible. Areas are not damage shares; the printed percentages give frequency-weighted damage shares among assigned modes.

Blue marks the dominant group of linked modes, lighter blue a possible link, and other colors groups with no link seen. Gray is reserved for firings assigned to no mode. Gate-closed firings (negative gate inputs, Q3/Q4) are excluded.

Per-firing input attribution is shown only when the stored text reproduces the original activation within 25%. “Usual readers” are the subcircuit’s average readers, not a trace of that individual firing. Blue tokens contributed to the gold one. Estimated to first order from up to 100 tokens of context.

See Appendix “Additional Analyses of Modality Profiles” for a more detailed typology of neuron profiles and their frequencies.

Mode Statistics in Detail

Here, we report the distributions of the metrics outlined above, showing the number of individual modes per neuron-pole, and the fraction of the pole’s KL contribution and firing associated with each of its modes.

To measure the KL damage for a pole’s modes, we pass the 10B-token corpus through the model with the pole ablated, recording for each token the KL divergence from the intact model’s output distribution. Each token, and its associated KL damage, is assigned to a mode according to the subcircuit it activates. We report each mode’s fraction of the pole’s total KL.

We measure firing fraction similarly, recording each mode’s fraction of a pole’s total firing for the 10B corpus, as measured by the mode’s subcircuit activations.

Below we show the results for all metrics over our entire sample population of 2,734 neurons.

Modes, firing and damage

Qwen3.6-35B-A3B · 5,468 neuron poles in the expanded study set

For poles with two or more readable modes, damage outside the main mode is the share of KL damage done by all modes other than the one with the most firings.

1Modes per pole

Readable modes per pole, all 5,468 poles, shaded by how much of the pole’s damage its main mode does. The closer a shade is to the one-mode bar, the more one mode dominates; the most dominated poles sit at the base of each bar.

Main mode’s share of the damage:100% (one mode) >99.9%99–99.9%90–99%50–90%<50% no readable mode

Scroll horizontally to view the full chart.

69% of all poles are dominated by one mode: 47% have a single computational function (2,556), and in 1,231 of the 2,673 poles with two or more modes (46%) the main mode still does at least 90% of the damage.

2Damage outside the main mode

Distribution over poles with 2+ modes, on a logit axis so both ends stay readable. Bars outside the range chosen below turn gray.

Damage outside the main mode (same shades as panel 1): <0.1%0.1–1%1–10%10–50%≥50%

Scroll horizontally to view the full chart.

Across 2,673 multi-mode poles the median is 14%. On this axis the poles form one broad, smooth spread with no gaps, peaking between 20% and 50%. 11% have less than 0.1% of their damage outside the main mode and 27% have half or more. 46% have 10% or less, and 29% sit between 3% and 30%.

3How firing divides among modes

Each mode’s share of the pole’s firing, modes ranked from most to least frequent. Poles in the chosen range.

middle 50% (box), 10th–90th percentile (whisker) median

Scroll horizontally to view the full chart.

Across 2,673 poles with 2+ modes, the most frequent mode takes a median 73% of the firing and the second 16%. In 74% of these poles one mode takes at least half the firing.

4Balance of damage per firing

How unequal damage per firing is across a pole’s modes: 0 = every mode does the same damage per firing; 1 = one mode does all of it. Poles in the chosen range.

beyond noise (p < .05) within noise noise only (firings reshuffled among modes)

Scroll horizontally to view the full chart.

Across 2,665 poles with 2+ modes (with ≥ 3 measured firings in 2+ modes), the median is 0.72 against 0.52 for reshuffled firings. In 37% (978) some modes do clearly more damage per firing than others.

Definitions and sampling

Damage outside the main mode. The main mode is the one with the most firings in the KL sample. A mode’s damage is mean KL per measured firing (summed over the firing token and the next 16) times its number of firings in the 5,000-firing sample, which holds 1,000 firings per band from p97 to the extreme band. So damage is weighted by firing count within that band-balanced sample, not by population frequency: a p97 firing counts the same as an extreme one. Residue and closed-gate groups are excluded. Mono poles have one readable mode, so the number is 0 by definition; they appear in panel 1 only.

Why it can be near 100%. The main mode is chosen by firing count, not by damage. When the most frequent mode is weak and a rarer mode fires hard, almost all the damage sits outside the main mode.

Firing share counts a mode’s firings among the pole’s real-mode firings in its 5,000-firing sample, 1,000 per band from p97 to the extreme band. It describes the top of the activation range, not population frequency.

Damage per firing (panel 4) is the mean over a mode’s measured gate-open firings, from a stratified KL sample of 150–500 firings per pole, using modes with at least 3 measured firings. The noise test reshuffles measured firings among modes, keeping each mode’s count, 1,000 times per pole. The outline shows one reshuffle per pole.

Figure 9: Modes, firing and damage across 5,468 neuron poles in the router-weighted five-band sample. Panel 1: The distribution of mode count among neurons. Even when neurons have multiple modes, the mode count is relatively low. Panel 2: The share of KL damage outside the most frequent mode. Panel 3: Firing rate per mode. In general, neurons are dominated by a single mode, sometimes fire on a secondary one, and only rarely fire on tertiary or further modes. Panel 4: Inequality measure for KL damage across modes (without taking into account frequency). Though many multimodal neuron-poles show equally causally-relevant modes, the population is highly weighted towards high levels of inequality, such that many secondary modes have almost no causal effect on the next token.

What do the mode statistics look like in practice? First, a plurality of poles (47%) have only a single mode, regardless of context, and among poles with multiple modes, most have fewer than five. Further, about half of multimodal poles have one dominant mode that accounts for at least 90% of their KL damage. Overall, our findings support the interpretation that a large majority of poles have either exactly one mode or one causally dominant mode.

Mode Relatedness

A factor of variation not captured by the statistics above is semantic similarity between modes within a pole. Often, we observe distinct modes with considerable semantic overlap, such as “Swedish compound common-noun stems” and “Recall of Swedish named entities”. To the extent that such similarity exists, it would further support the idea that poles often have consistent meanings.

To assess how related each mode within a neuron is, we embedded each mode description using Qwen-3-Embedding-4B. For each pair of modes within a neuron-pole, we computed the cosine similarity between these embeddings, obtaining a measure of relatedness for each mode pair. We computed the mean, KL-divergence-weighted mean (based on the mean KL divergence from ablation of the neuron in that mode), maximum, and minimum. Below, we show all four, beginning with the maximum. Roughly, the maximum can be interpreted as measuring how many poles might reasonably be said to have fewer than their ostensible number of modes, depending on similarity threshold.

Loading mode relatedness…

Figure 10: Relatedness of mode descriptions as computed by an embedding model. As can be seen, there is a broad, unimodal distribution of relatedness between different neurons’ constituent modes.

We see that the semantic consistency of a neuron can vary widely, but in many cases neurons have multiple modes that are quite related, added evidence of organization in a pole’s set of meanings.

How models disambiguate heterogeneous meanings: two circuit case studies

Let us consider Neuron L18_E219_N55, which has two unrelated modes: oral surgery and marine environments. When this neuron fires, it can only write to the residual stream in one direction; there are no different directions for different modes. How does the model know whether this neuron is signaling oral surgery or marine environments?

The architecture of the SwiGLU neuron offers a solution, allowing downstream reader neurons to disambiguate the firing of an upstream multimodal neuron. Consider a neuron n with modes m1 and m2. Often, a reader neuron’s up-projection receives input from n, while its gate projection receives inputs from contextual primers associated with, say, m1. Since the reader’s firing depends on both its up and gate, it will fire exactly when n fires, in mode m1.

In the case of Neuron L18_E219_N55, we can see exactly this mechanism play out. There are two sets of downstream readers–one set for the oral health concept, and another set for the marine environments concept. Both sets read Neuron L18_E219_N55 through their up-projections, but we also identify contextual primers for both the oral and marine modes; these feed readers’ gate projection, determining which mode is read.

Marine context

Choose a sentence, then remove N55, its context primers, or both to see how the downstream readers change.

, palatability, and gel strength. This research was carried out at the Laboratory of Nutrition and Feed Technology, Faculty of Marine Sciences and Fisheries, Hasanuddin University, Makassar, Indonesia. Experimental fishes used were tilapia size 7, 15

Scroll horizontally to explore the full circuit.

Marine side Oral side Listens in both Activation after removal ● Primer on · ○ Off · ✕ Removed · ↑ Fires harder after removal

A marine sentence. Primers on: marine primers (7 of 8 on). Listening: L20_E236_N283, L30_E219_N55.

Figure 11: Upstream neurons prime downstream readers to be receptive to L18_E219_N55 in marine or oral health contexts. Use the sentence selector to explore both modes.

In the marine context, the marine primers are active, activating the corresponding reader set. In the oral health context, the opposite happens: oral health neurons fire at both early and late layers, alerting the readers to treat L18_E219_N55’s signal as evidence for the oral health concept.

A set of random-direction baselines

It is reasonable to wonder: do random directions yield similarly interpretable activation examples, or are neurons unusually interpretable (Szegedy et al., 2014; Bau et al., 2017; Elhage et al., 2023; Heap et al., 2025; Hewitt & Liang, 2019; Makelov et al., 2023; Zimmermann et al., 2023)? We test this with a variety of random direction types, running them through the same example harvest, subcircuit clustering, and mode detection pipeline that we apply to the neurons themselves. Although random directions have been previously found to be selective for text patterns (Korznikov et al., 2026) we find that the neurons behave differently from random baselines in a number of ways:

• Neurons recover experts and layer function (as measured by KL recovered vs. complete ablation of the expert or layer) far more sparsely than random directions; 8 neurons recover as much function as 128-256 random directions.
• Neurons are wired geometrically to specific downstream readers; random directions are not. Random directions activate downstream neurons far more diffusely and less specifically.
• Neurons’ meanings narrow as they fire more strongly; random directions do not. The strongest firings for a neuron are dominated by its primary mode. Random directions remain heterogeneous at all activation ranges.

We provide plots and more detailed data in the appendix. To summarize, however, these results suggest that neurons are a highly meaningful basis for computation relative to random directions (even in the expert basis).

Summary: Neurons as Atomic Units of Computation

Neurons carry out multiple computations through subcircuits, which can be grouped into semantic modes. These modes are usually small in number, indicating that neurons are not intractably polysemantic units of computation, and suggesting that neurons can be described in concise, explicable terms.

Distinct computational roles need not make a neuron an unsuitable unit of analysis even if they are high in number. A neuron participates in different downstream computations – with different semantic interpretations – while remaining an atomic unit of computation. Its write direction is fixed, but the effect of its signal depends on which downstream readers are primed to read it, a property that we can extract through analysis, allowing us to enumerate computational modes. These modes are usually small in number and allow us to readily conduct analysis using neurons as units.

Contextual reuse therefore provides one way for heterogeneous responses to support useful computation. The relevant questions are when downstream consumers can distinguish these uses and what happens when their contexts overlap. Our examples demonstrate mechanisms that support this reuse, but do not establish how prevalent these mechanisms are or how reliably they resolve ambiguity across the model. We leave further analysis of these mechanisms to future work.

We find this to be strikingly good news for interpretability. Though we do not see a perfect one-to-one mapping of meaning to neurons, we do see that the number of functions that a neuron carries out is usually relatively small, and can be identified by the text and by looking at which downstream readers are being influenced by the neuron.

Evaluating Neuron Descriptions

We next consider how to evaluate our descriptions of these neurons. A good description should not be merely observational or correlational, but rather should allow us to make causal predictions about the units we are describing, including under intervention. We evaluate these descriptions through tasks that require increasingly precise predictions, from identifying whether a neuron fires to predicting its activation at each token. Our questions are therefore as follows:

• Input synthesis: Given a neuron description, can an LLM system create a text that controls the magnitude of a neuron fire, such that it exceeds the maximum found in our 10B text corpus?
• Detection: Given a sequence where the neuron fires, and a distractor sequence where it doesn’t, can an LLM judge equipped with the neuron description correctly identify the firing example?
• Classification: Given a sequence where the neuron is known to activate, and a set of mode descriptions for the neuron, can a blind LLM say, from text alone, which mode is operating? Can it do so for the subcircuit as well?
• Simulation: Given a sequence and a neuron description, can a model predict, for each token, what the activation level should be?

As activations increase in magnitude, prediction of neuron behavior becomes easier. When activations are weaker (e.g. p97/3), the neuron often fires on more distantly related topics than those that cause strong fires, which tend to be clearer. However, we are able to make predictions at every quantile within the effective range (p97/3), as follows:

At p97/3, we are able to detect from text whether a neuron should be firing. Activating examples at this quantile are often “blurrier” versions of strong activations. For example, a neuron that fires strongly on discussions of pet microchipping might fire weakly on contexts that discuss tracking wild animals with tags, or on technical documents that describe various types of microchips. Nevertheless, we are able to describe a neuron well enough that we can identify these cases at p97/3.

At p97/3, we are also able to classify which mode is active for a neuron by examining the inputs that made the neuron fire. By looking at what activates the neuron, we can determine which subcircuits and therefore which modes are responding to the text, even when it is not visible from surface features. We can do this with readers as well to a lesser extent.

At p99.9/0.1 and above, we are able to simulate token firing and recover KL divergence caused by neuron ablation. Here, we replace real activations with values that a simulator model (Jev, in this case) provides on a token-by-token basis for an entire sequence, given the description of the neuron. Perfect performance on this task would require in-depth knowledge of attention, inner circuit operations, and movement of information through the network over long contexts, so we do not expect a text description of the neuron to make it possible for a simulator to get a perfect score. Nevertheless, our simulator performs surprisingly well.

Results for all these tasks are summarized in the table below.

Scroll horizontally to compare all activation bands.

Metric	97th–99th percentile	99th–99.9th percentile	99.9th–99.99th percentile	99.99th+	Top 1,000 by |a| × rw	All bands
DetectionOpus 5.5 picks the firing snippet out of a pair, token marked · 496 poles (chance 0.50)
All distractors	0.858	0.892	0.927	0.955	0.953	0.914
Same-expert distractors	0.812	0.852	0.901	0.942	0.939	0.885
Random-text distractors	0.906	0.932	0.952	0.972	0.970	0.946
Classification · macro-F1Assign each held-out firing to one of the pole’s modes · 573 random multi-mode poles, the same firings for every method
Inputsthe firing’s selective gate drivers and opposers, the features its subcircuits are clustered on	0.957	0.972	0.981	0.988	0.990	0.968
Readersthe downstream neurons the firing moves; never used to build the modes	0.827	0.848	0.889	0.938	0.937	0.853
TextJev reads the mode descriptions and the snippet, firing token marked; no examples	0.650	0.703	0.787	0.878	0.881	0.707
Chanceguess modes in proportion to their shares	0.392	0.355	0.415	0.628	0.563	0.290
SimulationAn activation predicted from the description stands in for the neuron inside the model · 486 poles
Effect restoredshare of ablation damage undone, as a share of what the ceiling restores	Not evaluated		56%	64%	76%	73%†

Shading runs from chance (pale) to perfect (dark) within each row; grey rows are baselines.

Table 3: Neuron detection, mode classification, and simulation across activation strengths. Detection measures how accurately an LLM judge identifies the firing snippet from a pair, using same-expert or random-text distractors (chance: 50%). Classification measures how well upstream inputs, downstream readers, or text descriptions identify the active mode on the same held-out firings, scored by macro-F1. Simulation measures how much ablation-induced KL damage is restored by description-predicted activations, normalized to a 10-level oracle ceiling; its “All bands” result pools only the 99.9th–99.99th-percentile, 99.99th-percentile-and-up, and top-1,000 bands.

Agentic Neuron Science: Generating Descriptions

For generating the neuron descriptions underlying the experiments below, we use a frontier model (typically Claude Fable 5.0 or 5.1, Opus 5.5, or GPT-6 Sol) residing in a harness that allows it to conduct experiments and analyze neurons in detail (Shaham et al., 2024; Schwettmann et al., 2023a; Lu et al., 2024; Gottweis et al., 2025; Singh et al., 2024, Zimmermann et al., 2024, Colin et al., 2024). We refer to this system as the Neuron Scientist. We provide it with 200 documents where the neuron fired, divided among bands where the strongest neuron fire was in top-N, p99.99, p99.9 and so on down to p75. Activation values within each document are labeled. We then task the Neuron Scientist with producing a description that optimizes a metric like KL recovered through simulation, on a designated training set of texts. It is permitted to conduct a specified number of rounds until it achieves that goal. At each round, the Neuron Scientist develops a hypothesis; tests the hypothesis with experiments (like running texts through a server to collect activations, steering, constructing attribution graphs, or examining the neuron’s circuit); submits the description for evaluation against the target metric on a held out dev set of example texts; and receives feedback from the metric on where it failed. It then revises its hypothesis and tries again until it achieves its target or exhausts the designated number of rounds.

At the final step, a supporting agent writes a summary that details the findings and experiments from the research run.

Crucially, the numbers we report are computed on a separate set of test documents not used to optimize the descriptions.

Detection

A faithful description of a neuron should let one identify the inputs it fires on. To test this, we design a detection task. An LLM judge receives the label and description of each of the neuron pole’s modes, together with two unmarked text snippets from held-out documents. In one snippet the pole fires; in the other, the pole stays below its own 90th-percentile activation on every token. The judge must pick the snippet that makes the pole fire.

We evaluate detection on 500 randomly sampled neuron poles (496 with results), using held-out text from our 10-billion-token corpus with mixed pre-training and post-training data, and report accuracy separately for five activation percentile bands (see Appendix). Claude Opus 5.5 is right 86% of the time for the weakest firings (q97) and 95% for the strongest, against 50% chance. Controls drawn from text routed to the same expert are the harder case, lowering accuracy by 3–5 points, because the expert’s domain is present in both snippets.

Scroll horizontally to compare all activation bands.

Opus 5.5 · mode descriptions	p97–99	p99–99.9	p99.9–99.99	p99.99+	Top 1,000	All bands
All distractors	0.858	0.892	0.927	0.955	0.953	0.914
Same-expert distractors	0.812	0.852	0.901	0.942	0.939	0.885
Random-text distractors	0.906	0.932	0.952	0.972	0.970	0.946
Chance	0.50	0.50	0.50	0.50	0.50	0.50

Table 4: Opus 5.5 sees the pole’s modes (label and description) and two snippets from held-out documents, the 60 tokens before a token, the token itself (marked) and 8 after; in one the pole fires on the marked token; the other, the distractor, is a snippet where it stays below its own p90 throughout, drawn from text routed to the same expert or from random text. 496 neuron poles with results from the marked-token detection evaluation; up to 10 held-out firings per band per pole. Mean over poles; All bands pools each pole’s pairs.

Mode Classification

A neuron with several modes should affect downstream neurons differently depending on which mode is active. To test whether our modes capture these causal differences, we design a classification task and evaluate how well we can assign a firing example to the correct mode.

Inputs. Each firing is represented by a vector with one entry per upstream unit (earlier-layer neurons and attention components): the size of that unit’s direct contribution to the neuron’s gate at the firing token, kept for the 400 units that push it hardest towards firing and the 400 that push hardest against. Units that contribute in almost every firing are down-weighted (TF-IDF), and the vector is normalised to unit length. Each subcircuit is summarised by the average of these vectors over its firings in the fitting documents. A held-out firing gets a score for each subcircuit, the cosine similarity of its vector to that average. Each mode takes the score of its best-matching subcircuit, and the firing is assigned to the highest-scoring mode. The subcircuits were clustered on these same features, so this score shows how well the procedure recovers the modes from their own defining features, a near-ceiling reference rather than an independent test. It falls just short of perfect, because the modes come from a graph clustering of firings while this test uses a simpler nearest-average rule, and firings that sit between clusters can land in a neighbouring mode.

Readers. The same procedure in a different feature space: the firing’s first-order effect on every downstream routed neuron, meaning how much this neuron’s output at that token changes the pre-activation of each later-layer expert neuron the token is routed to. Readers play no part in building the subcircuits, so this is an independent test of whether the modes correspond to distinct circuit behaviour. Layer-39 neurons have no downstream routed readers, so they are excluded.

Text. An NRDM judge (Jev) receives the label and description of each of the pole’s modes, together with a snippet of the held-out document: the 60 tokens before the firing token, the token itself (marked) and the 8 tokens after it. It answers a multiple-choice question about which mode explains the marked firing, and returns a probability for each mode. To stop the judge from leaning on mode size, the modes appear in random order under neutral letters, and any sentence describing a mode’s size or rank is removed. The judge sees no example firings. The most probable mode is taken as the prediction, and the probabilities are used for separation. Claude Sonnet 5.5 given the same input scores the same. In many neuron-poles adjacent modes are present, often covering features that would co-occur in a text, which makes this a difficult test for the classifier.

Scroll horizontally to compare all activation bands.

Method	p97–99	p99–99.9	p99.9–99.99	p99.99+	Top 1,000	All bands
Inputs	0.957	0.972	0.981	0.988	0.990	0.968
Readers	0.827	0.848	0.889	0.938	0.937	0.853
Text	0.650	0.703	0.787	0.878	0.881	0.707
Chance	0.392	0.355	0.415	0.628	0.563	0.290

Table 5: Macro-F1, averaged over 573 neuron poles with two or more modes; up to 20 held-out firings per band per pole, with the same firings for every method and no example firings. “All bands” pools each pole’s firings across bands.

We find that much of the text-based classification error appears when the highest-firing neuron in the sample arrives via a high-ranked expert. Interestingly, classification based on text suffers, but based in inputs and readers it is relatively stable, suggesting that the text features become less prominent even though the subcircuit itself is still detectable.

Scroll horizontally to compare all methods.

Router rank of the expert	Firings	Inputs	Readers	Text (Jev, descriptions only)
1	11,375	0.986	0.943	0.856
2	7,226	0.982	0.923	0.812
3–4	12,250	0.978	0.900	0.764
5–6	11,526	0.975	0.881	0.722
7–8	10,445	0.973	0.865	0.695
Not in the recorded top 8	1,283	0.971	0.852	0.652

Table 6

Simulation

A good explanation should let us predict what a system will do (Hempel & Oppenheim, 1948), and understanding a neural system often means finding a description that is much smaller than the system but still captures how it works (Jonas & Kording, 2017; Lillicrap & Kording, 2019). In interpretability, this idea is known as simulatability, or meta-prediction: an explanation is good if it is short and allows someone – a person or a model – to predict the outputs of the model it explains (Lipton, 2016; Doshi-Velez & Kim, 2017; Ribeiro et al., 2018; Hase & Bansal, 2020; Hase et al., 2020; Colin, Fel et al., 2022; Chen et al., 2023).

This is a difficult task for a text-based description to pass. A simulator must infer many factors from text alone, including the operation of attention and the distribution of input features, often from hundreds of tokens away. A description often reveals whether a neuron will fire, as we saw in the previous experiments, but predicting how strongly it will fire is much harder because activation strength depends on how much of each input has accumulated across the context—information rarely apparent in the local text. In our measurements, features of the text explain only about 14% of the variance in firing size among a neuron’s firings, against 59% for the activity of its own gate inputs. Consistent with this, giving the simulator a summary of the neuron’s input activity and attention at each token raises the share of damage restored by about 10% versus the result we report below. This information varies with each input sequence and is not specified by the neuron’s general description.

Although the simulator lacks direct access to the internal signals, it may still capture a substantial part of that effect. To test this, we replace the neuron’s actual activations with predictions based on its description and the input text, then measure how much of the damage caused by removing the neuron these predictions repair. We give each neuron pole description to a non-autoregressive decision model (Jev), together with a full document in which a 58-token span is marked. For every token in the span, the model gives a probability for each level from 0 to 9, answering “how strongly does the neuron fire here?”. For each of 486 neuron poles we use about 60 held-out documents (and disjoint from those used to write the description), 20 from each of the three strongest activation bands: the 99.9th percentile, the 99.99th percentile, and the pole’s top 1,000 activations, where negative poles use the same bands on the negative side. In each span the sampled firing sits at token 45, so the span covers the 45 tokens before it and the 12 after.

We convert Jev’s probabilities over levels 0–9 into a single level by rounding their probability-weighted average, then map that level to an activation using a per-neuron lookup table fitted on separate documents. The table assigns each level its mean measured activation, adjusted so that higher levels never correspond to weaker activations, with no added offset or imposed minimum. We substitute these predictions for the neuron’s actual activations throughout the marked span, restricted to tokens routed to its expert and activations on the pole being evaluated, including cases where the gate is closed. To measure recovery, we compare the changes in the model’s next-token probability distributions when these activations are replaced by zeros versus simulated values. We measure both changes relative to the original model using KL divergence, summed across the span. Overall recovery is one minus the ratio of mean remaining damage under simulation to mean damage under zero ablation, giving greater weight to neurons whose removal causes more damage.

We compare the simulator with an ideal predictor that knows the true activation at each token and selects the closest value available in the same 10-level lookup table. This ideal predictor recovers 96% of the damage caused by ablation. The simulator achieves 73% of that recovery overall: 56% in the 99.9th–99.99th-percentile band, 64% at the 99.99th percentile and up, and 76% for the strongest 1,000 firings.

We find that we are able to recover a substantial amount of the KL damage via simulation:

Scroll horizontally to see all three activation bands.

Passages, damage, simulator and ceiling

Passages. For each neuron, about 20 held-out passages per band (29,090 in all), each with one sampled activation from that band at token 45: the neuron’s 1,000 strongest router-weighted activations in a 10-billion-token corpus, activations at the 99.99th percentile and up, or activations from the 99.9th to 99.99th percentile. Negative poles use the corresponding negative tail. Passages often hold other firings too, and all of them are scored.

Damage. Change in next-word predictions (KL divergence summed over the passage) when the neuron is set to zero on every word piece routed to its expert on the pole’s side, gate open or closed. Restored = damage minus the change that remains with the simulated activation in place. Shares are ratios of means over neurons, so neurons that do more damage count for more.

Simulator. A reader model (Jev) gives a probability for each level 0–9 at every word piece from the neuron scientist’s description alone; the expected level is rounded, and a per-neuron table fitted on separate documents turns the level into an activation, with no constant or floor. Nothing inside the model is read.

Ceiling. A perfect reader that, at every word piece, picks whichever of the 10 levels the same table maps closest to the true activation. It restores 96% of the damage overall (95–97% by band); percentages above the bars are the simulator’s share of it.

Figure 12: Damage from removing a neuron and the share restored by its simulated activation across the three strongest activation bands. Percentages are relative to a perfect reader with the same 10-level scale and activation table.

Given the high bar that simulation must fulfill, we find the level of simulatability we are able to attain quite significant.

Input Synthesis

A final way to verify our neuron descriptions is to determine whether the description serves as an effective seed for generating texts that cause a maximal neuron activation (Erhan et al., 2009; Olah et al., 2017; Nguyen et al., 2016b; Fel et al., 2023c; Poerner et al., 2018; Thompson et al., 2024; Simonyan et al., 2013; Yosinski et al., 2015; Geirhos et al., 2023; Ghiasi et al., 2022). We attempt this on a random set of 128 neurons in Qwen 3.6 35B-A3B, using Claude Opus and Fable agents to generate the constructed examples.

We implement this procedure as follows. We designate a controller agent, generally a model like Claude Opus 5.5, who directs a set of synthesizer agents. The synthesizer agents’ jobs are simply to construct sentences based on the rules that the controller provides them. The controller receives a description of the neuron; no examples are provided. Based on the description alone, the controller agent creates a set of rules for the synthesizer agents, which they use to create candidate sentences. These sentences are then tested against the actual model to see if they cause the neuron to activate. The results are fed back to the controller agent, which then creates a new set of rules for the synthesizers to generate an improved set of candidates. This is repeated for several rounds until it reaches parity with or exceeds the natural corpus skyline, or until the maximum number of rounds is exceeded.

For each neuron, we measure how many rounds it takes to exceed the maximum activation value found in the natural corpus of 10B tokens, and how strongly we are able to make the neuron fire. Intuitively, a better description should result in faster convergence to a maximally exciting text.

Texts written from descriptions exceed natural activation maxima

Individual neuron polesMedian over poles

Scroll horizontally to see all rounds.

This procedure succeeds in exceeding the natural corpus’ maximum 94% of the time. Though this method does not evaluate our ability to capture all of the different activation modes of a neuron, it does demonstrate that we are able to capture a strong sense of what the neuron responds to in its most causal modes through our descriptions.

We show some examples of the generated texts below.

× natural max

• 1.55×
  }'⏎⏎echo "Disk:"⏎df -h | awk 'NR>1 {printf "%s %s\n", $

  5

  , $6}'⏎⏎echo "Inodes:"⏎df -i | awk 'NR>1 {print $5,

  L24_E240_N188−In code, the name or index of a specific field, column or element being selected out of a data record.

• 1.46×
  t Agatha had sent Bertie down for a week to stay with Lady Pennyquick and her cousin Lady

  Mid

  L39_E115_N154−The start of a proper name, especially an unpredictable or invented one.

• 1.26×
  'Sudha Pratibha N

  and

  akumar, 78, of Chennai, passed away peacefully on March 3.

  L24_E106_N379−The subword «and» in a South Asian given name or surname starting "Nand-".

• 1.19×
  y, and on Saturday the dentist found two more cavities. And it keeps going: problems upon

  problems

  upon problems upon problems. I am so tired.

  L14_E218_N292−Rhetorical expressions of excess or overwhelming quantity: things piling up, one after another, too many to count or list.

• 1.04×
  Μπανγκόκ, Τζακάρτα, Μανίλα, Σιγκαπούρη και η

  Κ

  ουάλα Λουμπούρ— κάθε μία συμβάλλει με τον δικό της τρόπο στην οικονομική και

  L36_E144_N46+Kuala Lumpur, Malaysia, as an upcoming or current place name.

• 1.03×
  Why?!! Now I will never really

  know

  .

  L24_E132_N259+A future or modal clause claiming that a specific open question will never be answered.

• 1.02×
  tes in coal mines and is historically called firedamp, the miners' old name for it? Answer

  :

  methane, which seeps from coal seams and has caused countless mining explosions when igni

  L39_E79_N331+Methane and other alkane (-ane) hydrocarbon names. The neuron fires at the point where such a name is about to appear.

• 1.02×
  ，老橡树上有什么东西咕咕、咕咕、咕咕地叫个不停，黑暗中闪着两只圆圆的眼睛——是

  貓

  头鹰。

  L36_E137_N183+The opening piece of a multi-token bird name.

All 128 neuron poles

• 1.55×
  }'⏎⏎echo "Disk:"⏎df -h | awk 'NR>1 {printf "%s %s\n", $

  5

  , $6}'⏎⏎echo "Inodes:"⏎df -i | awk 'NR>1 {print $5,
• 1.46×
  t Agatha had sent Bertie down for a week to stay with Lady Pennyquick and her cousin Lady

  Mid
• 1.43×
  和页边距。喜欢竖排阅读体验的用户也可以尝试静读天下、超星阅读器或多看阅读，三者

  对

  繁体竖排小说支持较好。
• 1.41×
  want to achieve, and we design a plan that fits you. Because the plan is yours alone, you

  only

  pay for the sessions you actually need, and nothing is padded out to fill a term.
• 1.39×
  se resp) throws ServletException, IOException {⏎String scheme;⏎scheme=hreq.get

  Scheme

  ();⏎resp.getWriter().println("ok");⏎}⏎```
• 1.34×
  1971⏎United Arab Emirates | 1971 | 1971⏎⏎By 1

  9

  71, all five Gulf states discussed in this chapter—Kuwait, Qatar, Oman, Bahrain and the Un
• 1.32×
  任何风险由用户自行承担。⏎四、本软件更新内容及版本号信息可能存在滞后，本站概不保证该等信息之

  准确性

  ，具体以软件内置的“检查更新”功能为准。⏎五、本站不提供数据备份、系统恢复等售后服务
• 1.31×
  A 25-watt bulb emits about 250 lm. A 40-watt bulb emits about

  4
• 1.29×
  La rotazione (scorrimento circolare) è un'operazione sui bit di una parola:

  il

  bit che esce da un lato rientra dall'altro lato, a differenza dello scorrimento logico do
• 1.29×
  out departing from the scope of the invention, and the invention contemplates all obvious

  variations

  , modifications and equivalents of the structures and methods described herein.
• 1.26×
  'Sudha Pratibha N

  and

  akumar, 78, of Chennai, passed away peacefully on March 3.
• 1.25×
  ttributions were computed using the Captum library, treatment effects were estimated with

  the

  EconML package, and the identifying assumptions were validated via the DoWhy toolkit.
• 1.23×
  n sich selbst hat: das Gefühl, wertvoll zu sein und Respekt zu verdienen. Menschen wollen

  sich

  als Teil einer Gruppe fühlen und von anderen akzeptiert werden.
• 1.22×
  r masses.⏎⏎Problem 1. Fresh nickel powder reacts with carbon monoxide gas to form Ni(C

  O

  )4. What mass of Ni(CO)4 is produced from 25.0 g of nickel, assuming excess
• 1.22×
  .⏎⏎Brooks, F. P. (1995). Ten pounds in a five-pound sack: Reports on

  software

  engineering (Anniversary ed.). Addison-Wesley.⏎⏎Cook, R. D. (1977). Detection
• 1.20×
  陶然亭公园。石评梅的代表作有《涛语》《偶然草》等，散文集《石

  评

  梅选集》于1980年代重新出版。今天，山西平定县建有石评梅纪念馆。
• 1.20×
  learning to align and translate," in Proc. ICLR, 2015.⏎[11] H.

  Sak

  , F. de Chaumont Quitry, T. Sainath, and K. Rao, "Acoustic modelling with
• 1.19×
  列车实施超速防护；当列车实际速度超过允许速度时，系统自动施加紧急制动。司机通过司机室显示屏

  （

  DMI）查看列车当前速度、允许速度和目标距离。第十三条 列车进站前，司机应确认站台屏蔽门
• 1.19×
  Worked solution. Expand everything. Use the right-hand side e^x

  expanded

  about x=0, namely 1 + x + x^2/2.
• 1.19×
  y, and on Saturday the dentist found two more cavities. And it keeps going: problems upon

  problems

  upon problems upon problems. I am so tired.
• 1.18×
  N) vừa công bố báo cáo về tự do tôn giáo trên thế giới, cho thấy hàng triệu Kitô hữu đang

  bị

  bách hại vì đức tin. Tại nhiều quốc gia, các tín hữu bị bắt bớ, nhà thờ bị đóng cửa và
• 1.18×
  over the period.⏎⏎The change in Arizona's population is explained by three components

  :

  natural increase (the surplus of births over deaths), net international migration, and ne
• 1.18×
  s of the observations.⏎6. r measures the strength of a linear relationship.⏎7. r does

  not

  change when we change the units of measurement of x, y, or both.
• 1.18×
  忽見那女子躍身而起，劍鋒一轉，衣袖揚起如雲，剎那間，那白衣女子舞步

  躚

  然，宛若當年公孫大娘再現人間，滿座皆驚。
• 1.17×
  *w)⏎⏎3 2 2 2 2 2⏎o7 = ideal (x - y*w

  ,

  x y - z*w , x*y - z w)⏎⏎o7 : Ideal of R
• 1.17×
  arguments passed to the application.</param>⏎public static void Main(string[] args)⏎

  {⏎
• 1.16×
  on method for fuzzy system of linear equations, Applied Mathematics and Computation 163 (2

  0

  05) 553-563.⏎[3] S. Mitchell, R. Collins, A
• 1.16×
  import {Div

  iders

  } from 'antd';\nimport {Button} from 'antd';\n\nexport default () => <div><Dividers/><
• 1.16×
  helps the weekends. There is no mowing, no edging and no weekly fertiliser run.⏎⏎Soil

  improvement

  involves what you cannot see. Deep roots open channels for water and air, and earthworms
• 1.16×
  免责声明：本站部分内容来源于网络，如有侵权请联系删除。您的热心是对原创的最大

  支持

  。
• 1.15×
  Cử chỉ hùng

  hổ

  của hắn khiến ai cũng sợ.
• 1.15×
  2.10) Information about variants⏎Variants will be accepted: no⏎II.2.10) Information

  about

  variants⏎Variants will be accepted: no⏎II.2.13) Information about European Union fund
• 1.15×
  weak digestion. Ginger and bitter herbs are traditionally said to assist in the digestion

  of

  food.
• 1.14×
  Excel recognizes as a date. The way this function interprets the string is controlled by

  the

  regional settings in the Windows Control Panel. If the string is "6/9/2013", the function
• 1.14×
  안경렌즈 하드코트 처리란, 표면에 하드

  코

  트를 도포하여 내스크래치성을 강화하는 표면 처리 공정을 말합니다. 일반 플라스틱 렌즈(CR-3
• 1.13×
  he 36 hours of the storm. The whole beach was cut back and lowered between the first cross

  -section

  and the last. We ran the first cross-section, at 06:00 on the morning of the first day
• 1.13×
  主编：阎

  乐
• 1.12×
  Abbott, B. P., et al. (2017).

  ‘

  GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral". Physica
• 1.12×
  ⏎Translation of: Snap!⏎⏎This task implements the classic Sieve of Eratosthenes using

  Snap

  !, the visual, blocks-based programming language built on top of Scratch's design philosop
• 1.12×
  Problem A. Beautiful Array\n\nInput\nFirst line contains a

  number

  T, the number of test cases.
• 1.12×
  sicionEntrePantallasDeConfiguracionAvanzada As Integer = ParametrosGenerales.ObtenerTiempo

  Trans

  icionEntrePantallas()⏎
• 1.12×
  Among the Egyptians, Thoth is ever represented

  under

  the form of a man bearing the crook and the flail, and his sister Isis appears beside him
• 1.11×
  Kaleler birer birer düşmüş, halk açlıktan kırılıyordu; tehlikeye giren saltanat

  ını

  kurtarmak için genç sultan ordusunu topladı.
• 1.11×
  C17. El 17 de julio de 2014, según la investigación, el vuelo MH1

  7

  fue derribado.
• 1.11×
  From: Geoffrey R.

  Stone

  <gstone@uchicago.edu>\nTo: Faculty List\nSubject: Re: Workshop schedule\n\nDear colleague
• 1.10×
  onsider viscosity solutions of the fully nonlinear elliptic equation $$\mathcal{M}^{-}(D^{

  2

  }u) \le 0 \le \mathcal{M}^{+}(D^{2}u) \
• 1.10×
  d with curved hemostats to avoid the overlying intercostal vessels, and the subdermal air

  sac

  membrane, thin, translucent and avascular, was penetrated bluntly.
• 1.08×
  vieron vuestros padres, cuando estuvieron al otro lado del río, o a los dioses de los amor

  re

  os en cuya tierra habitáis; pero yo y mi casa serviremos a Jehová. 16 Entonces el pueblo r
• 1.08×
  seconds:⏎⏎Mar 11 09:14:02 myservice[1234]: http

  :

  TLS handshake error from 10.0.0.17:51234: remote error: tls
• 1.07×
  Sichuan Province is home to the world-ren

  owned

  giant panda.
• 1.07×
  The first loaf was too salty and the second was bland. As for the choice of flour

  ,

  I prefer a stone-ground whole wheat with a little rye blended in, because it holds moistu
• 1.07×
  58, Delphos St. John's 52. Shelby County Athletic League: Fort Loramie 6

  6

  , Russia 39; Botkins 52, Jackson Center 48; Anna 70, Fairl
• 1.07×
  Some reasons including heat

  ,

  pressure and friction can cause the material to deform.
• 1.07×
  a game on an array of n integers (Rai and Joshi 2019). The judge, who kept

  count

  of every even number Alice has taken, awards the win.
• 1.06×
  kg⏎⏎Solution: Recall what density is. Let us rebuild the formula from first principles

  .

  Density is a mass spread through a volume, so its units are those of mass divided by thos
• 1.06×
  The goal is to reverse that. A strengths coach does not ignore your weaknesses; she helps

  you

  manage them, by delegating them, by partnering with someone whose strengths cover your ga
• 1.06×
  铎与主教的共融，信友与牧者的共融，构成教会的合一。与罗马宗座的

  完全

  共融，是天主教会的标记。在圣体圣事中，我们彼此共融，与基督共融，与
• 1.06×
  $('#editor').keydown(function (evt) {⏎if (!evt.original

  Event

  ) {⏎return;⏎}⏎if (evt.originalEvent.keyCode == 27) {⏎
• 1.06×
  ilinoleoyl-sn-glycero-3-phosphoethanolamine;1,2-dilinoleoyl-sn-glycero-3-phosphoglycerol;1

  -l

  inoleoyl-2-hydroxy-sn-glycero-3-phosphocholine.
• 1.06×
  e class — deductions Romney refuses to name.⏎⏎Then there are his numerous entitlement

  plans

  , which have evolved over eighteen months of campaigning. Which version he would actually
• 1.06×
  network programming from the Blue Network through the 1940s. In 1955 it switched to a

  "

  good music" and news format.
• 1.06×
  이 책은 어려운 경제 이론을 초보자도 쉽게 이해할 수 있도록, 그림과 표를

  통해

  , 구체적인 예를 들어 친절하게 설명합니다.
• 1.06×
  nce for the financial sector is the subject of a new European framework. Regulation (EU) 2

  0

  22/2554 of the European Parliament and of the Council on digital operational resilience fo
• 1.06×
  The two transportable strontium lattice clocks were moved by road to

  the

  laboratory.
• 1.06×
  2008年11月12日 網站改版上線 http://www.example.tw/⏎•

  &

  2009年4月25日 新增RSS訂閱功能 http://www.example.tw/rss.xml⏎•
• 1.05×
  with the validation values from the specification.⏎⏎select⏎p_brand,⏎p_type,⏎p

  _size

  ,⏎count(distinct ps_suppkey) as supplier_cnt⏎from⏎partsupp,⏎part⏎where⏎p
• 1.05×
  fluorescent light hums and flickers. Is the problem more likely the tube, the starter or

  the
• 1.05×
  о,⏎точно този най-малко ми даде, как,⏎и когато го питах какво ли,

  ⏎

  той само се усмихна.
• 1.05×
  The word pachy

  lob

  us appears in the index of Don's General History of the Dichlamydeous Plants. The word pac
• 1.05×
  gle fibre, lower the micropipette slowly into the muscle until a fibre is successfully imp

  aled

  ; a brief buzz of capacitance compensation helps the tip penetrate the membrane.
• 1.05×
  ;⏎using System.Collections.Specialized;⏎using System.ComponentModel.Design;⏎using ZK

  Web

  ;⏎using ZKWeb.Web;⏎
• 1.05×
  .ndarray:⏎"""Build the adjacency matrix of a graph.⏎⏎A set has no order, so

  we

  first convert the vertex set to a sorted list;⏎the position of a vertex in that lis
• 1.05×
  $ is a perfect square they are rational. Also $\sqrt{\Delta} = 8$. We use $x = \

  frac

  {-b \pm \sqrt{\Delta}}{2a}$.
• 1.05×
  \n| 1 | Nick Drake | Pink Moon | Island | 1972 | Folk | [Spotify](

  https

  ://open.spotify.com/album/1Czp6sYvO9Z1qVzHcfUfp
• 1.05×
  电影《极速

  天使

  》由马楚成执导，汤唯、刘诗诗、韩彩英主演，讲述三位女赛车手的故事
• 1.04×
  [00:03.20] nama no koe de, kimi ni todoke⏎[00:06.10] な

  ま

  のこえで、きみにとどけ⏎[00:09.00] nama no kimochi wo kakusanai de
• 1.04×
  ure⏎- Fake Out⏎- High Jump Kick⏎- Zen Headbutt⏎- Ice Punch⏎⏎Set details: Fake

  Out

  lets Medicham chip a lead and break Focus Sash, while High Jump Kick is the main STAB. Ze
• 1.04×
  but Gangwon had already completed the survey last month.⏎⏎[핵심 문장 번역]⏎이들 지역 중 제주

  를

  제외한 나머지 지역은 이미 지난달 조사가 끝났다.
• 1.04×
  Μπανγκόκ, Τζακάρτα, Μανίλα, Σιγκαπούρη και η

  Κ

  ουάλα Λουμπούρ— κάθε μία συμβάλλει με τον δικό της τρόπο στην οικονομική και
• 1.04×
  CAMBRIDGE, MA: MIT PRESS, 1961.⏎⏎12. P. M. SeNG

  E

  , The Fifth Discipline: The Art and Practice of the Learning Organization (New York: Doubl
• 1.04×
  Storage. A whole loaf keeps for four days. The bread, once

  sliced
• 1.04×
  t me compare this with the given numbers. We computed the length PQ as 3 using the ratio.

  But

  the problem states that PQ is 2, so something is off. Let me go back to the similar trian
• 1.04×
  ers } from '../api/users';⏎⏎export function UserList() {⏎const { data, isLoading,

  is

  Fetching, error, refetch } = useQuery({⏎queryKey: ['users'],⏎queryFn: fetch
• 1.04×
  я Карамазовы. Книга первая.⏎—Sí —dijo Zósima—. Hubo solo un par de

  tales

  casas en toda la provincia, y la segunda estaba abandonada desde la guerra.
• 1.04×
  world-class education to anyone, anywhere loading external resources on our website ( see

  Figure

  1 )
• 1.03×
  corrosion inhibitor cost⏎cooling tower biocide cost⏎cooling tower chemicals bunnings

  ⏎
• 1.03×
  measured versus predicted:⏎⏎Sample 1: measured 66,751, predicted 26,60

  5

  .⏎Sample 2: measured 41,208, predicted 38,915.⏎
• 1.03×
  r the State Administrative Council and the ethnic armed organisations have agreed to sign

  a
• 1.03×
  Why?!! Now I will never really

  know

  .
• 1.03×
  360；第三步：两边同时减去 72，消去常数项。2x = 28

  8
• 1.03×
  with psychopy 2022.1.4 on macOS.⏎"""⏎# PsychoPy imports⏎from psych

  opy

  import visual, core⏎
• 1.03×
  13⏎13 = 13⏎Because the statement is true, 5 is a solution.⏎⏎CHECK YOUR

  SELF

  1⏎Is 4 a solution of the equation 3x - 2 = 10?⏎⏎Two equations
• 1.03×
  quèrent dans le château. Le comte tenta donc de jeter des vivres et des gens d'armes dans

  la

  place.
• 1.03×
  Testimonials from our students⏎⏎"Overall, I would say that, Professor Nguyen

  did
• 1.03×
  Better Business Bureau – Accredited Businesses⏎Royal SEPTIC

  and

  Plumbing⏎1420 S. Peoria Ave – (918) 555-0142
• 1.03×
  ку на колени. Поезд трогается.⏎Она закрывает глаза.⏎⏎* * *⏎⏎

  Когда

  она открывает их, долины уже нет. Женщина напротив чистит апельсин. Запах наполняет вагон
• 1.02×
  tes in coal mines and is historically called firedamp, the miners' old name for it? Answer

  :

  methane, which seeps from coal seams and has caused countless mining explosions when igni
• 1.02×
  ，老橡树上有什么东西咕咕、咕咕、咕咕地叫个不停，黑暗中闪着两只圆圆的眼睛——是

  貓

  头鹰。
• 1.02×
  is delay, yet dared not press the matter, for it was the custom of that country that none

  could

  see the Khan save by his express command. At length, on the ninth day, a chamberlain of h
• 1.02×
  largest relations of their kind and are proved by exhibiting a suitable open bisimulation

  up

  to injective substitution.
• 1.02×
  Tilskotet vart auka i fjor.

  Til

  samanlikning fekk skulane mindre. Tiltaka skal evaluerast neste år.
• 1.02×
  zation. For every well, a z-score was calculated as the measured value of that particular

  well

  minus the mean of all negative-control wells on the same plate, divided by the standard d
• 1.02×
  tember 2008. The Veteran has not alleged or demonstrated any prejudice with regard to the

  content
• 1.01×
  49.⏎⏎== External links ==⏎* Lecture by Smith on quantum optics, 1979, video in

  the

  Royal Institution archive⏎* John Smith at IMDb⏎
• 1.01×
  pockets among the frankincense, are tapped in the same way. Their bark, when wounded, ble

  eds

  a dark reddish-brown resin that is noticeably stickier and slower to set.
• 1.01×
  the Carnegie foundation in 1912.⏎⏎In operations news, the second week found Abby that

  she

  was coming in early to help set up the new inventory tracking system, and her dedication
• 1.01×
  filter builder and the search syntax.⏎/// </summary>⏎public interface IPredicate⏎

  {⏎string Name { get; }⏎string Symbol { get; }⏎bool Matches(object fieldValue,
• 1.01×
  10月28日。⏎⏎本文

  来自于

  经济观察网
• 1.01×
  `sudo systemctl status nginx` |⏎| 开机自启 | `update-rc.d nginx defaults` | `sudo

  systemctl
• 1.01×
  with trade and manufacturing categories.⏎⏎1. Categories: Dead people | Fictional axe

  and

  hammer wielders | Knife and tool makers
• 1.01×
  insgesamt zwanzig Sekunden und wurde als Siebzehnter gewertet.⏎⏎Stand der Weltmeisters

  chaft

  nach dem Rennen⏎⏎
• 1.01×
  ，芙蓉楼即为其送别友人之处。洪江市东接溆浦县，南连会同县，西

  界
• 1.01×
  املا، وان الجانب الاميركي لن يقبل بتأجيل جديد. واشارت مصادر ديبلوماسية الى ان المسؤولين

  الام

  يركيين ابلغوا رئيس المجلس ان الوقت ينفد.
• 1.00×
  ght to be teaching others. Instead, you need someone to teach you all over again the very

  elementary

  things about God's word. You are like babies who need milk and cannot eat solid food. 13
• 1.00×
  1867 at Lambeth Palace Chapel.⏎⏎Marriage.—On 19 June 1849

  ,
• 1.00×
  You cannot at the same time serve God

  and

  mammon.
• 1.00×
  Average Weather during January

  in

  Tunis, Tunisia\nThe average minimum temperature is 8.0°C. The amount of rain is normal wi
• 1.00×
  Экстракт центеллы запускает и поддерживает вы

  работку

  коллагена.
• 1.00×
  RAPP (1925–1932), VOAPP (1928–1932

  ),

  MAPP (1923–1932), VAPP (1920–1928
• 1.00×
  ntras?<|im_end|>⏎<|im_start|>assistant⏎Senamiestis laikomas Vilniaus miesto centru, o

  cent

  ras tęsiasi iki Gedimino prospekto.
• 0.94×
  states such as⏎// 'loading', 'loaded', 'interactive' and 'complete'.⏎if (newScript

  .readyState

  ) {⏎newScript.onreadystatechange = function () {⏎if (newScript.readyState ==
• 0.89×
  her, at last, she noticed, the first thin snow settling on the shoulders of a few people

  still

  hurrying home.
• 0.79×
  s not creation. It is the next best thing, and I am glad to have it.⏎⏎\begin{theorem}\

  label

  {thm:main}⏎There are infinitely many prime numbers; that is, for every integer $N$ there
• 0.79×
  了该镇的堤坝，迫使数千人在黑暗中撤离家园，许多人只穿着睡衣。 “都没有了，我们还剩下什么

  ？”

  一位年轻的母亲说。
• 0.71×
  пити до бази даних, і схоже, що другий колбек спрацьовує раніше

  ,

  ніж завершився перший. Через це в другому колбеку змінна result виявляється невизнач
• 0.68×
  Perfect!⏎/// When the

  current

  node is visited, stop.
• 0.58×
  it. A family car tends to earn its keep more on the boring trips than on the exciting ones

  ,
• 0.46×
  ⏎【出版单位】 河北教育出版社⏎【主办单位】 河北省教育厅⏎【卷】 十⏎【Volume】

  12⏎

We consider this a useful validation of the primary mode of each neuron-pole, especially at the higher activation levels, though it does not validate the full range of functions that a neuron can compute.

How we interpret these results

We can synthesize maximally-activating texts across neurons, and predict neuron activations to a significant degree. At p99.9/0.1 and above, we are also able to recover 73% of the perfect-reader ceiling for the difficult task of per-token simulation on a neuron's highest activation bands. Though we see that there is still room to improve these simulation metrics, we can clearly see that increased amounts of intelligence and compute consistently increase our scores. Moreover, there may be a ceiling on what is predictable through text: the description must not only describe what the neuron is detecting from text, but must also to some degree anticipate the intermediate model computations, including attention, that lead to a neuron firing.

We find these results quite compelling nevertheless. Not only can we enumerate a neuron’s modes, but we can also describe its behavior to a meaningful extent! This is far from the expectation we started out with and we think it has strongly positive implications for the field of interpretability.

Example neuron descriptions

So what do the neurons we’ve analyzed look like? We find a vast landscape of fascinating structures among them, encompassing a broad range of computations. We see evidence of universal primitives, collections of convenient heuristics, dataset artifacts, and many more phenomena. Alongside this article we are releasing a dataset of circuit diagrams, activation examples, and Neuron Scientist-authored pages for each of the stratified set of 2,734 representative neurons in Qwen 3.6 35B-A3B. To provide some initial examples:

• Helping people live more independently (L13_E133_N30): Responds to writing about technology and support that help older or disabled people communicate, manage daily activities, or live independently. The surrounding context matters: the response brings together information about the kind of writing and the benefits being described.
• Trying a new approach after getting stuck (L16_E9_N229): Responds when a problem solver has tried something, failed, and tentatively proposes another approach: “Hmm, this isn’t working. Maybe I should try…”. The response often lands on “Maybe” or “Perhaps”. Looking back with regret, as in “Maybe I should have used…”, produces little response.
• Registering a moment of self-questioning (L9_E98_N330): Responds when the model interrupts its reasoning with “Wait”. Increasing its influence can then push the model to settle on an answer too quickly, including accepting a planted wrong answer; pushing it in the opposite direction encourages more self-questioning. Its effect fits a signal that doubt has already been raised.
• Crossing into the next month or year (L4_E190_N436): Responds when a date moves into the next month or year relative to an earlier date in the text. It responds at “June” in “31 May – 1 June”, or at the final digit of “2016” after a date in December 2015. The earlier date can be hundreds of word pieces away. Responses are strongest just after the calendar changes; repeating the same month or moving backwards produces little response.
• Marking a finding as estimated or uncertain (L14_E2_N240): In research writing, responds to words that qualify a result before naming it: “estimated risk”, “predicted pathways”, “possible interactions”, or “self-reported” measurements. Replacing these with words such as “measured”, “observed”, or “actual” removes the response. Word order also matters: “the software predicted…” produces little response.
• Locating a place within its former borders (L24_E230_N452): Responds to the country, empire, or region in a place description when that place used to belong there but no longer does. Examples include “Vilna, Poland” and “Monongalia County, Virginia”. The same country or state name produces little response when it still contains the place, as in “Krakow, Poland” or “Fairfax County, Virginia”.
• Keeping track of “release” elsewhere in a document (L36_E110_N224): Its strongest responses occur in documents mentioning “release”, across uses such as a release lever, a mould-release coating, a press release, or a software release date. The response may appear on a space, number, or punctuation mark away from the word itself. Looking only at the immediate phrase misses the connection.
• Comparing someone’s condition with their age (L24_E121_N186): Responds to descriptions such as “mature for her age”, “I’m 24 but look 16”, and “wise beyond her years”. The comparison must involve age and an age-related quality, such as maturity, appearance, or health. Replacing age with weight or cost removes the response.

Alongside this writing, we provide a library of neurons for browsing and also release the low-effort labels for all 10 million neuron-poles.

Conclusion

Implications for neuron interpretability

Analyzing neural networks in the neuron basis has often been treated as unworkable. In this view, neurons mix so many unrelated functions, or distribute meaningful computations so diffusely, that studying individual units offers little insight into how the model actually works. Our work stands in contrast to this view, adding to a growing body of work that finds neurons are more interpretable than previously thought. In the model and activation regimes we study, we can interpret most neurons as having one to a handful of computational roles. We can also trace their contributions through sparse circuits.

Alongside Arora et al. (2026), we find that sparse but strong activations are responsible for the majority of a neuron's causal effect. Removing the weakest activations preserves substantial model function, and brief fine-tuning recovers benchmark performance even with approximately 88% of routed neuron activations removed. Our circuit experiments reveal further sparsity: for specific functions, a handful of neurons per layer can reproduce the effects of the entire MLP layers they occupy. These findings allow us to focus our interpretive efforts on the high-activation regimes where neurons are both most consequential and most readily understood.

Within these regimes, most neuron poles have a single or a small number of coherent, computational roles. Even when neuron poles have multiple roles, we can concisely describe and distinguish them through their upstream inputs and downstream readers. Heterogeneity is less common than previously thought, and less problematic when it does arise: understanding a neuron’s different roles and how the models use them can provide a useful explanation of its computation.

Our agent, the Neuron Scientist, turns these explanations into testable predictions. Its descriptions are precise enough that a simulator can predict neuron activations from the description alone, recovering approximately 73% of the perfect-reader ceiling for KL damage from ablation in the strongest activation regime. These results show that concise descriptions can capture a substantial share of the causal effects of the neuron firings we study.

We approached this work believing that neurons resist direct study and that, as a result, neural networks require some sort of transformation or untangling before we can understand how they work. Instead, we found that models can operate under considerable neuron sparsity; that most neurons have one or a few coherent computational roles; and that we can describe those roles well enough to simulate their firings. We also found that scaling agent-driven investigations reliably improves our explanations. This gives us optimism that continued direct study, with better agents, tools, and methods, can yield significant advances in interpretability.

The ability to devote substantial compute to interpretability in general, and to understanding neurons in particular, is a major change in our field. Agents are now capable enough to conduct experiments, inspect circuits, and test hypotheses. This lets us conduct detailed investigations of individual neurons at a scale that would be near impossible to achieve manually. Scaling these investigations could yield superlinear gains: understanding more neurons provides upstream and downstream context, which we have already shown makes interpretation easier. We see substantial room to improve the agent harness, experimental tools, and simulation methods, and to measure how the resulting understanding scales with compute.

Future Work

Much about neurons remains to be understood. We still need to understand how neurons acquire their computational roles during training, how heterogeneous modes arise, and what weak and Q3 activations contribute. We must also establish how far these findings generalize across models and behaviors. But neurons already show substantial promise as a basis for understanding circuits, steering and probe vectors, and higher-level behaviors such as reasoning. The next step is to connect these increasingly precise accounts of individual neurons into explanations of the larger computations they jointly implement.

This work is our first technical post in a series on native interpretability: studying the original network via its own units, rather than via a replacement for them, as outlined in “Neural Networks Are Already Clean”. We are especially interested in a few directions for future research: new ways to zoom out from single neurons to the larger computations in which they take part while staying faithful to the network’s own mechanisms; how neurons and circuits form during training; and how attention and neurons work together.

For a more extensive discussion of open questions in neuron interpretability, and what remains to be explained, see the appendix.

Acknowledgements

We are grateful to our friends and colleagues who contributed to this work and provided feedback on early drafts. We would particularly like to thank Aryaman Arora, Leon Bergen, Usha Bhalla, Dan Braun, Lucius Bushnaq, Michael Jae Byun, Oliver Clive-Griffen, Fatih Dinc, William Dorrell, Matt Feiszli, Thomas Fel, Tucker Fross, Atticus Geiger, Tarun Gogineni, Tal Haklay, Dron Hazra, Eric Ho, Thomas Icard, David Klindt, Andrew Lee, Jack Lindsey, Ekdeep Singh Lubana, Jack Merullo, Lee Sharkey, and Connor Watts.

Author Contributions

Curt Tigges: Performed much of the research on the LLM side, including developing sparsification methods, creating methods for identifying subcircuits and modes, designing agents to do neuron research, and conducting wide-scale studies of neuron populations. Identified how neurons’ multiple meanings can be disambiguated downstream. Developed KL-recovery-based simulation and other evaluation methods. Studied thousands of neurons. Wrote much of the post.

Matthew Kowal: Helped run some of the additional experiments for the sparsity and simulation sections. Helped with writing and organizing the post. Studied 10s of neurons.

Thomas McGrath: Provided guidance and ideas for experiments and research directions, provided feedback for writing and early outlines.

Owen Lewis: Helped with project organization, writing, and the design of some experiments and metrics.

Nick Cammarata: Started the project in October 2025. Created the first neuron papers. Built the first agents for describing neurons and circuits. Developed the blind maximum technique and first polysemanticity score measures. Studied thousands of neurons. Improved simulations. Improved the total neuron view and built the mode profiles. Did the Colorado neuron studies and wrote that section. Designed the header. Had a minority role in writing and editing the text more broadly.

References

1. Le et al. (2012). Building high-level features using large scale unsupervised learning.
2. Zeiler & Fergus (2014). Visualizing and Understanding Convolutional Networks.
3. Bau et al. (2017). Network Dissection: Quantifying Interpretability of Deep Visual Representations.
4. Olah et al. (2017). Feature Visualization.
5. Radford et al. (2017). Learning to Generate Reviews and Discovering Sentiment.
6. Goh et al. (2021). Multimodal Neurons in Artificial Neural Networks.
7. Nguyen et al. (2016). Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks.
8. Olah et al. (2020). Zoom In: An Introduction to Circuits.
9. Elhage et al. (2022). Toy Models of Superposition.
10. Arora et al. (2018). Linear Algebraic Structure of Word Senses, with Applications to Polysemy.
11. Park et al. (2023). The Linear Representation Hypothesis and the Geometry of Large Language Models.
12. Bricken et al. (2023). Towards Monosemanticity: Decomposing Language Models With Dictionary Learning.
13. Cunningham et al. (2023). Sparse Autoencoders Find Highly Interpretable Features in Language Models.
14. Geva et al. (2021). Transformer Feed-Forward Layers Are Key-Value Memories.
15. Dai et al. (2022). Knowledge Neurons in Pretrained Transformers.
16. Geva et al. (2022). Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space.
17. Gurnee et al. (2024). Universal Neurons in GPT2 Language Models.
18. Arora et al. (2026). Language Model Circuits Are Sparse in the Neuron Basis.
19. Dravid et al. (2026). Neuron Populations Exhibit Divergent Selectivity with Scale.
20. Feucht et al. (2026). Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Concepts.
21. Shabalin & Piotrowski (2026). Comparing Sparsified MLPs and Transcoders: Many Transcoder Features Are Neurons in Disguise.
22. Arora, Wu et al. (2025). Language Model Circuits Are Sparse in the Neuron Basis.
23. Hubel & Wiesel (1962). Receptive fields, binocular interaction and functional architecture in the cat's visual cortex.
24. Barlow (1972). Single Units and Sensation: A Neuron Doctrine for Perceptual Psychology?
25. Quiroga et al. (2005). Invariant visual representation by single neurons in the human brain.
26. Saxena & Cunningham (2019). Towards the neural population doctrine.
27. Ebitz & Hayden (2021). The population doctrine in cognitive neuroscience.
28. Rigotti et al. (2013). The importance of mixed selectivity in complex cognitive tasks.
29. Shazeer (2020). GLU Variants Improve Transformer.
30. Dauphin et al. (2017). Language Modeling with Gated Convolutional Networks.
31. Ramachandran et al. (2017). Searching for Activation Functions.
32. Elfwing et al. (2018). Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning.
33. Hendrycks & Gimpel (2016). Gaussian Error Linear Units (GELUs).
34. Touvron et al. (2023). LLaMA: Open and Efficient Foundation Language Models.
35. Yang et al. (2025). Qwen3 Technical Report.
36. Pearce et al. (2024). Bilinear MLPs enable weight-based mechanistic interpretability.
37. Sharkey (2023). A technical note on bilinear layers for interpretability.
38. Jayakumar et al. (2020). Multiplicative Interactions and Where to Find Them.
39. Veness et al. (2019). Gated Linear Networks.
40. Fishman et al. (2024). Scaling FP8 training to trillion-token LLMs.
41. Szegedy et al. (2014). Intriguing properties of neural networks.
42. Morcos et al. (2018). On the importance of single directions for generalization.
43. Zhou et al. (2018). Revisiting the Importance of Individual Units in CNNs via Ablation.
44. Leavitt & Morcos (2020). Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs.
45. Ghorbani & Zou (2020). Neuron Shapley: Discovering the Responsible Neurons.
46. Dhamdhere et al. (2018). How Important Is a Neuron?
47. Li et al. (2023a). The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers.
48. Liu et al. (2023a). Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time.
49. Mirzadeh et al. (2023). ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models.
50. Zhang et al. (2022). MoEfication: Transformer Feed-forward Layers are Mixtures of Experts.
51. Liu et al. (2024). Training-Free Activation Sparsity in Large Language Models.
52. Lee et al. (2024). CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models.
53. Jaszczur et al. (2021). Sparse is Enough in Scaling Transformers.
54. Ahmad & Scheinkman (2019). How Can We Be So Dense? The Benefits of Using Highly Sparse Representations.
55. Makhzani & Frey (2013). k-Sparse Autoencoders.
56. Song et al. (2024). ProSparse: Introducing and Enhancing Intrinsic Activation Sparsity within Large Language Models.
57. Zhang et al. (2024). ReLU² Wins: Discovering Efficient Activation Functions for Sparse LLMs.
58. Wang et al. (2024). Q-Sparse: All Large Language Models can be Fully Sparsely-Activated.
59. Song et al. (2023). PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU.
60. Alizadeh et al. (2023). LLM in a flash: Efficient Large Language Model Inference with Limited Memory.
61. Zhang et al. (2023). Emergent Modularity in Pre-trained Transformers.
62. Sun et al. (2024). Massive Activations in Large Language Models.
63. Dettmers et al. (2022). LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale.
64. Kovaleva et al. (2021). BERT Busters: Outlier Dimensions that Disrupt Transformers.
65. Puccetti et al. (2022). Outliers Dimensions that Disrupt Transformers Are Driven by Frequency.
66. Bondarenko et al. (2023). Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothing.
67. Olshausen & Field (2004). Sparse coding of sensory inputs.
68. Bills et al. (2023). Language models can explain neurons in language models.
69. Murphy (2002). The Big Book of Concepts.
70. Rosch et al. (1976). Basic objects in natural categories.
71. Murphy & Medin (1985). The role of theories in conceptual coherence.
72. Sloutsky & Deng (2019). Categories, concepts, and conceptual development.
73. Barsalou (1983). Ad hoc categories.
74. Klindt et al. (2026). A unifying framework from neural superposition to sparse interpretable codes.
75. Vig et al. (2020). Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias.
76. Geiger et al. (2025). Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability.
77. Mueller et al. (2025). The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysis.
78. Fusi et al. (2016). Why neurons mix: high dimensionality for higher cognition.
79. Marr (1982). Vision: A Computational Investigation into the Human Representation and Processing of Visual Information.
80. Lee & Messerschmitt (1988). Multiplexing.
81. Mante et al. (2013). Context-dependent computation by recurrent dynamics in prefrontal cortex.
82. Salinas & Thier (2000). Gain modulation: a major computational principle of the central nervous system.
83. Dreyer et al. (2024). PURE: Turning Polysemantic Neurons Into Pure Features by Identifying Relevant Circuits.
84. Gandelsman et al. (2024). Interpreting the Second-Order Effects of Neurons in CLIP.
85. Binder et al. (2016). Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers.
86. Traag et al. (2019). From Louvain to Leiden: guaranteeing well-connected communities.
87. Zheng et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena.
88. Paulo et al. (2024). Automatically Interpreting Millions of Features in Large Language Models.
89. Hill (1973). Diversity and Evenness: A Unifying Notation and Its Consequences.
90. Jost (2006). Entropy and diversity.
91. Simpson (1949). Measurement of Diversity.
92. Elhage et al. (2023). Privileged Bases in the Transformer Residual Stream.
93. Heap et al. (2025). Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformers.
94. Hewitt & Liang (2019). Designing and Interpreting Probes with Control Tasks.
95. Makelov et al. (2023). Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching.
96. Zimmermann et al. (2023). Scale Alone Does not Improve Mechanistic Interpretability in Vision Models.
97. Korznikov et al. (2026). Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines?
98. Shaham et al. (2024). A Multimodal Automated Interpretability Agent.
99. Schwettmann et al. (2023a). FIND: A Function Description Benchmark for Evaluating Interpretability Methods.
100. Lu et al. (2024). The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery.
101. Gottweis et al. (2025). Accelerating scientific discovery with Co-Scientist.
102. Singh et al. (2024). Rethinking Interpretability in the Era of Large Language Models.
103. Zimmermann et al. (2024). Measuring Per-Unit Interpretability at Scale Without Humans.
104. Colin et al. (2024). Local vs distributed representations: What is the right basis for interpretability?
105. Erhan et al. (2009). Visualizing Higher-Layer Features of a Deep Network.
106. Nguyen et al. (2016b). Synthesizing the preferred inputs for neurons in neural networks via deep generator networks.
107. Fel et al. (2023c). Unlocking Feature Visualization for Deeper Networks with MAgnitude Constrained Optimization.
108. Poerner et al. (2018). Interpretable Textual Neuron Representations for NLP.
109. Thompson et al. (2024). Fluent dreaming for language models.
110. Simonyan et al. (2013). Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps.
111. Yosinski et al. (2015). Understanding Neural Networks Through Deep Visualization.
112. Geirhos et al. (2023). Don't trust your eyes: on the (un)reliability of feature visualizations.
113. Ghiasi et al. (2022). Plug-In Inversion: Model-Agnostic Inversion for Vision with Data Augmentations.
114. Hempel & Oppenheim (1948). Studies in the Logic of Explanation.
115. Jonas & Kording (2017). Could a Neuroscientist Understand a Microprocessor?
116. Lillicrap & Kording (2019). What does it mean to understand a neural network?
117. Lipton (2016). The Mythos of Model Interpretability.
118. Doshi-Velez & Kim (2017). Towards A Rigorous Science of Interpretable Machine Learning.
119. Ribeiro et al. (2018). Anchors: High-Precision Model-Agnostic Explanations.
120. Hase & Bansal (2020). Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior?
121. Hase et al. (2020). Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?
122. Colin, Fel et al. (2022). What I Cannot Predict, I Do Not Understand: A Human-Centered Evaluation Framework for Explainability Methods.
123. Chen et al. (2023). Do Models Explain Themselves? Counterfactual Simulatability of Natural Language Explanations.
124. Gao et al. (2025). Weight-sparse transformers have interpretable circuits.
125. Heimersheim & Nanda (2024). How to use and interpret activation patching.
126. Zhang & Nanda (2024). Towards Best Practices of Activation Patching in Language Models: Metrics and Methods.
127. Meng et al. (2022a). Locating and Editing Factual Associations in GPT.
128. Mueller et al. (2025). MIB: A Mechanistic Interpretability Benchmark.
129. Pearl (2001). Direct and Indirect Effects.
130. Dunefsky et al. (2024). Transcoders Find Interpretable LLM Feature Circuits.
131. Paulo & Belrose (2025). Sparse Autoencoders Trained on the Same Data Learn Different Features.
132. Leask et al. (2025). Sparse Autoencoders Do Not Find Canonical Units of Analysis.
133. Fel et al. (2025). Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models.
134. Chanin et al. (2024). A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders.
135. Kantamneni et al. (2025). Are Sparse Autoencoders Useful? A Case Study in Sparse Probing.
136. Wu et al. (2025). AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders.
137. Pati et al. (1993). Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition.
138. Mallat & Zhang (1993). Matching pursuits with time-frequency dictionaries.
139. Olshausen & Field (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1?
140. Costa et al. (2025). From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit.
141. O’Neill et al. (2025). Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders.
142. Olah (2023). Distributed Representations: Composition & Superposition.
143. Mu & Andreas (2020). Compositional Explanations of Neurons.
144. Smolensky (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems.
145. Braun et al. (2025). Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition.
146. Bushnaq et al. (2025). Stochastic Parameter Decomposition.
147. Braun et al. (2024). Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning.
148. Brangus, 2026.
149. Jafari et al. (2025). RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patching.
150. Elhage et al. (2021). A Mathematical Framework for Transformer Circuits.
151. Xiao et al. (2023). Efficient Streaming Language Models with Attention Sinks.
152. Doshi et al. (2026). Bi-Orthogonal Factor Decomposition for Vision Transformers.
153. Voita et al. (2024). Neurons in Large Language Models: Dead, N-gram, Positional.
154. Chang & Bergen (2025). Bigram Subnetworks: Mapping to Next Tokens in Transformer Language Models.
155. Gurnee et al. (2023). Finding Neurons in a Haystack: Case Studies with Sparse Probing.
156. Reichardt & Bornholdt (2006). Statistical Mechanics of Community Detection.

Citation

Tigges, C., Kowal, M., McGrath, T., Lewis, O., and Cammarata, N., 2026. Neurons as an Interpretable Basis. Goodfire.

BibTeX:

@article{tigges2026neurons,
  author = {Tigges, Curt and Kowal, Matthew and McGrath, Thomas and Lewis, Owen and Cammarata, Nick},
  title = {Neurons as an Interpretable Basis},
  publisher = {Goodfire},
  year = {2026},
  note = {https://www.goodfire.com/research/neurons-as-an-interpretable-basis}
}

Footnotes

1. How many causally important modes show up in the population? Because we sample the same number of firings from each activation band, rarer high-activation firings are over-represented. To recover population frequency, we weight each firing by w = (the band's share of the population) / (the number of firings kept from that band), and replace KL and firing counts with their w-weighted versions before computing shares. This gives us a mean of 1.56 and a median of 1.04. ↩