Understand and debug your AI model
There is remarkable mathematical structure and geometry within neural networks. We help you uncover the hidden representations inside your model to remove the guesswork from AI training - going from alchemy to precision engineering.
Used by AI teams at
Understand the scientific foundations of neural networks so that we can intentionally design AI
We believe that AI is the most consequential technology of our time, yet today we train models with remarkably little understanding of the nature of their intelligence.
We’re an interpretability research lab dedicated to creating the science and technology to change that.
Interpretability for all types of AI models
LLMs
Life SciencesNovel methods to understand,
debug, and design your AI model
Understand
Reverse engineer the causal mechanisms of AI to reveal its internal structure, uncovering novel science and validating when predictions reflect true understanding.
Revealing the neural geometry inside models
Neural networks have rich structure in their activations, underpinning their internal algorithms. We’ve developed new methods to map and edit this geometry, unlocking deeper interpretability and better control of model behavior.
Discovering a novel class of Alzheimer's biomarkers
We identified a novel class of biomarkers for Alzheimer's detection by interpreting an epigenetic model, the first major finding in the natural sciences obtained from reverse-engineering a foundation model.
Explaining 4.2 million genetic variants
We achieved state-of-the-art performance in predicting whether and how genetic variants cause disease, releasing interpretable-by-design predictions for all 4.2 million variants in the NIH’s ClinVar database.
Debug
Precisely debug issues with model behavior, identify and remove confounders, and diagnose failures before they occur in production.
Reveal and shape what your model learns, before you train
Predictive data debugging lets us predict which behaviors RL on a preference dataset will amplify or suppress, trace that back to the responsible data, and reshape the data and/or training to prevent undesired effects.
Surfacing unexpected behaviors caused by a training run
Training often causes unexpected and undesired behaviors which show up only rarely, meaning evals often miss them. We can identify such behaviors by amplifying the diff between two checkpoints in logit space - a method now used for pre-deployment testing of frontier models.
Detecting performative chain-of-thought
We tracked “performative chain-of-thought”: when models “know” their final answer but continue to generate chain-of-thought anyways. We showed that probes can enable early exit from reasoning traces, saving up to 68% of tokens with minimal accuracy loss.
Design
Control training precisely to ensure your model learns what you want with less data and fewer off-target effects.
Reducing hallucinations with features as rewards
We cut hallucinations in an LLM by 58% by using interpretability to guide model training. Our approach was ~90x lower cost per intervention than LLM-as-judge, with no degradation in standard benchmarks.
Interpreting language model parameters
We’re pioneering a new approach to interpretability which breaks down the actual weights, rather than the activations, of a model. The resulting components reveal the model’s computational structure and let us make targeted edits.
Intentionally designing the future of AI
Our essay on intentional design describes our vision for using interpretability to guide model training – moving from guess-and-check to closed loop control.
