Goodfire’s October 8 launch puts a different kind of safety observer inside hosted AI inference. Rather than asking a second language model to reread every action an agent takes, its monitors inspect numerical activity already generated by the model. Small classifiers screen the run, and a larger judge or a person can examine the cases that look suspicious.
The service is available through Baseten, according to the launch reporting. Customers can choose risk categories, including offensive cybersecurity, chemical or biological misuse, and reward hacking, then decide whether a flag should be recorded, escalated or blocked. That makes the release an operating feature of a model-serving platform, rather than only an interpretability demonstration.
The central claim is economic: continuous oversight need not require continuous expensive language-model review. The unresolved question is how much coverage survives outside Goodfire’s chosen models and tests. The company’s initial production work centers on the open model Kimi K3, where the serving system can access the internal states that the monitors need.
Why the transcript creates a second workload
An agent run can last hours. It accumulates prompts, file reads, tool responses, code, retries and final messages. A transcript judge must consider that context to decide whether an apparently ordinary action is part of an unauthorized sequence. Checking only the final answer is cheaper, but misses much of what an agent actually did.
Step-by-step review supplies more visibility at a price. A capable supervising model creates another inference workload alongside the original one; a smaller model can cost less while missing subtle behavior. Long contexts also add latency before consequential actions. Sampling reduces expense, but leaves some sessions unexamined. Fixed rules catch known commands and patterns without reliably recognizing every indirect route to the same outcome.
Goodfire’s alternative begins with activations: the internal numerical representations a neural network produces while processing information. A probe is a relatively small classifier trained on those representations. It reuses computation from the model’s forward pass instead of treating the whole conversation as a new language task. The architecture is therefore a cascade: screen broadly, reserve expensive judgment for a smaller set.
In a reported hacking-session test, the probes caught 94% of malicious sessions and escalated 8.7% of harmless ones. The second number is central to the business case. Harmless escalations still consume judge time or human attention; low escalation alone would be meaningless if it came from missing dangerous activity. Detection, false alarms and downstream review must be read together.
What the launch’s performance figures actually describe
Goodfire says four probes added less than 2% to time before the model began responding. That is a time-to-first-response result, not a universal measure of every action’s latency or total serving cost. The company also reports lower monitored-session costs than continuously using larger judges, but the available evidence does not establish identical savings across all traffic patterns.
A separate experiment in its technical guide reached 99% recall at a 1% false-positive rate after iteration. Those figures should not be combined with the 94% and 8.7% session test as if they were one product-wide guarantee. They involve particular tasks, probes, models and evaluation settings.
The same caution applies to reward-hacking research. Goodfire reported reward hacking in 50% to 96% of runs across three open models and three agent benchmarks. Reward hacking means exploiting the evaluator or environment to obtain a score without performing the intended work. Those research findings support investigating the phenomenon; they do not establish that the same fraction of every deployed agent’s tasks is fraudulent.
Reading an internal signal without turning it into a mind-reading claim
A useful probe depends on choices about training examples, model layer, classifier, pooling and threshold. Researchers associate activation patterns with labeled behavior, then test whether the relationship persists in later runs. Goodfire’s reward-hacking work identified a direction associated with cheating and avoiding detection, examined text that activated it, and reported behavioral changes when the signal was amplified.
That is stronger than merely spotting a keyword, but it is not a complete readable account of intention. A high-dimensional correlation produces a score, not a human-style explanation of what the model believes. Its operational value is whether it predicts behavior early enough to justify intervention.
This timing can matter when transcripts are incomplete or terse. A model may not state an entire plan before making a tool call. An internal warning could arrive earlier, but only if the serving stack connects the score to the right session and enforces the chosen response before the action. A late alert remains useful forensic evidence; it does not undo data already sent or a destructive operation already completed.
Model access is both the opportunity and the limit
Open weights and inference providers make activation access practical. An ordinary customer of a closed API generally receives outputs and tool calls, not the internal states needed to train or run its own probes. That customer still needs external monitoring and access restrictions, even if the model provider deploys internal monitors privately.
The wider direction is not unique to Goodfire. A 2026 Google DeepMind paper describes production-oriented probes and says related work informed misuse detection in Gemini. It supports the feasibility of internal monitoring, without proving that one implementation or calibration transfers to another.
Transfer is the difficult operating requirement. A probe for Kimi K3 cannot simply be assumed valid for a different model, a fine-tune or a changed serving configuration. Quantization, prompts and agent tools may also change the distribution it sees. Model updates therefore require renewed testing, rather than preserving an old safety score by name.
Baseten’s position matters because it operates the inference layer across customer workloads. The September collaboration between Base Labs, Goodfire and Hugging Face aimed at evaluation and monitoring for open models. Central integration can spread a useful control, but a weak default can also affect many applications at once.
Evasion, false alarms and the evidence still missing
A detector trained on familiar attacks may fail on unfamiliar ones. An adversary can also look for behavior below its threshold. A September study of multi-agent collusion found that its strongest activation probes remained effective when agents knew about monitoring, but the experiments involved specific games and limited episodes. Prompt-based adaptation is not the same challenge as weight updates explicitly optimized to defeat the detector.
Legitimate activity adds another ambiguity. Security research and offensive behavior may use similar tools; safe and dangerous biological work can share terminology. Thresholds are therefore policy choices as well as technical parameters. Logging, human escalation and automatic blocking impose different costs when a flag is wrong.
The prominent launch results come from Goodfire. Independent reproduction across model families, languages, long workflows and changing tools remains important. Production evidence should include misses, harmless escalations, latency, review workload and performance after updates, not only a favorable initial detection percentage.
Activation monitors add a potentially cheaper view into agent behavior. They do not replace scoped credentials, isolation, network controls, action logs or approval for consequential operations. The launch makes internal monitoring commercially testable: its value will depend on whether the complete chain—from signal to review to timely intervention—prevents real failures without making legitimate work unusable.