Orca Security reports that Discovery AI cut input tokens by 93%, query cost by 90% and latency by 75% when it replaced a giant prompt with a nine-stage Amazon Bedrock pipeline. The useful detail is what stopped going into the model: most of a security schema that most questions did not need.
Discovery AI translates natural-language questions into Sonar filters. Sonar spans more than 900 infrastructure and application models, including resources, identities, vulnerabilities and their relationships. Analysts previously had to write precise JSON, know the model names and navigate 18 filtering operators. The interface reduces that language barrier without removing the need for a correct, authorized investigation.
The production case describes thousands of daily queries since March 2026. It is an AWS-and-Orca account of the company’s own system, not an independent comparison with other security vendors.
The prototype’s success exposed its production problem
The initial design put over 900 definitions, relationship types and examples into a large model’s prompt. Design partners and internal users liked the results, Orca says. But a comparable workload consumed about two million input tokens and took around 60 seconds. Multi-model, nested or multi-condition questions still needed refinement.
The redesigned flow begins by attaching organizational context, including connected cloud providers and request metadata. A deterministic matcher then recognizes structured identifiers: IP addresses, CVEs, hashes, names, URLs and alert IDs. Known patterns can return predefined filters without asking a language model to reason through an obvious mapping.
Other requests face guardrail screening for prompt injection, off-topic questions and unsupported inputs. The sequence separates a fast exact path from requests that need interpretation. It also creates identifiable failure points rather than hiding every responsibility in a single instruction.
That is the architectural change behind the headline. More stages can mean less generative work when each stage narrows what remains uncertain.
Retrieval and selection shrink the material that needs reasoning
Amazon Titan embeddings retrieve 20 semantically similar examples from a validated library. Cohere Rerank 3.5 reduces them to five. Orca says simple top-K retrieval often returned near-duplicates, while Maximal Marginal Relevance introduced diversity but could displace the most relevant example. Retrieval followed by reranking became its compromise.
An exact or high-confidence example can supply a known-good filter directly. Otherwise, examples guide generation. The confidence threshold matters: two questions can look similar while requiring different conditions.
A model-selection step reads compact descriptions of the available Sonar models—their purpose, fields and relationships—and selects those relevant to the question. Only the chosen full schemas and needed relationships reach generation. Anthropic Claude Haiku 4.5 handles selection and generation; Amazon Nova 2 Lite serves guardrail and validation tasks where Orca prioritizes lower-cost, faster inference.
The resulting filter is checked statically against model, field and operator definitions, then against the Sonar API. Structured errors identify the offending element for correction. Retries are bounded; the diagram’s final stage post-processes the response and constructs a Discovery URL for the generated filter rather than treating raw text as the product.
The reported input workload falls from about two million tokens to 140,000, the stated 93% reduction. Cost falls 90%, and approximately 60 seconds becomes 15 seconds, or 75% less latency. These figures compare Orca’s pipeline with its own original design. They are not a promise that any application will obtain the same savings.
A valid filter is not necessarily the right investigation
Schema checks reject invented fields or illegal operators. They do not prove that the filter captures every condition the analyst intended. A query can be valid while dropping an exception, a time boundary or a relationship.
“Show exposed databases” illustrates the ambiguity. Exposure can mean different network paths, resource types and accepted-risk definitions. A review interface should make assumptions or the generated filter visible. The case study explains generation and validation more fully than the analyst’s complete review experience.
Security data also includes internal asset names, vulnerabilities, configurations and account relationships. A prompt-attack classifier is not an authorization service. A system must limit which data reaches the model and run the final query under the requester’s appropriate permissions, not the assistant’s broadest backend access.
The linked AWS guidance recommends identity propagation and access enforcement outside the model. This avoids a confused deputy: a less-privileged person receiving information through a more-privileged intermediary. The public Orca account describes organizational context but does not detail every tenant-isolation and user-permission mechanism.
That absence is not evidence of a vulnerability. It is a limit on what the performance case can establish. Validation, intent and access are three separate tests, and efficiency percentages do not substitute for any of them.
The improvement loop relies on people and governed examples
Orca evaluates correctness, completeness and semantic equivalence against reference queries, with slices by question and model category. LLM-as-judge scoring supports iteration, while product managers, support engineers, researchers and customers flag errors.
Low-confidence production questions enter a daily labeling queue. Experts’ corrections can be retrieved in later runs without a code deployment. Orca’s AI director Or Gabay identifies this labeling interface as its most important long-term quality investment—a human feedback process, not just a better model.
That loop can improve coverage as analysts ask unfamiliar questions and cloud threats change. The linked MITRE cloud matrix supplies a framework for such investigations, but a static example bank would still decay as resources and attack paths evolve.
Corrections need consistent policy and review. An example library needs versioning, removal procedures, quality checks and safeguards against sensitive production information entering retrieval. Replacing a stage model through Bedrock may simplify integration, but changes still require regression tests across selection, retrieval, generation and validation.
Strong efficiency figures, incomplete public accuracy evidence
The case provides specific token, cost and time comparisons but not an evaluation dataset, sample size, pass rate or exact accuracy improvement. It says complex queries improved, without publishing enough detail to independently reproduce that claim.
The judge’s rubric and references also affect results. Human review agreement, retry-limit frequency, exact-match and example-reuse shares would help explain the pipeline’s behavior. Those rates are not disclosed.
Nor does the 15-second figure distinguish median from tail latency. “Comparable workload” depends on request mix, caching, output size, model choices and correction attempts. Thousands of daily requests show use but do not reveal unique analysts, accepted filters, abandoned questions, manual edits or time actually saved.
The broader lesson is therefore narrower than “nine stages beat one model.” Orca selected context, assigned models by task, reused verified patterns and made enforceable checks deterministic. Related AWS Security Lake work also translates investigation questions into structured queries, but Orca’s distinctive emphasis is reducing the large domain schema before generation.
This compiler-like division can make cost and failure causes more visible. It adds operational dependencies and evaluation work rather than making them disappear. Whether it remains useful depends on sustained accepted-query quality as schemas, users and model versions change.
Discovery AI is a substantial reported improvement over Orca’s own prototype. The next evidence needed is accuracy by difficult query category, sustained analyst adoption and regression results after changes. Those would connect cheaper inference to the real objective: faster security decisions that are valid, faithful to intent and permitted for the person asking.