Originales Skill-Dokument
Aus dem Quell-Repository gerendert; Überschriften, Beispiele, Code, Tabellen, Links und Bilder bleiben erhalten.
LLM Risk Assessment (2025)
Comprehensive evaluation of LLM applications against OWASP Top 10 for LLM Applications 2025. Follow the detailed procedure in plays/llm-risk-assess.md.
Steps
- Architecture & Threat Modeling
- Map LLM provider (OpenAI, Anthropic, local models)
- Document data flows: user input → preprocessing → prompt construction → LLM → output processing → actions
- Identify RAG components, tool integrations, memory systems
- Define trust boundaries and attack surfaces
- Automated Security Testing
- Run prompt injection probes (Garak, Giskard, custom scripts)
- Test output handling vulnerabilities
- Scan for secrets in prompts and configurations
- Validate vector database security
- Assess All 10 OWASP LLM 2025 Risks with attack scenarios:
- LLM01 Prompt Injection — Direct/indirect injection, jailbreaks, goal hijacking, delimiter bypasses
- LLM02 Sensitive Information Disclosure — Training data leakage, PII exposure, system info extraction, memorized secrets
- LLM03 Supply Chain — Model poisoning, malicious dependencies, insecure plugins, provenance issues
- LLM04 Data and Model Poisoning — Training data poisoning, RAG poisoning, embedding manipulation
- LLM05 Improper Output Handling — XSS, command injection, SQLi, path traversal via LLM outputs
- LLM06 Excessive Agency — Unauthorized tool calls, permission escalation, dangerous action chains
- LLM07 System Prompt Leakage — Prompt extraction attacks, secret disclosure, instruction reverse engineering
- LLM08 Vector and Embedding Weaknesses — Adversarial embeddings, retrieval poisoning, similarity attacks
- LLM09 Misinformation — Hallucinations, authoritative presentation, grounding failures, harmful domains
- LLM10 Unbounded Consumption — Token exhaustion, cost attacks, resource exhaustion, DoS
- Red Team Testing
- Attempt real-world attack scenarios
- Test defense bypasses and evasion techniques
- Validate guardrails and safety controls
Output
Comprehensive LLM security report:
- Architecture diagram with trust boundaries
- Risk matrix (all 10 categories with severity/status)
- Detailed findings with proof-of-concept examples
- Red team test results and bypass techniques
- Remediation roadmap with code examples
- Defense validation checklist
OWASP References
- OWASP Top 10 for LLM Applications 2025
- OWASP AI Exchange (owaspai.org)
- OWASP AI Testing Guide
- OWASP Cheat Sheet: Prompt Injection Prevention
- OWASP Prompt Injection Taxonomy (Arcanum)
