llm-optimization

LLM Token Optimization Skill

This skill strictly governs how you (the AI agent) MUST write code and prompt architectures that involve LLM interactions. If a user asks you to implement an AI feature, you are REQUIRED to follow these directives.

MANDATORY DIRECTIVES

1. PROMPT COMPRESSION (MANDATORY)

You MUST NEVER use conversational language or pleasantries in prompts ("Please", "I want you to", "Can you"). You MUST structure all instructions using YAML, Markdown lists, and declarative commands. Positive constraints ("Output ONLY JSON") are strictly preferred over negative constraints ("Do not output markdown").

❌ Bad Prompt Example (Do not use): "You are a helpful assistant. Please read the following text and I would like you to extract the names of the people. Also, summarize the text in one sentence. Make sure you return it in JSON format and don't write anything before or after the JSON."

✅ Optimized Prompt Example (Use this):

Task: Extract entities and summarize text.
Rules:
- Output valid JSON only.
- Schema: {"people": [str], "summary": str}
Input: <text>

2. STRICT OUTPUT FORMATTING (MANDATORY)

You MUST enforce the exact structure via API native parameters. Never rely solely on plaintext prompt instructions to get JSON.

OpenAI Implementation Snippet:

response = client.chat.completions.create(
    model="gpt-4o-mini",
    response_format={ "type": "json_schema", "json_schema": {...} }, # Mandatory for JSON
    messages=[{"role": "system", "content": "..."}]
)

Anthropic Implementation Snippet:

# Anthropic uses Tool Calling to force strict extraction schemas
response = client.messages.create(
    model="claude-3-5-haiku-20241022",
    tools=[{"name": "extract_data", "input_schema": {...}}],
    tool_choice={"type": "tool", "name": "extract_data"},
    messages=[...]
)

Gemini Implementation Snippet:

from google.genai import types
response = client.models.generate_content(
    model='gemini-2.5-flash',
    contents=...,
    config=types.GenerateContentConfig(
        response_mime_type="application/json",
        response_schema=MyPydanticModel
    )
)

3. CONTEXT PRUNING & SANITIZATION (CRITICAL RULE)

• Never pass raw HTML/XML, unprocessed logs, or massive payloads to the model uncleaned. You MUST write pre-processing code to strip tags, metadata, and whitespace.
• Stop-Word Removal: For datasets where grammar is irrelevant (e.g., massive log extraction), consider stripping stop-words ("the", "is", "a") prior to inference.
• RAG / Chunking: Instead of sending an entire document, you MUST use semantic search or targeted chunking to retrieve only the relevant paragraphs.

4. TOKEN BUDGETING & MODEL DOWNGRADE

Before proposing an LLM architecture to a user, you MUST estimate the token cost per call.

• Never default to flagship models (e.g., GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro) for simple extraction, classification, or formatting tasks.
• You MUST route highly structured tasks to GPT-4o-mini, Claude 3.5 Haiku, or Gemini 1.5 Flash.

5. PROMPT CACHING REQUIREMENTS

When building repeatable LLM pipelines:

• Place all static structural instructions, system prompts, schemas, and few-shot examples at the VERY BEGINNING of the prompt string array.
• Place dynamic user data (the variable context) at the VERY END.
• This specific ordering is required to maximize cache hit rates on Anthropic and Google systems, reducing costs by up to 90%.

6. TESTING / VALIDATION

Whenever you write an optimized prompt, you MUST accompany the code with a unit test or validation check. This proves that despite using compressed context and a downgraded/cheaper model, the result remains perfectly accurate.