qa-methodology

QA Methodology

Senior-to-principal QA and SDET methodology: test strategy, automation, regression, risk-based prioritization, exploratory testing, quality gates, AI code quality gates for agentic Spec-Driven Development, agentic eval design, career leveling, and SDET engineering.

Ownership

Core Principles

If it isn't tested, it's broken. Untested code is code whose failure mode hasn't been discovered yet.

Quality is a property of the process, not the artifact. Testing at the end doesn't create quality. Quality is designed in through strategy, automation, and gating throughout the cycle.

Test behavior, not implementation. Tests coupled to behavior survive refactoring; tests coupled to implementation break on it.

Risk drives priority. Not everything deserves equal test investment. Score probability × impact, then allocate accordingly.

Flaky tests are worse than no tests. A nondeterministic failure trains teams to ignore all failures. Quarantine on detection; rerun once, never twice.

Independent verification is non-negotiable. The implementing agent (or developer) must not self-verify. Separate session, fresh context, no shared priors.

Loading Guide

Scripts

Triggers

Load this skill when the task involves:

• Test strategy — designing what/how/priority to test for a project or feature
• Regression testing — building, selecting, or evolving regression suites
• CI triage — diagnosing CI failures, exit codes, flake-vs-real classification
• Test automation — framework selection, parallelism, flaky quarantine, ML selection
• Quality gates — gate design, blocking vs advisory, metrics, DORA
• Mutation-guided test hardening — diff-aware mutation scope, surviving mutants, weak assertions, and review evidence
• Risk-based testing — P×I scoring, risk registers, prioritization workshops
• Exploratory testing — SBTM charters, oracle heuristics, session debriefs
• Agentic evals — eval dataset design, judge bias, flaky-eval discipline, CI tiers
• SDD gate review — QA ownership at spec-driven gates, AC testability, independent verification
• SDET — test infrastructure engineering, gTAA, CI/CD integration, career scope

When not to use

Route to the named sibling skill instead:

• spec-driven-development — writing specs, running the SDD pipeline, gate verdict format, revision loops
• agent-evals-and-observability — eval framework governance, statistical comparisons, telemetry and privacy controls, grader implementation
• verification-methodology — collecting evidence and rendering verdicts against explicit pass/fail criteria
• release-engineering — composing test evidence into release-candidate readiness, promotion, go/no-go, production rollout, and rollback decisions; QA owns test strategy and gate semantics
• systematic-debugging — root-cause analysis of production incidents, bug reproduction, fault localization
• secure-software-engineering — security implementation, threat modeling, secure defaults, dependency evaluation
• playwright — operating the Playwright tool itself: authoring and running E2E specs, selector robustness, network mocking, headless scraping, and headed debugging

Stop and Exit Conditions

• Test strategy complete when: strategy document names risk tiers, level allocation, automation targets, and exit criteria for each tier.
• Risk assessment complete when: every identified risk has a P×I score, an owner, and a mitigation or acceptance decision recorded in the register.
• CI triage complete when: failure is classified (flake vs real, env vs code), root cause is localized, and a fix or escalation path is identified.
• Gate review complete when: every acceptance criterion maps to a verification method, the verifier is independent of the implementer, and evidence is attached.
• Bounded escalation: stop after three non-converging diagnostic passes and report the evidence collected so far.