How to use
- Copy the installation command and review the source package before installing it in your own agent environment.
- Check the source document for required tools, credentials and supported hosts.
- Describe your input and desired deliverable; review the output against the source material.
npx skills add https://github.com/aizech/clinical-skills --skill model-validationBefore you start
- Installation does not include a model subscription, API allowance or permission to access your files.
- Repository stars and installation counts describe activity, not an independent quality or security test.
Original skill document
Rendered from the source repository. Headings, examples, code, tables, links, and referenced images are preserved.
Model Validation Skill
Triggers
- "validate model performance"
- "external validation"
- "statistical analysis"
- "clinical validation"
- "model comparison"
- "regulatory submission"
- "performance benchmarking"
- "fairness audit"
Parameters
validation_type(required): Type of validation neededinternal- Retrospective internal datasetexternal- Prospective/out-of-distribution testingprospective- Clinical deployment studyregulatory- FDA/EMA submission prepfairness- Subgroup disparity analysiscomparison- Head-to-head model comparisonmodel_task(required): Model's intended usedetection- Sensitivity, specificity, PPV, NPVsegmentation- Dice, IoU, Hausdorff distanceclassification- Accuracy, AUC, F1 scoreregression- MAE, RMSE, correlationmodality(optional): Imaging modalityregulatory_path(optional): Target clearance pathway
Validation Framework
Performance Metrics
| Task | Primary Metrics | Secondary |
|---|---|---|
| Detection | Sensitivity, Specificity, AUC | PPV, NPV, FROC |
| Segmentation | Dice, IoU | Hausdorff, ASD |
| Classification | Accuracy, AUC, F1 | Sensitivity, Specificity |
| Regression | MAE, RMSE | Correlation, Bland-Altman |
Statistical Methods
- Confidence intervals (bootstrap, binominal)
- Significance testing (McNemar, DeLong for AUC)
- Power analysis for sample sizing
- Multiple comparison correction
- Subgroup interaction testing
Regulatory Standards
- FDA 510(k) predicate comparison
- FDA De Novo requirements
- EU MDR clinical evaluation
- IMDRF clinical evidence framework
- ACR-SIIM AI performance standards
Output Format
Returns structured JSON with:
- Validation protocol and methodology
- Required sample size with power analysis
- Statistical test selection and rationale
- Results template with standard metrics
- Interpretation guidelines
- Regulatory compliance checklist
Usage Examples
validation_type: external
model_task: detection
modality: CT
validation_type: regulatory
model_task: classification
regulatory_path: 510k