Come usare
- Copia il comando e controlla il pacchetto prima di installarlo nel tuo ambiente agente.
- Controlla strumenti, credenziali e ambienti richiesti nel documento originale.
- Descrivi input e risultato desiderato; confronta l’output con le fonti.
npx skills add https://github.com/aizech/clinical-skills --skill model-validationPrima di iniziare
- L’installazione non include abbonamenti, quote API o accesso ai file.
- Stelle e installazioni indicano attività, non un test indipendente di qualità o sicurezza.
Documento Skill originale
Contenuto dal repository con titoli, esempi, codice, tabelle, link e immagini preservati.
Model Validation Skill
Triggers
- "validate model performance"
- "external validation"
- "statistical analysis"
- "clinical validation"
- "model comparison"
- "regulatory submission"
- "performance benchmarking"
- "fairness audit"
Parameters
validation_type(required): Type of validation neededinternal- Retrospective internal datasetexternal- Prospective/out-of-distribution testingprospective- Clinical deployment studyregulatory- FDA/EMA submission prepfairness- Subgroup disparity analysiscomparison- Head-to-head model comparisonmodel_task(required): Model's intended usedetection- Sensitivity, specificity, PPV, NPVsegmentation- Dice, IoU, Hausdorff distanceclassification- Accuracy, AUC, F1 scoreregression- MAE, RMSE, correlationmodality(optional): Imaging modalityregulatory_path(optional): Target clearance pathway
Validation Framework
Performance Metrics
| Task | Primary Metrics | Secondary |
|---|---|---|
| Detection | Sensitivity, Specificity, AUC | PPV, NPV, FROC |
| Segmentation | Dice, IoU | Hausdorff, ASD |
| Classification | Accuracy, AUC, F1 | Sensitivity, Specificity |
| Regression | MAE, RMSE | Correlation, Bland-Altman |
Statistical Methods
- Confidence intervals (bootstrap, binominal)
- Significance testing (McNemar, DeLong for AUC)
- Power analysis for sample sizing
- Multiple comparison correction
- Subgroup interaction testing
Regulatory Standards
- FDA 510(k) predicate comparison
- FDA De Novo requirements
- EU MDR clinical evaluation
- IMDRF clinical evidence framework
- ACR-SIIM AI performance standards
Output Format
Returns structured JSON with:
- Validation protocol and methodology
- Required sample size with power analysis
- Statistical test selection and rationale
- Results template with standard metrics
- Interpretation guidelines
- Regulatory compliance checklist
Usage Examples
validation_type: external
model_task: detection
modality: CT
validation_type: regulatory
model_task: classification
regulatory_path: 510k