使用方法
- 复制安装命令,在自己的智能体环境安装前先检查源码包。
- 从原始文档核对所需工具、凭据和支持的宿主。
- 说明输入材料与期望交付物,并对照来源检查输出。
npx skills add https://github.com/aizech/clinical-skills --skill model-validation开始前需了解
- 安装不包含模型订阅、API 额度或访问文件的权限。
- 仓库星数和安装量反映活跃情况,不等于独立质量或安全测试。
仓库原始内容
按源仓库内容呈现,保留标题、案例、代码、表格、链接以及原文引用的演示图片。
Model Validation Skill
Triggers
- "validate model performance"
- "external validation"
- "statistical analysis"
- "clinical validation"
- "model comparison"
- "regulatory submission"
- "performance benchmarking"
- "fairness audit"
Parameters
validation_type(required): Type of validation neededinternal- Retrospective internal datasetexternal- Prospective/out-of-distribution testingprospective- Clinical deployment studyregulatory- FDA/EMA submission prepfairness- Subgroup disparity analysiscomparison- Head-to-head model comparisonmodel_task(required): Model's intended usedetection- Sensitivity, specificity, PPV, NPVsegmentation- Dice, IoU, Hausdorff distanceclassification- Accuracy, AUC, F1 scoreregression- MAE, RMSE, correlationmodality(optional): Imaging modalityregulatory_path(optional): Target clearance pathway
Validation Framework
Performance Metrics
| Task | Primary Metrics | Secondary |
|---|---|---|
| Detection | Sensitivity, Specificity, AUC | PPV, NPV, FROC |
| Segmentation | Dice, IoU | Hausdorff, ASD |
| Classification | Accuracy, AUC, F1 | Sensitivity, Specificity |
| Regression | MAE, RMSE | Correlation, Bland-Altman |
Statistical Methods
- Confidence intervals (bootstrap, binominal)
- Significance testing (McNemar, DeLong for AUC)
- Power analysis for sample sizing
- Multiple comparison correction
- Subgroup interaction testing
Regulatory Standards
- FDA 510(k) predicate comparison
- FDA De Novo requirements
- EU MDR clinical evaluation
- IMDRF clinical evidence framework
- ACR-SIIM AI performance standards
Output Format
Returns structured JSON with:
- Validation protocol and methodology
- Required sample size with power analysis
- Statistical test selection and rationale
- Results template with standard metrics
- Interpretation guidelines
- Regulatory compliance checklist
Usage Examples
validation_type: external
model_task: detection
modality: CT
validation_type: regulatory
model_task: classification
regulatory_path: 510k