huaweicloud/huaweicloud-skills

huawei-cloud-msmodelslim-model-adapt

|- Create basic Transformers model adapters for msModelSlim.

查看源码

使用方法

  1. 复制安装命令,在自己的智能体环境安装前先检查源码包。
  2. 从原始文档核对所需工具、凭据和支持的宿主。
  3. 说明输入材料与期望交付物,并对照来源检查输出。
npx skills add https://github.com/huaweicloud/huaweicloud-skills --skill huawei-cloud-msmodelslim-model-adapt

开始前需了解

  • 安装不包含模型订阅、API 额度或访问文件的权限。
  • 仓库星数和安装量反映活跃情况,不等于独立质量或安全测试。
仓库原始内容

按源仓库内容呈现,保留标题、案例、代码、表格、链接以及原文引用的演示图片。

Huawei Cloud msModelSlim Model Adapter

Overview

This skill guides how to create basic adapters for new models to run W8A8/W4A16 quantization workflows in msModelSlim.

Architecture: Model Analysis -> Adapter Creation -> Registration -> Verification (4 Steps)

Related Skills:

  • huawei-cloud-msmodelslim-model-analysis - Model structure analysis

before adapter implementation

  • huawei-cloud-ascend-profiler-db-explorer - Optional: Performance

analysis after deployment

Scope

Supported:

  • Decoder-only LLM
  • Understanding VLM (text/LLM backbone only)

Not supported:

  • Multimodal generation (Stable Diffusion/Flux/Wan)
  • Encoder-only models
  • Non-Transformers architectures

Architecture

text
┌─────────────────────────────────────────────────────────────┐
│              msModelSlim Model Adapter Skill                │
├─────────────────────────────────────────────────────────────┤
│  ┌──────────────────┐    ┌──────────────────────────────┐  │
│  │  Model Analysis │───▶│    Adapter Creation          │  │
│  │  - config.json  │    │    - LLM Adapter Template     │  │
│  │  - modeling_*.py│    │    - VLM Adapter Template     │  │
│  └──────────────────┘    │    - Required Interfaces     │  │
│                          └──────────────────────────────┘  │
│                                    │                       │
│                                    ▼                       │
│                          ┌──────────────────┐             │
│                          │  Registration    │             │
│                          │  & Installation  │             │
│                          └──────────────────┘             │
│                                    │                       │
│                                    ▼                       │
│  ┌──────────────────────────────────────────────────────┐ │
│  │                   Verification (4 Steps)              │ │
│  │  1. Generate Test Model → 2. Full Fallback Quant     │ │
│  │  3. Weight Verification → 4. Quant Description     │ │
│  └──────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘

Architecture Components

This skill involves the following cloud services and components:

  • msModelSlim: Huawei Cloud's model quantization framework for

efficient model compression

  • Transformers Library: Hugging Face Transformers for model loading

and processing

  • ModelScope: Model download and management platform
  • Ascend NPU: Target hardware for quantized model deployment

Use Cases

Typical Problem Scenarios:

  • Need to deploy LLM models with reduced memory footprint on Ascend NPU
  • Want to optimize inference speed without significant accuracy loss
  • Migrating models that don't have built-in msModelSlim support
  • Need W8A8/W4A16 quantization for decoder-only LLM or VLM text backbones

Typical User Phrases:

  • "How to quantize my custom LLM model for Ascend?"
  • "Create msModelSlim adapter for Qwen model"
  • "Implement W4A16 quantization workflow"
  • "Adapt my VLM text backbone for quantization"
  • "How to add quantization support for new models?"

Core Workflow

1. Preparation

  • Download Model: Recommended to use modelscope download for

non-weight files.

  • Example: `modelscope download --model <org>/<model> --local_dir

./models/<name> --exclude '*.safetensors'`

  • Analyze Model: Read config.json and modeling_*.py to confirm

structure and implementation.

2. Create Adapter

  • Use Templates:
  • LLM: assets/model_adapter_template.py
  • VLM: assets/vlm_model_adapter_template.py
  • Implement Interfaces: Implement handle_dataset, init_model,

generate_model_visit, generate_model_forward, enable_kv_cache.

  • Key Principles:
  • visit and forward must be strictly consistent.
  • MoE models recommended to unpack to pure linear layers.
  • See: Implementation Guide

3. Registration & Installation

  • Register model and entry in config/config.ini, then execute

bash install.sh.

4. Verify Adapter (Required)

  • Must execute four-step verification: Generate test model -> Full

fallback quantization -> Verify full fallback model matches float weights exactly and can load/save completely -> Verify actual quantization workflow works (including description file rule validation).

Common Scripts

Scripts located in scripts/ directory:

  • scripts/step1_generate_test_model.py
  • scripts/step2_run_quantization.py
  • scripts/step3_verify_weights.py
  • scripts/step4_verify_quant_description.py

Prerequisites

System Requirements

  • Python 3.8+
  • transformers >= 4.40.0
  • msmodelslim >= 1.0.0

Environment Check

Prerequisite check: Python3 + transformers + msmodelslim required ``bash python3 --version # Python3 >= 3.8 python3 -c "import transformers; print('OK')" # Transformers library python3 -c "import msmodelslim; print('OK')" # msModelSlim library If not installed: pip3 install --user transformers msmodelslim`

Reference Documents

DocumentDescription
Model Analysis GuideModel structure analysis guide
Implementation GuideAdapter implementation instructions
Registration GuideRegistration and installation guide
Verification GuideFour-step verification workflow
Interface ChecklistRequired interface implementation checklist
Core WorkflowCore workflow documentation
Acceptance CriteriaFunctional acceptance criteria
TroubleshootingCommon issues and solutions

Requirements

  • transformers >= 4.40.0 installed
  • msmodelslim >= 1.0.0 installed
  • Transformers model to be adapted
  • Understanding of target quantization scheme (W8A8/W4A16)

Core Commands

bash
# Create model adapter
python3 scripts/create_adapter.py \
  --model Qwen2-7B \
  --quantization W8A8

# Run four-step verification
python3 scripts/verify_adapter.py --adapter ./adapter.py

Parameter Confirmation

ParameterDescriptionRequired
modelModel name or pathYes
quantizationQuantization scheme (W8A8/W4A16)Yes
outputAdapter output pathNo

按任务相关推荐

来自同一仓库

更多 Skills

全部 Skills
huaweicloud
社区

huawei-cloud-devbridge-tunnel

Create and manage DevBridge development tunnels on Huawei Cloud to securely expose local development services to remote devices. Based on DevBridge CLI v0.2.x and API Key authentication. Use this skill when the user wants to: (1) install and configure the DevBridge CLI, (2) create/list/update/delete tunnels, (3) manage tunnel ports and protocols, (4) host local services through tunnels, (5) connect to remote tunnels from another device, (6) manage authentication and API Keys. Trigger: user menti

1小时安装量
3
GitHub Stars
59
最近更新
11月20日
huaweicloud
社区

huawei-cloud-find-skills

Invoke this skill to search, list, query, discover, browse, find and install any Huawei Cloud (华为云) agent skill. Triggers include: "华为云", "华为云有什么skill", "华为云相关skill", "华为云agent skill 市场", "华为云skill类目", "explore Huawei Cloud skills", "show Huawei Cloud skill categories", "does a Huawei Cloud skill exist for...", "which Huawei Cloud skills exist", "搜索华为云技能", "查询华为云skill列表", "有没有管理ECS/OBS/RDS的skill", "帮我找 XX 华为云skill", "介绍 XX Skill 内容", "华为云 XX Skill 具体做什么", "安装华为云Skill".

1小时安装量
3
GitHub Stars
59
最近更新
11月20日
huaweicloud
社区

huawei-cloud-terraform-installer

Installs Terraform CLI cross-platform with Huawei Cloud mirror support. Use this skill whenever the user mentions installing Terraform, setting up Terraform environment, or when IaC tasks detect missing Terraform. Trigger: install terraform, setup terraform, terraform environment, huaweicloud provider.

1小时安装量
6
GitHub Stars
59
最近更新
11月7日
huaweicloud
社区

huawei-cloud-billing-scout

Huawei Cloud BSS billing (not AWS/Azure/other clouds; refuses pricing quotes, real-name review, and any non-billing scope): balance, spend, attribution, reconciliation, coupons, stored-value cards, enterprise/partner billing. One-page briefing via hcloud. Use when the request is about Huawei Cloud billing. Triggers include: 华为云余额, 华为云欠费, 华为云本月花费, 华为云扣费原因, 华为云账单对账, 华为云资源包, 华为云优惠券, 华为云企业主账号, 华为云伙伴账务, 华为云服务编码, Huawei Cloud balance / bill / spend / reconciliation / coupon / stored-value card / BSS b

1小时安装量
5
GitHub Stars
59
最近更新
11月7日