Jak używać
- Skopiuj polecenie instalacji i sprawdź źródła przed instalacją we własnym środowisku agenta.
- W oryginale sprawdź wymagane narzędzia, dane dostępowe i obsługiwane środowiska.
- Opisz dane wejściowe i oczekiwany rezultat; porównaj wynik ze źródłami.
npx skills add https://github.com/runpod/runpod-plugins-official --skill runpod-flashPrzed rozpoczęciem
- Instalacja nie obejmuje subskrypcji modelu, limitu API ani uprawnień do plików.
- Gwiazdki i instalacje pokazują aktywność, a nie niezależny test jakości lub bezpieczeństwa.
Treść z repozytorium z zachowaniem nagłówków, przykładów, kodu, tabel, linków i obrazów.
Runpod SDK apps (Flash)
For an explicit Dockerfile/template project request, use runpod-build-template. Do not replace that deliverable with Flash unless the user chooses its code-first workflow.
Write Python locally, run it on remote Runpod GPUs/CPUs with rp flash dev, then rp flash deploy to ship. Flash is part of the runpod package (2.0): one SDK, one rp CLI, one login. The standalone runpod-flash package is deprecated and gets no fixes after October 29, 2026.
`rp --help` and `rp flash <command> --help` are authoritative for the command surface. This skill is the mental model, the decision rules, and the gotchas that help output does not carry. Confirm the installed version with pip show runpod before concluding a command or parameter is unavailable.
Which SDK is this project on?
Check before writing code. The two APIs do not mix.
| You see | Project is on | Do |
|---|---|---|
import runpod, runpod.App(...), @app.queue, rp flash ... | Runpod SDK 2.0 | This file + reference/ |
from runpod_flash import Endpoint, @Endpoint(...), flash dev, runpod-flash in deps | Legacy runpod-flash | reference/runpod-flash-legacy.md to maintain it, reference/migrate-from-runpod-flash.md to move it |
| A new project | Runpod SDK 2.0 | Start below |
Write new code on the SDK. For an existing runpod-flash project, propose migrating and check its Known differences first; don't migrate silently.
If the SDK path fails on an SDK bug, `runpod-flash` is the fallback: a traceback inside runpod/apps/, a worker that crashes before your function runs, or the same code working on runpod-flash. Upgrade runpod first; the steps to check for a fix and fall back are in reference/runpod-flash-legacy.md.
Worked examples first for anything multi-step. Flash appears in end-to-end paths: 03 variant B (whisper endpoint), 07 (network-volume handoff) and 08 (fine-tune → serve); the full index is runpod/golden-paths/README.md. Runnable single-file apps live in `examples/`.
Load on demand:
| Need | Read |
|---|---|
Install, rp login, rp flash init, every rp command | reference/setup-and-cli.md |
Decorator parameters, call styles, Queue/Api/Endpoint clients, volumes, Model, Secret, GPU & CPU names | reference/api.md |
| Worked patterns: choosing a model, warm model loading, CPU→GPU pipeline, fan-out, tasks, raw HTTP | reference/patterns.md |
Moving a runpod-flash project to the SDK | reference/migrate-from-runpod-flash.md |
Maintaining a runpod-flash project as it is | reference/runpod-flash-legacy.md |
Quick start: uv add "runpod>=2.0.0" (or pip install "runpod>=2.0.0") → rp login (or export RUNPOD_API_KEY=...) → rp flash init my-app → cd my-app && rp flash dev main.py.
The app model
import runpod
app = runpod.App("my-app") # names the group of resources
@app.queue(gpu="4090", dependencies=["torch"]) # queue-based serverless endpoint
def infer(x: list[float]):
import torch
return {"sum": torch.tensor(x, device="cuda").sum().item()}
@runpod.local_entrypoint # runs on YOUR machine
def main():
print(infer.remote([1.0, 2.0, 3.0])) # runs on a Runpod worker| Decorator | Creates | Use for |
|---|---|---|
@app.queue(...) on a function | Autoscaling queue-based endpoint | Request/response inference, batch jobs |
@app.api(...) on a class | Load-balanced endpoint serving routes (@runpod.get/@runpod.post, setup in @runpod.init) | HTTP services, multi-route APIs, load-once model state |
@app.task(...) on a function | One ephemeral Pod per call, terminated when it returns | Training, fine-tuning, long jobs |
Call styles on a queue or task handle: fn.remote(...) (blocks), await fn.remote.aio(...), fn.spawn(...) (returns a Job, or a TaskJob for a task), fn.stream(...) (queue generator functions only), fn.local(...) (runs on your machine). API classes are called by route: MyApi.post("/path", body).
Dev vs deploy
rp flash dev main.py— iterate. Creates temporary endpoints (dev-<app>-<resource>-<hash>),
runs the @runpod.local_entrypoint on your machine, streams the workers' print output to your terminal, and re-runs after every file save. Ctrl-C deletes the endpoints. There is no local HTTP server; the entrypoint is how you exercise the code.
rp flash dev main.py --once— run the entrypoint once, tear down, exit non-zero on failure.
The smoke test for CI and for "does the migrated code work".
rp flash deploy— ship. Builds an artifact (≤1,500 MB, torch excluded on built-in GPU
images) and creates persistent endpoints in the default environment (--env NAME for others). Re-run to update. --build-only packages without deploying, a cheap offline check that discovery and dependency resolution work.
Dev and deploy run the same module source, so module-level imports, constants and helpers behave identically in both. Deploy packages the project directory, but a dev session ships only the module's source, so keep the app in one file when you use rp flash dev.
Autonomous dev loop
A dev session is long-running. Three rules:
- Run it in the background with stdin closed (
< /dev/null). With no stdin it re-runs
the entrypoint on every file save, so you never need to press Enter.
- Capture its output to a log file and read it. Worker
prints, tracebacks,✗ remote execution failed: ...and the per-run
✓ done in ... / ✗ failed in ... lines land there.
- Stop it with `kill` (SIGTERM) or SIGINT, so it deletes its endpoints, and confirm the
log ends with cleaning up N deleted. A second signal stops waiting for a running entrypoint and goes straight to cleanup. kill -9 skips cleanup. When one run is enough, prefer --once; it cleans up on its own.
rp flash dev main.py < /dev/null > /tmp/rp-dev.log 2>&1 &
DEV=$!
# poll the log (bounded) for the run result: "✓ done in ..." or "✗ failed in ..."
for i in $(seq 1 120); do grep -qE "✓ done in|✗ failed in" /tmp/rp-dev.log && break; sleep 3; done
tail -40 /tmp/rp-dev.log
# edit main.py and save -> the session re-scans, refreshes endpoints, re-runs main()
# (each run appends a new result line; read the last one)
kill $DEV; wait $DEV; tail -3 /tmp/rp-dev.log # expect "cleaning up N deleted"For a single check, rp flash dev main.py --once is simpler and cleans up on its own. If a session died without cleanup, list leftovers with runpodctl serverless list and delete the dev-<app>-... endpoints this session created, by ID.
Choosing the resource shape
Default to plain Python on the Runpod runtime image: dependencies=[...] and system_dependencies=[...] install on the worker, so no Dockerfile is needed, and even large HuggingFace models stay in code (see reference/patterns.md).
- `image="org/image:tag"` on any decorator runs your function on that image; it needs
Python 3, not a Serverless handler. Use it for system libraries or a specific CUDA stack. registry_auth= names saved credentials for a private image (rp registry add).
- A prebuilt inference server image (vLLM, worker-comfyui) that already implements a
Serverless handler is not an SDK app. Deploy it with runpodctl or runpod-mcp and call it with runpod.Endpoint("ENDPOINT_ID").
- An existing endpoint by ID:
runpod.Endpoint("ENDPOINT_ID").run_sync({...}). - A resource in another deployed app:
runpod.Queue(app="media", name="transcribe")/
runpod.Api(app="media", name="inference").
Gotchas
- Queue arguments and results must be JSON-serializable. Queue jobs are plain JSON in dev
and deploy alike. A non-JSON argument fails in dev with arguments must be json-serializable and against a deployed endpoint with Python's own JSON TypeError. Pass URLs or base64, not objects. Whole-number floats can arrive as integers (1.0 → 1), so cast numeric inputs explicitly (torch.tensor(x, dtype=torch.float32)).
- One file per app in dev. A dev session ships only the module's source, so a sibling
local module imports in deploy but fails in dev. For dev sessions, keep helpers in the same file or in an installable package.
- Keep blocking work out of module scope. The SDK imports your module to discover
resources, locally and on every worker. Put heavy imports (torch) and work inside functions; guard local-only code with runpod.is_local().
- `gpu=` and `cpu=` are exclusive (
gpu and cpu are mutually exclusive). GPU strings can
be short ("4090", "H100") or a list of acceptable GPUs.
- Default `workers=(0, 3)` (runpod-flash was
(0, 1)). Set it explicitly when cost or
concurrency matters. idle_timeout is seconds (default 60).
- Auto GPU switching needs `workers` max ≥ 5 when you pass a list of GPUs.
- 10 MB payload limit on requests. Return large outputs via a volume or object storage URL.
- Volumes outlive everything. Dev sessions and
rp flash undeploydelete endpoints, not
the NetworkVolumes they created, and those keep billing. Queue and API resources mount one volume at /runpod-volume; read vol.path inside the remote function, never at import time. Load model weights through VolumeCache, not straight off the mount (reference/patterns.md).
- Tasks bill until they return. A
.spawn()ed task that you neverwait()orcancel()
keeps its Pod running after your script exits.
- Native CUDA libraries go in `dependencies` too, e.g. faster-whisper needs
nvidia-cublas-cu12 + nvidia-cudnn-cu12 or it silently falls back to CPU.
- Dev sessions and tasks need local Python 3.10–3.14. They ship code pickled by your
interpreter, so the worker matches your local version. Deploy falls back to 3.12 with a warning on other versions; rp flash deploy --python-version sets it explicitly.
- A set `RUNPOD_API_KEY` wins over `rp login`. A stale exported key causes 401s while the
saved login is fine. Check with runpodctl user, then unset RUNPOD_API_KEY or export a good key.
- The torch family comes from the GPU image; never pin it. Builds leave
torch,
torchvision, torchaudio and triton out of the artifact whether you list them or a dependency pulls them in (peft, accelerate, sentence-transformers), plus the wheels only torch needs. GPU workers use the image's build (torch 2.9.1+cu128), so a version pin on these packages is ignored. In an app that mixes CPU and GPU resources, a CPU resource whose dependencies need torch installs it at cold start; give it container_disk_gb room for that (the build warns when this applies).
- Dev sessions run the runtime image's own `runpod`. Deploys vendor your local package,
but rp flash dev workers import the runpod baked into the built-in image. An AttributeError on a runpod. name in a dev worker traceback means the image is behind your SDK; rp flash deploy runs your installed version.
- Teardown by app and environment:
rp flash undeploy --app APP --env default -y
removes one environment's endpoints; rp flash app delete APP --yes removes every environment (without --yes it prompts, and aborts with no terminal). Then delete volumes you no longer need.
- Queue and API names need 3+ characters to deploy. The name defaults to the function or
class name, and dev sessions don't check it, so class SD passes rp flash dev and fails rp flash deploy. Rename it or pass name=.
- Every endpoint gets your account API key. Deploy and dev put
RUNPOD_API_KEYin each
endpoint's template env, whether or not its code calls another resource, so anyone who can read the template can read the key.
- Re-running the same app soon after a dev session can fail. Dev sessions leave their
templates behind, and the platform holds them for up to about 90 minutes, so a new rp flash dev of the same app fails with "endpoint template names must be unique". Wait, or rename the runpod.App.
Resources
- Setup & CLI: reference/setup-and-cli.md · API: reference/api.md · Patterns: reference/patterns.md · Examples: examples/
- SDK source and upstream examples: https://github.com/runpod/runpod-python (
examples/apps/) - Docs: https://docs.runpod.io/sdk/overview · CLI: https://docs.runpod.io/sdk/cli · Migration: https://docs.runpod.io/sdk/migrate-from-flash