runpod/runpod-plugins-official

runpod-flash

Code-first serverless with the Runpod Python SDK (runpod 2.0): define a runpod.App with @app.queue / @app.api / @app.task, iterate on remote GPUs/CPUs with rp flash dev, ship with rp flash deploy, and maintain or migrate legacy runpod-flash projects (from r…

View source

How to use

  1. Copy the installation command and review the source package before installing it in your own agent environment.
  2. Check the source document for required tools, credentials and supported hosts.
  3. Describe your input and desired deliverable; review the output against the source material.
npx skills add https://github.com/runpod/runpod-plugins-official --skill runpod-flash

Before you start

  • Installation does not include a model subscription, API allowance or permission to access your files.
  • Repository stars and installation counts describe activity, not an independent quality or security test.
Original skill document

Rendered from the source repository. Headings, examples, code, tables, links, and referenced images are preserved.

Runpod SDK apps (Flash)

For an explicit Dockerfile/template project request, use runpod-build-template. Do not replace that deliverable with Flash unless the user chooses its code-first workflow.

Write Python locally, run it on remote Runpod GPUs/CPUs with rp flash dev, then rp flash deploy to ship. Flash is part of the runpod package (2.0): one SDK, one rp CLI, one login. The standalone runpod-flash package is deprecated and gets no fixes after October 29, 2026.

`rp --help` and `rp flash <command> --help` are authoritative for the command surface. This skill is the mental model, the decision rules, and the gotchas that help output does not carry. Confirm the installed version with pip show runpod before concluding a command or parameter is unavailable.

Which SDK is this project on?

Check before writing code. The two APIs do not mix.

You seeProject is onDo
import runpod, runpod.App(...), @app.queue, rp flash ...Runpod SDK 2.0This file + reference/
from runpod_flash import Endpoint, @Endpoint(...), flash dev, runpod-flash in depsLegacy runpod-flashreference/runpod-flash-legacy.md to maintain it, reference/migrate-from-runpod-flash.md to move it
A new projectRunpod SDK 2.0Start below

Write new code on the SDK. For an existing runpod-flash project, propose migrating and check its Known differences first; don't migrate silently.

If the SDK path fails on an SDK bug, `runpod-flash` is the fallback: a traceback inside runpod/apps/, a worker that crashes before your function runs, or the same code working on runpod-flash. Upgrade runpod first; the steps to check for a fix and fall back are in reference/runpod-flash-legacy.md.

Worked examples first for anything multi-step. Flash appears in end-to-end paths: 03 variant B (whisper endpoint), 07 (network-volume handoff) and 08 (fine-tune → serve); the full index is runpod/golden-paths/README.md. Runnable single-file apps live in `examples/`.

Load on demand:

NeedRead
Install, rp login, rp flash init, every rp commandreference/setup-and-cli.md
Decorator parameters, call styles, Queue/Api/Endpoint clients, volumes, Model, Secret, GPU & CPU namesreference/api.md
Worked patterns: choosing a model, warm model loading, CPU→GPU pipeline, fan-out, tasks, raw HTTPreference/patterns.md
Moving a runpod-flash project to the SDKreference/migrate-from-runpod-flash.md
Maintaining a runpod-flash project as it isreference/runpod-flash-legacy.md

Quick start: uv add "runpod>=2.0.0" (or pip install "runpod>=2.0.0") → rp login (or export RUNPOD_API_KEY=...) → rp flash init my-app → cd my-app && rp flash dev main.py.

The app model

python
import runpod

app = runpod.App("my-app")                     # names the group of resources

@app.queue(gpu="4090", dependencies=["torch"]) # queue-based serverless endpoint
def infer(x: list[float]):
    import torch
    return {"sum": torch.tensor(x, device="cuda").sum().item()}

@runpod.local_entrypoint                       # runs on YOUR machine
def main():
    print(infer.remote([1.0, 2.0, 3.0]))       # runs on a Runpod worker
DecoratorCreatesUse for
@app.queue(...) on a functionAutoscaling queue-based endpointRequest/response inference, batch jobs
@app.api(...) on a classLoad-balanced endpoint serving routes (@runpod.get/@runpod.post, setup in @runpod.init)HTTP services, multi-route APIs, load-once model state
@app.task(...) on a functionOne ephemeral Pod per call, terminated when it returnsTraining, fine-tuning, long jobs

Call styles on a queue or task handle: fn.remote(...) (blocks), await fn.remote.aio(...), fn.spawn(...) (returns a Job, or a TaskJob for a task), fn.stream(...) (queue generator functions only), fn.local(...) (runs on your machine). API classes are called by route: MyApi.post("/path", body).

Dev vs deploy

  • rp flash dev main.py — iterate. Creates temporary endpoints (dev-<app>-<resource>-<hash>),

runs the @runpod.local_entrypoint on your machine, streams the workers' print output to your terminal, and re-runs after every file save. Ctrl-C deletes the endpoints. There is no local HTTP server; the entrypoint is how you exercise the code.

  • rp flash dev main.py --once — run the entrypoint once, tear down, exit non-zero on failure.

The smoke test for CI and for "does the migrated code work".

  • rp flash deploy — ship. Builds an artifact (≤1,500 MB, torch excluded on built-in GPU

images) and creates persistent endpoints in the default environment (--env NAME for others). Re-run to update. --build-only packages without deploying, a cheap offline check that discovery and dependency resolution work.

Dev and deploy run the same module source, so module-level imports, constants and helpers behave identically in both. Deploy packages the project directory, but a dev session ships only the module's source, so keep the app in one file when you use rp flash dev.

Autonomous dev loop

A dev session is long-running. Three rules:

  • Run it in the background with stdin closed (< /dev/null). With no stdin it re-runs

the entrypoint on every file save, so you never need to press Enter.

  • Capture its output to a log file and read it. Worker prints, tracebacks, ✗ remote execution failed: ... and the per-run

✓ done in ... / ✗ failed in ... lines land there.

  • Stop it with `kill` (SIGTERM) or SIGINT, so it deletes its endpoints, and confirm the

log ends with cleaning up N deleted. A second signal stops waiting for a running entrypoint and goes straight to cleanup. kill -9 skips cleanup. When one run is enough, prefer --once; it cleans up on its own.

bash
rp flash dev main.py < /dev/null > /tmp/rp-dev.log 2>&1 &
DEV=$!
# poll the log (bounded) for the run result: "✓ done in ..." or "✗ failed in ..."
for i in $(seq 1 120); do grep -qE "✓ done in|✗ failed in" /tmp/rp-dev.log && break; sleep 3; done
tail -40 /tmp/rp-dev.log
# edit main.py and save -> the session re-scans, refreshes endpoints, re-runs main()
# (each run appends a new result line; read the last one)
kill $DEV; wait $DEV; tail -3 /tmp/rp-dev.log   # expect "cleaning up N deleted"

For a single check, rp flash dev main.py --once is simpler and cleans up on its own. If a session died without cleanup, list leftovers with runpodctl serverless list and delete the dev-<app>-... endpoints this session created, by ID.

Choosing the resource shape

Default to plain Python on the Runpod runtime image: dependencies=[...] and system_dependencies=[...] install on the worker, so no Dockerfile is needed, and even large HuggingFace models stay in code (see reference/patterns.md).

  • `image="org/image:tag"` on any decorator runs your function on that image; it needs

Python 3, not a Serverless handler. Use it for system libraries or a specific CUDA stack. registry_auth= names saved credentials for a private image (rp registry add).

  • A prebuilt inference server image (vLLM, worker-comfyui) that already implements a

Serverless handler is not an SDK app. Deploy it with runpodctl or runpod-mcp and call it with runpod.Endpoint("ENDPOINT_ID").

  • An existing endpoint by ID: runpod.Endpoint("ENDPOINT_ID").run_sync({...}).
  • A resource in another deployed app: runpod.Queue(app="media", name="transcribe") /

runpod.Api(app="media", name="inference").

Gotchas

  1. Queue arguments and results must be JSON-serializable. Queue jobs are plain JSON in dev

and deploy alike. A non-JSON argument fails in dev with arguments must be json-serializable and against a deployed endpoint with Python's own JSON TypeError. Pass URLs or base64, not objects. Whole-number floats can arrive as integers (1.0 → 1), so cast numeric inputs explicitly (torch.tensor(x, dtype=torch.float32)).

  1. One file per app in dev. A dev session ships only the module's source, so a sibling

local module imports in deploy but fails in dev. For dev sessions, keep helpers in the same file or in an installable package.

  1. Keep blocking work out of module scope. The SDK imports your module to discover

resources, locally and on every worker. Put heavy imports (torch) and work inside functions; guard local-only code with runpod.is_local().

  1. `gpu=` and `cpu=` are exclusive (gpu and cpu are mutually exclusive). GPU strings can

be short ("4090", "H100") or a list of acceptable GPUs.

  1. Default `workers=(0, 3)` (runpod-flash was (0, 1)). Set it explicitly when cost or

concurrency matters. idle_timeout is seconds (default 60).

  1. Auto GPU switching needs `workers` max ≥ 5 when you pass a list of GPUs.
  2. 10 MB payload limit on requests. Return large outputs via a volume or object storage URL.
  3. Volumes outlive everything. Dev sessions and rp flash undeploy delete endpoints, not

the NetworkVolumes they created, and those keep billing. Queue and API resources mount one volume at /runpod-volume; read vol.path inside the remote function, never at import time. Load model weights through VolumeCache, not straight off the mount (reference/patterns.md).

  1. Tasks bill until they return. A .spawn()ed task that you never wait() or cancel()

keeps its Pod running after your script exits.

  1. Native CUDA libraries go in `dependencies` too, e.g. faster-whisper needs

nvidia-cublas-cu12 + nvidia-cudnn-cu12 or it silently falls back to CPU.

  1. Dev sessions and tasks need local Python 3.10–3.14. They ship code pickled by your

interpreter, so the worker matches your local version. Deploy falls back to 3.12 with a warning on other versions; rp flash deploy --python-version sets it explicitly.

  1. A set `RUNPOD_API_KEY` wins over `rp login`. A stale exported key causes 401s while the

saved login is fine. Check with runpodctl user, then unset RUNPOD_API_KEY or export a good key.

  1. The torch family comes from the GPU image; never pin it. Builds leave torch,

torchvision, torchaudio and triton out of the artifact whether you list them or a dependency pulls them in (peft, accelerate, sentence-transformers), plus the wheels only torch needs. GPU workers use the image's build (torch 2.9.1+cu128), so a version pin on these packages is ignored. In an app that mixes CPU and GPU resources, a CPU resource whose dependencies need torch installs it at cold start; give it container_disk_gb room for that (the build warns when this applies).

  1. Dev sessions run the runtime image's own `runpod`. Deploys vendor your local package,

but rp flash dev workers import the runpod baked into the built-in image. An AttributeError on a runpod. name in a dev worker traceback means the image is behind your SDK; rp flash deploy runs your installed version.

  1. Teardown by app and environment: rp flash undeploy --app APP --env default -y

removes one environment's endpoints; rp flash app delete APP --yes removes every environment (without --yes it prompts, and aborts with no terminal). Then delete volumes you no longer need.

  1. Queue and API names need 3+ characters to deploy. The name defaults to the function or

class name, and dev sessions don't check it, so class SD passes rp flash dev and fails rp flash deploy. Rename it or pass name=.

  1. Every endpoint gets your account API key. Deploy and dev put RUNPOD_API_KEY in each

endpoint's template env, whether or not its code calls another resource, so anyone who can read the template can read the key.

  1. Re-running the same app soon after a dev session can fail. Dev sessions leave their

templates behind, and the platform holds them for up to about 90 minutes, so a new rp flash dev of the same app fails with "endpoint template names must be unique". Wait, or rename the runpod.App.

Resources

from this repository

More skills

All skills
runpod
Community

runpod

Start here when the Runpod lane is unclear: picks the skill for a task (runpod-mcp and the journey playbooks it routes to, such as runpod-serverless-deploy, runpod-pod-deploy, runpod-pod-doctor and runpod-cost-audit; runpodctl, runpod-flash, runpod-companion-clis, runpod-usage, runpod-templates, runpod-build-template, runpod-migrate, runpod-cloud-migrate) and indexes two dozen end-to-end worked examples (golden paths). Use it before any multi-step or provisioning task, even when the lane is clea

1h installs
3
GitHub stars
50
Updated
Nov 19
runpod
Community

runpod-build-template

Create or improve a complete Runpod template project from an application request, local code, or GitHub repository, with a documented Dockerfile and app-specific files for Pods or queue/load-balanced Serverless. Use runpod-templates for finding or operating existing official templates, runpod-mcp/runpodctl to register an already-built image as a template, runpod-flash for its code-first workflow, and runpod-companion-clis for standalone image build/push commands.

1h installs
3
GitHub stars
50
Updated
Nov 19
runpod
Community

runpod-cloud-migrate

Move a workload from another cloud or GPU platform to Runpod: Vast.ai, Lambda, CoreWeave, Paperspace, Modal, Replicate, Baseten, Together AI, Fireworks AI, fal, Hugging Face Inference Endpoints, AWS, Google Cloud or Azure. Inventories what the project uses, maps it to Runpod, plans, then migrates one workload at a time with verification. To move Runpod code between Runpod API versions, use runpod-migrate instead.

1h installs
3
GitHub stars
50
Updated
Nov 19
runpod
Community

runpod-companion-clis

Companion CLIs for Runpod workflows — HuggingFace, GitHub, Docker, and AWS. Use runpod-build-template for complete template project authoring, and the ComfyUI model-repair guide in runpod-templates for imported workflows lacking model download metadata.

1h installs
3
GitHub stars
50
Updated
Nov 19