Bringing a container
Deploy your own Dockerfile and HTTP server, and declare the endpoints the builder cannot see inside your image.
Introduction
Bring a container when the environment matters as much as the model: a CUDA build you have already fought with, a pipeline with system packages in it, or a server that works and that you would rather not rewrite.
You provide a Dockerfile, a container.yaml beside it, and whatever files the Dockerfile copies. Everything after the build is identical to a code app: the same versions, the same scaling, the same invoke routes.
All of it travels in one archive of at most 64 entries, and a directory entry counts as one. The CLI packs files only, so through the CLI the limit is 64 files.
Your server is the app
The image runs your CMD and your server answers requests, exactly as it does on your machine.
FROM python:3.12-slim
WORKDIR /app
COPY app.py /app/app.py
ENV PORT=8080
EXPOSE 8080
CMD ["python", "/app/app.py"]What the platform requires of it is short. Bind the port you declare, answer POST /{endpointPath} for every endpoint you declare, and serve the two probe routes.
The platform's own containers reserve ports 8000, 8888, 8889 and 9090. Binding one of those collides with something already running beside you, so pick anything else.
vLLM images
The platform runs one kind of image differently. vLLM labels every image it publishes, and the builder reads that label off the image your final stage builds on. When the label is there, the platform installs NVIDIA Dynamo next to the image's own vLLM and puts the Dynamo launcher first on PATH as vllm, so the vllm serve in your CMD starts a Dynamo worker and a Dynamo frontend on the port you declared. The platform leaves your image and its packages as they are.
What happens next depends on the vLLM inside your image:
| What the builder finds | What you get |
A vLLM in the range Dynamo supports, 0.28.x today | Your server under Dynamo, with the two flags below |
| vLLM-Omni, a different server with the same label | Your image as you wrote it, without Dynamo and without those flags |
| A vLLM outside that range | A refused build, with the range and the Dynamo version in the log |
The supported range moves with the Dynamo version the platform installs, so a refused build is what names the range in force. The build fails before any worker starts, so you find out while you are building and not on your first request.
The platform adds two flags on the Dynamo path. --enable-sleep-mode lets the platform move the model off the card between calls and restore it at the same addresses. --max-num-seqs arrives when your app sets MAX_CONCURRENCY, and if you pass your own --max-num-seqs the platform keeps your value.
The platform splits the flags in your CMD three ways. Every engine flag reaches the engine as written. The frontend gets the ones that shape a response, such as --host, --port, the chat template and the tool-call parser. The platform drops the ones that do not affect the response, including logging, CORS, the docs routes and --api-key, because it authenticates in front of the pod. Any other server-side flag stops the worker before it starts, and the log lists each flag it refused, so remove flags such as --lora-modules, --root-path and the --ssl-* pair from your image.
8000 is vLLM's default port and one of the four the platform reserves, so a vLLM image has to move its port. Running your own container deploys vllm/vllm-openai this way.
Declaring your endpoints
A code app has its endpoints read out of the source. A builder cannot look inside your image and find them, so a container app declares them itself.
configVersion: 2
port: 8080
probes:
readiness: /ready
liveness: /health
timeouts:
requestSeconds: 60
startupSeconds: 300
endpoints:
- path: generate
default: true
input:
type: object
additionalProperties: false
required: [prompt]
properties:
prompt: { type: string }
steps: { type: integer }
- path: upscaleport is where your server listens. probes are the routes the platform calls to decide whether a worker is ready for traffic and whether it is still alive. timeouts bound one request and the startup that precedes it.
default: true marks the endpoint a root invocation reaches, one that names no endpoint path. An app marks one at most, and an app with exactly one endpoint is its own default without the mark.
An endpoint can also set target, the absolute route your server answers on, when it differs from the public path: path: chat with target: /v1/chat/completions. Both keys need configVersion: 2, and version 1 refuses them with a 422.
Only port and endpoints are required. The example above spells probes and timeouts out, but every one of those values is the default you get by omitting them.
| Setting | Default | Most you can ask for |
probes.readiness | /ready | Any absolute HTTP path |
probes.liveness | /health | Any absolute HTTP path |
timeouts.requestSeconds | 60 | 900 |
timeouts.startupSeconds | 300 | 2400 |
endpoints | none, you declare them | 20 per app |
| Schema size | none | 200 properties across every input and output together |
The startup clock starts when your container starts, and the platform adds its own 30 minutes. startupSeconds is the budget for your container to answer its readiness route, and ready also covers the first checkpoint write, so what is enforced is your value plus 30 minutes. A worker that has still not answered by then is restarted and comes back in loading, so a slow startup reads as a loop rather than as a failure.
The platform waits requestSeconds plus 30 seconds for each answer. A request still running after that fails its task with inference timed out after N seconds, and the late answer is discarded.
Declaring your schemas
This is the real difference between the two paths. A code app derives its request schema from your handler signature. A container app declares it, in JSON Schema, per endpoint.
An endpoint with an input schema gets the same treatment as a code app: a mismatched body is rejected before it reaches your container. An endpoint with none, like upscale above, is a passthrough and takes whatever JSON you send it.
You can declare an output schema too, which documents the response rather than gating it.
Both paths converge here. The builder turns your container.yaml into the same manifest a code build produces, so the queue manager routes, validates and generates documentation without knowing which kind of app it is looking at.
Readiness is your cold start
A worker takes no traffic until its readiness route answers. That is where you load your model, and returning 503 until you are done is the whole mechanism.
def do_GET(self):
if self.path == "/ready":
self.send_response(200 if model_loaded else 503)
self.end_headers()Each poll gives your route about five seconds to answer. The platform allows six for the response and asks your server for five, so a route that replies in a millisecond and one that takes four seconds are read the same way. A route that takes longer counts as not ready, which is the right reading while it is still loading, and it is the reason a readiness check should answer from a flag your loading code sets rather than by doing work of its own.
Getting this wrong is the most common container failure. A server that reports ready before its weights are loaded takes a request it cannot serve, and the caller sees a timeout rather than a cold start.
Testing before you deploy
The container path has an advantage the code path does not: it is just your image, so you can run the whole thing locally with Docker before Runware ever sees it.
docker build -t my-app .
docker run --rm -p 8080:8080 my-app
curl http://localhost:8080/ready
curl -X POST http://localhost:8080/generate \
-H 'Content-Type: application/json' \
-d '{"prompt": "a red bicycle"}'Local testing leaves one gap, the envelope. Runware wraps your payload as {taskId, payload} and hands your server the payload member alone, so a local curl sends the inner object directly. See Invoking an app.
What you give up
Endpoint hardware is still app-level. Every endpoint your container declares shares one queue and one worker pool, and a hardware or scaling key inside an endpoint entry is a 422. Two endpoints that genuinely need different GPUs are two apps.
And the schema is yours to maintain. A code app's schema is generated from its handler, so the two always agree. A container.yaml can drift, and nothing will tell you until a request is rejected that should have worked.