---
title: Compute and scaling | Runware Docs
url: https://runware.ai/docs/serverless/compute-and-scaling
description: Choose a GPU, set the bounds the platform scales between, and change any of it on a running app without a rebuild.
relatedDocuments:
  - https://runware.ai/docs/serverless/pricing
  - https://runware.ai/docs/serverless/monitoring
  - https://runware.ai/docs/serverless/versions
---
## Introduction

An app's `configuration` says **what hardware its workers run on and how many of them there may be**. The platform stays inside those bounds and decides the rest.

```json
{
  "gpuType": "rtx-pro-6000",
  "maxWorkers": 4,
  "idleTtlSecs": 300,
  "scalingDelaySecs": 90,
  "minWorkers": 0
}
```

**Four of these are required**: `gpuType`, `maxWorkers`, `idleTtlSecs` and `scalingDelaySecs`. The rest have defaults.

That is the API. **The CLI fills three of them in**, so a deploy needs only `--gpu-type`: `--max-workers` is 1, `--idle-ttl` is 60 seconds and `--scaling-delay` is 10.

## Choosing a GPU

`gpuType` is a lowercase catalog code such as `rtx-pro-6000` or `l40s`. It must match an id from `GET /v1/gpu-types` **that currently has capacity**, so the list you get back is what you can deploy on rather than everything that exists.

Every app must pin a GPU type. Omitting it **fails with a `422` before a build starts**, because an unpinned app would fall through to whatever default the deployer renders.

`fallbackGpuType` gives a **substitute GPU type** for the app. Placement uses it when the pool behind `gpuType` has no recorded inventory, or when no complete block is free there. It stays unused when placement is merely contending for a busy pool, when a scan reaches its limit, or when your credit does not cover `gpuType`.

**A substitute the platform cannot use is dropped when the version deploys, with no error.** The app then rolls out in the preferred pool without one. A type that is not admitted does this, and so does a type that does not support a group when `gpusPerWorker` is above one.

A worker placed on the substitute **reports it as its own `gpuType`**, and one app's workers can sit on both types at once. **The substitute bills at its own catalog rate**, including when that rate is higher than the type you preferred, and a commitment bought for `gpuType` does not cover it.

The value has to **differ from `gpuType` and resolve to capacity in the same cluster**, and it has to grant the `gpusPerWorker` you asked for. Send an empty string to clear it.

## More than one GPU per worker

The cluster grants a worker's GPUs as **one group**, so `gpusPerWorker` is a group size: `1`, `2`, `4` or `8`.

The size also has to be one the cluster behind your `gpuType` grants. A size it does not grant returns **`422` naming `/configuration/gpusPerWorker`**, because the pod would have nowhere to run.

Not every GPU type can have a group. `GET /v1/gpu-types` reports **`multiGpuSupport` on each entry**, and a type that reports `false` has exactly one GPU per worker.

> [!WARNING]
> Raising it above `1` needs a redeploy, not only a new number: **an image built for a single rank has every GPU it was granted and uses one**.

> [!NOTE]
> GPU is the only compute type. `computeType` defaults to it and **naming `cpu` is a `422`**.

## Bounds, not instructions

The platform starts and stops workers for you. You set the edges.

<table><tbody><tr><td>Field</td><td>What it does</td><td>Default</td></tr><tr><td><code>minWorkers</code></td><td>The floor. <code>0</code> allows the app to release every worker when nothing is arriving</td><td><code>0</code></td></tr><tr><td><code>maxWorkers</code></td><td>The ceiling, and your spend limit. <strong>Required</strong></td><td>none</td></tr><tr><td><code>idleTtlSecs</code></td><td>How long the app waits with an empty queue before it scales to zero. It applies only while <code>minWorkers</code> is 0. <strong>Required</strong></td><td>none</td></tr><tr><td><code>scalingDelaySecs</code></td><td>How long demand must stay below the running workers before workers are removed. Values below 90 act as 90. <strong>Required</strong></td><td>none</td></tr><tr><td><code>minAvailableWorkers</code></td><td>Idle workers held above current demand, so a burst arrives at workers that are already up. Must be lower than <code>maxWorkers</code></td><td>none</td></tr><tr><td><code>availableWorkersPct</code></td><td>The same buffer as a percentage of current demand, rounded up. When both are set, the larger applies</td><td>none</td></tr><tr><td><code>startupTimeoutSecs</code></td><td>How long readiness has before the pod is failed, from 1 to 2400</td><td>platform</td></tr><tr><td><code>requestTimeoutSecs</code></td><td>How long the platform waits on one forwarded request, from 1 to 900. Container apps only</td><td>platform</td></tr></tbody></table>

> [!NOTE]
> **`requestTimeoutSecs` bounds the worker, not your HTTP wait.** `invoke-sync` still returns `504` at the platform deadline so you can go on polling the task. A code app stores and returns the field but does not apply it, because it keeps the platform read ceiling.

> [!WARNING]
> On a container app **a later deploy overwrites both from `container.yaml`**, out of its `timeouts.requestSeconds` and `timeouts.startupSeconds`. Setting them through the API and then deploying that document puts the document's values back.

**Scale-up waits for a minute of steady demand**, so a spike that passes inside it is served by the workers already up. An app woken from zero serves on one worker until that worker finishes its first task, and while the workers accept fewer than two tasks from the queue in 5 seconds, as with long tasks on few workers, the platform asks for **one worker per queued task**, up to `maxWorkers`.

> [!NOTE]
> The buffer only applies **while work is queued**. An idle app still scales down to `minWorkers`, so the floor is what keeps capacity through a quiet stretch. While demand stays steady, the platform skips a buffer below about a tenth of current demand.

> [!NOTE]
> **Your credit balance can lower the ceiling.** When your organization's credit cannot back every worker up to `maxWorkers` at deploy time, the platform applies a lower ceiling and reports it on the app as `effectiveMaxWorkers`. The app scales only up to it. `maxWorkers` keeps the value you set, and a top-up raises a running app's ceiling back without a redeploy.

## The trade you are actually making

Scaling to zero is the reason this product is cheap, and the reason a cold request is slow. **A request arriving at an app with no workers waits for one to start and for your model to load.**

The levers are few and they pull against each other.

**`minWorkers` above zero** keeps capacity ready and removes the cold start for the traffic that fits inside it. You pay for that GPU whether or not anything arrives.

**A buffer** keeps spare workers ahead of a burst while traffic is flowing, and releases them when the queue empties.

**A longer `idleTtlSecs`** keeps the last worker through a gap in traffic on an app that scales to zero, so the next request after a quiet spell is fast. You pay for the idle time you did not use. **With a `minWorkers` floor it is `scalingDelaySecs` that keeps workers through a dip**, because the floor is what the app scales back to.

**A longer `scalingDelaySecs`** keeps workers through a dip in traffic, so the next wave arrives at workers that are already up. You pay for them while they wait. **Capacity ahead of a rising wave comes from `minWorkers`**, the only lever that has workers up before the demand arrives.

Caching weights on a [volume](https://runware.ai/docs/serverless/volumes) attacks the same problem from the other side: **a worker that already has the weights on disk starts faster than one that has to fetch them**, and that is where most of a cold start goes.

## Changing it on a running app

Configuration is stored on the app, **separate from your source**, and can be changed while the app is serving.

```text
runware serverless apps scale <appId> --max-workers 16
```

Every field except `computeType` can be patched this way. **A field you omit is kept**, so a patch touches only what you name.

> [!WARNING]
> Clearing a value by sending `null` is not supported. **Omit the field instead**, which leaves it unchanged. The one exception is `fallbackGpuType`, which an empty string clears.

There are two consequences. **Changing `gpuType` affects only workers created after the change**, so the ones already running finish on the hardware they started on. And a patch is rejected with a `422` if the new type has no capacity on offer, which is the same check that applies at create.

On a running app, **a change that raises `minWorkers` or moves to another `gpuType` is checked against your credit balance** and returns `402` when the balance cannot back it. A change that lowers capacity is never refused that way, though the rollout it starts reads the balance again like any other.

## What it costs you in versions

A configuration change **produces a new version but not a new build**. The image is carried forward untouched, so tuning your scaling is fast and cheap, and the version history still records exactly what changed.

That is also why **your scaling survives a redeploy of your code**: the values are stored on the app.