---
title: Writing a model | Runware Docs
url: https://runware.ai/docs/serverless/writing-a-model
description: Declare endpoints with the decorator, load your weights once, and let the build derive the request schema from your handler signatures.
relatedDocuments:
  - https://runware.ai/docs/serverless/deploying
  - https://runware.ai/docs/serverless/examples
  - https://runware.ai/docs/serverless/invoking
  - https://runware.ai/docs/serverless/bringing-a-container
---
## Introduction

A code application is **one Python class in one file**. You write the class, mark the methods you want callable, and mark the class itself. The platform builds the image, starts the worker, and routes each request to the matching method.

```python
from runware_serverless import endpoint, serve

@serve
class ImageTools:
    def load(self) -> None:
        self.pipe = load_my_pipeline()

    @endpoint
    def generate(self, prompt: str, steps: int = 4) -> dict:
        steps = 4 if steps is None else steps
        return {"image": self.pipe(prompt, steps).encode()}

    @endpoint
    def upscale(self, image: str, scale: int = 2) -> dict:
        scale = 2 if scale is None else scale
        return {"image": self.pipe.upscale(image, scale).encode()}
```

That file serves two endpoints, `generate` and `upscale`, on one worker.

## load

`load` runs **once per worker, before any request reaches it**. Put your weights, your pipeline and anything else expensive here, so the cost is paid at startup and callers get a warm worker.

Everything you assign to `self` in `load` is **available to every handler** for the life of that worker.

> [!WARNING]
> **Keep your heavy imports out of the top of the file.** The build imports your file to find your endpoints, and it does that without those dependencies installed, so a module-level `import torch` fails the build. Import inside `load` or inside a handler, which is what every example with a real model does, or put the imports in a [`with imports():` block](#serving-time-imports) when a name has to be visible at module scope.

> [!WARNING]
> **`load` is required, even when there is nothing to load.** The build identifies your class by looking for one that defines `load` alongside its handlers, so a class without it is not a candidate, and the build fails with no deployment class found when none is left. Write `def load(self) -> None: pass` when you have nothing to put in it.

There is no matching shutdown hook. `load` is **retried when the GPU is out of memory at startup**, waiting for a larger allocation each time. The retry re-instantiates your class, so **both `__init__` and `load` have to be safe to run twice**.

## Serving-time imports

`with imports():` is the other place a serving-time import can go, and the one to use when a name has to be visible outside a method.

```python
from runware_serverless import endpoint, imports, serve

with imports():
    from my_model.manager import InferenceManager

@serve
class MyModel:
    manager: "InferenceManager | None" = None

    def load(self) -> None:
        self.manager = InferenceManager()

    @endpoint
    def predict(self, prompt: str) -> dict:
        return {"text": self.manager.run(prompt)}
```

At serving time the block runs and **binds its names at module scope**, so `load` and your handlers use them without importing again. At build time the platform suppresses whatever the block raises, so the build imports your file with none of your dependencies installed.

**The quotes on the annotation are needed.** At build time the import was suppressed, so the name is unbound, and a class-level annotation is evaluated when the class body runs. A module with `from __future__ import annotations` can leave them off, because every annotation in it is already a string.

**The block holds imports and nothing else**, at the top level of your file. A statement in it that fails for its own reason is suppressed at build time too, and silently. A block inside a function binds its names as locals, where no handler can reach them.

**A file with a block needs those dependencies to import at all.** A test suite that imports your model file needs them in its dev group, which importing inside `load` does not ask of you.

## The endpoint decorator

`@endpoint` marks a method as callable. Bare, **the path is the method name** with each underscore replaced by a hyphen, so `remove_background` serves `remove-background`.

It accepts two optional keyword arguments:

- `path` sets a route the method name cannot produce, such as `@endpoint(path="v1/chat/completions")`.
- `default=True` marks the endpoint **a root invocation calls**, one sent without an endpoint path. An app marks one at most.

A positional argument such as `@endpoint("upscale")`, a keyword other than those two, a second `default=True`, or a path that is not legal **fails the build**, before a deployment exists.

> [!NOTE]
> A class with no `@endpoint` methods falls back to `predict`, so a single-endpoint model can skip the decorator. **An app with exactly one endpoint is the default**, so you can invoke it without naming the path. **The moment you declare one endpoint, only declared paths are served**, and `predict` returns `404` like any other path you did not declare.

One app declares **at most 20 endpoints**. Going over fails the build.

### Endpoints you inherit

Decorated methods are found **through the whole inheritance chain**, so a shared base class serves its endpoints to every model built on it.

```python
class ImageBase:
    @endpoint
    def upscale(self, image: str, scale: int = 2) -> dict: ...

@serve
class ImageTools(ImageBase):
    def load(self) -> None: ...

    @endpoint
    def generate(self, prompt: str) -> dict: ...
```

`ImageTools` serves both `generate` and `upscale`. **An override serves its own implementation** whether or not you decorate it again, so removing the decorator from an override keeps the path served.

## Request schemas

**The build reads each handler's signature** and records one manifest entry per endpoint.

A parameter **without a default becomes required**. A parameter **with a default becomes optional**. The schema is **closed**, so a body with a field outside the signature is rejected.

For the class above, a request to `generate` must include `prompt` and may include `steps`. Sending `image` to `generate` **fails with a `422`** before your worker sees it, because `image` belongs to `upscale`.

> [!WARNING]
> An omitted optional **parameter** arrives as **an explicit `null`, not as a missing key**, so a Python default alone does not apply. Apply the default inside the handler, as `steps = 4 if steps is None else steps` does above. A parameter's default is not published, so the platform has no value to send. A field on a model is different, and the Pydantic section below says how.

Every parameter has to be **fillable by name**, because each request is dispatched as `handler(model, **fields)`. A positional-only parameter, `*args` or `**kwargs` fails the build.

## Typed requests with Pydantic

A parameter annotated with a Pydantic model is **validated into that model before your handler runs**, and the model's JSON Schema becomes that field's schema in the contract.

```python
from pydantic import BaseModel
from runware_serverless import endpoint, serve

class Transcribe(BaseModel):
    audio_url: str
    language: str | None = None

@serve
class Whisper:
    def load(self) -> None:
        self.pipe = load_my_pipeline()

    @endpoint
    def transcribe(self, req: Transcribe) -> dict:
        return {"text": self.pipe(req.audio_url, language=req.language)}
```

`req` arrives as a `Transcribe` instance, not a dictionary, so **the annotation is a contract the platform enforces**. It behaves the way the same signature does on a FastAPI route.

A field publishes its `default` and `description`, and a parameter cannot. The platform sends the published default, so **a field with a default arrives as that default, not as `null`**, and your handler does not have to apply it again. Declare a model even for a single optional flag.

Every field you declare on a model is a field the platform can reject before your worker is woken. With a bare `dict` the caller has to guess the body, and nothing about its contents is checked for them.

> [!NOTE]
> An annotation the build cannot resolve **becomes "anything", and the build still succeeds**. A type that only exists at serving time, imported under `if TYPE_CHECKING`, is the usual cause: the build reads your file without your heavy dependencies installed, so the name is missing at build time. The endpoint still works, and its schema stops describing that field.

Returning a model declares your response shape in the same way. **Only the request side is coerced**: the value your handler returns goes out as it is, and the declared type is not enforced. A response type that is not JSON needs `encode_response` below.

> [!WARNING]
> Coercion applies to **a model, or a model that may be null**. It does not reach a model inside a container: a parameter annotated `list[Item]` arrives as a list of plain dictionaries, not a list of `Item`. Declare a wrapper model with the list as a field when you want a batch validated.

### Images that arrive as base64

An image arrives at an endpoint as a base64 string, and `runware_serverless.schema` **converts that string to a `PIL.Image` before your handler runs**. `Base64RGBImage` and `Base64RGBAImage` are the two annotations, and they differ only in the alpha channel: the first converts to RGB, the second keeps the channels that arrived.

```python
from pydantic import BaseModel
from runware_serverless import endpoint, serve
from runware_serverless.schema import Base64RGBImage

class UpscaleRequest(BaseModel):
    image: Base64RGBImage
    scale: int = 2

@serve
class Upscaler:
    def load(self) -> None:
        self.pipe = load_my_pipeline()

    @endpoint
    def upscale(self, req: UpscaleRequest) -> dict:
        return {"image": self.pipe(req.image, req.scale)}
```

The field publishes as **a string of format `base64`**, and a payload that is not valid base64, or that decodes to something that is not an image, is refused with a validation error on that field.

> [!WARNING]
> **These work on a field of a model, not on a parameter.** Binding validates a parameter against a Pydantic model, and these types are PIL images, so a bare `def upscale(self, image: Base64RGBImage)` is never coerced and your handler receives the string exactly as it arrived.

They are the one part of the package that has dependencies, Pillow and Pydantic, so they come as an extra. Add it in the app directory, where the package is installed.

```bash
uv add --dev 'runware-serverless[schema]'
```

Without `uv`, install it with the Python inside the app directory's `.venv`, at `.venv/bin/python` or `.venv\Scripts\python` on Windows. That route does not touch `pyproject.toml`, so record the extra yourself: find the `runware-serverless` entry in the `dev` list under `[dependency-groups]` and write it as `runware-serverless[schema]`.

## serve

`@serve` registers the class and **returns it unchanged**, so the file stays importable for the class it defines. Your own tests can import `ImageTools` and call its methods directly without any of the serving machinery running.

Use it as a decorator above the class, or call `serve(ImageTools)` after the class body for the same result.

Endpoints are resolved **when the module is imported**, not on the first request, so a class with nothing to serve fails in the build, with the reason in its log.

**A model file where `serve` registered no class fails the build too**, before the manifest is written, so a missing decorator shows up in the build log.

### Encoding the response

The schema is derived from your handler's return type. An image model that returns a PIL image cannot declare its wire format, and encoding inside the handler would **make the published schema wrong**.

**`encode_response` does that conversion.** It maps the handler's results to what goes on the wire, and the signature keeps describing what the handler returns.

```python
@serve(encode_response=to_base64_png)
class FluxDev:
    ...
```

Use it when your response needs a transport encoding. **Leave it unset otherwise.** Every use of it is a deliberate difference between the published schema and the wire format.

## One queue and one worker pool

Every endpoint on an app **shares one queue and one worker pool**. Hardware and scaling are set once, on the app, and apply to all of them.

The platform enforces that at three points: **the decorator accepts only those two keywords**, the create and update routes reject an `endpoints` field, and a container configuration with a `hardware` key returns `422`.

Two endpoints that need different hardware are **two apps**.

## Dependencies

List what your model imports beyond the standard library. There are three places to put that list, and the build reads **the first one it finds**: a `requirements` attribute on your model class, then whatever you passed as `--requirement` on the deploy, then a `pyproject.toml` and `uv.lock` in your source directory.

The attribute keeps the list **beside the code that needs it**, so a file that moves between projects keeps its dependencies.

```python
@serve
class MyModel:
    requirements = ["torch==2.5.1", "diffusers>=0.31"]

    def load(self) -> None:
        ...
```

It has to be a list or tuple of strings. **The name is reserved**, so a handler called `requirements` is refused and has to be renamed. Declaring the list on the class and on the deploy at the same time **fails the build** instead of picking one, because picking one would silently drop the other.

**Leave `runware-serverless` out of it.** The platform's client libraries are installed into the image at build time and again when the container starts, so a version pinned in your requirements would drift from the services it talks to.

**System libraries come from the base image, not from this list.** The build starts from `python:3.12-slim`, which has an interpreter and little else, so `ffmpeg` and `libGL` are missing at runtime. **`--base-image` on the deploy pins one that has them**, and it has to include Python 3.12 or newer, which the build checks. [Bringing a container](https://runware.ai/docs/serverless/bringing-a-container) is the other way.

OpenCV is the common case: `opencv-python` needs `libGL`, and `opencv-python-headless` does not.

## What the build includes

The build **includes importable code only**: a `.py`, `.so` or `.pyd` file, or a directory with an `__init__.py`. A folder of loose files beside your model file is not a package, so the build skips it.

Each included entry is capped at **100 MiB**. Weights belong on a [volume](https://runware.ai/docs/serverless/volumes) or in the mounted cache, fetched in `load`.

> [!WARNING]
> A model file that reads a data file next to itself, say a `prompts.txt` at the root, **will not find it in the worker**. Move it inside a package that ships, or read it at runtime from a mounted path.

The build log lists every skipped entry with the reason.

The [examples](https://runware.ai/docs/serverless/examples) are working apps you can deploy as they are.