---
title: "ControlNet: Structural guidance for image generation | Runware Docs"
url: https://runware.ai/docs/learn/controlnet
description: Provides structural control over generation using conditioning inputs like edge maps, depth maps, and pose detection.
relatedDocuments:
  - https://runware.ai/docs/learn/text-to-image
  - https://runware.ai/docs/learn/loras
---
## Introduction

ControlNet provides **precise structural control** over the generation process by using conditioning images (guide images) to direct how the model creates specific aspects of the output. The guide image is [conditioning](https://runware.ai/docs/learn/glossary#conditioning) like a prompt is, read at every [denoising](https://runware.ai/docs/learn/denoising) step, which is why it constrains the whole result rather than one region of it.

While ControlNet originated in image generation, the underlying concept of conditioning a model with structural guidance has expanded to video generation as well. Video-aware ControlNet models can enforce consistent poses, depth, or edges across frames to maintain temporal coherence.

The conditioning mechanisms can interpret various types of visual guidance: edge maps (like Canny or MLSD) for structural guidance, depth maps for spatial composition, pose detection for human positioning, and segmentation maps for object placement.

**Set 1**:

![Girl with a Pearl Earring by Johannes Vermeer](https://runware.ai/docs/assets/controlnet-source.Bw8HuSyM_Z14YdbB.jpg)

*Original*

![Canny edge detection map of Girl with a Pearl Earring, showing white outlines on a black background](https://runware.ai/docs/assets/controlnet-canny-preprocess.B3pmGmdR_Z1ebCl5.jpg)

*Canny edge map*

![A futuristic woman with iridescent skin wearing a chrome headwrap and pearl earring in cyberpunk style](https://runware.ai/docs/assets/controlnet-canny-output.BzN4wVBz_1eax9l.jpg)

*runware:25@1*

> **Prompt**: A futuristic woman with iridescent skin wearing a sleek chrome headwrap and minimalist pearl earring, soft glowing light, cyberpunk aesthetic, gentle expression, clean background, elegant and surreal

**Set 2**:

![Girl with a Pearl Earring by Johannes Vermeer](https://runware.ai/docs/assets/controlnet-source.Bw8HuSyM_Z14YdbB.jpg)

*Original*

![Depth map of Girl with a Pearl Earring, showing grayscale depth information](https://runware.ai/docs/assets/controlnet-depth-preprocess.B3WL3I5V_2fU80s.jpg)

*Depth map*

![A futuristic woman with iridescent skin wearing a chrome headwrap and pearl earring in cyberpunk style](https://runware.ai/docs/assets/controlnet-depth-output.Dzbkf5V2_I06nM.jpg)

*runware:27@1*

> **Prompt**: A futuristic woman with iridescent skin wearing a sleek chrome headwrap and minimalist pearl earring, soft glowing light, cyberpunk aesthetic, gentle expression, clean background, elegant and surreal

**Set 3**:

![Girl with a Pearl Earring by Johannes Vermeer](https://runware.ai/docs/assets/controlnet-source.Bw8HuSyM_Z14YdbB.jpg)

*Original*

![OpenPose skeleton overlay of Girl with a Pearl Earring, showing detected body keypoints](https://runware.ai/docs/assets/controlnet-pose-preprocess.BNdXhuEJ_2b6oNM.jpg)

*OpenPose image*

![A futuristic woman with iridescent skin wearing a chrome headwrap and pearl earring in cyberpunk style](https://runware.ai/docs/assets/controlnet-pose-output.CbAHrTCL_ZXvdDE.jpg)

*runware:29@1*

> **Prompt**: A futuristic woman with iridescent skin wearing a sleek chrome headwrap and minimalist pearl earring, soft glowing light, cyberpunk aesthetic, gentle expression, clean background, elegant and surreal

**Set 4**:

![Girl with a Pearl Earring by Johannes Vermeer](https://runware.ai/docs/assets/controlnet-source.Bw8HuSyM_Z14YdbB.jpg)

*Original*

![Blurred version of Girl with a Pearl Earring used as structural guidance](https://runware.ai/docs/assets/controlnet-blur-preprocess.DqIA6R-7_Z25Pope.jpg)

*Blurred image*

![A futuristic woman with iridescent skin wearing a chrome headwrap and pearl earring in cyberpunk style](https://runware.ai/docs/assets/controlnet-blur-output.CDs5su17_TEEKl.jpg)

*runware:28@1*

> **Prompt**: A futuristic woman with iridescent skin wearing a sleek chrome headwrap and minimalist pearl earring, soft glowing light, cyberpunk aesthetic, gentle expression, clean background, elegant and surreal

**Set 5**:

![Girl with a Pearl Earring by Johannes Vermeer](https://runware.ai/docs/assets/controlnet-source.Bw8HuSyM_Z14YdbB.jpg)

*Original*

![Grayscale version of Girl with a Pearl Earring used as tonal guidance](https://runware.ai/docs/assets/controlnet-grayscale-preprocess.vM_5Rj4Y_86us6.jpg)

*Grayscale image*

![A futuristic woman with iridescent skin wearing a chrome headwrap and pearl earring in cyberpunk style](https://runware.ai/docs/assets/controlnet-grayscale-output.Dbg8RORj_NCgWG.jpg)

*runware:30@1*

> **Prompt**: A futuristic woman with iridescent skin wearing a sleek chrome headwrap and minimalist pearl earring, soft glowing light, cyberpunk aesthetic, gentle expression, clean background, elegant and surreal

## How it works

Each ControlNet model is **trained to work with a specific type of preprocessed guidance image**. The workflow involves two steps:

1. Preprocess your reference image using our [ControlNet preprocessing tools](https://runware.ai/docs/models) to generate the appropriate guidance image (edge map, depth map, pose skeleton, or other guidance type).
2. Provide this preprocessed guidance image as the `guideImage` parameter along with the corresponding ControlNet model and settings. During generation, the system uses this guidance to influence the output, balancing structural control with your text prompt based on the `weight` parameter.

This preprocessing + inference workflow gives you explicit control over how the structural guidance is prepared and applied.

## Request structure

The `controlNet` parameter is an array that can contain multiple ControlNet models. Each model can have its own settings.

```json
[
  {
    // other parameters...
    "controlNet": [{
      "model": "<controlnet-model-air>",
      "guideImage": "56f8916f-1a33-49cb-b67f-2c4f48472563",
      "startStep": 1,
      "endStep": 10,
      "weight": 1.0,
      "controlMode": "balanced"
    }]
  }
]
```

## Weight and timing

The `weight` parameter controls **how strongly the ControlNet guidance influences** the generation process. Higher values force the output to follow the guide image more closely, while lower values allow the text prompt to dominate.

The timing parameters determine **when the ControlNet guidance is applied** during the generation process. The `startStep`/`startStepPercentage` and `endStep`/`endStepPercentage` parameters define the specific steps when guidance begins and ends (e.g., steps 1-10 of a 30-step generation).

These timing controls offer strategic advantages:

- Starting guidance later (higher `startStep`) allows more creative initial formation before structural guidance kicks in.
- Ending guidance earlier (lower `endStep`) lets your prompt take control for final detailing.

Different timing strategies produce distinctly different results. A common approach is applying ControlNet only during the first 30-50% of steps to lock in composition, then letting the prompt and model handle textures and fine detail in the remaining steps.

## Control mode

The `controlMode` parameter determines how the ControlNet guidance is applied relative to the base model's generation process:

- **`balanced`**: Equal influence between the ControlNet guidance and the text prompt. A safe default.
- **`prompt`**: Gives more weight to the text prompt, useful when you want the structure as a loose guide rather than a hard constraint.
- **`controlnet`**: Gives more weight to the guide image, useful when structural accuracy is more important than prompt creativity.

## Tips

1. **Match the preprocessor to your intent.** A Canny edge map preserves fine outlines (good for architecture, product shots), while a depth map preserves spatial relationships without edge detail (good for scenes and compositions).
2. **Use lower weights for creative work.** Weights of 0.4-0.6 let the model interpret the guide loosely, which works well for artistic output. Weights above 0.8 enforce strict adherence, better suited for technical or reproduction tasks.
3. **Combine multiple ControlNets.** You can stack a depth map with a pose skeleton to control both scene layout and character positioning simultaneously. When stacking, lower the individual weights (0.3-0.5 each) to prevent the guides from competing.
4. **Preprocess at the right resolution.** The guide image should match your target output dimensions. Mismatched resolutions can cause alignment issues between the guide and the generated content.