Introducing a new way to review your PRs.

Read the announcement

comfy-org / comfyui

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface. The fastest local inference engine in the world.

View on GitHub

13 documents · Updated Nov 19, 2026

overview

Knowledge Base updated: 2026-11-19

Knowledge Base Index

1. Orientation

ComfyUI is a Python application for graph-based generative media workflows. The highest-level flow is: the server accepts workflow prompts, validates a node graph, loads/patches models as needed, executes nodes through the prompt executor, and streams progress/results back over HTTP/WebSocket APIs.

Main parts:

  • main.py / server.py — process startup, PromptServer, HTTP routes, WebSocket protocol, middleware, and API plumbing.
  • execution.py / comfy_execution/** — prompt graph validation, scheduling, caching, job lifecycle, and progress/media enrichment.
  • nodes.py, comfy_extras/**, comfy_api_nodes/** — built-in, extra, and hosted/provider-backed node implementations.
  • comfy/** — model loading, memory/device management, samplers, conditioning, encoders, and model architecture implementations.
  • comfy_api/** — versioned public/internal API abstractions used by nodes and integrations.
  • app/** and related asset directories — application services for settings, assets, users, custom nodes, models, cloud/deploy features, and persistence.
  • comfy_config/**, utils/**, tests, and project config — configuration, CLI/options, utilities, scripts, and validation structure.

Critical state lives in:

  • Filesystem model/input/output/custom-node trees — model discovery and generated media assets (models/, input/, output/, custom_nodes/, plus path configuration in folder_paths.py).
  • Application database and migrations — app-level persisted entities and schema evolution (app/database/**, alembic_db/**, alembic.ini).
  • Execution caches and job state — in-memory/runtime prompt execution state under comfy_execution/** and execution.py.

2. Architecture diagrams

flowchart TD
  User[Browser or API Client]

  subgraph Server[Startup and API Server]
    Main[main.py]
    PromptServer[server.py PromptServer]
    Protocol[protocol.py]
    Middleware[middleware]
    InternalAPI[api_server]
  end

  subgraph Execution[Prompt Execution]
    Executor[execution.py]
    Graph[comfy_execution graph and validation]
    Jobs[comfy_execution jobs progress cache]
  end

  subgraph Nodes[Node Surfaces]
    CoreNodes[nodes.py]
    ExtraNodes[comfy_extras]
    HostedNodes[comfy_api_nodes]
  end

  subgraph Models[Core ML Runtime]
    Loading[model loading and management]
    Sampling[sampling and conditioning]
    Encoders[text image audio encoders]
    LDM[ldm and specialized models]
  end

  subgraph AppServices[Application Services]
    App[app services]
    DB[app database and alembic]
    Assets[input output models custom_nodes]
  end

  subgraph APIPlatform[Versioned API Platform]
    ComfyAPI[comfy_api]
  end

  User --> PromptServer
  Main --> PromptServer
  PromptServer --> Middleware
  PromptServer --> Protocol
  PromptServer --> InternalAPI
  PromptServer --> Executor
  Executor --> Graph
  Graph --> Jobs
  Executor --> CoreNodes
  Executor --> ExtraNodes
  Executor --> HostedNodes
  CoreNodes --> Loading
  ExtraNodes --> Sampling
  HostedNodes --> ComfyAPI
  Loading --> Encoders
  Loading --> LDM
  Sampling --> LDM
  App --> DB
  App --> Assets
  PromptServer --> App
  Loading --> Assets
sequenceDiagram
  participant Client
  participant Server as server.py PromptServer
  participant Validation as comfy_execution.validation
  participant Queue as execution.py / jobs
  participant Nodes as node registry
  participant Models as comfy model runtime

  Client->>Server: submit prompt graph
  Server->>Validation: validate graph and inputs
  Validation-->>Server: validated graph or errors
  Server->>Queue: enqueue execution job
  Queue->>Nodes: execute node graph
  Nodes->>Models: load/patch/sample/encode as required
  Models-->>Nodes: tensors, latents, media outputs
  Nodes-->>Queue: node outputs and progress
  Queue-->>Server: status, previews, results
  Server-->>Client: HTTP/WebSocket updates
flowchart LR
  Paths[folder_paths.py and configured dirs]
  Detection[model_detection and supported_models]
  Loaders[sd.py diffusers_load checkpoint_convert]
  Patcher[model_patcher and patcher_extension]
  Memory[model_management memory_management multigpu]
  Encoders[CLIP text vision audio encoders]
  Samplers[sample.py samplers k_diffusion]
  Outputs[latents images audio video]

  Paths --> Detection
  Detection --> Loaders
  Loaders --> Patcher
  Loaders --> Encoders
  Patcher --> Memory
  Memory --> Samplers
  Encoders --> Samplers
  Samplers --> Outputs
flowchart TD
  Registry[nodes.py node registry]
  Core[Core built-in nodes]
  Extras[comfy_extras nodes]
  Hosted[comfy_api_nodes provider nodes]
  API[comfy_api abstractions]
  Execution[execution.py]
  External[External provider APIs]

  Registry --> Core
  Registry --> Extras
  Registry --> Hosted
  Hosted --> API
  Hosted --> External
  Execution --> Registry

3. Glossary

  • PromptServer — the server-side HTTP/WebSocket orchestration surface implemented primarily in server.py.
  • Prompt graph — the submitted node graph validated and executed by execution.py and comfy_execution/**.
  • Comfy API — the versioned API abstraction package under comfy_api/**, distinct from the HTTP server routes.
  • Hosted API nodes — provider-backed nodes under comfy_api_nodes/** that call external services through shared API utilities.
  • LDM — the collection of latent diffusion and specialized model architecture implementations under comfy/ldm/**.

4. Codebase-wide invariants

  • Core, extra, and hosted nodes are separated by path: built-ins live in nodes.py, optional local extensions live under comfy_extras/**, and provider/API-backed nodes live under comfy_api_nodes/**.
  • The prompt execution engine is split between the legacy/root entrypoint execution.py and supporting graph, validation, cache, job, progress, and enrichment modules under comfy_execution/**.
  • Versioned API abstractions are isolated under comfy_api/**; HTTP server routing and middleware are owned separately by server.py, api_server/**, and middleware/**.
  • Model assets and generated media are filesystem-oriented: representative roots include models/, input/, output/, custom_nodes/, with path resolution centralized by folder_paths.py.
  • Test coverage is split between integration-style tests in tests/** and unit-focused tests in tests-unit/**, with shared test configuration in pytest.ini.

5. Routing table