Introducing a new way to review your PRs.

Read the announcement

comfy-org / comfyui

The most powerful and modular diffusion model GUI, api and backend with a graph/nodes interface. The fastest local inference engine in the world.

View on GitHub

13 documents · Updated Nov 19, 2026

docs / prompt-execution-graph

Prompt Execution Graph

Overview

This area owns the backend path that turns a validated prompt graph into scheduled node execution, cached outputs, progress events, queue history, and enriched output assets. The main orchestration lives in execution.py: validate_prompt() validates the submitted graph before it is queued, PromptQueue stores pending/running/history entries, and PromptExecutor.execute_async() drives execution for the selected output nodes. The server-facing contract is narrow: comfy_execution/server_protocol.py defines the send_sync() and queue_updated() methods that execution code expects, while server.py wires /prompt through execution.validate_prompt() and PromptQueue.put() and exposes job views through comfy_execution/jobs.py.

Routing/covers for index use: route prompt validation, graph scheduling, node execution, cache behavior, progress state, queue/history, /api/jobs normalization, and output media/asset enrichment reviews here when the touched paths are execution.py or comfy_execution/**. Server route changes belong here only when they change the execution contract with PromptQueue, PromptExecutor, or comfy_execution/jobs.py; pure HTTP middleware or unrelated server APIs do not.

Validation starts in execution.validate_prompt(). It finds output nodes from nodes.NODE_CLASS_MAPPINGS, honors partial execution targets, rejects missing/unknown node classes, and recursively validates every upstream dependency with validate_inputs(). validate_inputs() checks required inputs, linked input shape, return-type compatibility via comfy_execution/validation.py::validate_node_input(), primitive coercions, min/max and combo constraints, custom node validators (VALIDATE_INPUTS or v3 validate_inputs), dependency cycles, and v3 dynamic input finalization through comfy_api.latest._io. Loop-aware v3 nodes are additionally checked by comfy_execution/validation.py::validate_loops(), which pairs Start/End loop boundaries, writes _loop_body and _loop_end back onto start nodes, and rejects unpaired, ambiguous, escaping, or self-accumulating loop structures.

Graph runtime state is represented by comfy_execution/graph.py::DynamicPrompt. It keeps the original prompt immutable while allowing override_node() and ephemeral nodes produced by dynamic subgraph expansion. Ephemeral nodes carry parent/display metadata, so progress and UI events can report display_node, parent_node, and real_node even when the actual executing node id is synthetic. comfy_execution/graph_utils.py::GraphBuilder is the helper node authors use to produce these subgraphs; execution.execute() sets its default prefix per node/list call before invoking a node so generated ids are stable and unique.

Scheduling is handled by comfy_execution/graph.py::ExecutionList, a topological dissolve over the dynamic prompt. PromptExecutor.execute_async() creates it with the prompt outputs that must run, and ExecutionList.add_node() walks upstream strong links while skipping lazy inputs unless they are explicitly required. Each loop iteration stages one ready node with stage_node_execution(), preferring output nodes, async nodes, and nodes near outputs for better perceived UI latency. A node can return ExecutionResult.PENDING when lazy inputs become required, when async tasks are still running, or when a dynamic subgraph is inserted; the executor then unstages it so newly-added blocks can dissolve before retrying.

Node execution itself is in execution.execute(). It first checks caches.outputs; on a hit it replays cached UI through comfy_execution/asset_enrichment.py::emit_cached_output(), finishes progress, updates the ExecutionList edge cache, and skips the function call. On a miss it builds input lists with get_input_data(), injecting hidden prompt/dynprompt/extra metadata/unique id/auth/usage/execution-list values for both legacy and v3 node APIs. _async_map_node_over_list() then maps the node function over list inputs, sets comfy_execution/utils.py::CurrentNodeContext for consumers that need the active prompt/node/list index, supports coroutine node functions, and propagates ExecutionBlocker values. get_output_from_returns() normalizes legacy dict returns, v3 _NodeOutputInternal, subgraph expansions, execution blockers, list outputs, and UI payloads.

Dynamic subgraph returns are not treated as final output immediately. execution.execute() adds each generated node to DynamicPrompt, creates child subcaches under the expanding node, queues any generated output nodes, links generated result references back to the parent, and records pending_subgraph_results. When the parent is retried after children run, those links are resolved from ExecutionList caches and merged as the parent output. Async node returns follow a similar pending path: unfinished tasks are stored in pending_async_nodes, an external block is added to ExecutionList, and an asyncio callback releases the block after the tasks complete.

Caching is split by execution.CacheSet into output caches and object caches. Output caches use comfy_execution/caching.py::CacheKeySetInputSignature, which hashes the class type, IS_CHANGED/v3 fingerprint_inputs() value from execution.IsChangedCache, stable constants, and ordered ancestor references; object caches use CacheKeySetID so node class instances are reused by node id. The default HierarchicalCache supports subcaches for ephemeral subgraphs. LRUCache evicts by generation count, RAMPressureCache evicts by RAM pressure and tensor/model size heuristics, and NullCache disables reuse. External providers registered through comfy_execution/cache_provider.py can look up or store output cache entries when the cache key can be deterministically JSON-canonicalized and contains no self-unequal values such as NaN.

PromptExecutor.execute_async() also owns model and memory lifecycle. It starts comfy.model_patcher.PromptModelTracker, sets the latent preview method, resets interrupts at prompt start, initializes progress state, tells external cache providers about prompt start/end, runs under torch.inference_mode(), calls comfy.model_management.cleanup_models_gc() before execution, handles RAM-pressure cache release between nodes, and unloads models when smart memory is disabled. The execution_list is constructed with a model-output callback (self.prompt_model_tracker.add) so linked model outputs can be tracked as they become available.

Progress reporting is centralized in comfy_execution/progress.py. reset_progress_state() creates a ProgressRegistry for the current prompt and PromptExecutor.execute_async() registers a WebUIProgressHandler. execution.execute() marks node start/finish and node internals can call get_progress_state().update_progress(). The WebUI handler sends progress_state websocket events with active node states plus display/parent/real ids, and sends binary preview images with metadata when the client advertises supports_preview_metadata. CLIProgressHandler offers the same registry contract through tqdm progress bars.

Queue and job lifecycle are split between execution.PromptQueue and comfy_execution/jobs.py. PromptQueue is a heap-backed queue with an RLock/Condition, tracks currently_running, stores bounded history, broadcasts queue updates, and atomically interrupts a running prompt by id via interrupt_if_running(). Completed prompts are stored with PromptExecutor.history_result, which contains UI outputs and per-node meta. comfy_execution/jobs.py normalizes queue/history tuples into /api/jobs responses, maps prompt status messages into pending/in-progress/completed/failed/cancelled, picks preview outputs with media-over-text priority, counts previewable assets, validates canonical UUID job ids, and implements state-aware cancellation that interrupts running jobs or dequeues pending jobs.

Output enrichment happens immediately before UI/history emission in execution.execute(). Raw UI payloads have asset_preview references removed by comfy_execution/asset_enrichment.py::take_asset_previews(), then comfy_execution/media_enrichment.py::enrich_output_with_media_metadata() adds best-effort media metadata to on-disk file entries even when the assets system is disabled. register_executed_outputs() deep-copies and registers output files with app.assets.manager.AssetManager, adding asset ids and preview ids for websocket clients and history; the cache stores the media-enriched output without asset ids, so emit_cached_output() can strip/re-register ids for the current job on cache hits. comfy_execution/preview_tonemap.py::linear_to_preview() is a shared helper for turning scene-linear tensors into bounded sRGB PIL previews used by save/upload preview paths.

Coupling

  • If the prompt tuple layout changes in execution.PromptQueue, update comfy_execution/jobs.py and the server queue/history readers together. jobs.normalize_queue_item() expects a sensitive-data-stripped 5-tuple (priority, prompt_id, prompt, extra_data, outputs), while server.py queues a 6-tuple including sensitive data and removes it before exposing job APIs.
  • If node input schemas or v3 dynamic input behavior change in nodes or comfy_api.latest._io, verify both execution.validate_inputs() and execution.get_input_data(). Validation and execution separately finalize v3 inputs, filter dynamic paths, inject hidden inputs, and map list inputs; schema drift can validate successfully but fail at runtime if only one side is updated.
  • If comfy_execution/graph_utils.py::is_link() changes, verify execution.validate_inputs(), execution.get_input_data(), comfy_execution/caching.py, comfy_execution/graph.py, and comfy_execution/validation.py. Link recognition drives type validation, dependency scheduling, cache signatures, loop graph walks, and subgraph output resolution.
  • If node return shapes change, update execution.get_output_from_returns() and cache/job consumers together. UI dicts, v3 _NodeOutputInternal, ExecutionBlocker, list outputs, and expand subgraphs are normalized before cache storage, websocket executed messages, and history/job summaries see them.
  • If cache key semantics change in comfy_execution/caching.py, verify external providers in comfy_execution/cache_provider.py and forced rerun handling in execution.py. Local cache keys can contain unhashable/self-unequal sentinels that intentionally miss; external providers are skipped for those keys so they do not return hits that local equality would not.
  • If output file entry fields (filename, type, subfolder, format, metadata, asset_preview) change, update comfy_execution/media_enrichment.py, comfy_execution/asset_enrichment.py, and comfy_execution/jobs.py together. Media metadata, asset registration, cached replay, preview selection, and previewable counts all parse the same UI output dictionaries.
  • If progress metadata changes, keep comfy_execution/progress.py aligned with comfy_execution/graph.py::DynamicPrompt. Websocket progress and binary preview metadata depend on display/parent/real node id mapping for ephemeral subgraphs.

Risks

  • execution.validate_inputs() and execution.execute() both implement v3 dynamic/list input handling, and recent history includes 7c8fbc6d Fix dynamic input validation and lazy scheduling (#16377). If a dynamic input or lazy input rule is changed in one path but not the other, prompts can pass validation and then stall or fail during scheduling.
  • execution.get_output_from_returns() has an inline TODO warning that UI values from nodes that also return expand may be cached before subgraphs complete. If a dynamic-expansion node starts emitting UI output, cached history or websocket output can describe an incomplete parent result.
  • execution.PromptQueue.interrupt_if_running() relies on holding the queue mutex while setting the global nodes.interrupt_processing() flag. If cancellation code bypasses this method, a cancel can race with prompt completion and interrupt the next prompt instead of the intended job.
  • comfy_execution/jobs.py preview selection has churn in recent history (7374157e fix(jobs): prefer media over text for job preview_output, a683fa6e Add previewable_outputs_count to /api/jobs). If new output media keys are added only to node UI payloads, /api/jobs can undercount or choose a misleading preview until PREVIEWABLE_MEDIA_TYPES, extension fallbacks, and normalization are updated.
  • comfy_execution/media_enrichment.py and comfy_execution/asset_enrichment.py both resolve output paths but use different path normalization (realpath containment for media metadata, abspath containment for asset registration). If output directories can contain symlinks, metadata extraction may skip a file that asset registration accepts, producing history entries with asset ids but no media metadata.
  • comfy_execution/caching.py::RAMPressureCache.ram_release() recursively sizes cached outputs and evicts model patcher outputs aggressively; recent history includes a2afcdb8 comfy-execution: cache: cache errors is RAM cache sizing scan (#16314). New cached output container types should expose _comfy_cache_tensors() or they may be undercounted, retained too long, or evicted unpredictably under RAM pressure.