Celery: Distributed task queue for Python
Asynchronous task queue with distributed worker execution.
Learn more about Celery
Celery is a Python library for managing distributed task queues and asynchronous job execution. It operates by having client applications place task messages onto a broker, which then routes those messages to worker processes for execution. The system supports multiple message brokers (AMQP, Redis, SQS) and allows horizontal scaling through multiple workers and brokers. Common deployment patterns include background job processing, scheduled task execution, and workload distribution across server clusters.
Multi-broker support
Celery integrates with multiple message broker backends including RabbitMQ/AMQP, Redis, and AWS SQS, allowing users to select infrastructure based on their deployment requirements rather than being locked to a single system.
Language interoperability
While written in Python, Celery's message protocol can be implemented in other languages. Implementations exist for Node.js, PHP, Go, and Rust, enabling task queue communication across polyglot systems.
Flexible scheduling and routing
Celery provides task scheduling capabilities, message routing options, and support for task prioritization and rate limiting, allowing fine-grained control over job execution patterns and worker allocation.
from celery import Celery
app = Celery('tasks', broker='redis://localhost:6379')
@app.task
def send_email(recipient, subject, body):
# Send email logic here
return f"Email sent to {recipient}"
# Call task asynchronously
result = send_email.delay('user@example.com', 'Hello', 'Welcome!')Top in Backend & APIs
Related Repositories
Discover similar tools and frameworks used by developers
Dragonfly
In-memory data store with Redis/Memcached API compatibility and multi-threaded C++.
Supabase
PostgreSQL backend with auto-generated APIs and real-time subscriptions.
PHPMailer
SMTP client and MIME message builder for PHP.
Ghost
Headless CMS built with Node.js for blogging, memberships, subscriptions, and newsletters.
HikariCP
Zero-overhead JDBC connection pooling for production Java applications.