Help
Celery with Django
#
Find similar titles
- (rev. 7)
- KyujinLee
Structured data
- Category
- Programming
- Database
Table of Contents
Overview #
When developing software products, queue management is one of the most important things. There are many solutions that try to solve queue management issues differently. 'Celery' is written in Python language and fully supports asynchronous i/o based operations. In this page we introduce the basic usage. First, let's study Celery's features.
Celery #
Celery is asynchronous task queue management system which is based on distributed messaging. All its operations are in realtime since it supports scheduling.1
- Easy to integrate
- Multiple broker support
- Many other languages supported
- Webhooks supported
Brokers #
Offially supported brokers:
- RabbitMQ
- Redis
- MongoDB
- Beanstalk
- Amazon SQS
- CouchDB
- IronMQ
- other databases (using SQLAlchemy or the Django ORM) are supported in status experimental.2
Django-celery #
Django ORM can be used to store task results which handled by Celery. Django integration gives functionality to query over Celery task results and handling them nicely.
Requirements #
To install django-celery just typing pip install django-celery is enough as it includes also Celery itself. Since Celery has no its built-in message broker, it is recommended to have RabbitMQ or any message brokers that are supported by Celery itself. The default recommended broker is RabbitMQ.
How to install #
Install using pip:
$ pip install django-celery
Or through easy_install:
$ easy_install django-celery
Configuration #
To keep task files and configuration files organized, it is better to know Django project's structure itself. Modern Django project has similar structure.
- proj/
- proj/__init__.py
- proj/settings.py
- proj/urls.py
- manage.py
And then, recommended way to write configuration file can be proj/proj/celery.py:
from __future__ import absolute_import
import os
from celery import Celery
# set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')
from django.conf import settings # noqa
app = Celery('proj')
# Using a string here means the worker will not have to
# pickle the object when using Windows.
app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)
@app.task(bind=True)
def debug_task(self):
print('Request: {0!r}'.format(self.request))
Before getting to work with Celery integrated with our Django project, we need to migrate DB models first.
$ python manage.py migrate djcelery
Run Celery itself:
$ python manage.py celery worker -l info
Usage example #
After making migrations, we can use asynchronous task management and backgorund task running. Following is very simple example.
from djcelery import celery
@celery.task
def add(x, y):
print(x + y)
