Install celery with rabbitmq on Django

UPDATE 1 AUGUST 2019 


After a lot of headaches at lost I found a great video  tutorial by Mike Hibbert on how to install rabbitmq with celery and setup Django to use it . It worked like a charm on Django 1.7 1.8 running on Ubuntu . The server OS is Debian 9.

First install celery on machine (no on virtualhost):

root@vps:~# apt-get  install rabbitmq-server

Then define rabitmq user, password and vhost and set permissions:

root@vps:~# rabbitmqctl add_user djuser djpass

root@vps:~# rabbitmqctl add_vhost /djangovhost

root@vps:~# rabbitmqctl set_permissions -p /djangovhost djuser ".*" ".*" ".*"

Restart the rabbitmq server:

root@vps:~# /etc/init.d/rabbitmq-server stop
[ ok ] Stopping message broker: rabbitmq-server.
root@vps:~# /etc/init.d/rabbitmq-server start
[ ok ] Starting message broker: rabbitmq-server.
root@vps:~# 

Install celery on django virtualenv:

(.djenv)root@vps:/root$ pip install celery  

also add these lines to settings.py

#Celery configs
BROKER_HOST = "127.0.0.1"
BROKER_PORT = 5672
BROKER_VHOST = "/djangovhost"
BROKER_USER = "djuser"
BROKER_PASSWORD = "djpass"

Make a celery execution script:

nano /path/to/django/celery.sh

Add this script:

#!/bin/bash

CELERY_BIN="/home/john/.djenv/bin/celery"


# App instance to use
CELERY_APP="myapp"

# Where to chdir at start.
CELERYD_CHDIR="/path/to/django"

# Extra command-line arguments to the worker
CELERYD_OPTS="--time-limit=300 --concurrency=1"

# %n will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"

# Workers should run as an unprivileged user.
#   You need to create this user manually (or you can choose
#   a user/group combination that already exists (e.g., nobody).
CELERYD_USER="john"
CELERYD_GROUP="john"

# If enabled pid and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1

export SECRET_KEY="somesecretstring"

----------------

Now you need to deamonize the script using systemd



nano /etc/systemd/system/celery.service

[Unit]
Description=Celery  Daemon
#After=network.target
StartLimitIntervalSec=0
[Service]
Type=simple
User= john
Group= www-data
ExecStart=/path/to/django/celery.sh
#Restart=always
#RestartSec=1
Restart=on-failure
# Configures the time to wait before service is stopped forcefully.
TimeoutStopSec=300
[Install]
WantedBy=multi-user.target


--------

systemctl enable celery

systemctl start celery


check the status

systemctl status celery

The result should be like:

● celery.service - Celery  Daemon
   Loaded: loaded (/etc/systemd/system/celery.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Thu 2019-08-01 22:30:32 CDT; 33min ago
  Process: 28796 ExecStart=/path/to/django/celery.sh (code=exited, status=0/SUCCESS)
 Main PID: 28796 (code=exited, status=0/SUCCESS)



Make sure that the daemon is started

Note: if the damon is not started, you may need to: chomod o+x celery.sh


Now check if celery is actually handling the queue;



celery -A myapp worker -l info

(.djenv)john@server$ celery -A myapp worker -l info

Voila!