Email sending in Mautic 5: Cron, Queue and Daemons

Similarly to my Email Setup in Mautic 5 blogpost, I created a video, that takes a deep dive into the inner workings of the Email queue. You learn how Mautic sends emails, and what options you have to set up the background processes, that actually send out those emails. You’ll learn abuot all the ways Mautic sends emails, how to schedule emails and how queue works.

Once you watched the video, you will have a lot more clearity how Mautic uses queues and you can decide if this is something you want to do.

Now, that we covered theory, let’s talk action.

Enable the Queue in Mautic 5

Mautic used to send email using file queue, which had it’s limitations. Symfony 5.4 enables database queue amond others, and it is a huge improvement. In this blogpost I will only cover database queue, and with time I’ll add Redis, Rabbit MQ and Beanstalk as well.

In order to enable Database queue we have to set doctrine in the queue management interface under Configuration > Queue Settings > Queue for email (SMS and push messages).

This means, that Mautic knows: use your own database to manage the queue. Once you set this up, all emails sent will land in the messenger_messages table in your database.

In order to get these messages out to the recipients, you’ll need to run the following command:

bin/console messenger:consume email

But this command is very different from commands like mautic:segments:update. Unlike the other mautic commands, this will not stop working after it did it’s job, but stay focused and keep looking out for new messages in the queue. Theoretically you would need to run it only once (ever) and will be sending until the end of times. (Or next restart, or when you stop it manually or it dies any other way.)

This is a huge improvement, because now all emails can go out within seconds, no need to wait for cron cycles.

I think we all agree it’s not a good idea to run this command without proper supervision, and hope it runs by itself. It can be killed by other processes, system glitch or any other unforseen issue. We have to make sure it restarts if needed. There are 2 ways to do this.

Set up Cronjobs

The good old cronjobs come in handy again, we can use them to take this command. Our goal is to run the service all the time, so I suggest to run it every minute with a runtime of 50 seconds. This way it works 50 seconds out of 60, and during this time sends all the incoming emails.

You can do this by setting a time limit:

bin/console messenger:consume email --time-limit=50

This ensures your messenger service runs almost all the time, but never multiple copies at the same time.

Please remember your messenger will send emails as fast as possible. If you need to limit sending speed, you will need to make adjustments to this startegy. You can do it either via cronjobs or config in your mail transport. I will have a tutorial coming up regarding mastering the email sending speed via queue. You can sign up to this course here.

Set up a Daemon

You can call daemons also a service, but I think daemon is a lot cooler. Without going into details they are programs, running in the background with specific premissions and tasks. Daemons are demi gods, who don’t die, so you can configure them to respawn ones they are killed. Pretty cool 🙂

To set up a messenger daemon you need to add it to your systemd. (Not that d at the end is for daemon.)

sudo nano /etc/systemd/system/php-mautic-consumer.service

Add the following content to the file, make sure you are using the right path to Mautic and your php.

[Unit]
Description=Mautic 5 Messenger Queue Consumer
After=network.target
[Service]
ExecStart=/usr/bin/php /var/www/html/mautic/bin/console messenger:consume email
WorkingDirectory=/var/www/html/mautic
Restart=always
User=www-data
Group=www-data
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=php-mautic-consumer
[Install]
WantedBy=multi-user.target

Reload the changes:

sudo systemctl daemon-reload

Enable this service to start on boot

sudo systemctl enable php-mautic-consumer

This is how you would start the service if needed

sudo systemctl start php-mautic-consumer

This is how you check if the status is alive

sudo systemctl status php-mautic-consumer

Obsolete cron command in Mautic 5

If you still have your mautic:send:email command from your previous Mautic 4 installation, you can remove it now, it is obsolete in Mautic 5.