Zapier is great if you want to create a proof of concept automation or test your workflows. However if you want to scale up your automation, Zapier can be super-expensive. And all it does is take data from one place and puts it into another place. If you use Zapier for couple of simple repetitive jobs, its better to cut out the middleman and use your own code.
Update:
I’ve updated the connector file, so you’ll get the updated version if you download it again.
Also: There is a video version of this blog, you can find it here: The Marketing Automation Show
This is what we gonna do
We will create few line of code, that captures the incoming data from a webhook, process it if needed, and post it to our Mautic using simple authentication.
You can download the whole code (commented and everything) if you register to my website. Don’t worry it’s free. If you already registered, please log in!
Understand the basics
You don’t need to be a programmer to do this, with the help of google and Stackoverflow you can write the script, that replaces Zapier.
Here are the key expressions you need to know. You need to know what you are using in order to research more, or if you don’t want to look stupid if you ask questions from others in the forums.
Here we go:
A webhook is like a package of information sent to your script over the internet. It can be different format or structure. The good thing, is that you know what to expect, when you write your code to receive it. Check the other party’s website to understand what form of data to expect.
Basic auth is the form of authentication we use to talk to our Mautic instance. It is super simple, just like a login, only with a line of code.
A Curl call is similar to a webhook. You can use it to post the data you to your Mautic.
API stands for Application Programming Interface, and in our case it is the part, that accepts our package and places the data in the right place.
All Zapier does is accept the webhook, and use basic auth to reach your Mautic via curl formatted according to the API docs.
Not rocket science.
Based on comments I would like to lay out the plan, what we intent to do.
Version 1:
We’d like to create a connector file and save it in our WordPress, or our other system, that we use with Mautic. Not in our Mautic install. This can be any folder, that you can reach from the internet.
Version 2:
We can save a file in our Mautic’s install as well, but have to be careful, as Mautic will restrict filenames you can use. If you make an index.php, than you are fine, anything else will break.
So you could make a /connector/index.php file in your mautic subfolder and you would be set. Your system looks like this then:
Okay, before we get started, let’s turn on the API settings in Mautic. I’d create a new user called API as well, just to keep the simple login and programmatic login separate.
Post to Mautic via the API
What? Why to start from the last step?
Multiple reasons. I want to make sure the API works, and I also want to you to have instant success.
In this example we will crate a new contact and tag them. This is a great way to start a campaign for them.
Let’s declare our variables:
<?php $loginname = 'apiuser'; // Loginname of your API user $password = 'yourpasswordhere'; // This is the password of the API user $siteurl = yoursiteurl.tld; // example: mymautic.com
The data you would like to transfer is called payload. We will set it for now, and later replace with the data we get from the webhook.
$email = 'captain_enterprise-d@starfleet.com'; $firstname = 'Jean-Luc'; $lastname = 'Pickard'; $tag = 'purchased';
This is our curl call:
$curl = curl_init();
// Set some options - we are passing in a user agent too here
curl_setopt_array($curl, array(
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_URL => "https://".$loginname.":".$password."@".$siteurl."/api/contacts/new",
CURLOPT_USERAGENT => 'Mautic Connector',
CURLOPT_POST => 1,
// posting the payload
CURLOPT_POSTFIELDS => array(
'firstname' => $firstname,
'lastname' => $lastname,
'email' => $email,
'tags' => $tag
)
));
curl_exec($curl);Save the code in a php file, place it in your server (outside of your Mautic folders), and run it.
Our new contact will be placed into Mautic tagged with the appropriate tag.
Cool.
Capture and process a webhook (woocommerce example)
Now I have a bad news and good news. The bad news is, that each service you want to connect with might have a different webhook structure. The good news is, that we need to build the interface only once. With a little practice and the help of the internet you’ll be able to catch any webhook payload in no time.
In our example we will capture a woocommerce webhook.
As a first step, you can should see how the payload looks like. You can do it in 3 easy steps:
- create a bin at https://requestbin.com/
- add the bin address as your webhook address
- Fire up the webhook. In our case that would be making a woocommerce order
If all goes fine, you’ll be able to see an incoming webhook payload in your bin.
We can capture the entire webhook’s payload with just one command.
// receive the webhook
$HTTP_RAW_POST_DATA = file_get_contents('php://input'); The hard part is to make sense of this incoming data. Teaching you how to decode and loop through JSON content is beyond the scope of this tutorial, but I’ll show you how I did it with couple of commands in case of this woocommerce webhook:
// decode the incoming data $data = json_decode($HTTP_RAW_POST_DATA,true);
Now we can grab the different values. In case of woocommerce we will use the billing information:
$email = $data['billing']['email']; $firstname = $data['billing']['first_name']; $lastname = $data['billing']['last_name'];
Obviously this info comes to the front of our script, and replaces the payload we declared before. The tag is static, you don’t need to replace it. Maybe I’ll create a second part to this article where we tag our contact based on what they have purchased.
Add logging
I am huge fan of logging, so why don’t you add a small logfile to your script. It makes debugging easier. Add this at the end of your code:
file_put_contents("webhooklog.txt",($email','.$firstname','.$lastname','.PHP_EOL),FILE_APPEND);You can download the whole code (commented and everything) if you register to my website. Don’t worry it’s free. If you already registered, please log in!
If you have questions, let me know in the comments.
Alex Hammerschmied
says:
Hi Joey,
thanks for the tutorial.
Sadly it doesn’t work for me. When I call the connector.php file from the terminal everything seems to work, but it never goes to Mautic. What am I doing wrong?
Also I found, that at the beginning of the code the <?php tag is missing here on the blogpost.
And to get the log working it gave me an error everytime i wanted to use the ',' commas between the values. Without the commas it works.
Cheers,
Alex
jos0405
says:
Hey Alex,
Thanks for noticing. I added the opening php tag.
Turns out curl_exec was missing (Duuh…)
I added it, and modified the downloadable code as well!
Tom
says:
Hey Joey,
This is great. However I am little lost.
I have placed the codes in the example outside of the Muatic install directory but where do I put the connector.php file?
– I have API and Enable HTTP basic auth Turned on.
I called my PHP file under hook.php. When I run it, it show blank and no contact created.
Thanks in advance for your help.
jos0405
says:
Hello Tom!
Leave the connector.php also outside.
Is your hook.php pushing in certain data to your connector.php?
Make sure you are using the right method: POST or GET depending on what you accept in connector.php
Tom
says:
Hi. I need to do some data mapping. User signs up for a Digital Access Pass product. DAP sends a webhook to Mautic and adds contact to a segment. The First Name field will be different. Email might be the same. I need to map at least those two fields. Would I use this code for that?
jos0405
says:
Hello Tom,
Yeas, you can use use it for that.
Capture the values from your provider and push it forward to Mautic.
You’ll need to do the followings:
1. Modify the code to be able to capture the payload by aligning the fieldnames and POST/GET method.
2. Create the custom fields in Mautic according to your needs (Add purchase date, product, etc.)
3. Modify the code to include your newly created fields to be able to pass the data
Tom Von Deck
says:
There are so many missing instructions here, both in the article and in your reply. This isn’t even close to actionable. I’m a techie, but I’m not a php developer.
jos0405
says:
Oh yes, you need to understand basic php to be able to edit a php code properly. When you check my blog you can see a tutorials, where you just need to be a marketer, but this one is for those, who fiddle with the code.
Tom
says:
OK. I am a fiddler with code. But… there are missing steps here. For example: “create a bin at https://postb.in/.” Without any other context to work with, I go to that URL and get nothing but a generic 404 error from the browser. There’s nothing I can do on that website if it doesn’t exist. If it did exist, would there be instructions on how to create a bin? See what I mean? There’s no thorough step-by-step anything here.
Mike
says:
For Tom. A simple search for something similar to postb.in will give you https://hookbin.com. Use this to check your payload.
jos0405
says:
Hi Tom,
I’ll make some additions to this post, as there are many questions. You could use:
https://requestbin.com/
Just use the huge blue button.
Mike
says:
Hey Joey
Great article!! Thank you for posting and explaining. I too am a little lost here.
I am trying to understand where to put this script in order to receive the Webhooks. I have no problem understanding the JSON.
My example is i am able to define Shopify to send off a Webhooks to my mautic instance, how do I get the connector to interpret the data – what I mean is where do I send the Webhooks from Shopify to ? would it be my base mautic url and then I fire up the connector script on a cron or manually ?
jos0405
says:
Hello Mike, I added the schematics.
You would call a URL either on your mautic server (in a newly created folder, your connector file would be called index.php)
OR you can just use another site you use (WP site if you have) where you save this connector file.
The point is, you need to tell your third party app where the webhook should be sent to, for example:
http://mautic.mysite.com/connector/index.php
There are certain tools, that require a confirmation from your connector (“do you really exist”), which would require a bit of coding. If you have this case, hit me up, I’ll point you to the right direction.
jos0405
says:
This part is where the magic happens:
You capture the complete payload:
$data = json_decode($HTTP_RAW_POST_DATA,true);
You extract the values important for you:
$email = $data[‘billing’][’email’];
$firstname = $data[‘billing’][‘first_name’];
$lastname = $data[‘billing’][‘last_name’];
But this example is for wooCommerce. Do you have that or Shopify?
Mike
says:
Hey Joey – please delete all off the above – I made such a schoolboy error. You going to laugh, I wrote the name of my mautic address incorrectly. Spent about 2 hours playing with this for such a simple mistake.
So now i see the payload coming in, however it is not processing properly.
This is the error I am getting in mautic.error log:
2021/01/12 13:44:55 [error] 7011#7011: *5962 FastCGI sent in stderr: “PHP message: PHP Notice: Undefined variable: tag in /var/www/mautic/connector/index.php on line 33” while reading response header from upstream, client: 199.199.199.199, server: mymautic.com/connecto, request: “POST /connector/index.php HTTP/1.1”, upstream: “fastcgi://unix:/run/php/php7.3-fpm.sock:”, host: “mymautic.com/connecto”, referrer: “https://mymautic.com/connector/index.php”
I then figured out that the undefined variable was the ‘tags’ that was not defined in the JSON.
So i commented this line out and then got a permission error:
2021/01/12 14:59:16 [error] 7011#7011: *6794 FastCGI sent in stderr: “PHP message: PHP Warning: file_put_contents(/var/cache/prod/lastUpdateCheck.txt): failed to open stream: Permission denied in /app/bundles/CoreBundle/Helper/UpdateHelper.php on line 197” while reading response header from upstream, client: 99.99.99.99, server: https://mymautic.site, request: “GET /s/dashboard HTTP/1.1”, upstream: “fastcgi://unix:/run/php/php7.3-fpm.sock:”, host: “https://mymautic.site”, referrer: “https://mymautic.site”
So here i chowned the entire directory, ran everything again and got no errors but nothing has been created in Mautic.
Any pointers ?
jos0405
says:
Hello Mike,
The first error sais you have an undefined variable. What is in line 33?
Second error: your script doesn’t have the right to write that txt file. Remove it, and try again.
Mike
says:
Hey Joey.
Line 33 was the // ‘tags’ => $tag which i commented out.
I managed to get all the errors to disappear.
1. Commented out the tags variable
2. chown -R www-data:www-data on entire /mautic directory
Ran an event in wordpress, no errors in the mautic.error log but nothing happening either, no new contact created.
My problem is getting no errors now.
I also tried to add the loggin you described:
file_put_contents(“webhooklog.txt”,($email’,’.$firstname’,’.$lastname’,’.PHP_EOL),FILE_APPEND);
However on the simple test file this throughs the following error:
PHP Parse error: syntax error, unexpected ”,” (T_CONSTANT_ENCAPSED_STRING) in /root/test.php on line 34
and line 34 is the log entry you have suggested that i put at the very end of the file.
jos0405
says:
Remove the comma from the line in question, cause by commenting out the tag you are creating the error
Mike
says:
will do.
I also noticed this line: CURLOPT_URL => “https://”.$loginname.”:”.$password.”@”.$siteurl.”/api/contacts/new”,
Now i checked in my mautic directory and i do not have a directory called api.
All i did at the begging was install the /lib directory from the api. Do i need to install the entire API ?
I do not fully understand your last comment of where to remove the comma ?
When i add this line:
file_put_contents(“webhooklog.txt”,($email’,’.$firstname’,’.$lastname’,’.PHP_EOL),FILE_APPEND); Then I am getting the error.
Mike
says:
ok still driving me mad. I thought the extra code would give me some debuggin and went and researched why it is throwing the errors, so i found in order to add the logging as you have suggested I needed to declare another variable to contain the ones I am sending.
Went and did this:
$data = ($firstname, $lastname, $email, PHP_EOL)
file_put_contents(“webhooklog.txt”,implode(‘ ‘,$data).”\n”,FILE_APPEND);
So this worked and I am getting logs of what is being inputted into Mautic. This is with the basic php test file running when I am defining the variables in the file ( captain_enterprise-d@starfleet.com). But ALAS i went and changed the password to see if i get error logs and this is not the case. So logging is great to know what has been added, but still I am stuck on trying to debug this.
I am not giving up yet….
jos0405
says:
Hello Mike, based on your and other’s comments I reliased, that this type of content I’m preparing has to include more information.
I went a bit nuts with the video this came out: https://youtu.be/Cyoi6AEJTxI
Mike
says:
Hey Joey – amazing man! Thanks for the effort. I am busy following your video and I see that I do not have the api directory in my mautic/ root directory.
How do I get this ? I did download the api and followed instructions which was to copy of the /lib/ library to mautic folder.
Inside lib I do see a directory called Api with a capital “A”
I am guessing this is where I am falling flat.
Mike
says:
Joey – got it working!! Tried it on a few other of our installs including shopify and works like a dream!! Do not ask how i did it, lots of tinkering, but got it done. Thanks for this and for your youtube video!!
jos0405
says:
Hey, glad to hear that! 🙂
roman
says:
are you use https://github.com/escopecz/mautic-form-submit ?
jos0405
says:
Hello Roman!
Thx for mentioning that script. No, I don’t use it, even if it makes posting to a form easier. Api is my weapon of choice 🙂
Mike
says:
Hi Joey,
I have a question semi related to the post.
Could you explain how to extract data from the woocommerce json post in getting product name.
“line_items”: [
{
“id”: 472,
“name”: “Product1″….}
So how would we get the product name ?
I have tried the following:
$product_names = $data[‘line_items’][‘name’];
however does not seem to work.
Thanks,
Mike
jos0405
says:
Hey Mike!
I apologize, didn’t see this comment.
You need to cycle through the posed values:
First get the number of products:
$noOfProducts = sizeof($data[‘line_items’]);
Then create a cycle to get all of them:
while($noOfProducts>$i){
$productId = ($data[‘line_items’][$i][‘product_id’]);
$i++;
}
I hope that works!
Alvaro Nalda
says:
Hi Joey,
good job with the article and all your inputs.
I’m starting with Mautic and i’m a bit lost.
The goal that i have is to have follow up mailings after lead purchases.
So the steps i find are:
1. Push data to Mautic from ecommerce (Magento): I do get that using the API i can update the contact custom fields like last_order_date, last_order_products
2. Get the contact through the process of a campaign: Maybe post the data to a form campaign?
2.1. In the campaign send a mailing X days after the last_order_date and depending on the products of the order send a content or other
Doubts:
* I don’t how to solve the problem when a contact have multiple purchases and the last data is overwritten.
Do you know any good practice to perform this workaround?
Thank you in advance.
jos0405
says:
Sorry I missed this one.
1. create a segment, where last_order_date < -14 days (14 days ago) 2. launch the communication with this segment If someone shops again, you won't send them email, cause the last order date will be more recent. If that is what you want.
Aufani
says:
How to send webhook POST from mautic campaign to other apps?
The plan is.
When a contact’s in a campaign, it’ll send webhook to create account on humhub installation.
Thanks
jos0405
says:
You can use Mautic webhooks for that. Go to Settings > Webhooks
Marcos Lopes
says:
How do I pull birth data and address data?
// grab woocommerce values
$email = $data[‘billing’][’email’];
$firstname = $data[‘billing’][‘first_name’];
$lastname = $data[‘billing’][‘last_name’];
$address_1 = $data[‘billing’][‘address_1’];
$address_2 = $data[‘billing’][‘address_2’];
$city = $data[‘billing’][‘city’];
$state = $data[‘billing’][‘state’];
$country = $data[‘billing’][‘country’];
$birthdate = $data[‘billing’][‘billing_birth_date’];
$tag = ‘WooO’;
jos0405
says:
Hi, this is a woocommerce related quistion, not really Mautic 🙂
Vincent
says:
Hi Joey,
Thanks. Great article. That helped me understand more or less the concept.
I have read a couple of times, but still don’t get to know how to secure the mautic API login name and password that are written in plain text in the .php file.
Could you please explain a little more? Thank you.
Cheers.
jos0405
says:
Depends. What do you mean by secure?
Robert Lefkowitz
says:
Hi Joey,
Thanks for all of your great information and the video, but I’m still falling on my face. I’ve created your simple php file to insert Captain Pickard, but it isn’t working. I’ve put the php file in my WordPress folder, and I know that I’m calling it, because each time that I do, I see another 2 lines added to webhooklog.txt. The first line, which should contain the response from Mautic is always blank. The next line contains the contact data I was trying to add (Captain Pickard). No new contact is getting created though. I’ve tried intentionally sending a bad password, which I would expect should result in an error message, but I still have a blank line in webhooklog.txt for the Mautic response. Am I missing something?
Thanks,
Robert
jos0405
says:
Hi,
Did you clear the cache after you allowed API basic login?
Robert A Lefkowitz
says:
Yes. That didn’t help.
I ended up installing N8N to do the job.
Thank you for all the help you provide to the community.
Robert