What is a webhook — detailed explanation with code examples

• A webhook allows an application to automatically provide data to another application in real-time, through an HTTP request.
• Instead of having to check for updates periodically, it allows one application to notify another application of events or updates in real-time.

How does this happen?

When an event occurs in the source application, it sends an HTTP POST request to a predefined URL in the target application, containing information about the event. The target application then receives and processes the information, to take appropriate actions.

Where do we use webhooks?

In many different applications, including messaging services, web applications, and e-commerce platforms, to notify other applications of events such as new messages, order updates, or user activities. They can also be used to automate processes, trigger actions, and integrate different applications.

Why is it named as a webhook?

“Web” refers to the World Wide Web (commonly and wrongly known as the internet), which is the platform over which webhooks operate.

“Hook” refers to the idea that the webhook is a way to hook or connect two applications or systems together. When an event occurs in the source application, it “hooks” into the target application by triggering a request, allowing the target application to receive and process the information.

Together, the name “webhook” describes a mechanism for connecting web-based applications or systems in a way that allows them to exchange real-time information and automate processes.

The difference between webhooks, and checking for updates periodically (an alternative method to webhooks)

Example: You have an e-commerce website that sells products online. You want to keep your customers informed about the status of their orders, such as when an order is placed, shipped, or delivered.

Method 1: Checking for updates periodically

One way to keep your customers informed is by periodically checking the order status and sending notifications to them via email or SMS. For example, you could set up a cron job to run every few minutes to check the order status in the database and send notifications if the status has changed.

This approach has a few drawbacks. It puts unnecessary load on your server since you’re making periodic requests to the database even when there are no updates. And there may be delays between when the order status changes and when the notification is sent, since you’re only checking periodically.

Method 2: Webhooks

Alternatively you could use a webhook. When an order status changes from “pending” to “shipped”, your e-commerce platform can send a request to your server with the updated information. Your server can then process the request and send a notification to the customer immediately. This is what a webhook is.

This approach has several benefits. First, it reduces the load on your server since you’re only receiving updates when there are actual changes. Second, it provides real-time updates to your customers, which can improve their experience and reduce customer support inquiries.

So the main difference between webhooks and periodic updates is that webhooks provide real-time updates, while periodic updates require periodic checks that can be less efficient and less timely. Webhooks enable systems to communicate efficiently and trigger actions automatically based on real-time events, reducing delays and improving overall system performance.

a short video on periodical checks versus webhooks

Implementing Periodical Checks (with Cronjobs) versus Implementing Webhooks in Python

Method 1: Webhooks

Let’s implement a functionality that sends an email notification to a user when they reach a certain number of points in a game. We’ll implement this functionality using a webhook and a cronjob in Python.

We’ll assume that our game platform has a webhook that sends a notification to our server when a user reaches a certain number of points.

Flask code for the “target application” that listens for incoming webhooks and sends an email notification to the user:

from flask import Flask, request
import smtplib

app = Flask(__name__)

@app.route('/webhook', methods=['POST'])
def webhook():
    data = request.json
    user_id = data['user_id']
    points = data['points']

    if points >= 100:
        send_email_notification(user_id)

    return 'OK'

def send_email_notification(user_id):
    # code to retrieve user's email address
    email = get_user_email(user_id)

    # code to send email notification
    server = smtplib.SMTP('smtp.gmail.com', 587)
    server.starttls()
    server.login('your_email@gmail.com', 'your_password')
    server.sendmail('your_email@gmail.com', email, 'Congratulations! You have reached 100 points.')
    server.quit()

def get_user_email(user_id):
    # code to retrieve user's email address from database
    return 'user_email@example.com'

if __name__ == '__main__':
    app.run()

This code listens for incoming webhooks on the /webhook endpoint. When a webhook is received, it checks if the user has reached 100 points, and if so, sends an email notification to the user using the send_email_notification() function.

Flask code for the “source application” with the webhook to send a notification to our target application on the /webhook route:

import requests
import json

data = {
    'user_id': '123',
    'points': 150
}

headers = {
    'Content-Type': 'application/json'
}

response = requests.post('http://localhost:5000/webhook', data=json.dumps(data), headers=headers)

print(response.status_code)

We’re using the requests library to send a POST request to our Flask application's /webhook route. We're passing in a JSON payload that includes the user_id and points data.

Note that in a real-world scenario, the webhook would be implemented by the game platform itself and would be configured to send notifications to our Flask application when the appropriate event occurs (in this case, when a user reaches a certain number of points). We’re simulating the webhook here by manually sending a request to our application.

Method 2: Periodical Checks using Cronjobs

We’ll create a Python script that runs periodically using a cronjob. The script will check the points for each user in the database and send an email notification if they have reached 100 points.

import smtplib
import sqlite3

def check_points():
    conn = sqlite3.connect('database.db')
    cursor = conn.cursor()

    # select all users with points >= 100
    cursor.execute('SELECT user_id, email FROM users WHERE points >= 100')
    users = cursor.fetchall()

    for user in users:
        user_id, email = user
        send_email_notification(email)

    conn.close()

def send_email_notification(email):
    # code to send email notification
    server = smtplib.SMTP('smtp.gmail.com', 587)
    server.starttls()
    server.login('your_email@gmail.com', 'your_password')
    server.sendmail('your_email@gmail.com', email, 'Congratulations! You have reached 100 points.')
    server.quit()

if __name__ == '__main__':
    check_points()

This code connects to a SQLite database and selects all users with points greater than or equal to 100. It then loops through each user and sends an email notification using the send_email_notification() function.

To run this script periodically, we can add a cronjob that runs the script every 5 minutes:

*/5 * * * * python /path/to/script.py

This will run the script every 5 minutes, checking for users who have reached 100 points and sending them email notifications if necessary.

Wait, CronJobs? How do we set that up again…

1. Create a Python script that contains the code you want to run periodically. (we did that above, in our example, the script is called script.py )
2. Make sure that any dependencies required by your Python script are installed.
3. Determine the absolute path to the script.py file on your server.
4. Open your computer’s crontab file using the command crontab -e. This will open the file in a text editor.
5. Add a new line to the crontab file that specifies the frequency and command to run your Python script. For example, to run the script every 5 minutes, you would add the following line:

*/5 * * * * python /path/to/script.py

6. Replace /path/to/script.py with the absolute path to your script.py file.

7. Save and close the crontab file. Your script will now run automatically at the specified frequency.

Note: The cronjob will only run if the server is running continuously, so it may not be suitable for certain use cases. Additionally, it’s important to ensure that your script is idempotent, meaning that running it multiple times will not cause unintended side effects or duplicate data.

What’s a crontab file in the Step 7?

A crontab file is a configuration file that contains instructions for the cron daemon on a Unix-based system. The cron daemon is a background process that runs continuously and executes tasks at specified intervals, known as cronjobs.

The crontab file specifies the frequency and commands for each cronjob. It’s a simple text file with one cronjob per line, and each line consists of six fields that specify the schedule for the command to be executed. The fields are as follows:

* * * * * command to be executed
- - - - -
| | | | |
| | | | ----- Day of the week (0 - 7) (Sunday is both 0 and 7)
| | | ------- Month (1 - 12)
| | --------- Day of the month (1 - 31)
| ----------- Hour (0 - 23)
------------- Minute (0 - 59)

Here are some examples of crontab entries:

• 0 0 * * * /usr/bin/backup.sh - runs the script /usr/bin/backup.sh every day at midnight.
• */10 * * * * /usr/bin/check-status.py - runs the Python script /usr/bin/check-status.py every 10 minutes.
• 30 6 * * 1-5 /usr/bin/report.sh - runs the script /usr/bin/report.sh at 6:30 AM every weekday.
• 0 0 1 * * /usr/bin/rotate-logs.sh - runs the script /usr/bin/rotate-logs.sh at midnight on the first day of every month.

In each of these examples, the first five fields specify the schedule for the command to be executed, and the last field is the command itself. These entries would be added to the crontab file using the crontab -e command.

Cron Daemon? How do we set that up?

1. On most Unix-based systems, the cron daemon is preinstalled. You can check if it’s installed by running the command which cron. If it’s not already installed, you can install the cron daemon yourself using the package manager for your operating system.
2. Create a crontab file for the user that will run the cronjobs (we gave a few examples for that file above). To do this, run the command crontab -e.
3. Add cronjobs to the crontab file using the format described in my previous answer. Each line should contain one cronjob.
4. Save and exit the crontab file. The cron daemon will automatically read the crontab file and execute the cronjobs at the specified intervals.

Note: Each user has their own crontab file, so if you want to run cronjobs as a different user, you will need to switch to that user’s account and create a crontab file for them. Additionally, the cron daemon will only run if the system is running continuously, so it may not be suitable for certain use cases.

That’s all for today on Webhooks!

Last summer I wanted to learn on webhooks. I Googled the term for a while and read quite a bit, however it wasn’t totally clear to me.

Recently I tried asking it to ChatGPT. I was surprised how lean ChatGPT’s explanation was. Then I decided to make a blog post on it.

Prompts and the answers are not entirely in the post, since I’ve edited the text. However, the code pieces are unedited for example.

We’re going to places with AI. Let’s see what happens.

-cover image for the post-