How Cloud Cannon uses webhooks

NOTE: This is a guest post by Mike Neumegen, co-founder of Cloud Cannon, a CMS built on top of Dropbox syncing.

Using the delta Dropbox API is a great way of tracking file changes from users. Until now the best way of receiving file changes was to poll the delta API as fast as possible. With the addition of webhooks to the Dropbox API, there’s a faster way. Now you can receive instant notification when a user changes files in their Dropbox.

We’ve been using webhooks at Cloud Cannon since its beta in April and have learned how to really leverage them.

Cloud Cannon is an easy way to get websites online and updatable for clients. Our customers put static website files in Dropbox, we download them to our server and turn them into a live website with a CMS. Webhooks are the fastest and most efficient way to get these files on our servers and keep them in sync.

Setting up webhooks is easy. First, you configure a URL endpoint which Dropbox will make a POST request to every time your users update files. Now it’s just a matter of performing delta calls on those users to get the set of changes.

With webhooks, Dropbox must call a public URL so it can be difficult to test in your local development environment. We get around this by using ngrok which exposes our local web server on the internet. Unfortunately this solution doesn't work for multiple developers. We ended up making a node.js app for local development which polls the delta API and simulates a webhook call.

You don’t want anyone calling your webhook endpoint. There’s a security check you should perform to make sure it’s actually Dropbox calling the webhook. The ugly part of getting this going is the hashing algorithm. Here are a couple examples of the validation:

Ruby: 

def valid_dropbox_request?(message)
    digest = OpenSSL::Digest::SHA256.new
    signature = OpenSSL::HMAC.hexdigest(digest, APP_SECRET, message)
    request.headers['X-Dropbox-Signature'] == signature
end

Node.js:

var crypto = require('crypto');
 
function isValidRequest(message, request) {
    var signature = request.headers['x-dropbox-signature'],
        hash = crypto.createHmac('SHA256', APP_SECRET).update(message).digest('hex');
 
    return signature == hash
};

When we were using the delta polling method for updating files it was easier to predict and control the load on our servers. If the load got too high we could slow down the polling rate. With webhooks it’s much trickier to predict load. If 10,000 users start concurrently update files, the webhook endpoint could be saturated with incoming requests. We’ve reduced this risk by processing the webhook calls as fast as possible. This is important because it reduces the chance the endpoint will get overloaded. Also, Dropbox will cut the connection if it doesn’t receive a response within 10 seconds. To get this speed we perform the delta calls for users in the webhook, then put the file operations on a queue to process later.

The switch to webhooks has meant we can provide a far superior experience to our customers. Before our users would drag files into Dropbox and wait a couple of seconds for them to appear on Cloud Cannon, now it’s instant.