Converting the Dropbox Activity Log into Common Event Format Events

In Dropbox Business accounts key events and actions are recorded and stored in an activity log.

This log records activities such as a modifying a file, an admin changing policies, or a new user being added to the team. Accessible via the admin console or Dropbox Business APIs, this log can serve as an authoritative source to audit and monitor activity within a team. Each individual event of the log contains contextual information and metadata to help sort and categorize events.

The activity log is a great source of data for security and event management (SIEM), or data-loss prevention (DLP) applications. These types of applications are geared to identify security issues, policy violations, and provide analytics on event data. 

Specifically, SIEM applications usually require aggregation of events from a range of different devices and inputs. Because of this it can be difficult to consolidate information, and complex to integrate events from all different sources. Targeting this problem, cyber security and SIEM analytics provider, MicroFocus ArcSight developed the Common Event Format (CEF). The format’s goal is to promote interoperability between log generating sources by establishing a standard log and audit message format for SIEM applications. 

Converting events in Dropbox to the CEF standard can make information easier for SIEM apps to work with. In this blog post we’ll explore:

1. How to pull events from the Dropbox activity log
2. A high-level overview of the common event format
3. Mapping the activity log to the common event format
4. Review and use a script to parse the activity log into the common event format

Exclusive to the Dropbox Business API, team events can be pulled via the activity log endpoints (team_log/get_events and team_log/get_events/continue). Before interacting with these endpoints you’ll need an access token with the appropriate permissions. If you don’t already have one, create a Dropbox app with one of the following access types: “Team Auditing” or “Team Member File Access”.

Calling the activity log

When using the Dropbox API to interact with the activity log endpoints there are four important parameters to consider: limit, account_id, time, or category. While these parameters are optional, they play an essential role in controlling the results of the activity log. If no parameters are used, the endpoints will return the full list which may be bulky to work with. You can review the endpoint documentation here for the details on what these parameters do.

Below are a couple cURL of examples showing how you might use these parameters.

Objective: Return 10 events, filtered on file operation events

Objective: Return all events for account id dbid:AACaXNrKenjisuUqxju9ppCK5B4J1rknytk from a specific date range

The response of the team_log endpoints is made up of a list of entries under the event field, a cursor, and has_more flag. Each entry in the event field contains information such as the event_type, actor, origin, or assets involved. An important note is that each event entry will always have the timestamp, event_category, event_type, and details fields available. However, depending on the type of event, information like origin and assets may not be available.

If the has_more flag is true you’ll also be returned a cursor used to paginate through additional events that weren’t displayed in the initial call. The cursor is passed to the team_log/get_events/continue to return the next page of activity. The cursor is a string that can be saved and used to poll for new events. However, cursors do expire (usually valid for a few days) and should be updated with every call to the team_log endpoints. Be prepared to handle reset exceptions if the cursor expires. You’ll want to store the cursor somewhere when you’re paginating through events, and when you want to poll for new events based on the original event query.

We’ll cover a simplified overview of the CEF here, but please refer to Micro Focus ArcSight’s official documentation for more detailed descriptions of the format and guidelines.

At a high-level the CEF format defines a syntax for standardizing logs from multiple sources. The format is broken up into two distinct parts: headers which define important metadata and the extension, which is a collection of key value pairs. Each individual header is separated by a pipe operation, “|”,  with the final section containing the extension.

Once compiled, events are sent to a designated server via syslog, a protocol used to send logs to a central storage place. You can read more about the syslog protocol here. The full CEF event sent to a syslog server would look similar to the to the one below:

CEF:Version|Device Vendor|Device Product|Device Version|Device Event Class ID|Name|Severity|[Extension]

The most important headers to consider are Device Event Class ID, Name and Severity.  These headers are variable and provide information on the particular event. The remaining headers are static values that detail information about ArcSight and Dropbox.

Device Event Class ID: This maps to unique identifiers per event-type.

Name: A human readable description of the event that occurred.

Severity: A string or integer reflecting the importance of the event. For our use we’ll be using the integer representation. CEF defines the integer severity range as: 0-3 = Low, 4-6 = Medium, 7-8 = High, and 9-10 = Very-High

The extensions at the end of the header are specific keys that ArcSight has defined that map to certain data types. For example, the key src maps directly to IPv4 addresses. This would be represented as src=10.0.0.1. Multiple extensions are separated by spaces, and any event can have any number of key-value pairs in any order.

Let’s pull all of this together into a sample Dropbox team event in the CEF format:

CEF:0|Dropbox|Dropbox Audit Log|1|logout|Signed out|6|duser=example@dropbox.com duid=dbmid:AAACCXX cat=logins rt=Feb 24 2016 22:45:43 end=Feb 24 2016 22:45:43 src=10.10.0.111

As you can see the first three headers detail the CEF version and information about Dropbox. Device Vendor and Device Product would be different based on the specific logging device.

Now that we’ve covered how to work with the Dropbox activity log and the common event format, let’s take a look at a mapping of event details from Dropbox to CEF key names.

Not all information for an event is mapped to CEF key names, as the activity log provides details that do not readily map back to CEF. These CEF key names are sourced from MicroFocus’ documentation. 

As we mentioned earlier, Severity is an important header that is assigned to an event. Based on the perceived impact a specific event has on a team, we assign each event a severity according to what category the event falls under. These severity ranks were determined based on the impact they might have on a team. For example, someone commenting on a document may be low impact and severity, but changing two-factor authentication settings or team policies would have a high security impact and severity. These mappings can be changed to match the definitions of severity unique to every organization.

Let’s combine our new knowledge to create a script that will pull events from the activity log, convert it into the CEF format and send those events to a designated syslog server when you pass in a host and a port.

Note: The script is compatible with Python versions 3.0+. You’ll also need a Dropbox Business access token with either the “Team Auditing” or “Team Member File Access” permissions.

At the most basic level all you need to run the script is your access token, but you can adjust the events pulled with optional arguments that correspond to the activity log endpoint parameters. You can access the files for the script here.

To get started from the provided files open up the “parserSettings.ini” and paste your Dropbox Business access token next to dfbToken=. Since we are hardcoding an access token here, if you are using version control be mindful about storing this value, especially if using a public repository. In those situations, or in a production environment, consider using the dotenv library along with a .gitignore file to create local environment variables. Additionally if you need to adjust and read in settings the “parserSettings.ini” file can be a good place to set this, for example with the severity mapping.

Below are a few examples that use optional arguments with the provided “cefParser.py” script. Remember to execute the samples from the same directory the script is located in.

Objective: Generate a CSV of CEF events for all single sign on events:

python cefParser.py --output --category sso

In this example passing in the argument --output will generate a .CSV file of the events pulled. --category or -c designates you want to filter on an event category and sso (single sign on) is the category filter.

Objective: Send all category of events to designated syslog server

python cefParser.py --host 121.121.01.10 --port 1234

Here you’re sending all events to the host and port you’ve designated, by using the --host and --port arguments.

Objective: Send all CEF events from 2018 up to 2019 to designated syslog server

python cefParser.py --host 121.121.01.10 --port 1234 --start_time 2018-01-01T00:00:00Z --end_time 2019-01-01T00:00:00Z

Using the --start_time and --end_time arguments you can filter events from a certain time period.

To see how to use the full list of arguments and the details of each you can run python cefParser.py --help.

When running the script it will show updates in terminal when it is pulling events, writing them to a CSV or sending them to the syslog server. As part of the final output the script will return the event cursor that can be used for any subsequent calls or polling.

Here’s an example of a raw event you would get from the Dropbox event log:

Here’s what that same event looks like after being converted into CEF:

CEF:0|Dropbox|Dropbox Activity Log|1|file_add|Added files and/or folders|3|duser=ti@dbxexamples.com duid=dbmid:AADOKK2wbsDF52k9LWixze38RY1tNFSRvCI cat=file_operations rt=Dec 09 2016 17:51:38 end=Dec 09 2016 17:51:38 cs1={'.tag': 'file_add_details'}, cs1Label=Details of event cs2={'.tag': 'user', 'user': {'.tag': 'team_member', 'account_id': 'dbid:AAXCPeiWpyqDu7DFMevhIJXQGLGGAXOZ8EvU', 'display_name': 'Tahsin Islam', 'email': 'ti@dbxexamples.com', 'team_member_id': 'dbmid:AADOKK2wbsDF52k9LWixze38RY1tNFSRvCI'}} cs2Label=Details of event actor cs3={'geo_location': {'city': 'Unknown', 'region': 'Unknown', 'country': 'Unknown', 'ip_address': '172.21.101.82'}, 'access_method': {'.tag': 'end_user', 'end_user': {'.tag': 'web', 'session_id': 'dbwsid:1'}}} cs3Label=Details of the event origin src=172.21.101.82 cs4=[{'.tag': 'folder', 'path': {'contextual': '/Share145', 'namespace_relative': {'ns_id': '3159137245'}}, 'file_id': 'id:jDYRsZKZImAAAAAAAAAAZg'}] cs4Label=Details of event assets

The Dropbox activity log is a robust, powerful tool that exposes information about events that occur in Dropbox. By converting those events into the CEF standard, the information becomes easier for SIEM applications to work with. This article included links to a repo with a sample script to convert events from the Dropbox activity log into the CEF standard. If you want to learn more about the CEF standard format you can explore the documentation here.

For more Dropbox tutorials and technical guides, check out our previous blog posts or our reference page. If you need technical support for the Dropbox API, then you can use the support request form, post on our developer forum, or @ us on Twitter.

Build with Dropbox today at www.dropbox.com/developers