[BETA] Cloud Import Data Preparation Guide

🚧

Cloud Import is currently in beta

Cloud Import is currently in closed beta and not adding new projects. Please reach out to [email protected] with questions

This document specifies the file format expected by Mixpanel’s Cloud Import for loading data into Mixpanel. Cloud Import is capable of transferring the following models into Mixpanel:

Events track meaningful actions that a user performs in the application. Each event is timestamped and immutable by default. A profile is a collection of information about a user entity. They are mutable and constantly change to reflect the most recent information about a user.

File Format

Cloud Import expects all the different data models to be structured in NDJSON format. This format allows us to import complex properties (such as dictionaries), and allows the data model to remain sparse.

Event

Each event is represented as a new line as a JSON object. The format is similar to our Ingestion API request format with few changes. The event should have the following attributes.

event: Name for the event (UTF-8 string, required))
properties: JSON dictionary representing a collection of properties for the event.

The properties are a key-value JSON object and can contain any valid UTF-8 string as a name and a valid JSON value. Cloud Import expects the following special properties in an event JSON.

  • time (required)
    The time an event occurred. The value should be a UNIX seconds timestamp (seconds since midnight, Jan 1, 1970, UTC). Cloud Import will discard events without time.

  • distinct_id (optional)
    The value of distinct_id is a string and it uniquely identifies the user. It can be up to 255 characters in length. While distinct_id is not required for an event to be tracked to Mixpanel, we strongly recommend using this unique identifier for each user as part of an event. This is necessary for all unique user-based analysis (reports such as funnels, flows and retention all require distinct_id). Put in other words, unless you are unable to attribute a given event data to a single user (e.g. you’re tracking a daily summary event for your entire user base), you should assign distinct_ids to events.

  • $insert_id (optional)
    The value of $insert_id is 36 character case-sensitive alphanumeric characters (including -) that uniquely identifies the event. $insert_id is used to deduplicate events within Mixpanel. It also enables the customer to send a new version of the event if an event is modified. If the value of $insert_id is less than 36 characters, it will be used as-is; if it is greater, it will be truncated to 36 characters. For events without $insert_id or values containing illegal characters, A random UUID will be generated for that event.

Deduplication

When multiple events are sent with the same event name, distinct_id, insert_id, and date (based on UNIX time above). Mixpanel will choose the most recent event (based on upload time) that is imported into Mixpanel. For overwriting, events should have the same $insert_id, distinct_id, event name and UTC day. If any value is different we will create a new event and not overwrite.

Event Deletion

We don’t have a Cloud Import path for deleting events yet. Please get in touch through email for custom delete scenarios.

Example Event

{
    "event": "Ring Acquired",
    "properties": {
        "distinct_id": "Sauron",
        "$insert_id": "ae4094aa-2e7b",
        "time": 1585744507,
        "location": "Mount Doom"
    }
}

People Profiles

Each user profile is represented in a new line in NDJSON. They have the following attributes.

  • $distinct_id
    The value of $distinct_id is a string and it uniquely identifies the user. It can be up to 255 characters in length.

  • Any of the user profile operations in Update Operations

Example People Profile

{
    "$distinct_id": "Sméagol",
    "$set": {
        "race": "Hobbit",
        "alilas": "Gollum",
        "name": "Sméagol",
        "email": "[email protected]"
    }
}

Sample Data

{"event": "Birth", "properties": {"distinct_id": "Isildur", "time": 1601427723}}
{"event": "Birth", "properties": {"distinct_id": "Sméagol", "time": 1601522222}}
{"event": "Birth", "properties": {"distinct_id": "Bilbo", "time": 1601629423, "location": "Shire"}}
{"event": "Birth", "properties": {"distinct_id": "Frodo", "time": 1601706584, "location": "Shire"}}
{"event": "Death", "properties": {"distinct_id": "Sauron", "time": 1601431446, "location": "Mount Doom", "cause": "homicide"}}
{"event": "Death", "properties": {"distinct_id": "Isildur", "time": 1601384521, "location": "Gladden Fields", "cause": "homicide"}}
{"event": "Death", "properties": {"distinct_id": "Sméagol", "time": 1601814444, "location": "Mount Doom", "cause": "accident"}}
{"event": "Ring Acquired", "properties": {"distinct_id": "Sauron", "time": 1601251200, "location": "Mount Doom"}}
{"event": "Ring Acquired", "properties": {"distinct_id": "Isildur", "time": 1601431446, "location": "Mount Doom", "previous_ring_bearer": "Sauron"}}
{"event": "Ring Acquired", "properties": {"distinct_id": "Sméagol", "time": 1601559543, "location": "Gladden Fields", "previous_ring_bearer": "Déagol"}}
{"event": "Ring Acquired", "properties": {"distinct_id": "Bilbo", "time": 1601661989, "location": "Misty Mountains", "previous_ring_bearer": "Sméagol"}}
{"event": "Ring Acquired", "properties": {"distinct_id": "Frodo", "time": 1601720421, "location": "Shire", "previous_ring_bearer": "Bilbo"}}
{"event": "Ring Acquired", "properties": {"distinct_id": "Sméagol", "time": 1601813532, "location": "Mount Doom", "previous_ring_bearer": "Frodo"}}
{"$distinct_id": "Sauron", "$set": {"race": "Ainur", "kingdom":"Mordor", "name": "Sauron", "email": "[email protected]"}}
{"$distinct_id": "Isildur", "$set": {"race": "Men", "kingdom": "Gondor", "name": "Isildur", "email": "[email protected]"}}
{"$distinct_id": "Sméagol", "$set": {"race": "Hobbit", "alias": "Gollum", "name": "Sméagol", "email": "[email protected]"}}
{"$distinct_id": "Bilbo", "$set": {"race": "Hobbit", "family": "Baggins", "name": "Bilbo Baggins", "email":"[email protected]"}}
{"$distinct_id": "Frodo", "$set": {"race": "Hobbit", "family": "Baggins", "name": "Frodo Baggins", "email":"[email protected]"}}

Updated 5 days ago


[BETA] Cloud Import Data Preparation Guide


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.