Cloud Import Data Preparation Guide

This document specifies the file format expected by Mixpanel’s Cloud Import for loading data into Mixpanel. Cloud Import is capable of transferring the following models into Mixpanel:

Events track meaningful actions that a user performs in the application. Each event is timestamped and immutable by default. A profile is a collection of information about a user entity. They are mutable and constantly change to reflect the most recent information about a user.

File Format

Cloud Import expects all the different data models to be structured in NDJSON format. This format allows us to import complex properties (such as dictionaries), and allows the data model to remain sparse.

Event

Each event is represented as a new line as a JSON object. The format is similar to our Ingestion API request format with few changes. The event should have the following attributes.

event: Name for the event (UTF-8 string)
properties: JSON dictionary representing a collection of properties for the event.

The properties are a key-value JSON object and can contain any valid UTF-8 string as a name and a valid JSON value. Cloud Import expects the following special properties in an event JSON.

  • time
    The time an event occurred. The value should be a UNIX seconds timestamp (seconds since midnight, Jan 1, 1970, UTC). Cloud Import will discard events without time.

  • distinct_id
    The value of distinct_id is a string and it uniquely identifies the user. It can be 255 characters in length.

  • $insert_id
    The value of $insert_id is 36 character case-sensitive alphanumeric characters (with -) that uniquely identifies the event. This insert_id is used to deduplicate events within the distributed system of mixpanel. It also enables the customer to send a new version of the event if an event is modified.

Deduplication

When multiple events are sent with the same event name, distinct_id, insert_id, and date (based on UNIX time above). Mixpanel will choose the most recent event (based on upload time) that is imported into Mixpanel. For overwriting, events should have the same $insert_id, distinct_id, event name and UTC day. If any value is different we will create a new event and not overwrite.

Event Deletion

We don’t have a Cloud Import path for deleting events yet. Please get in touch through email for custom delete scenarios.

Example Event

{
    "event": "Ring Acquired",
    "properties": {
        "distinct_id": "Sauron",
        "$insert_id": "ae4094aa-2e7b",
        "time": 1585744507,
        "location": "Mount Doom"
    }
}

People Profiles

Each user profile is represented in a new line in NDJSON. They have the following attributes.

  • $distinct_id
    The value of $distinct_id is a string and it uniquely identifies the user. It can be up to 255 characters in length.

  • Any of the user profile operations in Update Operations

Example People Profile

{
    "$distinct_id": "Sméagol",
    "$set": {
        "race": "Hobbit",
        "alilas": "Gollum",
        "name": "Sméagol",
        "email": "[email protected]"
    }
}

Sample Data

{"event": "Birth", "properties": {"distinct_id": "Isildur", "time": 1601427723}}
{"event": "Birth", "properties": {"distinct_id": "Sméagol", "time": 1601522222}}
{"event": "Birth", "properties": {"distinct_id": "Bilbo", "time": 1601629423, "location": "Shire"}}
{"event": "Birth", "properties": {"distinct_id": "Frodo", "time": 1601706584, "location": "Shire"}}
{"event": "Death", "properties": {"distinct_id": "Sauron", "time": 1601431446, "location": "Mount Doom", "cause": "homicide"}}
{"event": "Death", "properties": {"distinct_id": "Isildur", "time": 1601384521, "location": "Gladden Fields", "cause": "homicide"}}
{"event": "Death", "properties": {"distinct_id": "Sméagol", "time": 1601814444, "location": "Mount Doom", "cause": "accident"}}
{"event": "Ring Acquired", "properties": {"distinct_id": "Sauron", "time": 1601251200, "location": "Mount Doom"}}
{"event": "Ring Acquired", "properties": {"distinct_id": "Isildur", "time": 1601431446, "location": "Mount Doom", "previous_ring_bearer": "Sauron"}}
{"event": "Ring Acquired", "properties": {"distinct_id": "Sméagol", "time": 1601559543, "location": "Gladden Fields", "previous_ring_bearer": "Déagol"}}
{"event": "Ring Acquired", "properties": {"distinct_id": "Bilbo", "time": 1601661989, "location": "Misty Mountains", "previous_ring_bearer": "Sméagol"}}
{"event": "Ring Acquired", "properties": {"distinct_id": "Frodo", "time": 1601720421, "location": "Shire", "previous_ring_bearer": "Bilbo"}}
{"event": "Ring Acquired", "properties": {"distinct_id": "Sméagol", "time": 1601813532, "location": "Mount Doom", "previous_ring_bearer": "Frodo"}}
{"$distinct_id": "Sauron", "$set": {"race": "Ainur", "kingdom":"Mordor", "name": "Sauron", "email": "[email protected]"}}
{"$distinct_id": "Isildur", "$set": {"race": "Men", "kingdom": "Gondor", "name": "Isildur", "email": "[email protected]"}}
{"$distinct_id": "Sméagol", "$set": {"race": "Hobbit", "alias": "Gollum", "name": "Sméagol", "email": "[email protected]"}}
{"$distinct_id": "Bilbo", "$set": {"race": "Hobbit", "family": "Baggins", "name": "Bilbo Baggins", "email":"[email protected]"}}
{"$distinct_id": "Frodo", "$set": {"race": "Hobbit", "family": "Baggins", "name": "Frodo Baggins", "email":"[email protected]"}}

Updated 2 days ago

Cloud Import Data Preparation Guide


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.