Developing with Lytics / Lytics API Documentation

Catalog

Catalog exposes info about the schema of the data in Lytics.

  • Entities are Object Types, the primary one being a User although most accounts also have Content and Campaign as well. Custom object types may be created.

  • Streams Streams are data sources collected from api or Integrations (aka Work).

  • Schema Per entity type is list of field-types. Think of it as describing columns on the user table in a database.

Streams

/api/schema/_streams

List of Streams (raw data sources) and which fields they contain.

This shows information like name, data-type (introspected), first-seen, cardinality (number of unique values).

This data is created using probablistic data-structures and also is sampled after 40m events seen per stream. So it may not be exactly correct for cardinalty or unique values after 40 million.

# Example of getting the streams and field's for an account and saving to file
curl -s "$LIOAPI/api/schema/_streams?access_token=$LIOKEY" > /tmp/streams.json

# create a list of stream field names for a stream
jq '.data[] | select(.stream == "cm_activity") | [.fields[].name] | join(",") ' < /tmp/streams.json
GET

Streams

/api/schema/_streams

Get list of streams, field info they contain.

Response 200

Headers
Content-Type: application/json
Body
{
    "data":[
        {
            "stream":"events",
            "hidden": false,
            "ct": 37,
            "curct": 0,
            "last_msg_ts": "1401216931893",
            "last_update_ts": "1421167622683",
            "fields":[
                {
                    "avg": 4,
                    "card": 22,
                    "is_array": true,
                    "name": "utm_medium",
                    "sample": [
                        "cpc",
                        "adwords",
                        "blog",
                        "pricing-page",
                        "social",
                        "integrations-page",
                        "email",
                        "Integrations-page",
                        "docs",
                        "search",
                        "CPC",
                        "twitter",
                        "social_organic",
                        "card",
                        "social_paid",
                        "sponsored",
                        "mailchimp",
                        "facebook",
                        "infographic",
                        "lytics-reporting",
                        "retargeting",
                        "referral"
                    ],
                    "smart": {
                        "coverage": 0,
                        "email": 0,
                        "noun": 13,
                        "time": 0,
                        "uid": 0,
                        "value": 0,
                        "verb": 13,
                        "ip": 0,
                        "phone": 0,
                        "url": 0,
                        "analyses": 315
                    },
                    "sumlen": 146221,
                    "type": "string",
                    "usage": 32484,
                    "first_seen": "2016-06-08T16:11:06Z",
                    "last_seen": "2016-06-24T18:00:40.871025224Z",
                    "hidden": false,
                    "column": [
                        {"table": "campaign", "as": "utm_mediums"},
                        {"table": "user", "as": "utm_mediums"}
                    ]
                }
            ]
        }
    ]
}

Stream

/api/schema/_streams/{stream}

Get Single Stream by stream name.

Parameters
streamstring (optional)
Ex: web_data
Name of the stream
POST

Stream Hide

/api/schema/_streams/{stream,hidden}

The POST endpoint allows Stream to be hidden/un-hidden. A hidden stream is not visible from the Application Data > Streams Web UI.

CAVEAT's

  • This does not delete the data, only hides it from ui.

  • If queries (LQL) still map this data into profiles it will still be added.
# Hide a temporary stream
curl -XPOST 'https://api.lytics.io/api/schema/_streams' \
    -H 'Content-type: application/json' \
    -H "Authorization: $LIOKEY" \
    -d '{
        "stream": "my_temp_upload",
        "hidden": true
     }' | jq '.'
Parameters
streamstring (required)
Ex: my_temp_upload
Name of the stream To Delete
hiddenboolean (required)
Ex: true
hide stream from Web UI

Request

Headers
Authorization: {API Token}

Response 204

POST

Field Hide

/api/schema/_streams/{stream,name,hidden}

This POST endpoint allows specific fields belong to a stream to be hidden/un-hidden. A hidden field is not visible from the Application Data > Streams Web UI.

CAVEAT'S

  • This does not delete the data, only hides the field in the UI.

  • If queries (LQL) stil map this field into profiles, it will still be added
# Hide the 'first-name' field from the 'default' stream
curl -XPUT 'https://api.lytics.io/api/schema/_streams/default' \
    -H 'Content-type: application/json'
    -H 'Authorization: $LIOKEY' \
    -d '{
        "name": "first_name",
        "hidden": true
    }' | jq '.'
Parameters
streamstring (required)
Ex: default
Name of the stream containing the field to delete
namestring (required)
Ex: first_name
Name of the field to delete
hiddenboolean (required)
Ex: true
hide field from Web UI

Response 204

Schema

/api/schema/{table}

Field level info for entity/table (usually a user). but you can try content as well.

# Get the schema for the user table
curl -s -XGET 'https://api.lytics.io/api/schema/user' \
    -H 'Content-type: application/json' \
    -H "Authorization: $LIOKEY"  | jq '.'
Parameters
tablestring (required)
Ex: user
Name
GET

Schema Fields

/api/schema/{table}

Get Field information about a NodeType(user). Eventually, we will extend beyond user.

Response 200

Headers
Content-Type: application/json
Body
{
    "data": {
        "fields": [
              {
                "as": "created_ts",
                "is_by": false,
                "type": "date",
                "shortdesc": "Created Date",
                "longdesc": "First time usage for this user",
                "froms": [
                  "default"
                ],
                "identities": [
                  "min"
                ]
              },
              {
                "as": "content",
                "is_by": false,
                "type": "string",
                "shortdesc": "Site Content",
                "longdesc": "Specific piece of content viewed",
                "froms": [
                  "default"
                ],
                "identities": [
                  "content"
                ]
              },
              {
                "as": "location",
                "is_by": false,
                "type": "string",
                "shortdesc": "Location",
                "longdesc": "",
                "froms": [
                  "linkedin_data"
                ],
                "identities": [
                  "location"
                ]
              }
        ]
     }
}

FieldInfo

/api/schema/{table}/fieldinfo{?limit,fields}

Estimates of field level info for table user segment fields.

Parameters
tablestring (required)
Ex: user
Table name (generally "user")
limitinteger (optional)
Ex: 20
number of unique values to return per field
fields[]string (optional)
Ex: name,email,id
list, or comma separated list of string fields to include
GET

FieldInfo For Table

/api/schema/{table}/fieldinfo{?limit,fields}

Get Field level info about a NodeType(user). Contains the possible values contained in each field.

Response 200

Headers
Content-Type: application/json
Body
{
    "data": {
        "fields":[
              {
                "field": "region",
                "terms_counts": {
                  "02": 704,
                  "07": 857,
                  "AZ": 671,
                  "CA": 18064,
                  "CO": 654,
                  "FL": 1537,
                  "GA": 822,
                  "H9": 758,
                  "IL": 1254,
                  "KS": 1104,
                  "MA": 1301,
                  "NC": 626,
                  "NJ": 892,
                  "NY": 4035,
                  "ON": 1097,
                  "OR": 12158,
                  "PA": 1040,
                  "TX": 2726,
                  "VA": 1894,
                  "WA": 1948
                },
                "more_terms": true,
                "ents_present": 70489,
                "ents_absent": 65796,
                "high_card": false,
                "approx_cardinality": 331
              },
              {
                "field": "utm_mediums",
                "terms_counts": {
                  "Integrations-page": 1102,
                  "adwords": 4434,
                  "banner": 1496,
                  "blog": 3413,
                  "card": 10,
                  "cpc": 2923,
                  "email": 6051,
                  "ppc": 220,
                  "search": 8762,
                  "social": 12513,
                  "parnter": 2314
                },
                "more_terms": true,
                "ents_present": 29595,
                "ents_absent": 116690,
                "high_card": false,
                "approx_cardinality": 27
              },
              {
                "field": "zd_yymm",
                "terms_counts": {
                  "1409": 5,
                  "1410": 12,
                  "1411": 4,
                  "1412": 4,
                  "1501": 3,
                  "1502": 10,
                  "1503": 26,
                  "1504": 20,
                  "1505": 3
                },
                "more_terms": false,
                "ents_present": 72,
                "ents_absent": 136213,
                "high_card": false,
                "approx_cardinality": 9
              },
              {
                "field": "openct",
                "terms_counts": {
                  "1": 75,
                  "10": 18,
                  "108": 3,
                  "11": 7,
                  "12": 5,
                  "152": 3,
                  "19": 8,
                  "2": 42,
                  "20": 15,
                  "24": 3,
                  "26": 10,
                  "3": 11,
                  "32": 3,
                  "35": 3,
                  "4": 25,
                  "5": 16,
                  "51": 15,
                  "6": 17,
                  "8": 5,
                  "9": 11
                },
                "more_terms": true,
                "ents_present": 328,
                "ents_absent": 135957,
                "high_card": false,
                "approx_cardinality": 98,
                "stats": {
                  "mean": 15.621951219512194,
                  "sd": 27.978602557504388,
                  "min": 1,
                  "max": 264,
                  "n": 328
                }
              },
              {
                "field": "mc_lastopen_ts",
                "terms_counts": {},
                "more_terms": false,
                "histograms": [
                  {
                    "data": {
                      "1446336000000": 2,
                      "1446595200000": 2,
                      "1447891200000": 333,
                      "1448150400000": 348,
                      "1448409600000": 9,
                      "1448668800000": 8
                    },
                    "start": "1446401936511",
                    "end": "1448993936511",
                    "interval": "3d"
                  },
                  {
                    "data": {
                      "1442188800000": 35,
                      "1442793600000": 3,
                      "1445212800000": 71,
                      "1445817600000": 2,
                      "1446422400000": 4,
                      "1447632000000": 340,
                      "1448236800000": 351,
                      "1448841600000": 7
                    },
                    "start": "1441131536511",
                    "end": "1448993936511",
                    "interval": "week"
                  },
                  {
                    "data": {
                      "1417392000000": 3,
                      "1420070400000": 2,
                      "1425168000000": 2,
                      "1427846400000": 43,
                      "1430438400000": 5,
                      "1433116800000": 24,
                      "1435708800000": 41,
                      "1438387200000": 48,
                      "1441065600000": 38,
                      "1443657600000": 73,
                      "1446336000000": 702
                    },
                    "start": "1417457936511",
                    "end": "1448993936511",
                    "interval": "month"
                  }
                ],
                "ents_present": 1377,
                "ents_absent": 134908,
                "high_card": false,
                "approx_cardinality": 1212
              }

        ]
     }
}

FieldSuggest

/api/schema/{table}/fieldsuggest/{field}

Suggest Values for a field for auto-complete in webui

Parameters
tablestring (required)
Ex: user
Table name (generally `user`)
fieldstring (required)
Ex: email
field name to do lookup on.
GET

FieldSuggest

/api/schema/{table}/fieldsuggest/{field}

Suggest Values for a field for auto-complete in webui

Response 200

Headers
Content-Type: application/json
Body
{
    "data": {
        "fields": :[
          {
            "name": "_cc",
            "sample": ["US"],
            "type": "string",
            "usage": 4
          }
        ]
     }
}