Catalog
Catalog exposes info about the schema of the data in Lytics.
Entities are Object Types, the primary one being a User although most accounts also have Content and Campaign as well. Custom object types may be created.
Streams Streams are data sources collected from api or Integrations (aka Work).
Schema Per entity type is list of field-types. Think of it as describing columns on the user table in a database.
Streams
List of Streams (raw data sources) and which fields they contain.
This shows information like name, data-type (introspected), first-seen, cardinality (number of unique values).
This data is created using probablistic data-structures and also is sampled after 40m events seen per stream. So it may not be exactly correct for cardinalty or unique values after 40 million.
# Example of getting the streams and field's for an account and saving to file
curl -s "$LIOAPI/api/schema/_streams?access_token=$LIOKEY" > /tmp/streams.json
# create a list of stream field names for a stream
jq '.data[] | select(.stream == "cm_activity") | [.fields[].name] | join(",") ' < /tmp/streams.json| Parameters | ||
|---|---|---|
| streams | undefined (required) Ex: comma separated list of streams to return. If empty, the response returns all streams. | |
Streams
Get all the streams information
Response 200
Headers
Content-Type: application/json
Body
{
"data":[
{
"stream":"events",
"hidden": false,
"ct": 37,
"curct": 0,
"last_msg_ts": "1401216931893",
"last_update_ts": "1421167622683",
"fields":[
{
"avg": 4,
"card": 22,
"is_array": true,
"name": "utm_medium",
"sample": [
"cpc",
"adwords",
"blog",
"pricing-page",
"social",
"integrations-page",
"email",
"Integrations-page",
"docs",
"search",
"CPC",
"twitter",
"social_organic",
"card",
"social_paid",
"sponsored",
"mailchimp",
"facebook",
"infographic",
"lytics-reporting",
"retargeting",
"referral"
],
"smart": {
"coverage": 0,
"email": 0,
"noun": 13,
"time": 0,
"uid": 0,
"value": 0,
"verb": 13,
"ip": 0,
"phone": 0,
"url": 0,
"analyses": 315
},
"sumlen": 146221,
"type": "string",
"usage": 32484,
"first_seen": "2016-06-08T16:11:06Z",
"last_seen": "2016-06-24T18:00:40.871025224Z",
"hidden": false,
"column": [
{"table": "campaign", "as": "utm_mediums"},
{"table": "user", "as": "utm_mediums"}
]
}
]
}
]
}
Stream
Update a stream by stream name.
| Parameters | ||
|---|---|---|
| stream | string (optional) Ex: web_data | Name of the stream |
Stream Hide
The POST endpoint allows Stream to be hidden/un-hidden. A hidden stream is not visible from the Application Data > Streams Web UI.
CAVEAT's
This does not delete the data, only hides it from ui.
If queries (LQL) still map this data into profiles it will still be added.
# Hide a temporary stream
curl -XPOST 'https://api.lytics.io/api/schema/_streams' \
-H 'Content-type: application/json' \
-H "Authorization: $LIOKEY" \
-d '{
"stream": "my_temp_upload",
"hidden": true
}' | jq '.'| Parameters | ||
|---|---|---|
| stream | string (required) Ex: my_temp_upload | Name of the stream To Delete |
| hidden | boolean (required) Ex: true | hide stream from Web UI |
Request
Headers
Authorization: {API Token}
Response 204
Field Hide
This POST endpoint allows specific fields belong to a stream to be hidden/un-hidden. A hidden field is not visible from the Application Data > Streams Web UI.
CAVEAT'S
This does not delete the data, only hides the field in the UI.
If queries (LQL) stil map this field into profiles, it will still be added
# Hide the 'first-name' field from the 'default' stream
curl -XPUT 'https://api.lytics.io/api/schema/_streams/default' \
-H 'Content-type: application/json'
-H "Authorization: $LIOKEY" \
-d '{
"name": "first_name",
"hidden": true
}' | jq '.'| Parameters | ||
|---|---|---|
| stream | string (required) Ex: default | Name of the stream containing the field to delete |
| name | string (required) Ex: first_name | Name of the field to delete |
| hidden | boolean (required) Ex: true | hide field from Web UI |
Response 204
Schema
Field level info for entity/table (usually a user). but you can try content as well.
# Get the schema for the user table
curl -s -XGET 'https://api.lytics.io/api/schema/user' \
-H 'Content-type: application/json' \
-H "Authorization: $LIOKEY" | jq '.'| Parameters | ||
|---|---|---|
| table | string (required) Ex: user | Name |
Schema Fields
Get Field information about a NodeType(user). Eventually, we will extend beyond user.
Response 200
Headers
Content-Type: application/json
Body
{
"data": {
"fields": [
{
"as": "created_ts",
"is_by": false,
"type": "date",
"shortdesc": "Created Date",
"longdesc": "First time usage for this user",
"froms": [
"default"
],
"identities": [
"min"
]
},
{
"as": "content",
"is_by": false,
"type": "string",
"shortdesc": "Site Content",
"longdesc": "Specific piece of content viewed",
"froms": [
"default"
],
"identities": [
"content"
]
},
{
"as": "location",
"is_by": false,
"type": "string",
"shortdesc": "Location",
"longdesc": "",
"froms": [
"linkedin_data"
],
"identities": [
"location"
]
}
]
}
}
FieldInfo
Estimates of field level info for table user segment fields.
| Parameters | ||
|---|---|---|
| table | string (required) Ex: user | Table name (generally "user") |
| limit | integer (optional) Ex: 20 | number of unique values to return per field |
| fields | []string (optional) Ex: name,email,id | list, or comma separated list of string fields to include |
FieldInfo For Table
Get Field level info about a NodeType(user). Contains the possible values contained in each field.
Response 200
Headers
Content-Type: application/json
Body
{
"data": {
"fields":[
{
"field": "region",
"terms_counts": {
"02": 704,
"07": 857,
"AZ": 671,
"CA": 18064,
"CO": 654,
"FL": 1537,
"GA": 822,
"H9": 758,
"IL": 1254,
"KS": 1104,
"MA": 1301,
"NC": 626,
"NJ": 892,
"NY": 4035,
"ON": 1097,
"OR": 12158,
"PA": 1040,
"TX": 2726,
"VA": 1894,
"WA": 1948
},
"more_terms": true,
"ents_present": 70489,
"ents_absent": 65796,
"high_card": false,
"approx_cardinality": 331
},
{
"field": "utm_mediums",
"terms_counts": {
"Integrations-page": 1102,
"adwords": 4434,
"banner": 1496,
"blog": 3413,
"card": 10,
"cpc": 2923,
"email": 6051,
"ppc": 220,
"search": 8762,
"social": 12513,
"parnter": 2314
},
"more_terms": true,
"ents_present": 29595,
"ents_absent": 116690,
"high_card": false,
"approx_cardinality": 27
},
{
"field": "zd_yymm",
"terms_counts": {
"1409": 5,
"1410": 12,
"1411": 4,
"1412": 4,
"1501": 3,
"1502": 10,
"1503": 26,
"1504": 20,
"1505": 3
},
"more_terms": false,
"ents_present": 72,
"ents_absent": 136213,
"high_card": false,
"approx_cardinality": 9
},
{
"field": "openct",
"terms_counts": {
"1": 75,
"10": 18,
"108": 3,
"11": 7,
"12": 5,
"152": 3,
"19": 8,
"2": 42,
"20": 15,
"24": 3,
"26": 10,
"3": 11,
"32": 3,
"35": 3,
"4": 25,
"5": 16,
"51": 15,
"6": 17,
"8": 5,
"9": 11
},
"more_terms": true,
"ents_present": 328,
"ents_absent": 135957,
"high_card": false,
"approx_cardinality": 98,
"stats": {
"mean": 15.621951219512194,
"sd": 27.978602557504388,
"min": 1,
"max": 264,
"n": 328
}
},
{
"field": "mc_lastopen_ts",
"terms_counts": {},
"more_terms": false,
"histograms": [
{
"data": {
"1446336000000": 2,
"1446595200000": 2,
"1447891200000": 333,
"1448150400000": 348,
"1448409600000": 9,
"1448668800000": 8
},
"start": "1446401936511",
"end": "1448993936511",
"interval": "3d"
},
{
"data": {
"1442188800000": 35,
"1442793600000": 3,
"1445212800000": 71,
"1445817600000": 2,
"1446422400000": 4,
"1447632000000": 340,
"1448236800000": 351,
"1448841600000": 7
},
"start": "1441131536511",
"end": "1448993936511",
"interval": "week"
},
{
"data": {
"1417392000000": 3,
"1420070400000": 2,
"1425168000000": 2,
"1427846400000": 43,
"1430438400000": 5,
"1433116800000": 24,
"1435708800000": 41,
"1438387200000": 48,
"1441065600000": 38,
"1443657600000": 73,
"1446336000000": 702
},
"start": "1417457936511",
"end": "1448993936511",
"interval": "month"
}
],
"ents_present": 1377,
"ents_absent": 134908,
"high_card": false,
"approx_cardinality": 1212
}
]
}
}
FieldSuggest
Suggest Values for a field for auto-complete in webui
| Parameters | ||
|---|---|---|
| table | string (required) Ex: user | Table name (generally `user`) |
| field | string (required) Ex: email | field name to do lookup on. |
FieldSuggest
Suggest Values for a field for auto-complete in webui
Response 200
Headers
Content-Type: application/json
Body
{
"data": {
"fields": :[
{
"name": "_cc",
"sample": ["US"],
"type": "string",
"usage": 4
}
]
}
}