Data Streams
All data sent to Lytics must be sent through a data stream. Data streams are silos of raw data containing key-value pairs organized by source that can be used in the mapping of user fields. Until the raw event of a data stream is mapped to a user field it will not be available for use to build audiences. This gives Lytics the ability to filter, aggregate, and merge raw data into user fields in a non-destructive way.
Viewing your data streams
You can view information about your data streams in your Lytics dashboard by navigating to Data > Data Streams. The primary purpose of this section is to verify that data is successfully being received by Lytics. If your account has multiple data streams you can use the dropdown menu located above the graph to view a different stream.
Many integrations have multiple streams. For instance, it is common for email integrations to have an activity stream as well as a user stream. Integrations streams should be prefixed to help identify the source. You can find the streams for an integration under the documentation for that integration.
Data ingress graph
The event ingress graph shows the number of events collected on a stream for the selected time period (past day, week, month, 3 months, and year) and interval (hourly, daily, weekly, and monthly). Above the graph you will find the time the last message from this stream was received, the source of the data stream, and the number of fields in the stream.
NOTE: Last message received strives for real time reporting but can lag under a number of conditions including during bulk imports. If a data stream is not updating as expected please contact Lytics Support for assistance.
Raw keys table
Below the event ingress graph is the raw keys table. An event may contain any number fields containing a key-value pair. Each record in this table is a unique raw key seen on the stream.
The table has the following information on keys:
| Column | Description |
|---|---|
| Name | The name of the key. |
| Predicted Type | The assumed data type for the value, determined by sampling the values received. |
| First Seen | The date the key was first seen, according to the date on the event. |
| Last Seen | The last time a key was seen, according to the date on the event. |
| Times Seen | The number of events that contained the key. |
| Unique Values | The number of different values seen. |
| Times Used | The number of user fields that use the key. |
In addition to these seven columns, each record in the table can be clicked to open up a set of sample values. This can be used to verify that values are being collected and they match the expected data.
NOTE: If a key has many different values, the modal may not display all the values for the chosen key.
The table can be filtered in three ways: used vs. unused, common vs. uncommon, and text search.
| Filter | Description |
|---|---|
| Used | A raw key that is mapped to a user field. |
| Unused | Raw keys that are collected and stored, but never mapped to user fields. |
| Common | Raw keys that have been seen more often on events relative to other raw keys based on the times seen value. |
| Uncommon | Raw keys that are seldom seen on events relative to other raw keys. |
Hiding Keys
Raw event keys can be hidden but it is important to note that once hidden, keys cannot be made visible again through the user interface - only through the API. It is recommended that a list is kept of hidden keys in the event one needs to be resurfaced at a later date. If you need assistance, please contact Lytics Support with the key name and your account ID.
To hide a key:
- Select the checkbox next to the name of the key or keys.
- Click Remove selected key.
Monitoring data streams
As mentioned in the previous sections, the Data Streams section is primarily used to monitor the collection of raw data.
The data ingress graph displays the time and volume of raw events for each stream. You can use it to evaluate when data has stopped being sent to Lytics entirely or when the volume of events changed.
The raw keys table can be used to check on individuals keys including when they were last seen, the type of data being recorded, and the sample values being recorded.