The Data Streams section of Lytics can be found under Data at Data > Data Streams.
Data streams can be thought of as containers for different data. When data gets sent to Lytics, it is also always additionally sent to a data stream. The default data stream, is aptly named
default. Keeping data organized in this manner allows for clean filtering, aggregation, and most importantly, merging.
The most important detail to consider about data streams when it comes to data management is that they represent raw data. The graph visualizes the volume of data of this raw data and lists the keys it contains. Remember that audience membership is based on user fields which are essentially keys that have been mapped for use. This means that seeing data in data streams does not mean the data is available for use in audiences. It must first be mapped into user fields before it can be used.
Note: the number of keys in a data stream can be extremely large. This is normal and does not impact performance. Furthermore, any keys comprising less than 0.1% of the data volume will be hidden from display to reduce clutter.
The primary purpose of the data streams tool is to verify that data is in fact being collected by Lytics. The easiest way to confirm this is by reading the event ingress graph.
The graph shows the number of events collected on a stream in bi-hourly intervals. Standard intuition applies here: when there are no bars, no data is being collected.
Above the graph are a few helpful facts.
Fields are important to understand. They represent the smallest workable unit of data in Lytics. The table below the graph is designed to help discover and validate individual fields.
Underneath the event ingress graph is the raw events table. Each record in this table is a unique raw field seen on the stream. All events are packets of keys and values. A raw field represents the key.
The table has the following information on fields:
In addition to these seven columns, each record in the table can be clicked to open up a set of sample values seen for the raw field.
The table can be filtered in three ways: used vs. unused, common vs. uncommon, and text search.
An active Lytics account that is collecting data from a variety of sources will have multiple data streams continuously collecting data. Use the dropdown on the top right of the Data Streams page to switch streams and verify that data for the stream is being collected as expected.
Note: Many integrations have multiple streams. For instance, it is common for email integrations to have an activity stream as well as a user stream. Learn the integration prefixes to identify all the streams for an integration. Also keep in mind that the Data Source mentioned above the ingress graph will be the name of the integration for the stream.