Azure: Import Data

Importing custom data stored in Azure Blob Storage to Lytics allows you to leverage Insights provided by Lytics data science to drive your marketing efforts.

• Implementation Type: Server-side.
• Type: REST API.
• Frequency: One-time or scheduled batch (can be hourly, daily, weekly, or monthly depending on configuration).
• Resulting Data: Raw events or, if custom mapping is applied, new user profiles or existing profiles with new user fields.

This integration uses the Storage Service API to read the CSV file stored in Azure Container. Each run of the job will proceed as follows:

1. Finds the file selected during configuration using Blob Storage service of Azure.
2. If found, reads the contents of the file.
3. If configured to diff the files, it will compare the file to the data imported from the previous run.
4. Filters fields based on what was selected during configuration.
5. Sends event fields to the configured stream.
6. Schedule the next run of the import if it is a scheduled batch.

Fields imported via CSV through Azure Storage Service will require custom data mapping. For assistance mapping your custom data to Lytics user fields, please reach out to Lytics support.

Follow these steps to set up and configure a CSV Import from Azure Storage in the Lytics platform. If you are new to creating jobs in Lytics, see the Jobs Dashboard documentation for more information.

1. Select Microsoft Azure from the list of providers.
2. Select the Import Data job type from the list.
3. Select the Authorization you would like to use or create a new one.
4. Enter a Label to identify this job you are creating in Lytics.
5. (Optional) Enter a Description for further context on your job.
6. From the Stream box, enter or select the data stream you want to import the file(s) into.
7. From the Container drop-down, select the Azure container with the file you want to import.
8. In the File drop-down, select the file to import. Listing files may take up to a couple minutes after the bucket is chosen.
9. (Optional) In the Custom Delimiter box, enter the delimiter used. For tab delimited files enter "t". Only the first character is used (e.g. if "abcd" was entered, only "a" would be used as a delimiter).
10. (Optional) From the Timestamp Field drop-down, select the name of the column in the CSV that contains the timestamp of an event. If no field is specified, the event will be time stamped with the time of the import.
11. (Optional) Use the Fields input to select the fields to import. Leave empty to import all fields. If no field names appear, the Custom Delimiter may need to be adjusted. Also check to ensure the CSV file has an appropriate header row.
12. (Optional) Select the Keep Updated checkbox to run the import on a regular basis.
13. (Optional) Select the Diff checkbox on repeating imports to compare file contents to the previous file contents and import only rows that have changed. This is useful when full data dumps are provided.
14. Click on the Show Advanced Options tab to view additional Configuration options.
15. (Optional) In the Prefix text box, enter the file name prefix. You may use regular expressions for pattern matching. The prefix must match the file name up to the timestamp. A precalculated prefix derived from the selected file will be available as a dropdown.
16. (Optional) From the Time of Day drop-down, select the time of day for the import to be scheduled after the first import. This only applies to the daily, weekly, and monthly import frequencies. If no option is selected the import will be scheduled based on the completion time of the last import.
17. (Optional) From the Timezone drop-down, select the time zone for the Time of Day.
18. (Optional) From the File Upload Frequency drop-down, select the frequency to run the import.
19. Click Start Import.

NOTE: For continuous imports, files should be in the following format: timestamp.csv. The job will understand the sequence of files based on the timestamp. If no next file is received, the continuous import will stop and a new export will need to be configured.