Custom Integrations: CSV Import
Lytics allows you to import data via a CSV file from data sources we do not natively integrate with.
- Implementation Type: Server-side.
- Implementation Technique: File based transfer.
- Frequency: Batch, frequency can be configured.
- Resulting data: User profiles once the imported data is mapped via LQL.
This integration parses CSV files and imports fields and associated data. Please see Custom Data Ingestion for more information on file naming, field formatting, headers, timestamps, etc. Once the integration is started it will:
- Connect to the SFTP server.
- Navigate to the configured folder.
- Read the selected CSV file.
- Import the fields that are chosen during configuration.
Once you choose the file to import during configuration, Lytics will read the file and list all the fields that can be imported. You may select fields that you wish to import.
Follow these steps to set up and configure a CSV import workflow in the Lytics platform.
- Navigate to the Integrations section and select the Custom Integrations tile.
- Click the New Workflow button.
- Select Import CSV from the list of workflows.
- Select the authorization you would like to use.
- From the Stream input, select the data stream where the data will be posted or enter a name to create a new stream.
- Using the Folder drop-down, select the folder where the CSV file is located.
- Using the File drop-down, select the file to import, or if this is a continuous import of dated files e.g.,
file_20160218.csv, choose the earliest dated file to import, and all files will be imported from that date forward.
- In the Custom Delimiter text field, enter the delimiter of the file. Default delimiter is a comma,
,. For tab delimited files enter
- Using the Timestamps input, select one or more columns in the CSV that contain the timestamp of an event. Sort in order of importance (e.g., "Updated" should be above "Created"). If no fields are specified, the event will be timestamped with the time of the import. Timestamp selection is unavailable for zip files larger than 100MB.
- Using the Fields input, select fields to import. If left empty all fields will be imported. If no field names appear, check to ensure the CSV file has an appropriate header row or the delimiter may need to be changed. Field selection is unavailable for zip files larger than 100MB.
- Check the Keep Updated checkbox to run this import continuously.
- Using the File Upload Frequency drop-down, select how often to check for a new file.
- Using the Time of Day drop-down, select the time of day to start import.
- Using the Timezone drop-down, select the timezone for time of day you selected above.
- Check the Diff checkbox to compare file contents to the previous file contents during continuous import and import only rows that have changed. This is useful when large amounts of data remain unchanged in each file.
- Click Start import.