AWS S3: Export Activity Data

Export events into S3 so you can access, archive, or run analysis on Lytics events in the AWS S3 ecosystem.

• Implementation Type: Server-side.
• Type: REST API.
• Frequency: One-time or scheduled batch which may be hourly, daily, weekly, or monthly depending on configuration.
• Resulting Data: Raw events exported to a CSV file format.

This integration uses the Amazon S3 API to read the CSV file selected. Each run of the job will proceed as follows:

1. Create a file with the configured file name.
2. Start a multi-part upload in S3.
3. Read events from the stream selected. If the export is configured to run periodically, only the events since the last run of the export will be included.
4. Compress event data, if selected in the export's configuration.
5. Write the event data to S3.
6. Check that the upload has completed after the last event before the start of the export is written.

The fields included depend on the raw event in Lytics data. All fields in the selected stream will be included in the exported CSV. To see a list of fields in a stream, select the stream name in the dropdown from Data > Data Streams in the Lytics platform.

Follow these steps to set up and configure an export of event data to AWS S3 in the Lytics platform.If you are new to creating jobs in Lytics, see the Jobs Dashboard documentation for more information.

1. Select Amazon Web Services from the list of providers.
2. Select the Export Activity Data job type from the list.
3. Select the Authorization you would like to use or create a new one.
4. Enter a Label to identify this job you are creating in Lytics.
5. (Optional) Enter a Description for further context on your job.
6. In the S3 Bucket Name drop-down list, select the bucket you wish to export to from the drop down. If you do not select an option from this drop-down you must enter a bucket in the text field below.
7. (Optional) In the S3 Bucket Name (Alt) box, you can enter the name of the bucket you want to export to if it is not available in the drop-down above.
8. From the Directory drop-down list, select the directory that you would like to save your files in.
9. Use the Data Streams to Export input to select data streams to export. A stream is a single source/type of data (you may choose more than one). If none are selected, all streams are exported.
10. (Optional) In the Email to box,enter an email address, to recieve a link to the completed file.
11. (Optional) From the Compress drop-down list, select a compression method for the file.
12. Additional Configuration options are available by clicking on the Show Advanced Options tab.
13. In the Filename box, enter the name of the destination file. You can use {TIMESTAMP} in this name, and it will be replaced with the timestamp at the time of the export. By default, the filed is exported with the name lytics-events-{TIMESTAMP}.
14. (Optional) From the Timestamp format drop-down list, select the format for the timestamp in the filename. MM = month, mm = minute
15. Select the Include Headers checkbox to include field names as the first row of the CSV (this is selected by default).
16. (Optional) Select the Keep Updated checkbox to run the export repeatedly.
17. (Optional) From the File Export Frequency drop-down list, select the frequency to run the export, if Keep Updated is selected.
18. (Optional) From the Time of Day drop-down list, select the time of day for the export work to be scheduled, if Keep Updated is selected. Note: The export will run once within a few minutes of clicking Start Export, each run after that will be started at the selected time.
19. (Optional) Timezone drop-down list, select the timezone for the Time of Day.
20. Click Start Export.