Documentation / Product / Integrations / Amazon Web Services / AWS S3

AWS S3: CSV/JSON User Export

A wide ecosystem of tools can read JSON or CSV files from Amazon S3. This export allows you to bring Lytics user profiles based on your identity resolution strategy into other tools or to simply store as an archive of user data.

Integration Details

  • Implementation Type: Server-side.
  • Type: REST API.
  • Frequency: One-time or scheduled batch which may be hourly, daily, weekly, or monthly depending on configuration.
  • Resulting Data: User profile data written to a CSV or JSON format.

This integration uses the Amazon S3 API to upload data to S3. Each run of the workflow will proceed as follows:

  1. Create file(s) with the format selected in configuration.
  2. Start a multi-part upload in S3
  3. Scan through the users in the audience selected. Filter user fields based on the export configuration, and write to the file(s) created.
  4. Compress event data, if selected in the export configuration.
  5. Upload the User data files to S3.
  6. Check that the upload has completed after all users have been written.

Fields

The fields exported to the S3 file will depend on the Fields to Export option in the workflow's configuration Any user field in your Lytics account may be available for export.

Configuration

Follow these steps to set up and configure an export of user profiles to S3 in the Lytics platform.

  1. From Data > Integrations Select the Amazon Web Services tile. aws
  2. Click on New workflow.
  3. Select CSV/JSON User Export.
  4. Select the authorization you created during the authorization step.
  5. (Required) From the Audience Name drop-down list, select the Lytics audience you want to export.
  6. (Required) From the S3 Bucket Name drop-down list, select the bucket you wish to export to from the drop down. If you do not select an option from this drop-down you must enter a bucket in the text field below.
  7. (Optional) In the S3 Bucket Name (Alt) box, you can enter the name of the bucket you want to export to if it is not available in the drop-down above.
  8. (Required) From the Directory drop-down list, select the directory that you would like to save your files in.
  9. (Optional) In the Email to box,enter an email address, to recieve a link to the completed file.
  10. (Required) From the File Type drop-down list, select the type of file(s) you want to export.
    • CSV: the file will be a flat CSV, list and map fields will be joined by &
    • Multi-CSV: multiple files will be created; scalar values and non-scalar values will be in diffenent files. A separate file will be created for each non-scalar field that maps the various values to its user. Each user will be assigned a join key that can be used to join data accross multiple files. This join key can is based on the BY fields of the user and can change between exports.
    • JSON: The users will be exported as a JSON file.
  11. Select the Additional Segments checkbox to add an additional field containing the full list of audiences the user is a member of. Selecting this option may extend the time it takes for the export to complete. AWS S3 export user cfg-1
  12. Additional Configuration options are available by clicking on the Show Advanced Options tab.
  13. (Optional) From the Compress drop-down list, select a compression method for the file.
  14. (Required) In the Filename box, enter the name of the destination file. You can use {TIMESTAMP} in this name, and it will be replaced with the timestamp at the time of the export. By default, the filed is exported with the name lytics-events-{TIMESTAMP}.
  15. (Optional) From the Timestamp format drop-down list, select the format for the timestamp in the filename. MM = month, mm = minute
  16. (Required) Select the Include Headers checkbox to include field names as the first row of the CSV (this is selected by default).
  17. Use the Fields to Export input to select the user fields to include in the exported file. If left blank all fields will be included.
  18. In the User Limit box, enter the maximum number of users to export. If left blank all users will be exported.
  19. (Optional) Select the Keep Updated to run the export repeatedly.
  20. (Optional) From the File Export Frequency drop-down list, select the frequency to run the export, if Keep Updated is selected.
  21. (Optional) From the Time of Day drop-down list, select the time of day for the export work to be scheduled, if Keep Updated is selected. Note: The export will run once within a few minutes of clicking Start Export, each run after that will be started at the selected time.
  22. (Optional) Timezone drop-down list, select the timezone for the Time of Day. AWS S3 export user cfg adv
  23. Click the Start Export button when you're ready to begin the export.