Understanding Lytics / Integrations

Amazon Web Services

The Amazon Web Services S3 integration provides the ability to import and export files to S3 storage. For more information on how to integrate with Kinesis, see our docs on AWS Kensis.

This article covers:

Before you Begin

Make sure that you have already created an Amazon S3 account. Amazon also provides additional information to help you get set up with S3.

Authentication

Before you can share data between Lytics and S3 you need to set up an authorization. You may authorize in one of two ways:

  • Providing AWS keys - for more information see Amazon's documentation on secret and access keys
  • Delegate Access To Lytics with AWS Identity and Access Management (IAM) - for more information see Amazon's documentation on IAM.
  1. From Data > Integrations Select the Amazon Web Services tile. aws
  2. Navigate to the Authorizations section authorizations tab
  3. If you want to authorize using AWS keys:
    1. Select Enter AWS Keys.
    2. Enter your Access Key, Secret Key, and a Description for the authorization. authorization
    3. Click Authorize to save this authorization.
  4. If you want to authorize by delegating access to Lytics with IAM, enter the following policy in your S3 bucket:
    {
    "Statement": [
     {
        "Sid": "Grant Lytics Access",
        "Effect": "Allow",
        "Principal": {
           "AWS": "arn:aws:iam::358991168639:root"
        },
        "Action": [
           "s3:GetBucketLocation",
           "s3:ListBucket",
           "s3:PutObject",
           "s3:GetObject",
           "s3:DeleteObject"
        ],
        "Resource": [
           "arn:aws:s3:::YOUR_BUCKET",
           "arn:aws:s3:::YOUR_BUCKET/*"
        ]
     }
    ]
    }

Exporting Event Data

Summary
FrequencySingle export or daily
Exports toCSV file in your S3 bucket
Namedefault: lytics-events-{TIMESTAMP}
Identifiersnone
Mapped Fieldsno
Typeadd, no removal

Running this exports raw data stream events to a CSV which will be transferred to your S3 bucket.

  1. From Data > Integrations Select the Amazon Web Services tile. aws
  2. Click on New workflow.
  3. Select CSV Event Export.
  4. Select the authorization you created during the authentication step.
  5. Complete the required fields:

    • S3 Bucket Name: Select the bucket you wish to export to from the drop down.
    • S3 Bucket Name (Alt): If no buckets appeared in the previous drop down, you can enter the name of the bucket in this field.
    • Directory: Choose the directory that you would like to save your files in.
    • Data Streams to Export: Choose data to export, a stream is a single source/type of data (you may choose more than one). If none are selected, all streams are exported.
    • Email to: An email address which will receive an email with the export link.
    • Compress: Select a compression method for the file.

    configuration

  6. You may also choose to configure these advanced options:
    • Filename: Enter the name of the destination file. You can use {TIMESTAMP} in this name, and it will be replaced with the timestamp at the time of the export. By default, the filed is exported with the name lytics-events-{TIMESTAMP}.
    • Timestamp format: The format for the timestamp in the filename. MM = month, mm = minute
    • Include Headers: Selecting this includes field names as the first row of the CSV (this is selected by default).
    • Keep Updated Allows you to receive an updated version of the CSV daily.
    • Time of Day & Timezone: Let you select the exact time the CSV should be exported for a continuous export.
  7. Click the Start Export button when you're ready to begin the export.

Exporting User Data

Summary
FrequencySingle export or daily
Exports toCSV or JSON file in your S3 bucket
Namedefault: lytics-users-{TIMESTAMP}
Identifiersnone
Mapped Fieldsno
Audience ExportsUser selected
Typeadd & remove

Export an audience of users in either CSV or JSON format to your S3 bucket.

  1. From Data > Integrations Select the Amazon Web Services tile. aws
  2. Click on New workflow.
  3. Select CSV/JSON User Export.
  4. Select the authorization you created during the authentication step.
  5. Complete the required fields:

    • Audience Name: Select the Lytics audience to export.
    • S3 Bucket Name: Select the bucket you wish to export to from the drop down.
    • S3 Bucket Name (Alt): If no buckets appeared in the previous drop down, you can enter the name of the bucket in this field.
    • Directory: Choose the directory that you would like to save your files in.
    • Email to: An email address which will receive an email with the export link.
    • File type: Select either CSV or JSON as the file type.
    • Additional Segments: Selecting this adds an additional column containing the full list of audiences the user is a member of. Selecting this option may affect the performance of the workflow (the time it takes to export).

    configuration

  6. You may also choose to configure these advanced options:
    • Compress: Select a compression method for the file.
    • Filename: Enter the name of the destination file. You can use {TIMESTAMP} in this name, and it will be replaced with the timestamp at the time of the export. By default, the file is exported with the name lytics-users-{TIMESTAMP}.
    • Timestamp format: The format for the timestamp in the filename. MM = month, mm = minute
    • Include Headers: (CSV only) Selecting this includes field names as the first row of the CSV (this is selected by default).
    • Fields to Export: Use this select to choose only specific fields to export. If it is left blank, then all user fields will be exported.
    • User Limit: You can set a maximum number of users to export. Leave this field blank to export all users.
    • Keep Updated: Allows you to receive an updated version of the file daily.
    • Time of Day & Timezone: Let you select the exact time the file should be exported for a continuous export.
  7. Click the Start Export button when you're ready to begin the export.

Importing Data

Summary
FrequencySingle export, hourly, or daily
StreamsUser selectable
User fieldsUser defined
Provider fieldsNone
User activityUser defined
Campaign contentUser defined
Click URL paramUser defined

You can import data from a CSV or JSON file hosted on your S3 bucket.

  1. From Data > Integrations Select the Amazon Web Services tile. aws
  2. Click on New workflow.
  3. Select Import CSV if you wish to import a CSV file, or Import JSON to import a JSON file.
  4. Select the authorization you created during the authentication step.
  5. Complete the required fields:

    • Stream: Input or select the data stream you want to import to.
    • Bucket: Choose the bucket to import from. If there is an error fetching buckets, your credentials may not have permission to list buckets.
    • Bucket Name (Alt): If no buckets appeared in the previous drop down, you can enter the name of the bucket in this field.
    • File: Choose the file to import. Listing files may take up to a couple minutes after the bucket is chosen
    • Custom Delimiter: (CSV only) If a custom delimiter is used, specify it here. For tab delimited files enter "t".
    • Timestamp Field: (CSV only) Select the name of the column in the CSV that contains the timestamp of an event. If no field is specified, the event will be timestamped with the time of the import.
    • Fields: (CSV only) Select fields to import. Leave empty to import all fields. If no field names appear, the delimiter may need to be adjusted. Also check to ensure the CSV file has an appropriate header row.
    • Keep Updated: Enables the file to be imported daily by default. To import hourly, select that option under File Upload Frequency
    • Diff: (CSV only) On continuous imports, compare file contents to the previous file contents and import only rows that have changed. This is useful when full data dumps are provided.

    configuration

  6. You may also choose to configure these advanced options:

    • Time of Day & Timezone: Lets you select the exact time the file should be imported for a continuous import.
    • File Upload Frequency: If this is a continuous import, the import will check for new files daily by default. To run hourly you must select the hourly option. If you expect to only import files weekly or monthly, select those options instead.
  7. Click the Start Import button when you're ready to begin the export.

NOTE: For continuous imports, files should be in the following format: prefix_timestamp.csv or prefix_timestamp.json. The workflow will understand the sequence of files based on the timestamp. If no next file is received the continuous import will stop and a new export will need to be configured.