Google Cloud Pub/Sub: Import Events

Importing user and activity data from Cloud Pub/Sub in real-time results in new users or existing user profiles supplemented with activity data. You can use this data to build and refine your existing Lytics audiences to power better, cross-channel campaigns.

The integration is similar to AWS Kinesis Streaming or Azure Event Hubs. It requires you to set up a Cloud Pub/Sub Topic, which you publish JSON messages to. Lytics then listens (subscribes) to this Topic and receives those messages.

• Implementation Type: Server-side.
• Implementation Technique: REST API.
• Frequency: Real-time.
• Resulting data: Raw event data, user fields, and content (if mapped).

This integration utilizes the Cloud Pub/Sub API methods of receiving events from a publisher. Specifically we use the Golang library. You are the publisher who sends (publishes) events through a user-created object known as a Topic. Lytics manages an object called a Subscription, which receives the data (events) from the Topic you created. The continuously running job works like this:

1. Through the service account token you created in the authorization step, Lytics creates a new client, which manages and receives events using the subscription.
2. Lytics verifies the Topic ID you provide in the job configuration, and creates a connection to the Subscription associated with this Topic. If no Subscription exists or is provided, Lytics creates a new Subscription in the following format: lytics\_subscription\_RANDOM_WORKFLOW_ID.
3. The Topic is the "publisher", whereas the Subscription is the "receiver". This means you must publish/send data to this Topic from your account, which Lytics will then receive and process.
4. The job will continue to get events in real-time as soon as events are published on the Topic.
5. The events are sent to a user-selected stream which then gets added to user profiles or creates new user profiles.

Data imported by this job will be sent to a new or existing data stream in your account which you can define as part of the configuration. You will need to map the raw event data into user or content fields using LQL. You may reach out to Lytics Support for assistance with mapping your custom data.

Follow these steps to set up and configure an import of events from Google Cloud Pub/Sub into Lytics. If you are new to creating jobs in Lytics, see the Jobs Dashboard documentation for more information.

1. Select Google CLoud from the list of providers.
2. Select the Import Data (Cloud Pub/Sub) job type from the list.
3. Select the Authorization you would like to use or create a new one.
4. Enter a Label to identify this job you are creating in Lytics.
5. (Optional) Enter a Description for further context on your job.
6. In the Stream text box, either select or type in the stream you would like to send these events to.
7. Enter the Topic ID you would like Lytics to receive events from.
8. (Optional) Enter an existing Subscription Name that you would like to receive events on or leave it blank and Lytics will create a new one named lytics_subscription_RANDOM_ID.
9. By default, after 100 bad messages (usually because they are not JSON), the job will error. If you would like to change the maximum number of bad messages before the job errors, enter a number under Max Bad Messages.
10. Click Start Import.