Processing plugin

There are two types of plugins: transformer and enrichment. You can choose to have only one transformer, and zero or multiple enrichment.

Built-in plugins

Below plugins are provided by Clickstream Analytics on AWS.

Plugin name	Type	Description
UAEnrichment	enrichment	User-agent enrichment, use ua_parser Java library to enrich User-Agent in the HTTP header to ua_browser,ua_browser_version,ua_os,ua_os_version,ua_device
IpEnrichment	enrichment	IP address enrichment, use GeoLite2 data by MaxMind to enrich IP to city, continent, country

The UAEnrichment uses UA Parser to parse user-agent in Http header

The IpEnrichment plugin uses GeoLite2-City data created by MaxMind, available from https://www.maxmind.com

Custom plugins

You can add custom plugins to transform raw event data or enrich the data for your need.

Note

To add custom plugins, you must develop your own plugins firstly, see Develop Custom Plugins.

To add your plugins, choose Add Plugin, which will open a new window, in which you can upload your plugins.

1. Enter the plugin Name and Description

2. Choose Plugin Type

3. Enrichment: Plugin to add fields into event data collected by SDK (both Clickstream SDK or third-party SDK)

4. Transformation: A plugin used to transform a third-party SDK’s raw data into solution built-in schema

5. Upload plugin java JAR file

6. (Optional) Upload the dependency files if any

7. Main function class: fill the full class name of your plugin class name, e.g. com.company.sol.CustomTransformer

Develop Custom Plugins

The simplest way to develop custom plugins is making changes based on our example project.

1. Clone/Fork the example project.

   git clone https://github.com/awslabs/clickstream-analytics-on-aws.git
   cd examples/custom-plugins

   • For enrichment plugin, please refer to the example: custom-enrich/

   • For transformer plugin, please refer to the example: custom-sdk-transformer/

2. Change packages and classes name as you desired.

3. Implement the method public Dataset<row> transform(Dataset<row> dataset) to do transformation or enrichment.

4. (Optional) Write test code.

5. Run gradle to package code to jar ./gradlew clean build.

6. Get the jar file in build output directory ./build/libs/.