How to Publish Your own Events (Data) to WSO2 Analytics Platform (BAM, CEP)

We collect data via a Sensor API (a.k.a. agents), send them to servers: WSO2 CEP and WSO2 BAM, process them, and do something with the results. You can find more information about the big picture from the slide deck http://www.slideshare.net/hemapani/introduction-to-large-scale-data-analysis-with-wso2-analytics-platform.  This post describes how you can collect data.

We provide a one Sensor API to publish events for both batch and realtime pipelines. The Sensor API is available as Java clients (Thrift, JMS, Kafka), java script clients* ( Web Socket and REST) and 100s of connectors via WSO2 ESB. Let’s see how we can use the java thrift client to publish events.

  1. First of all, you need CEP or BAM running. Download, unzip, and run WSO2 CEP or WSO2 BAM (via bin/wso2server.sh).
  2. Now, let’s write a client. Add the jars given in Appendix A or add POM Dependancies given in Appendix B to your Maven POM file to setup the classpath.
  3. The Java client would look like following. 
  4. Just like you create tables before you put data into a database, first you define streams before sending events to WSO2 Analytic Platform. Streams are a description of how your data look like (a.k.a. Schema). Then you can publish events. In the code, the “Event Data” is an array of objects, and it must match the types and parameters given in the event stream definition.
You can find an example client from samples/producers/pizza-shop from WSO2 CEP distribution. 

Appendix

Appendix A: Dependency Jars
You can find the jars from the location ${cep.home}/repository/components/plugins/ of CEP or BAM pack.
  1. org.wso2.carbon.logging_4.2.0.jar
  2. commons-pool_1.5.6.*.jar
  3. httpclient_4.2.5.*.jar
  4. httpcore_4.3.0.*.jar
  5. commons-httpclient_3.1.0.*.jar
  6. commons-codec_1.4.0.*.jar
  7. slf4j.log4j*.jar
  8. slf4j.api_*.jar
  9. axis2_1.6.1.*.jar
  10. axiom_1.2.11.*.jar
  11. wsdl4j_1.6.2.*.jar
  12. XmlSchema_1.4.7.*.jar
  13. neethi_*.jar
  14. org.wso2.securevault_*.jar
  15. org.wso2.carbon.databridge.agent.thrift_*.jar
  16. org.wso2.carbon.databridge.commons.thrift_*.jar
  17. org.wso2.carbon.databridge.commons_*.jar
  18. com.google.gson_*.jar
  19. libthrift_*.jar
Appendix B: Maven POM Dependancies 
 Add the following WSO2 nexus repo and dependencies to pom.xml at corresponding sections.
<repository>
<id>wso2-nexus</id>
<url>http://maven.wso2.org/nexus/content/groups/wso2-public/</url>
</repository>
<dependency>
    <groupId>org.wso2.carbon</groupId>
    <artifactId>org.wso2.carbon.databridge.agent.thrift</artifactId>
    <version>4.2.0</version>
</dependency>
<dependency>
    <groupId>org.wso2.carbon</groupId>
    <artifactId>org.wso2.carbon.databridge.commons.thrift</artifactId>
    <version>4.2.0</version>
</dependency>
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s