Complex event processing: True integration of Things to Internet

There is a subtle difference between building internet of things and “Thingify the Internet”. Using advanced IPv6 compatible protocol any device gets an IP address, and one can start from there: Simple machine to machine communication. Add a transport layer and you have your processes communicating with each other. Having stated that, this draws quite some consequences: We can’t rely on the infrastructure in between. The IP packet travels through the Internet, and we don’t know who is handling it. This means any computational processing needs to be performed at the edges: Either on the Smart Thing (sensor node) or on the (cloud)server. This draws the question, of where to do the processing? E.g. how much context processing do we want to do on the Smart Thing? How much on the server? A simple answer to is to limit the communication effort, and try to do as much processing on the server! SmartThings/sensor nodes usually have energy and resource constraints. So it will depend on the application developer: Depending on the scenario, he/she will have to balance Smart Item processing and communication efforts. However, in order to give the application developer, the freedom to balance, we must support them as much as possible from the server side. And that is where Complex Event Processing (CEP) jumps in!

So, How does CEP work?

In a truly micro service oriented world, we have this global service anybody can subscribe to and use: The application developer writes the CEP scripts (e.g. in EPL, Event Processing Language), and orders her SmartThings to publish their messages to the CEP service. The CEP service then executes the relevant scripts and does message/event processing, transforming simple data points (in time), into enriched information Such derived data streams can further be extended to advanced engines to generate complex reports and apply data science

Key architecture ingredient: Low Latency Requirement

Most IOT deployments are event driven: each part of the system reacts to the events happening in the other parts, potentially generating new events. Complex event processing (CEP) engines in charge of interpreting, filtering, and combining primitive events to identify higher level composite events. Hence to handle such interactions, CEPs are required to have low latency.

Before we close, lets touch upon a few use cases leveraging CEP:

1. Network Management

Suddenly thousands of alarms go off at a data center. A complex event processing engine filters the events and infers a new event that an external network link has gone down. Operators work to switch over to a backup telecom link

2. Safety

Multiple sensors in a moving vehicle sense an object on the road ahead. The vehicle immediately warns the driver and works to detect what they object is. It is confirmed from analysis of video data that it is a pedestrian and the driver still hasn't applied brakes. The vehicle decides to apply maximum brake force automatically. If a collision is about to occur the vehicle deploys pedestrian airbags timed to minimize injury.

3. Financial

A payment platform receives thousands of payment events per minute during peak hours. A CEP engine is used to infer possible fraudulent payments from patterns of purchases.

4. Algorithmic Trading

A trading algorithm watches social media and industry new sources for information that can influence the price of stocks. It seldom acts on such information alone but may seek confirmation from other events such as large trades with a particular signature. If it discovers bullish social media chatter followed by large, aggressive buys it infers that the stock is in the early seconds of news driven momentum. Such strategies are dangerous as aggressive buying is no confirmation that news is genuine or meaningful to the stock price.