How-to for real-time alerting, analytics and reporting at scale with Apache Kafka and Apache Ignite

Are you tasked to build a system or upgrade an existing architecture to a solution capable of handling unbound streams of data, do real-time alerting, storing always growing terabytes and petabytes of data and, finally, act on the data within milliseconds SLAs. Sounds like a challenging but rather a common task? Yes! Should you as an engineer or architect build it up off the ground? Unlikely!

What if we tell you that by integrating Apache Kafka with Apache Ignite you’ll solve all of the requirements faster and easier. A battle-tested recipe is simple - take Kafka Connect and have your data stream through Kafka pipelines, add a pinch of KSQL to act on the streams with SQL in real-time with zero delays, rinse and flush the preprocessed data in Ignite as in-memory databases and get further insights by analyzing your hot and cold datasets.

In this talk, Denis and Viktor will demonstrate how to implement the solution in practice, will explain architectural reasoning and the benefits of real-time integration, share common usage patterns. As always, we'll build a streaming data pipeline using nothing but our bare hands, Apache Ignite, Kafka Connect, and KSQL.