Storm is an obvious choice for high-throughput processing but it's a little out of its territory when you also need to maintain low latency. It doesn't have to be that way, though. We at Forter devised a few components and practices to help us build scalable high-throughput and low-latency storm topologies for our real-time fraud detection system. We'd like to share some of the lessons we learned and our ways of handling the issues. From low-latency join bolts to "escape-routes" for sub-topologies, in this talk we'll reveal all the nitty gritty of how we keep our pipeline processing under one second while maintaining data integrity, robust monitoring and code simplicity.

 

MP3