Lucid Streaming; how to take full advantage of data streaming
- Data streaming, Stream Processing, Real-time analytics, operational analytics — what is this? What’s the difference?
Most important use cases for data streamingThere are lots of misconceptions especially for the MDS crowd (not as much enterprise) between fast batch vs streamingMemory-first processing (in-memory) vs disk space batch jobsChange data capture (and only capture of change)Data warehouses are now tying to support streaming more (like Snowflake)This will be a big deal to make it so that more streaming can happen Streaming warehouses (Rockset, Materialize) vs data streamingLineage - transformed data - can I trust this data I'm looking atHow does data streaming and lineage come together? What’s unique about lineage in a streaming context?If time: what does it mean to do streaming data products in a data mesh context?