Dagger or Data Aggregator is an easy-to-use, configuration over code, cloud-native framework built on top of Apache Flink for stateful processing of real-time streaming data. With Dagger, you don’t need to write custom applications or manage resources to process data in real-time. Instead, you can write SQLs to do the processing and analysis on streaming data.
At Gojek, Data Platform team use Dagger for feature engineering on realtime features. Computed features are then ingested to Feast for model training and serving. Dagger powers more than 200 realtime features at Gojek. This talk will about the end to end architecture and how Dagger and Feast work together to provide a cohesive feature engineering workflow.