Posts

Introducing Mm FLaNK... An Apache Flink Stack for Rapid Streaming Development From Edge 2 AI

Image
Introducing Mm FLaNK...  An Apache Flink Stack for Rapid Streaming Development From Edge 2 AI See:  https://blog.cloudera.com/announcing-support-for-apache-flink-with-the-ga-of-cloudera-streaming-analytics/




Source:   https://github.com/tspannhw/MmFLaNK

Stateless NiFi:https://www.datainmotion.dev/2019/11/exploring-apache-nifi-110-parameters.html

To show an example of using the Mm FLaNK stack we have an Apache NiFi flow that reads IoT data (JSON) and send it to Apache Kafka.   An Apache Flink streaming application running in YARN reads it, validates the data and send it to another Kafka topic.  We monitor and check the data with SMM.    The data from that second topic is read by Apache NiFi and pushed to Apache Kudu tables.

Mm FLaNK Stack (MXNet, MiNiFi, Flink, NiFi, Kafka, Kudu)



 First, we rapidly ingest, route, transform, convert, query and process data with Apache NiFi.   Once we have transformed it into a client, schema-validated known data type we can stream it to Kafka for additiona…