DP-900 Part 13

Real time analytics

Data Processing : Converting of raw data to meaningful information.

1. Batch Processing :

Group of stored data, processed as batch, data records are stored before processing.

Example : Credit card billing

1. Stream Processing :

Data is processed in real time as it occurs. For dynamic data generation, and critical operations.

Example : Stock market change

Batch vs Stream

1. Batch handles large dataset while stream handles individual record

2. Batch has access to all datasets, while stream has access to recent data

3. Batch returns result in few hours, while stream in few second

Architecture of stream processing

• Event generates some data
• Data is captured in streaming source for processing
• Event data is processed, query operators on event data
• result are stored on output

Real time analytics in Azure

• Azure stream analytics
• Spark structured analytics
• Azure data explorer

Thanks for reading...