System Block Diagram of an AI Data Processing Pipeline
System Block Diagram of an AI Data Processing Pipeline (Ingestion -> Feature Store -> Training -> Inference) has 4 layers: Data Sources & Ingestion, Processing & Feature Engineering, Model Lifecycle, Serving & Feedback.
Key modules include Data Sources, Batch Ingestion, Streaming Ingestion, Data Lake / Object Storage, ETL / Data Cleaning.
Key modules
- Data Sources
- Batch Ingestion
- Streaming Ingestion
- Data Lake / Object Storage
- ETL / Data Cleaning
- Feature Engineering
- Feature Store
- Model Training