ArchGen

Azure Data Architecture

Focus: Lakehouse + Streaming + Governance. Key areas: Azure SQL, Dynamics API, Blob Storage.

Use this as a block diagram of the system when explaining architecture.

Preview
Edit this example
Diagram caption: Azure Data Architecture (Lakehouse + Streaming + Governance) has 4 layers: Sources & Ingestion, Processing & Modeling, Storage & Serving, Governance & Security.

Prompt

Data architecture diagram on Azure. Ingest data from Azure SQL, Dynamics 365, and application logs in Blob Storage; stream IoT telemetry via IoT Hub and Event Hubs. Use Azure Data Factory for batch ingestion, Databricks for lakehouse processing, and Stream Analytics for real-time enrichment. Store raw and curated data in ADLS Gen2 with Delta Lake tables and publish curated marts to Synapse Analytics. Expose analytics through Power BI and secure APIs with Azure API Management. Add governance with Microsoft Purview, Key Vault for secrets, and RBAC with Azure AD; include data quality checks and monitoring.
Highlights
  • Layer details · Sources & Ingestion: Modules include Source Systems, Batch Ingestion, Streaming Intake.
  • Module responsibilities · Sources & Ingestion / Source Systems: Emit business data; Provide APIs; Capture raw events
  • Layer details · Processing & Modeling: Modules include Lakehouse Processing, Real-time Enrichment, Warehouse Modeling.

Overview

Azure Data Architecture (Lakehouse + Streaming + Governance) has 4 layers: Sources & Ingestion, Processing & Modeling, Storage & Serving, Governance & Security.

Layer details

Show all (4)
  • Sources & Ingestion: Modules include Source Systems, Batch Ingestion, Streaming Intake.
  • Processing & Modeling: Modules include Lakehouse Processing, Real-time Enrichment, Warehouse Modeling.
  • Storage & Serving: Modules include Data Lake, Analytics Warehouse, BI & APIs.
  • Governance & Security: Modules include Catalog & Lineage, Access Control, Monitoring & Quality.

Module responsibilities

Show all (12)
  • Sources & Ingestion / Source Systems: Emit business data; Provide APIs; Capture raw events
  • Sources & Ingestion / Batch Ingestion: Load batch datasets; Normalize schemas; Apply checkpoints
  • Sources & Ingestion / Streaming Intake: Ingest real-time events; Route streams; Enforce retention
  • Processing & Modeling / Lakehouse Processing: Transform data; Optimize storage; Handle merges
  • Processing & Modeling / Real-time Enrichment: Process streams; Compute KPIs; Publish derived data
  • Processing & Modeling / Warehouse Modeling: Build curated marts; Version transformations; Document datasets
  • Storage & Serving / Data Lake: Persist raw data; Support replay; Optimize cost
  • Storage & Serving / Analytics Warehouse: Serve BI queries; Scale concurrency; Enforce governance
  • Storage & Serving / BI & APIs: Expose insights; Serve datasets; Control access
  • Governance & Security / Catalog & Lineage: Enable discovery; Track provenance; Define ownership
  • Governance & Security / Access Control: Protect data; Enforce least privilege; Manage keys
  • Governance & Security / Monitoring & Quality: Detect failures; Validate datasets; Notify teams

Key flows

Show all (3)
  • Batch flow: Data Factory pulls source data into ADLS, Databricks transforms it into Delta tables, and curated marts are published in Synapse for BI.
  • Streaming flow: IoT events land in Event Hubs, Stream Analytics enriches them, and results are written to Delta and served through Power BI APIs.
  • Governance flow: datasets are registered in Purview, access is enforced via Azure AD RBAC, and quality checks gate promotion to gold zones.