Pipelines

AirGapKit pipelines are composable, zero-trust workflows that run entirely in the browser.

High-level flow

Upload → Parse → Import → Validate → Transform → Mask → Profile → Diff → Sync → Unify → Lint

• Upload: Collect raw files or streams from the user.
• Parse: Normalize CSV/XLSX/JSON into structured rows.
• Import: Map incoming fields to your schema, apply defaults, and prep metadata.
• Validate: Apply schema rules, constraints, and lint-like checks.
• Transform: Cleanse, reshape, and enrich data before storage.
• Mask: Apply PII masking and tier-aware redaction in-browser.
• Profile: Summarize distributions, completeness, and anomalies.
• Diff: Compare incoming data to a reference set for change detection.
• Sync: Stage rows for upsert into your destination.
• Unify: Deduplicate and merge entities across sources.
• Lint: Enforce style and rule hygiene to keep ingestion consistent.

TODO: Link each phase to the corresponding hero or support module once the deep dives are written.