Appearance
2. Run the first scan
A scan is a discovery run: a job the worker executes in stages, each resumable. Start it with Save and scan in the connect sheet, Scan now in the explorer, the Home page's primary action, or the API (POST …/sources/{id}:scan → 202 with a job id).
Stages
| Stage | What it does |
|---|---|
| Harvest | Reads schemas, tables, views (with their definitions), columns, keys, indexes, comments and row counts into an immutable schema snapshot. Catalog queries never name tables outside the scope. |
| Profile | Bounded statistics per column. Tables up to 100,000 rows are read in full; larger ones are sampled at min(100,000 rows, 1%) with repeatable sampling where the engine supports it. Columns that look sensitive are profiled without values: counts, lengths and pattern signatures only — no min/max, top values, mean or deviation. |
| Relationships | Candidate joins from declared foreign keys, primary/unique keys and names; probes check value inclusion and fan-out on sampled distinct values; each candidate gets a score and its evidence. |
| Classify | Table roles (fact, dimension, bridge, snapshot, SCD, lookup, staging, event…), grain, column semantic types and roles, default aggregation and additivity, sensitivity and PII category. |
| Generate model | Lays accepted and high-confidence proposals onto a proposals branch with a change request (step 4). |
| Strategy | Recomputes the strategy report. |
Progress streams to the UI over Server-Sent Events (/api/v1/jobs/{jobId}/events: progress, stage, completed, failed). Cancel stops in-flight source queries within seconds; Retry resumes a failed or cancelled run from its first unfinished stage (profiling resumes per table).
With an LLM, and without
If the tenant has an LLM enabled, the Relationships stage asks it to review uncertain candidates (confidence moves by at most ±0.25) and the Classify stage asks for business names and descriptions. Without one, heuristics decide alone — every stage works without AI. On the benchmark schemas (Chinook, Northwind, AdventureWorksLT, AdventureWorksDW) the heuristics-only relationship precision is 1.00 with recall 1.00 (Northwind 0.92).
Explore the result
Discovery (/sources/:id/explore) shows the snapshot: a schema tree with role tags (F/D/B), a table header with its role and grain, stat tiles, and per column the semantic type, null ratio, default aggregation and sensitivity.
Rescans
Proposals belong to the source, not to one snapshot. A rescan refreshes undecided proposals, supersedes those the new scan no longer supports, keeps your decisions and edits, and remembers rejections (a rejected join returns only if its confidence rose by 0.2 or more).
Scheduled drift scans are a lighter, metadata-only variant: see Drift, notifications and webhooks.