Appearance
Connectors
Eight source kinds ship in v1.0. Every connector implements the same contract — test, metadata, bounded profiling, relationship and grain probes, query execution and a SQL dialect — and passes the shared conformance suite against a real engine or emulator where one exists.
Matrix
Kind (kind) | Driver | Authentication | Connection fields | Since |
|---|---|---|---|---|
PostgreSQL (PostgreSql) | Npgsql (PostgreSQL License) | User / password | Host, port (5432), database, user, TLS (require by default) | M2 |
SQL Server / Azure SQL (SqlServer) | Microsoft.Data.SqlClient (MIT) | User / password | Host, port (1433), database, user, TLS, trust server certificate | M2 |
MySQL / MariaDB (MySql) | MySqlConnector (MIT) | User / password | Host, port (3306), default database, user, TLS (require / verify-full / disable) | M8 |
Oracle (Oracle) | Oracle.ManagedDataAccess.Core (Oracle Free Use terms) | User / password | Host, port (1521), service name, user, TLS (TCPS, system trust store) | M8 |
Amazon Redshift (Redshift) | Npgsql | User / password | Endpoint, port (5439), database, user, TLS | M8 |
Snowflake (Snowflake) | Snowflake.Data (Apache-2.0) | Key pair (unencrypted PKCS#8 PEM, preferred) or password | Account host (*.snowflakecomputing.com), database, warehouse, optional role, user | M8 |
Google BigQuery (BigQuery) | Google.Cloud.BigQuery.V2 (Apache-2.0) | Service-account JSON key | Project ID, service account, key, optional location, max GB billed per query (default 10), API endpoint | M8 |
Databricks SQL (Databricks) | Statement Execution REST API over HttpClient (no driver) | Personal access token or OAuth M2M (client credentials) | Workspace host, SQL warehouse ID, catalog (Unity Catalog), client id or user, token or client secret | M8 |
Behaviour per kind
| Kind | Read-only enforcement | Sampling | Keys | Notes |
|---|---|---|---|---|
| PostgreSQL | default_transaction_read_only=on, server-side statement_timeout | TABLESAMPLE SYSTEM/BERNOULLI … REPEATABLE | Declared | — |
| SQL Server | ApplicationIntent=ReadOnly | TABLESAMPLE above 1M rows, systematic every k-th row below | Declared | Time grains via DATEADD/DATEDIFF (works before 2022) |
| MySQL | Read-only session, max_execution_time (MariaDB max_statement_time), backslash escapes forced | RAND(1729) | Declared | Reads every database the user can see; privileges granted through roles are not seen by the least-privilege check |
| Oracle | Read-only transaction | Engine sampling | Declared | Schemas are non-Oracle-maintained users; upper-case names shown in lower case (objects created quoted in all lower case can't be addressed); DATE → timestamp; no wallets yet |
| Redshift | Relies on grants | RANDOM() (not repeatable) | Declared, not enforced; sort/dist keys shown as informational indexes | Only SQL valid on both Redshift and PostgreSQL; row estimates from svv_table_info. Tested on PostgreSQL with capability flags; live-cluster checklist in ADR-0014 |
| Snowflake | Session STATEMENT_TIMEOUT_IN_SECONDS; statements tagged anchor-keel | SAMPLE … SEED (1729) | Declared (informational) | Large results download from the account's cloud storage via pre-signed URLs |
| BigQuery | Jobs carry maximumBytesBilled; profiling and probes dry-run first and refuse statements above the limit; label app=anchor-keel | TABLESAMPLE SYSTEM (no seed: re-profiles are not deterministic) | NOT ENFORCED constraints | Key file must be service_account, match the account and name Google's token_uri |
| Databricks | Statements on the configured warehouse with a time limit, then cancelled | Engine sampling | Informational | Never follows redirects; results inline (JSON_ARRAY), never external links; Unity Catalog has no row counts, so tables are counted exactly |
Network rules
- Direct mode (all kinds): private, loopback, link-local and cloud-metadata addresses are refused (
Sources.BlockedHost), and resolved addresses are re-checked at connect time. - Cloud kinds only reach their provider's hosts —
*.snowflakecomputing.com,bigquery.googleapis.com(and regionalbigquery.{region}.rep.googleapis.com),*.cloud.databricks.com,*.azuredatabricks.net,*.gcp.databricks.com— plusKeel:Connectors:AllowedCloudHostsfor private links; anything else isSources.HostNotAllowed. - Private agent mode: see Private agent (in progress).
Dialects
The query compiler generates SQL for all eight kinds; golden suites cover 80 queries (plus policy queries) per dialect.
| Dialect | Quoting · parameters | Specifics |
|---|---|---|
| PostgreSQL | "x" · @p (typed) | date_trunc, weeks start Monday |
| SQL Server | [x] · @p | TOP (n), DATEADD/DATEDIFF grains, UTF-16 hashes |
| MySQL | `x` · @p | TIMESTAMP() grains, <=>, SHA2 |
| Oracle | "X" · :p | FETCH FIRST, TRUNC grains (ISO week), STANDARD_HASH |
| Redshift | "x" · @p | PostgreSQL syntax without ESCAPE, SHA2 |
| Snowflake | "x" · :n | IS NOT DISTINCT FROM, SHA2 |
| BigQuery | `x` · typed @p | DATE_TRUNC(x, WEEK(MONDAY)), UNION DISTINCT, TO_HEX(SHA256()) |
| Databricks | `x` · :p | <=>, sha2 |
Known gaps
- Azure SQL with Entra managed identity (F02 lists it) is not implemented: SQL Server uses user / password.
- Query-log harvesting is not implemented for any kind; Test connection only reports whether a query log is readable.
- Cloud warehouses are verified with emulators and fakes in CI (BigQuery emulator, a fake Databricks server, Snowflake unit and golden tests). The live checks (
LiveCloudConnectorTests) run only when account secrets are provided and are not wired into CI yet;docs/10-devops.mdhas the manual pre-release checklist. - Lineage parsing of backtick-quoted and Snowflake-specific view text is not supported yet.
- Oracle's driver licence must ship with self-hosted packages (a third-party notices file is part of packaging work).