> ## Documentation Index
> Fetch the complete documentation index at: https://docs.emergence.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Source Setup

> How to connect a PostgreSQL database to CRAFT as a data source for querying and governance.

# Connect a PostgreSQL Database

This guide walks you through registering a PostgreSQL database as a data connection in CRAFT. Once connected, the database is available to solutions like Data Insights (for NL-to-SQL queries) and Data Governance (for profiling and enrichment).

## Prerequisites

<Info>
  Before you begin, ensure you have:

  * A running CRAFT instance
  * A valid JWT token with the `developer` or `admin` role
  * Network connectivity between the platform and your PostgreSQL instance
  * A PostgreSQL user with read access to the target schemas
</Info>

## How Data Connections Work

Data connections are managed by the **Assets** service (reachable at `/api/assets` through the platform gateway). When you register a connection:

1. The connection metadata (host, port, database name) is stored in the Assets database
2. Credentials (username, password) are stored securely via the platform Secrets API (Infisical or ESO + GCP Secret Manager) -- never in the Assets database
3. Solutions retrieve the connection at runtime via the Assets SDK and use the credentials to establish a live database session
4. All connections are scoped to an **organization** and **project** for multi-tenant isolation

<Warning>
  Data connection credentials are never exposed in API responses. They are injected at runtime only when a solution needs to establish a connection.
</Warning>

## Step 1: Prepare Your Database

Ensure your PostgreSQL instance is configured to accept connections from the platform.

<Steps>
  <Step title="Create a read-only user">
    ```sql theme={null}
    -- Connect to your PostgreSQL instance as a superuser
    CREATE USER emergence_reader WITH PASSWORD 'your-secure-password';

    -- Grant read access to the target schema
    GRANT USAGE ON SCHEMA public TO emergence_reader;
    GRANT SELECT ON ALL TABLES IN SCHEMA public TO emergence_reader;

    -- Ensure future tables are also accessible
    ALTER DEFAULT PRIVILEGES IN SCHEMA public
      GRANT SELECT ON TABLES TO emergence_reader;
    ```
  </Step>

  <Step title="Configure network access">
    Update `pg_hba.conf` to allow connections from the platform's network:

    ```text theme={null}
    # Allow connections from the Kubernetes cluster CIDR
    host    your_database    emergence_reader    192.0.2.0/24    scram-sha-256
    ```

    Reload the configuration:

    ```bash theme={null}
    sudo systemctl reload postgresql
    ```
  </Step>

  <Step title="Verify connectivity">
    From a pod in the platform's Kubernetes cluster, test the connection:

    ```bash theme={null}
    psql -h <db-host> -U emergence_reader -d your_database -c "SELECT 1;"
    ```
  </Step>
</Steps>

## Step 2: Register the Data Connection

Use the Assets API to register the connection in the platform.

```bash theme={null}
curl -X POST "https://<platform-host>/api/assets/data" \
  -H "Authorization: Bearer $TOKEN" \
  -H "X-Project-ID: <your-project-id>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "production-analytics",
    "description": "Production analytics database (read-only)",
    "connection_type": "postgres",
    "database_config": {
      "host": "db.example.com",
      "port": 5432,
      "database": "analytics",
      "ssl_mode": "require"
    },
    "credentials": {
      "username": "emergence_reader",
      "password": "your-secure-password"
    }
  }'
```

A successful registration returns `201 Created` with the resource:

```json theme={null}
{
  "resource_uri": "data:<org-id>:<project-id>:production-analytics",
  "name": "production-analytics",
  "connection_type": "postgres",
  "status": "PENDING",
  "project_id": "<project-id>",
  "owner_id": "<user-id>",
  "tags": [],
  "created_at": "2026-05-31T23:40:43.698045Z"
}
```

<Note>
  * The platform constructs `resource_uri` as `data:<org-id>:<project-id>:<name>` (or a kebab-case `resource_uri` override if you supply one in the request).
  * The `credentials` field is encrypted and stored via the platform Secrets API upon registration. It will **not** appear on subsequent `GET` responses (use `GET /assets/data/{resource_uri}/secret` if a solution needs to read the credentials at runtime).
  * For object-storage connections (`s3`, `gcs`, `minio`), use `storage_config` instead of `database_config`. For filesystem connections (`nfs`, `smb`), use `filesystem_config`. The complete list of supported `connection_type` values is `GET /api/assets/data/types`.
</Note>

## Step 3: Test the Connection

Verify the platform can reach your database. There are two flavors of verify:

```bash theme={null}
# Verify an already-registered connection (idempotent, no body):
curl -X POST \
  "https://<platform-host>/api/assets/data/<resource-uri>/verify" \
  -H "Authorization: Bearer $TOKEN" \
  -H "X-Project-ID: <your-project-id>"
```

To verify a candidate connection **before** registration (e.g. as part of a
UI wizard), `POST /api/assets/data/verify` with the same body shape as the
create request — no `name`/`description` needed:

```bash theme={null}
curl -X POST "https://<platform-host>/api/assets/data/verify" \
  -H "Authorization: Bearer $TOKEN" \
  -H "X-Project-ID: <your-project-id>" \
  -H "Content-Type: application/json" \
  -d '{
    "connection_type": "postgres",
    "database_config": { "host": "db.example.com", "port": 5432, "database": "analytics" },
    "credentials": { "username": "emergence_reader", "password": "..." }
  }'
```

Both flavors return the same shape:

```json theme={null}
{
  "success": true,
  "message": "Connection verified",
  "latency_ms": 15.2,
  "details": { "server_version": "17.2", "schemas": ["public", "analytics"] }
}
```

A failed verify returns `success: false` and a `message` describing the
failure mode (`details` and `latency_ms` may be `null`).

## Step 4: Use the Connection with Data Insights

Once registered, the connection's `resource_uri` (returned in Step 2) is
what you pass to other platform services. The Data Insights solution
consumes it from its chat UI / SDK; the underlying HTTP contract lives at
`POST /api/talk2data/chat/messages` (paired with `/api/talk2data/chat/sessions`
for session management — see the [Data Insights › Text-to-SQL guide](/data-insights/text-to-sql)
for the full message-and-session model). The Data Insights UI handles
session creation implicitly when a user opens a new chat against this
connection.

## Connection Configuration Reference

<Tabs>
  <Tab title="PostgreSQL">
    | Parameter  | Type    | Required | Description                                                                       |
    | ---------- | ------- | -------- | --------------------------------------------------------------------------------- |
    | `host`     | string  | Yes      | Database hostname or IP                                                           |
    | `port`     | integer | No       | Port number (default: 5432)                                                       |
    | `database` | string  | Yes      | Database name                                                                     |
    | `schema`   | string  | No       | Default schema (default: `public`)                                                |
    | `ssl_mode` | string  | No       | SSL mode: `disable` (no TLS) or `require` (TLS, encrypt-only). Default `disable`. |
  </Tab>

  <Tab title="Credentials">
    | Parameter  | Type   | Required | Description       |
    | ---------- | ------ | -------- | ----------------- |
    | `username` | string | Yes      | Database username |
    | `password` | string | Yes      | Database password |
  </Tab>
</Tabs>

## Security Considerations

<AccordionGroup>
  <Accordion title="Credential Storage">
    Credentials are stored via the platform Secrets API (backed by Infisical or ESO + GCP Secret Manager). They are encrypted at rest and organization-scoped. Credentials are injected at runtime only when a solution needs to establish a connection.
  </Accordion>

  <Accordion title="Network Security">
    Configure your database firewall to accept connections only from the platform's Kubernetes cluster CIDR. Use `ssl_mode: require` for production deployments so traffic is TLS-encrypted between the platform and your database. Certificate-pinning modes (`verify-ca`, `verify-full`) are not currently accepted by the Assets API.
  </Accordion>

  <Accordion title="Principle of Least Privilege">
    Always create a dedicated read-only user for the platform. Never use a superuser or an account with write privileges unless the solution explicitly requires write access.
  </Accordion>
</AccordionGroup>

## Troubleshooting

<AccordionGroup>
  <Accordion title="Connection test fails with timeout">
    Verify network connectivity between the Kubernetes cluster and your database. Check firewall rules, security groups, and `pg_hba.conf` configuration. Ensure the database host is resolvable from within the cluster.
  </Accordion>

  <Accordion title="Authentication failure">
    Confirm the username and password are correct. Check that the user exists in the target database and has the `CONNECT` privilege. Verify the `pg_hba.conf` entry allows the authentication method being used.
  </Accordion>

  <Accordion title="SSL connection required">
    If your database requires SSL, set `ssl_mode: require`. Certificate-pinning modes (`verify-ca`, `verify-full`) are not currently accepted by the Assets API — track this if your database requires server-cert verification at the client.
  </Accordion>
</AccordionGroup>

## Next Steps

<CardGroup cols={2}>
  <Card title="Chat with Data" icon="comments" href="/data-insights/chat-with-data">
    Learn how to ask natural-language questions against your connected database.
  </Card>

  <Card title="Data Profiling" icon="magnifying-glass" href="/data-governance/data-profiling">
    Profile your connected database to understand data quality and structure.
  </Card>

  <Card title="Backup & Restore" icon="box-archive" href="/guides/backup-restore">
    Learn how to back up your platform data including connection configurations.
  </Card>

  <Card title="Network Security" icon="shield" href="/security/network-security">
    Review network security policies for data connections.
  </Card>
</CardGroup>