> ## Documentation Index
> Fetch the complete documentation index at: https://docs.emergence.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# CRAFT Data Insights Overview
> Multi-agent conversational data analytics using the A2A protocol for natural-language to SQL generation, analysis, and visualization.
# Data Insights Overview
Ask natural-language questions about your data and get instant SQL-generated answers, visualizations, and insights. **Data Insights** coordinates a team of AI agents that generate SQL, execute queries, analyze results, and produce visualizations -- all in real time via streaming.
Data Insights is built on the **A2A (Agent-to-Agent) protocol**, an open standard for inter-agent communication using JSON-RPC 2.0 over Server-Sent Events (SSE). Each agent is independently deployable and discoverable via Agent Cards.
## Who Is It For
Ask questions about your data in plain language and get instant answers, visualizations, and reports -- no SQL or technical skills required.
Explore data through multi-turn conversations, generate visualizations, and export results. The AI handles SQL generation so you can focus on analysis.
Get AI-powered insights from enterprise data to support strategic decisions. Generate on-demand reports and discover trends through natural-language queries.
Validate SQL generation, review query plans, and configure data connections. Monitor agent performance and query execution through observability tooling.
## How It Works
```mermaid theme={null}
%%{init: {'theme': 'base', 'themeVariables': {'lineColor': '#555555', 'fontFamily': 'sans-serif', 'edgeLabelBackground': '#ffffff', 'actorBkg': '#56B4E9', 'actorBorder': '#555555', 'actorTextColor': '#000000', 'actorLineColor': '#555555', 'signalColor': '#555555', 'signalTextColor': '#555555', 'noteBkgColor': '#F0E442', 'noteBorderColor': '#555555', 'noteTextColor': '#000000', 'activationBkgColor': '#E69F00', 'activationBorderColor': '#555555'}}}%%
sequenceDiagram
participant U as User
participant SVC as Talk2Data Service
participant IA as Insights Agent
participant SQL as Text2SQL Agent
participant PG as PostgreSQL
U->>SVC: Natural-language question
SVC->>IA: Route message (A2A)
IA->>SQL: Delegate SQL generation (A2A)
SQL->>SQL: Analyze schema + generate SQL
SQL->>PG: Execute validated query
PG-->>SQL: Result set
SQL-->>IA: Query results (A2A artifact)
IA->>IA: Analyze results, generate insights
IA-->>SVC: Insights + visualizations (SSE stream)
SVC-->>U: Real-time streamed response
```
A user types a natural-language question in the chat interface, such as "What were the top 10 products by revenue last quarter?"
The Talk2Data Service (REST + SSE gateway) creates a session, routes the message to the Insights Agent, and establishes an SSE stream back to the client.
The Insights Agent runs an agentic loop: it reasons about the question, selects tools, and delegates SQL generation to the Text2SQL Agent.
The Text2SQL Agent analyzes the database schema, generates SQL using an LLM, validates it with `sqlglot`, and executes the query against the user's data connection.
Query results, analysis, and visualizations stream back to the user in real time via A2A events (`TaskStatusUpdateEvent` for progress, `TaskArtifactUpdateEvent` for results).
## Architecture
Building a similar solution? Use the [Solution Developer Guide › Starter Templates](/guides/solution-dev/starter-templates) — the Tier 3 template mirrors this layered agent shape with copy-pasteable scaffolds.
The solution follows a layered agent architecture:
```text theme={null}
Client / React UI
|
Talk2Data Service (REST + SSE gateway, port 8080)
|
Insights Agent (agentic loop, reasoning, tool orchestration, port 8002)
|--- Text2SQL Agent (NL-to-SQL generation, validation, execution, port 8001)
|--- Coding Agent (LLM-generated Python code execution, port 8004)
|--- MCP Plotly Server (visualization tools via MCP, port 8000)
```
Text2SQL and Coding agents communicate with the Insights Agent via A2A, while the MCP Plotly Server is accessed via MCP. The Insights Agent dynamically discovers both sub-agents and MCP tools on each request.
## A2A Protocol Integration
All inter-agent communication uses the A2A protocol:
Each agent publishes a JSON manifest at `/.well-known/agent-card.json` that describes its identity, skills, and capabilities. Agent Cards enable automatic discovery by the Talk2Data service and Insights Agent at request time.
Key fields include:
* **name** -- human-readable agent name
* **skills** -- list of capabilities with input/output schemas
* **capabilities** -- supported features (streaming, multi-turn, etc.)
* **endpoint** -- the agent's A2A service URL
A2A messages contain typed parts:
| Part Type | Usage |
| ---------- | --------------------------------------------------------- |
| `TextPart` | Natural-language text (questions, analysis, explanations) |
| `DataPart` | Structured data (datasource configs, query parameters) |
| `FilePart` | Binary artifacts (charts, exported files) |
Agents emit events for real-time frontend updates:
| Event | Purpose |
| ------------------------- | -------------------------------------------------------------- |
| `TaskStatusUpdateEvent` | Progress messages ("Analyzing schema...", "Generating SQL...") |
| `TaskArtifactUpdateEvent` | Final results (data tables, charts, text analysis) |
Every pipeline step emits a status event before starting work, providing real-time visibility into the agent's reasoning process.
The A2A `context_id` maps to the session ID for multi-turn conversation state. This enables agents to maintain context across multiple questions in the same conversation.
## Pipeline Framework
Agent workflows are built on the `commons.pipeline.Pipeline` state-machine framework:
* **Steps** are named functions that perform a unit of work
* Each step returns a `Transition` object with a `goto` target (next step, `break`, or `error`)
* The pipeline supports **cooperative cancellation** for graceful shutdown
* Unexpected exceptions are wrapped in `StepError` for structured error handling
The Text2SQL agent uses this framework for its generate-validate-execute flow:
```text theme={null}
generate_sql -> validate_sql -> execute_query -> format_results
```
## LLM Integration
Data Insights uses **LiteLLM** for provider-agnostic LLM access:
| Feature | Details |
| ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Client** | `commons.llm.LLMClient` wrapping LiteLLM |
| **Model format** | `provider/model` (e.g., `vertex_ai/gemini-3.5-flash`, `vertex_ai/claude-opus-4-8`, `openai/gpt-5.5`) |
| **Observability** | Langfuse LLM tracing auto-enabled when `LANGFUSE_HOST` is set. Chat dispatch and the pipeline executor are auto-instrumented with `@observe` decorators, A2A trace context propagates across services so a single conversation produces one unified trace, trace IDs are seeded deterministically from `turn_id`, and per-iteration spans are emitted inside agent loops. See [Langfuse Setup](/guides/langfuse-setup) for configuration and [Langfuse Overview](/deployment/observability/langfuse) for the full tracing model. |
| **Configuration** | Per-service LLM env vars (`TALK2DATA_TEXT2SQL_LLM_MODEL`, `TALK2DATA_INSIGHTS_LLM_MODEL`, etc.) via `pydantic_settings.BaseSettings` |
Credentials are never hardcoded. All API keys and connection strings are loaded from environment variables or `.env` files via `pydantic_settings.BaseSettings`.
## Database Schema
The `talk2data` database schema stores conversation state and artifacts:
| Table | Purpose |
| -------------------------- | -------------------------------------------------------- |
| **sessions** | Chat sessions with user and project context |
| **conversation\_messages** | Individual messages within a session |
| **artifacts** | Generated outputs (SQL queries, results, visualizations) |
| **feedback** | User feedback on agent responses |
* **Primary keys**: UUID strings
* **Timestamps**: `DateTime(timezone=True)` with UTC
* **ORM**: SQLAlchemy 2.0+ async with `asyncpg` driver
* **Migrations**: Alembic in `packages/common-db/`
## REST Endpoints
The Talk2Data Service exposes the following REST and SSE endpoints:
| Endpoint | Purpose |
| ------------------------------ | ----------------------------------------------------------- |
| `GET /talk2data/chat/sessions` | List active chat sessions |
| `POST /talk2data/chat/*` | Start or continue a chat session (SSE streaming) |
| `POST /talk2data/v1/sample` | Preview rows from a data connection table (no LLM involved) |
The `/talk2data/v1/sample` endpoint is useful for data exploration before composing a question — it fetches a configurable number of rows from a named table via an existing data connection, using a fully qualified table name (`database.schema.table`). See [Text-to-SQL](/data-insights/text-to-sql) for request and error details.
## Platform Integration
Data Insights integrates with the platform layer for:
| Capability | Platform Service |
| ------------------ | ---------------------------------------------------------------- |
| **Authentication** | Governance (JWT validation via the platform identity provider) |
| **Authorization** | Governance (permission checks via the authorization service SDK) |
\| **Data connections** | Assets (configured database connections) |
\| **Asset management** | Assets (data connections, artifacts, files, models) |
The integration uses auto-generated Python SDKs from the platform's OpenAPI specs. Data Insights never implements its own permission checks.
## Next Steps
Learn how to use the conversational interface for data analysis.
Understand how natural-language questions are converted to SQL queries.
See how Data Insights agents are registered and discovered.
Configure data connections for your databases.