> ## Documentation Index
> Fetch the complete documentation index at: https://docs.emergence.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Secrets Management

> Overview of secrets management options for CRAFT, including ESO + GCP Secret Manager and Infisical as equal deployment alternatives.

# Secrets Management

CRAFT supports two secrets management backends. Both operate through the same **Governance Secrets API** abstraction, applications access secrets through a unified interface regardless of which backend is deployed.

## Secrets API Abstraction

The Governance service exposes a Secrets API that proxies secret read/write operations to the configured backend. This decouples application code from the underlying secrets provider.

```text theme={null}
Application → Governance Secrets API → [ ESO + GCP Secret Manager | Infisical ]
```

## Option A: ESO + GCP Secret Manager

**Recommended for cloud deployments on GCP.** Uses the [External Secrets Operator](https://external-secrets.io/) (ESO) to sync secrets from GCP Secret Manager into Kubernetes Secrets, with [Stakater Reloader](https://github.com/stakater/Reloader) for automatic pod restarts on secret changes.

### Architecture

```mermaid theme={null}
%%{init: {'theme': 'base', 'themeVariables': {'lineColor': '#555555', 'fontFamily': 'sans-serif', 'edgeLabelBackground': '#ffffff'}}}%%
flowchart LR
    TF["Terraform<br/>(GKE module)"] -->|provisions| SM["GCP Secret Manager"]
    SM -->|synced by| ESO["External Secrets<br/>Operator"]
    ESO -->|creates| KS["Kubernetes Secrets"]
    KS -->|mounted in| Pod["Application Pod"]
    RLD["Stakater Reloader"] -->|watches| KS
    RLD -->|restarts| Pod

    classDef infra fill:#0072B2,stroke:#555555,color:#fff
    classDef process fill:#E69F00,stroke:#555555,color:#000
    classDef data fill:#009E73,stroke:#555555,color:#000
    classDef app fill:#56B4E9,stroke:#555555,color:#000
    class TF,ESO,RLD infra
    class SM process
    class KS data
    class Pod app
```

### Components

| Component                     | Purpose                                                                      |
| ----------------------------- | ---------------------------------------------------------------------------- |
| **GCP Secret Manager**        | Cloud-native secret store, encrypted at rest, audit-logged, IAM-controlled   |
| **External Secrets Operator** | Kubernetes operator that syncs GCP SM secrets into K8s Secrets on a schedule |
| **ClusterSecretStore**        | Cluster-scoped ESO resource that configures the GCP SM connection            |
| **ExternalSecret**            | Namespace-scoped resource mapping a GCP SM secret to a K8s Secret            |
| **Stakater Reloader**         | Watches K8s Secrets for changes; triggers rolling restarts of annotated pods |

### Secret Naming Convention

Secrets in GCP Secret Manager follow a `<service>-<environment>` naming pattern. Specific names are environment-specific.

### Authentication

ESO authenticates to GCP Secret Manager using [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity), no service account keys are stored in the cluster.

### Rotation

When a secret value is updated in GCP Secret Manager:

1. ESO detects the change on its next sync interval (configurable, default 1h)
2. The K8s Secret is updated
3. Stakater Reloader detects the change (via annotation `reloader.stakater.com/auto: "true"`) and triggers a rolling restart

For immediate rotation, manually trigger an ESO sync or restart the affected deployment.

<Warning>
  Secrets injected as environment variables require a pod restart to take effect, even after ESO syncs the updated value to the Kubernetes Secret. Stakater Reloader (included in reference infrastructure) automates this by triggering rolling restarts when annotated Secrets change. For security incidents, manually restart the affected deployment immediately after triggering an ESO sync. Consider reducing the default 1-hour sync interval for high-sensitivity secrets.
</Warning>

### Deployment

See [Infrastructure Secrets](/deployment/infrastructure/secrets) for the step-by-step deployment guide.

***

## Option B: Infisical

**Recommended for on-premises and cloud-agnostic deployments.** [Infisical](https://infisical.com/) provides application-level envelope encryption with a self-hostable server and native Kubernetes authentication.

### Architecture

Infisical runs as a subchart within the em-runtime Helm release and stores secrets in its own PostgreSQL database (`infisical`). Services authenticate via Infisical's Kubernetes machine identity.

### Components

| Component               | Purpose                                                                 |
| ----------------------- | ----------------------------------------------------------------------- |
| **Infisical server**    | Self-hosted secret management platform (deployed as Helm subchart)      |
| **Infisical SDK**       | Python/TypeScript SDK used by em-runtime services at startup            |
| **Machine Identity**    | Kubernetes service account-based authentication (no static credentials) |
| **Envelope encryption** | Secrets encrypted at rest with a per-secret encryption key hierarchy    |

### Configuration

Infisical is enabled by default in the em-runtime Helm chart:

```yaml theme={null}
infisical:
  enabled: true
  resources:
    requests:
      cpu: "250m"
      memory: "512Mi"
```

To use an external Infisical instance instead of the subchart:

```yaml theme={null}
infisical:
  enabled: false

governance:
  extraEnvVars:
    - name: INFISICAL_URL
      value: "https://infisical.example.com"
    - name: INFISICAL_PROJECT_ID
      value: "your-project-id"
```

### Rotation

Infisical supports secret rotation via its dashboard or API. Service pods must be restarted to pick up new secret values (unlike ESO + Reloader which handles this automatically).

<Warning>
  Infisical requires manual pod restarts to pick up rotated secrets. For regulated deployments, configure Stakater Reloader (included in reference infrastructure) to watch Infisical-managed Kubernetes Secrets and trigger automated rolling restarts on change. Document the rotation procedure including verification steps.
</Warning>

***

## Comparison

| Attribute              | ESO + GCP Secret Manager  | Infisical                          |
| ---------------------- | ------------------------- | ---------------------------------- |
| **Deployment model**   | Cloud-native (GCP)        | Self-hosted or cloud-agnostic      |
| **Cloud dependency**   | Requires GCP              | No cloud dependency                |
| **Encryption at rest** | GCP KMS (managed)         | Envelope encryption (self-managed) |
| **Auto-rotation**      | Yes (Stakater Reloader)   | Manual pod restart required        |
| **Audit logging**      | GCP Cloud Audit Logs      | Infisical audit logs               |
| **On-prem support**    | No                        | Yes                                |
| **Helm chart**         | ESO Helm chart (separate) | Infisical subchart (included)      |
| **Auth mechanism**     | GCP Workload Identity     | Kubernetes machine identity        |

## Choosing a Backend

* **GKE (GCP cloud deployment)**: ESO + GCP Secret Manager is the default and is provisioned by Terraform
* **On-premises or non-GCP cloud**: Infisical (included in Helm chart as subchart)
* **Multi-cloud (EKS/AKS)**: Either, ESO supports AWS Secrets Manager and Azure Key Vault; Infisical is cloud-agnostic

<Info>
  Both backends are supported and maintained. The choice depends on your deployment environment, not the platform version.
</Info>

## Related

<CardGroup cols={2}>
  <Card title="Infrastructure Secrets" icon="server" href="/deployment/infrastructure/secrets">
    Step-by-step guide to deploying ESO or Infisical in your cluster.
  </Card>

  <Card title="Data Classification" icon="shield" href="/security/data-classification">
    How credentials are classified and protected within the platform.
  </Card>

  <Card title="Helm Configuration" icon="gear" href="/deployment/helm/configuration">
    Helm values for enabling and configuring secrets backends.
  </Card>

  <Card title="Network Security" icon="network-wired" href="/security/network-security">
    Network-level controls that protect secret access paths.
  </Card>
</CardGroup>