Run the Docker Compose stack

Use this how-to to run a local OpenBao Observability reference stack with a three-node OpenBao Raft cluster, PostgreSQL, Prometheus, Loki, Grafana Alloy, and Grafana. The stack is for local evaluation and contract validation.

[!WARNING] This stack uses HTTP, a local static seal key, deterministic local credentials, and unauthenticated metrics access inside the Compose network. You must not use it for production, shared environments, or sensitive data.

Before you begin

Install Docker with Docker Compose.
Run commands from the repository root.
Generate the latest rule artifacts before you start the stack.
Reset the Compose volumes when you switch from an older single-node stack.

Start the stack

Generate Prometheus Operator and native Prometheus rule files.
```
make generate
```
Remove old local Compose volumes when you change the OpenBao topology.
```
make compose-reset
```
Start the Compose services.
```
make compose-up
```
Check that the services are running.
```
docker compose --project-directory examples/docker-compose -f examples/docker-compose/compose.yaml ps -a
```
Expected services:
- openbao-seal-init
- openbao-node0
- openbao-node1
- openbao-node2
- openbao-audit-canary
- postgres
- prometheus
- loki
- alloy
- grafana

Start the audit archive profile

Use this optional profile when you want to exercise the OpenBaoAuditArchiveDegraded alert end to end in the local stack.

Start the stack with the audit archive health exporter.
```
make compose-audit-archive-up
```
This command uses examples/docker-compose/compose.audit-archive.yaml, builds the example exporter, starts a local status writer, and runs Prometheus with prometheus.audit-archive.yml.

Check the exporter endpoint.

curl -fsS http://127.0.0.1:19110/metrics

Expected output includes:

openbao_audit_archive_enabled{backend="compose-demo",pipeline="openbao-audit-archive"} 1
openbao_audit_archive_delivery_success{backend="compose-demo",pipeline="openbao-audit-archive"} 1

Check that Prometheus sees the archive health target.

curl -fsS -G http://127.0.0.1:19090/api/v1/query \
  --data-urlencode 'query=up{job="openbao-audit-archive-health"}'

Stop the profile with the matching target when you no longer need it.
```
make compose-audit-archive-down
```

Open the local endpoints

Service	URL	Purpose
OpenBao node 0	`http://127.0.0.1:18200`	Raft bootstrap node and expected active node.
OpenBao node 1	`http://127.0.0.1:18201`	Raft follower.
OpenBao node 2	`http://127.0.0.1:18202`	Raft follower.
PostgreSQL	`127.0.0.1:15432`	Dynamic database secret backend for local lease activity.
Prometheus	`http://127.0.0.1:19090`	Metrics, recording rules, and alerts.
Loki	`http://127.0.0.1:13100`	Local log backend.
Alloy	`http://127.0.0.1:12345`	Collector status UI.
Audit archive health	`http://127.0.0.1:19110`	Optional archive health exporter profile.
Grafana	`http://127.0.0.1:13000`	Explore metrics, logs, and dashboards.

Grafana uses admin / admin by default. Change the local password in examples/docker-compose/.env when you need a different local credential. The stack provisions the generated dashboards in the OpenBao folder. Start with OpenBao overview, then use the focused HA/Raft, audit, operational logs, auth and identity, token and lease lifecycle, database secrets, Transit, PKI, secret engines and mounts, runtime and storage, namespaces and scale, Kubernetes platform, and SLO and availability dashboards when you need deeper context.

Understand the local OpenBao setup

The stack starts openbao-seal-init first. That one-shot container writes a 32-byte static seal key into a named Docker volume if the key does not already exist.

openbao-node0 starts next and performs self-initialization. Only node 0 has an initialize block. The self-initialization creates:

the userpass auth method,
the approle auth method,
a KV v2 secret/ mount,
a KV v1 kv-v1/ mount,
a database secrets database/ mount backed by the local PostgreSQL service,
a Transit transit/ mount and local payments key,
a PKI pki/ mount, local root CA, and observability-dot-local role,
a local compose-admin policy,
local app-reader, app-writer, and identity-auditor policies,
a local openbao-metrics policy,
a local audit-canary policy,
the demo-admin, demo-reader, and demo-writer users,
an observability-app AppRole and secret ID,
a PostgreSQL readonly dynamic credential role,
demo identity entities, entity aliases, and internal identity groups,
a sample secret/data/observability/audit-canary secret,
a sample secret/data/apps/payments/api secret, and
a deterministic local audit canary token named openbao-observability-audit-canary-token, and
a deterministic local metrics token named openbao-observability-metrics-token.

openbao-node1 and openbao-node2 use retry_join to join node 0. They do not initialize OpenBao, and they unseal through the shared local static seal key.

Prometheus scrapes all three OpenBao nodes. The local OpenBao listener enables unauthenticated_metrics_access so standby nodes expose metrics in this local profile. Use a private metrics-only listener or equivalent network controls for production all-node scraping.

openbao-audit-canary reads secret/data/observability/audit-canary every 60 seconds with a token that only has the audit-canary policy. This creates a known audited request for the OpenBaoAuditCanaryMissing alert.

The optional audit archive profile starts a local status writer and the audit archive health exporter. The status writer updates a demo archive status file every 30 seconds. The exporter exposes the reference openbao_audit_archive_* metrics that drive the OpenBaoAuditArchiveDegraded alert.

Verify the result

Check OpenBao health on each node.

curl -sS http://127.0.0.1:18200/v1/sys/health
curl -sS http://127.0.0.1:18201/v1/sys/health
curl -sS http://127.0.0.1:18202/v1/sys/health

This check does not use -f because standby nodes return a standby health status code while still reporting an initialized and unsealed node.

Expected output includes:

{
  "initialized": true,
  "sealed": false,
  "version": "2.5.4"
}

export BAO_ADDR=http://127.0.0.1:18200
bao login -method=userpass username=demo-admin password=openbao-observability

Inspect the local auth and identity demo data.
```
bao auth list
bao list identity/entity/name
bao list identity/group/name
bao read auth/approle/role/observability-app/role-id
```
Expected output includes the userpass/ and approle/ auth methods, demo-admin, demo-reader, demo-writer, platform-team, and payments-team.
Read a local dynamic database credential and inspect its lease.
```
bao read database/creds/readonly
bao list sys/leases/lookup/database/creds/readonly
```
Expected output includes a lease ID under database/creds/readonly/.
Run the production-like fixture scenario.
```
make fixtures-scenarios
```
The scenario performs userpass and AppRole logins, KV v1 and KV v2 activity, identity activity, token create/lookup/renew/revoke operations, root and namespace database credential lease lookup/renew/revoke operations, Transit encrypt/decrypt operations, PKI certificate issue/revoke operations, minimal nested namespace KV activity, and expected denied requests.
Check Raft peers.
```
bao operator raft list-peers
```
Expected output includes node0, node1, and node2 as voters.
Check Autopilot state.
```
bao operator raft autopilot state
```
Expected output includes a healthy cluster and a failure tolerance of 1 after Autopilot converges.

Check Prometheus readiness.

curl -fsS http://127.0.0.1:19090/-/ready

Expected output:

Prometheus Server is Ready.

Check the OpenBao scrape targets.

curl -fsS -G http://127.0.0.1:19090/api/v1/query \
  --data-urlencode 'query=up{job="openbao"}'

Expected output includes three up series with value 1.

Check Raft recording rules.

curl -fsS -G http://127.0.0.1:19090/api/v1/query \
  --data-urlencode 'query=openbao:raft_peers:max'

curl -fsS -G http://127.0.0.1:19090/api/v1/query \
  --data-urlencode 'query=openbao:autopilot_failure_tolerance:max'

Expected output includes peer count 3 and failure tolerance 1 after the rule evaluation interval passes.

Check Loki stream labels.

curl -fsS http://127.0.0.1:13100/loki/api/v1/label/log_stream/values

Expected output includes:

["openbao.audit","openbao.operational"]

Check the audit canary event.

curl -fsS -G http://127.0.0.1:13100/loki/api/v1/query \
  --data-urlencode 'query=count_over_time({log_stream="openbao.audit"} | json request_path="request.path" | request_path="secret/data/observability/audit-canary" [5m])'

Expected output includes at least one canary event after the openbao-audit-canary service has completed a read.

Check Grafana health.

curl -fsS -u admin:admin http://127.0.0.1:13000/api/health

Expected output includes "database": "ok".

Validate dashboard queries against the local backends.
```
make validate-dashboard-queries
```
Expected output confirms that dashboard PromQL and LogQL queries validate against Prometheus and Loki.

Query the data

In Grafana, open Dashboards, select the OpenBao folder, and open OpenBao overview, OpenBao HA/Raft, OpenBao audit overview, OpenBao operational logs, OpenBao audit investigation, or OpenBao auth and identity, OpenBao token and lease lifecycle, OpenBao database secrets, or OpenBao secret engines and mounts.

Use the provisioned Prometheus data source to run these PromQL queries:

up{job="openbao"}

openbao:core_in_flight_requests:max

openbao:core_handle_request:rate5m

openbao:core_handle_request:avg5m

openbao:core_handle_login_request:avg5m

openbao:core_check_token:avg5m

openbao:raft_peers:max

openbao:autopilot_node_healthy:min

openbao:token_count:max30m

openbao:expire_num_irrevocable_leases:max

Use the provisioned Loki data source to run these LogQL queries:

{log_stream="openbao.operational"}

{log_stream="openbao.operational"} |~ "\"@level\":\"(warn|error)\""

{log_stream="openbao.audit"}

Use this query when you need auth and identity audit events:

{log_stream="openbao.audit"} | json request_path="request.path" | request_path=~"(auth/.*|sys/auth/.*|identity/.*)"

Use this query when you need token and lease lifecycle audit events:

{log_stream="openbao.audit"} | json request_path="request.path" | request_path=~"(auth/token/.*|sys/leases/.*)"

Use this query when you need secret engine and mount activity:

{log_stream="openbao.audit"} | json request_path="request.path" | request_path=~"(secret|kv-v1|database|transit|pki|sys/mounts)(/.*)?"

Use this query when you need the audit canary event:

{log_stream="openbao.audit"} | json request_path="request.path" | request_path="secret/data/observability/audit-canary"

Use this query when you need an audit request ID drilldown:

{log_stream="openbao.audit"} | json request_id="request.id" | request_id=~"<request_id>"

Use this query when you need audit event volume by Raft node:

sum by (node_id) (count_over_time({log_stream="openbao.audit"}[5m]))

Filter by Raft node when you need one node’s logs:

{log_stream="openbao.operational", node_id="node1"}

Change local settings

Create a local environment override when you need different ports, images, or Grafana credentials.

cp examples/docker-compose/.env.example examples/docker-compose/.env

Then edit examples/docker-compose/.env and restart the stack.

make compose-down
make compose-up

The .env file is ignored by Git.

Stop the stack

Stop containers and keep named volumes:

make compose-down

Stop containers and remove named volumes, including the local Raft data and static seal key:

make compose-reset

Stop containers and remove volumes for the optional audit archive profile:

make compose-audit-archive-reset

Troubleshooting

Prometheus has no OpenBao targets

Regenerate the rule files and restart the stack.

make generate
make compose-down
make compose-up

Check the OpenBao node logs if Prometheus still does not scrape the nodes.

docker compose --project-directory examples/docker-compose -f examples/docker-compose/compose.yaml logs openbao-node0
docker compose --project-directory examples/docker-compose -f examples/docker-compose/compose.yaml logs openbao-node1
docker compose --project-directory examples/docker-compose -f examples/docker-compose/compose.yaml logs openbao-node2

The Raft cluster does not reach three voters

Check that node 0 initialized before node 1 and node 2 started.

docker compose --project-directory examples/docker-compose -f examples/docker-compose/compose.yaml logs openbao-node0

Then inspect the follower logs for retry join errors.

docker compose --project-directory examples/docker-compose -f examples/docker-compose/compose.yaml logs openbao-node1 openbao-node2

Run make compose-reset when a local Raft volume contains a failed or stale test cluster.

Loki has no OpenBao streams

Check that Alloy is tailing the per-node OpenBao files.

docker compose --project-directory examples/docker-compose -f examples/docker-compose/compose.yaml logs alloy

The Alloy logs include start tailing file for audit and operational log files when collection is active.

Grafana has no data sources

Check Grafana provisioning logs.

docker compose --project-directory examples/docker-compose -f examples/docker-compose/compose.yaml logs grafana

The stack mounts provisioning files from examples/docker-compose/grafana/provisioning.

The audit archive health target is down

Use the audit archive profile targets when you start and stop the stack.

make compose-audit-archive-up

Then check the status writer and exporter logs.

docker compose --project-directory examples/docker-compose \
  -f examples/docker-compose/compose.yaml \
  -f examples/docker-compose/compose.audit-archive.yaml \
  --profile audit-archive logs audit-archive-status-writer audit-archive-health

What’s next

Inspect generated rule files in generated/prometheus/.
Inspect prefix-specific rule variants in generated/prometheus/vault-prefix/ and generated/prometheus/openbao-prefix/.
Inspect Prometheus Operator rule artifacts in generated/prometheusrules/.
Use contracts/alerts/ as the source of truth for local alert changes.
Use OpenBao Raft and Autopilot health when a Raft or Autopilot alert fires.
Use Audit archive degraded when the optional archive health profile reports degraded delivery.

Source: OpenBao documents static seal configuration in the OpenBao static seal documentation . OpenBao documents self-initialization in the OpenBao self-initialization documentation . OpenBao documents integrated storage and Raft join behavior in the OpenBao integrated storage documentation . OpenBao documents the Prometheus metrics endpoint in the OpenBao telemetry documentation . OpenBao documents configuration-defined audit devices in the OpenBao declarative audit documentation . Grafana documents local file collection in the Alloy file source documentation .