Persistent PostgreSQL data lost after docker compose down/up in CrewAI setup

[markusrock]

I’m trying to add a persistent PostgreSQL database to my CrewAI Docker Compose stack so that data survives container restarts. Despite defining a volume and wiring up the DATABASE_URL, every time I run:

docker compose down
docker compose up -d

all database tables and rows are gone. It’s as if the database never persisted or CrewAI never actually connected to it.

---

Environment

• Docker Compose v2.38.2 on Ubuntu 24.04
• Docker Engine (installed from Docker’s official repo)
• CrewAI FastAPI service (custom Dockerfile, built from ./crewai)
• Compose network: ki_net

---

Relevant snippets

docker-compose.yml

version: "3.8"

services:
  crewai:
    build: ./crewai
    image: ai-crew/crewai:latest
    command: uvicorn main:app --host 0.0.0.0 --port 8002
    environment:
      OPENAI_API_KEY: sk-… 
      DATABASE_URL: postgres://crewai:supersecret@db:5432/crewai
    ports:
      - "8002:8002"
    volumes:
      - ../ai-crew-data/projects:/app/projects
      - ../ai-crew-data/agents:/app/agents
    depends_on:
      - db
    networks: [ki_net]

  chromadb:
    image: chromadb/chroma:latest
    volumes:
      - ../ai-crew-data/chroma-data:/chroma/data
    networks: [ki_net]

  db:
    image: postgres:15
    restart: always
    environment:
      POSTGRES_USER: crewai
      POSTGRES_PASSWORD: supersecret
      POSTGRES_DB: crewai
    volumes:
      - pgdata:/var/lib/postgresql/data
    networks: [ki_net]

networks:
  ki_net:
    driver: bridge

volumes:
  pgdata:

.env

OPENAI_API_KEY=sk-…
STUDIO_PORT=8501

---

Steps to reproduce

1. Clone my repo and ensure the above docker-compose.yml is in ~/ai-crew/.

2. Start the stack:

   docker compose up -d --build

3. Create some test data in CrewAI (e.g. via POST to /generate/ or manual script).

4. Shut down the stack:

   docker compose down

5. Bring it back up:

   docker compose up -d

6. Query the database or check CrewAI behavior → all previously created data is gone.

---

Expected behavior

• PostgreSQL data under pgdata should persist across restarts.
• CrewAI should continue reading from the existing database without resetting schemas or data.

---

Actual behavior

• After bringing the containers back up, PostgreSQL initializes from scratch.
• No tables or previous rows exist.
• It appears either the named volume isn’t being used, or CrewAI is pointing to an in‑memory or alternate storage.

---

What I’ve tried so far

• Verified pgdata volume exists (docker volume ls shows ai-crew_pgdata).
• Inspected volume contents (docker run --rm -v ai-crew_pgdata:/data busybox ls /data) → data directory is empty after restart.
• Confirmed DATABASE_URL is correctly passed into the CrewAI container.
• Checked logs for both db and crewai services—no obvious errors or migrations running.
• Explicitly created tables in main.py via SQLAlchemy on startup; these, too, disappear after down/up.
• Manually executed docker inspect on the db container to verify mount points.

---

Questions

1. Am I missing a Compose flag or option to ensure the named volume is correctly reused?
2. Could CrewAI be auto-dropping/recreating schemas on startup (e.g. via alembic migrations)?
3. Any tips to debug whether the container is actually mounting the volume or initializing an ephemeral store?

Any insight or suggestions would be greatly appreciated!

[strnad]

Hello, the application expects the DB_URL environment variable, but your Docker Compose file supplies DATABASE_URL. Because DB_URL is missing, the service falls back to its local SQLite file, which is removed whenever the container is recreated—hence the lost data.