Skip to content

Dev: Write up database core tables and schema - #6

Merged
naman0r merged 16 commits into
mainfrom
3-db-schema
Oct 10, 2026
Merged

naman0r merged 16 commits into
mainfrom
3-db-schema

Conversation

@naman0r

@naman0r naman0r commented Sep 24, 2026 •

Copy link
Copy Markdown

ℹ️ Issue

Closes #3

📝 Description

Adds the BHO database schema. The doc describes each table, and the migration creates the tables in Postgres.

  1. docs/database-schema.md lists every table and column, with the spreadsheet cell each column comes from.
  2. db/migrations/0001_init_schema.ts is a Kysely migration. It creates users, stations, imports, daily_observations, hourly_observations, scheduled_observations, daily_records, and audit_log, and adds station 19737-02. Deleting a day also deletes its hourly and scheduled rows. down drops everything.
  3. A trigger on the four weather tables writes one audit_log row per insert, update, or delete. To record who made a change, the writer calls set_config('app.user_id', ..., true) and set_config('app.import_id', ..., true) in its transaction.
  4. db/types.ts has the Kysely type for each table.
  5. db/client.ts has createDb(), which returns date columns as 'YYYY-MM-DD' strings. By default pg returns a Date at local midnight, so 2026-07-01 is 04:00Z on a laptop in Eastern time and 00:00Z in Lambda.
  6. Adds kysely@^0.27.0 to the root package.json, the same version the Lambdas use.

Only audit_log.action is an enum. Values that may change, like imports.status, are text.

✔️ Verification

Ran the migration against Postgres 16 in Docker with Kysely's Migrator and FileMigrationProvider pointed at db/migrations. Until #11 adds yarn db:migrate, that is the way to run it. Checked that:

  • up, down, and up again succeed, and down leaves no tables, type, or function behind.
  • Deleting a day deletes its hourly and scheduled rows.
  • Inserts, an update, and the cascaded deletes each wrote an audit_log row with the row's key and the user.
  • With TZ=America/New_York, createDb() reads 2026-07-01 back as the string "2026-07-01".

🏕️ (Optional) Future Work / Notes

  • hourly_observations.mountainVis (1@1.5) stays text. The client says it is mountain number @ visibility on a 1-3 scale. Once the weather group says which number is which mountain, split it into mountainId and mountainVisibility with a mountains table.
  • On 4 days in July the peak gust has two times (28 WNW @ 1631E, 1651E), and July 19 has two directions. peakGustDir and peakGustTime hold one value each.
  • daily_observations.remarks reads G47-G49, but notes also appear in G50-G54 on some days.
  • importId is required on daily_observations and daily_records, which assumes only uploads write weather data. If staff can edit values in the app, it needs to be nullable.
  • Re-uploading a month logs every row as an update, even when only importId changed.

Comment thread docs/database-schema.md Outdated
Comment thread docs/database-schema.md Outdated
naman0r and others added 11 commits October 5, 2026 18:36
Entities and generated migration for users, imports, daily/hourly/scheduled
observations, daily records and audit log, plus docs/database-schema.md
mapping each column to its source cell in the example sheets.
Client confirmed more stations are coming, so stationId joins the keys of
the three daily tables now rather than in a later migration. Every row
comes from an upload, so importId is not null on observations and records.
Keying records by station now avoids changing their primary key later if
records become per-station. The hourly notes now say what column H's
fractions and column I's weather codes mean.
@naman0r
naman0r marked this pull request as ready for review October 6, 2026 01:14
github-actions Bot added a commit that referenced this pull request Oct 6, 2026

@Rayna-Yu Rayna-Yu left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

Non-blocking: One thing to note though, is I think pg parses a Postgres date at the server's local midnight. So that would be UTC in Lambda and Eastern for our laptops. This shouldn't be an issue because we are planning on running everything in Lambdas anyways, but something that we may need to be careful about.

@CharlesTChapman CharlesTChapman left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. All entities and fields match sample data well and relational structure seems logical.

github-actions Bot added a commit that referenced this pull request Oct 8, 2026
@naman0r
naman0r merged commit ac7bba0 into main Oct 10, 2026
3 checks passed
github-actions Bot added a commit that referenced this pull request Oct 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dev : Write up database core tables and schema

4 participants