SSE (Server-Sent Events) Integration/Consideration

[MasterUser43]

Currently, LQD-DAQ relies on HTTP polling (GET /snapshot) to retrieve telemetry. At a hardware acquisition rate of 500 Hz, polling creates network overhead and potential latency affecting safety monitoring.

So, guessed solution instead of websockets would be SSE i.e.:

Prototype and implement a unidirectional Server-Sent Events (SSE) stream. This allows the DAQ to "push" fresh sensor snapshots to the dashboard immediately upon processing, rather than waiting for the client to ask for them.

This should maintain all currently added safety-related changes (e.g., reporting hardware 'stale' during reconnection). On basic example look up, probably would add /stream endpoint for asynchronous generator + have engine.py trigger a broadcast event per batch processed (should be down sampled for GUI stability, one of the main benefits).

Current test model may fail b/c of new 'persistent' connection model.

And considering SDEC-API related stuff, should consider a telemetry envelope of some kind:

{
  "origin": "LQD-DAQ",
  "version": "1.0",
  "timestamp": 1783957108.38,
  "payload": { 
     "PC": 101325.0,
     "POT": 1723689.5,
     "is_stale": false
  }
}

For uniformity, versioning, or, at the bare, useful for consideration.

TL;DR:
Modify api.py & likely engine.py.

Probably going to bork the test suite.

Basically, see and prototype the extent that SSE would be useful for DAQ communication.

[ETSells]

I like the design. Will draw up an ICD for that external json structure so its common to both of us and dashboard can reuse the mechanism we implement for it on the front-end side. Ship it!

[ETSells]

@MasterUser43 what's the timebase for that timestamp? measurement time? or send time from LQD-DAQ?

also I'm modifying the telemetry envelope to include a "messageType" field so we can just broadcast every incoming message from the API instead of requiring polling.

{
  "origin": "SDEC-API",
  "version": "1.0",
  "timestamp": 1783957108.00,
  "messageType": "dashboard-dump",
  "payload": { 
      <content>
  }
}

[ETSells]

Possible reference implementation for the API side: SunDevilRocketry/SDEC-API#24

[MasterUser43]

For LQD-DAQ, timestamp represents Measurement Time, pulled from hardware scan clock s.t. on my end, makes calculation easy (integration & impulse).

messageType is also neat for me to dump. So yippie on that.

Heads up that hardware t is Relative Time while API uses Unix Time (time.time()).

[ETSells]

We should try to unify the timebases eventually. Right now, API uses:

1. Unix time (time.time()) if it's the serial/wired path (might actually be monotonic, i forgor)
2. Measurement time w/ flight computer timebase if it's the LoRa path.

I will likely switch the serial path over to FC timebase once I start including timestamps in wired dump packets. Unifying FC timebase to UTC will be done in SunDevilRocketry/Flight-Computer-Firmware#299 as discussed.

[ETSells]

Outer level SSE interface (from SDEC-API):

{
  "origin": "SDEC-API",
  "version": "1.0",
  "timestamp": 1783957108.00,
  "messageType": "dashboard-dump",
  "payload": { 
      <content>
  }
}

Payload example for data stream (let me know if changes are needed):

{
  "origin": "LQD-DAQ",
  "version": "1.0",
  "timestamp": 1783957108.00,
  "messageType": "system-state",
  "payload": { 
    sensors: [
      "LOx P": {
        "readout": 2700000.15,
        "type": "pressure",
        "status": "nominal"
      },
      "K P": {
        "readout": 1600000.95,
        "type": "pressure",
        "status": "warning"
      },
      "K Orifice A": {
        "readout": 3605954.25,
        "type": "pressure",
        "status": "alert"
      }
      "GN2 T": {
        "readout": 21.7,
        "type": "temperature",
        "status": "nominal"
      }
    ],
    valves: [
      "LOx Purge": {
        "normal": "open"
        "status": "open"
      },
    ],
    derived: [
      "LOx MFR": {
        "readout": 2700,
        "type": "mass rate",
        "status": "nominal"
      }
    ],
  }
}

[MasterUser43]

Outer Layor is fine

Payload for LEGACY will likely be:

{
  "streaming": true,
  "is_stale": false,
  "sequence": {"active": true, "name": "fire", "elapsed_s": 5.21},
  "sensors": {
    "pt0": {"readout": 1723689.5, "type": "pressure", "status": "NOMINAL"},
    "tc0": {"readout": 21.7, "type": "temperature", "status": "NOMINAL"},
    "pos_lox_main": {"readout": 412, "type": "position", "status": "NOMINAL"}
  },
  "valves": {
    "lox_main": {"status": "open", "moving": false},
    "lox_press": {"status": "open", "moving": false}
  },
  "derived": {}
}

Defined below as:

streaming     bool    T7 hardware connection alive right now

is_stale      bool    true if latest data batch is older than 1.0s - independent 
                      of streaming, can be true b/f a comms drop is even declared

sequence      obj     currently running autosequence, if any
  .active     bool    true while fire/abort is executing
  .name       string  "fire" | "abort" | "" if none running
  .elapsed_s  float   seconds since this sequence started

sensors       obj     keyed by channel tag, one entry per PT/TC/position channel
  .readout    float   calibrated engineering-unit value
  .type       string  "pressure" | "temperature" | "position"
  .status     string  NOMINAL | CAUTION | WARNING

valves        obj     keyed by actuator tag, one entry per actuator
  .status     string  "open" | "closed"
  .moving     bool    true while a commanded stepper move is still in flight
                      (meaningful only for pulse_stepper actuators; always
                      false for solenoids, where the DIO write is the state)

derived       obj     reserved, always empty for legacy (no live mass flow/
                      mixture ratio, no orifice geometry data for legacy).

[ETSells]

I think this should work! can I get a list of sensor/valve names for this? also, for "pt0", can we rename these to be more meaningful, like "lox_inlet_tc" or "lox_bleed_pres"