Universal PowerSync Diagnostics (POC) - #1094
khawarizmus wants to merge 53 commits into
Conversation
🦋 Changeset detectedLatest commit: 170d577 The changes in this PR will be included in the next version bump. This PR includes changesets to release 13 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
simolus3
left a comment
simolus3
left a comment
There was a problem hiding this comment.
To share more context here (I've also shared that offline with more context @khawarizmus). I think the idea is awesome, my initial architectural complaints are:
- It's not clear whether we need a browser extension, so remove that from this PR which is already very big (you can open a follow-up PR for that separately).
- I think a JSON-based protocol over "any two way channel" is the wrong abstraction here. It e.g. forces us to re-invent an RPC protocol when some implementations we'll use (say the Dart VM service protocol) are already RPC based.
- None of this should require any changes to an existing SDK package (aside from forwarding core diagnostics events, but even that should likely be a separate PR to validate that the basic integration can work without SDK changes). We can likely inject an agent by writing a vite devtools package instead. Or initially, we could also expose it as a seperate package a user would depend on to expose the channel (e.g. a
enablePowerSyncDiagnostics(port: MessagePort, db: BasePowerSyncDatabase)). - Related to 2 and 3, the interface the diagnostics tool expects an SDK to provide should be an actual TypeScript interface with async methods and event listeners. It is the responsibility of each integration (vite,
postMessageiframe, Dart) to figure out how to implement that interface by doing the serialization itself.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
simolus3
left a comment
simolus3
left a comment
There was a problem hiding this comment.
I didn't take a look at the UI yet, but the architecture of how devtools are served and integrated looks pretty good at this point. I'll also try to scaffold a Dart integration over the vm service protocol.
9c6336a to
bc0edf9
Compare
dc754e0 to
395f91d
Compare
…lient implementations # Conflicts: # pnpm-lock.yaml # Conflicts: # pnpm-lock.yaml
…ge with components and playground
…d indexes; add SchemaTab to diagnostics UI
…line module imports and setup
…ding columns, indexes, and table options
…ream, UI components, and schema inspection features
…for a cleaner separation
…ion alongside currentOptions
…t and enhance connection handling in extension
…ostMessageTransport, enhance package metadata, and add Chrome types
…nents with icons and status bar
…play and integrate virtual scrolling for upload queue
…t detail component, and improved data handling features
…sizable columns, and theme synchronization
Add dependency-free `shapes.ts` and `integration.ts` as the tool-owned protocol; move PROTOCOL.md into diagnostics-core and rewrite it to interface + shapes.
…et the host own the theme
…e, hold hint, host theme channels
…ools into the Node demo, fix changesets
…e code; fix README facts
…ified Dart details
875f838 to
17ebaf3
Compare
simolus3
left a comment
simolus3
left a comment
There was a problem hiding this comment.
I did not review UI package code, I only tested it in the node demo and by integrating it into the Dart SDK. I reviewed SDK changes and the diagnostics-core package in detail, those look good to me.
I think having a vite web demo somewhere in this repository would be helpful to try this with a real web SDK. Perhaps example-vite which already uses a recent vite version? Some of our other demos use very old dependencies which might be annoying to upgrade.
I assume there's currently no way to surface diagnostics events in the node SDK, right? I think that's okay for now, but it should be documented somewhere.
| /** | ||
| * Enables the core extension's diagnostics event stream — detailed per-bucket download stats | ||
| * (including per-bucket `target_count`) and inferred column types — delivered as | ||
| * `HandleDiagnostics` instructions. Off by default. | ||
| */ | ||
| diagnostics?: boolean; |
There was a problem hiding this comment.
This documentatation comment should probably mention that these diagnostics are only used internally, and exposed in via devtools? We could also add a link to devtools documentation once we have it.
Otherwise users might expect these to show up in public APIs.
There was a problem hiding this comment.
This was addressed
| * exist, so it adds nothing to a production bundle beyond the references themselves. | ||
| */ | ||
| const databases = new Set<WebPowerSyncDatabase>(); | ||
| const listeners = new Set<(databases: readonly WebPowerSyncDatabase[]) => void>(); |
There was a problem hiding this comment.
To save a tiny bit of duplicate code, we could use a BaseObserver singleton for this.
| }); | ||
| // PowerSync DevTools: a live view of this client in the browser. Development only. | ||
| if (process.env.NODE_ENV !== 'production') { | ||
| const devtools = await enablePowerSyncDiagnostics(db); |
There was a problem hiding this comment.
For me, this needed a db as any because connector and connectionOptions are (deliberately!) not part of the public CommonPowerSyncDatabase interface.
Can we have createIntegration and enablePowerSyncDiagnostics take a generic CommonPowerSyncDatabase and cast to BasePowerSyncDatabase internally?
There was a problem hiding this comment.
This was addressed
khawarizmus
added
the
Product Visibility
| } | ||
|
|
||
| /** Maps a numeric SDK log level to its name. */ | ||
| function logLevelName(level: number): string { |
There was a problem hiding this comment.
This should import LogLevels from @powersync/common instead of hardcoding them.
There was a problem hiding this comment.
This was addressed
| * Serves the app's first live database to any iframe that asks. Call once per page. | ||
| * Returns a function that stops serving. | ||
| */ | ||
| export function serveDiagnostics(): () => void { |
There was a problem hiding this comment.
I couldn't figure out ift this is still used, it looks like nothing imports it anymore? Has this been replaced by client.ts which is what the vite plugin injects?
There was a problem hiding this comment.
This is still used by Nuxt, the module loads it for Nuxt DevTools 3, which has no devframe hub (packages/nuxt/src/runtime/plugin.client.ts imports @powersync/diagnostics/page).
if (import.meta.dev && moduleOptions.useDiagnostics && moduleOptions.diagnosticsTransport === 'page') {
import('@powersync/diagnostics/page').catch((error) => {
console.error('[powersync-diagnostics] failed to load the diagnostics client', error);
});
}…els, registry on BaseObserver, document internal diagnostics
…upabase-todolist; upgrade to Vite 8
I have added an example in the
We decided to add the diagnostics event functionality to Node instead of suppressing it. I have also moved the packages to |
…oduction installs without dev dependencies start
This branch has not been deployed
3 participants
This PR implements a live-attach diagnostics tool built once and reused across different hosts. It shows the live state of the PowerSync client side.
How it works
The agent runs next to your database. It reads the database and pushes snapshots. The UI rebuilds its state from these snapshots. On the web, the agent runs in your app page. Vite DevTools loads it after you trust the tab. On node, the agent runs in the same process as the database. A React Native app connects to the standalone window over the network (RN has not been tested yet).
The MCP endpoint exposes the same functions as the tools, allowing AI agents to directly query the client app and retrieve diagnostic information.
New packages
@powersync/diagnostics-core. The protocol. It is one interface,SdkIntegration, plus the data shapes. The UI talks only to this interface. Each SDK implements it. The package also has the JavaScript agent and the iframe bridge.@powersync/diagnostics-ui. The UI. Rendered from anSdkIntegration. It runs in an iframe or in a standalone window.@powersync/diagnostics. The host integration, built on devframe. It gives a dock in Vite DevTools, a tab in Nuxt DevTools, a dev server for Node apps, a standalone window via CLI (powersync-devtools), and MCP tools for AI agents.SDK changes
@powersync/common. adiagnosticsoption onconnect(). It turns on the SQLite core diagnostics stream. This gives pre-bucket download stats and inferred column types.@powersync/shared-internals: the sync client keeps the connector for the whole connection. It forwards the core diagnostics events.@powersync/web. a new@powersync/web/devtoolssubpath. It lists the open databases in the page. The agent finds them without any app code.@powersync/nuxt. breaking.NuxtPowerSyncDatabaseis removed. UsePowerSyncDatabasefrom@powersync/weband passconnect(connector, { diagnostics: true }). The module shows the PowerSync tab on Nuxt DevTools 3 and the dock on Nuxt DevTools 4.Tested by hand
compatibilityVersion: 5.devtools: true) and with Vite 7 (@vitejs/devtoolsplugin).Claude code was used to steer the discovery and development. The code was reviewed, tested and modified manually.