Network Analyzer as an Admin Panel page, incl. per-account traffic observation

[sven-n]

Is your feature request related to a problem? Please describe.

The network analyzer (src/Network/Analyzer) is a WinForms tool which acts as a man-in-the-middle proxy. That works well for development, but an admin who wants to look at the traffic of a suspicious player usually can't set such a proxy up on the server machine. The traffic is already flowing through our own servers — we just don't expose it anywhere.

Status: phase 0 is done (#896), phase 1 is done (#897).

Describe the solution you'd like

A /network-analyzer page in the admin panel:

• A collapsible connection list on the left, grouped by server type (connect server, game server, chat server) and by server instance.
• An entry shows the account and character name where we know it, and falls back to the remote endpoint (IP:port) where we don't.
• Selecting an entry starts capturing and shows the packets in a live-updating grid, with the same field extraction the WinForms tool does. The protocol version is applied automatically — no manual version dropdown.
• An account flag IsNetworkObservationActive makes the game server archive the traffic of that account's sessions on the file system. Archived sessions are browsable in the same page — while the player is online and after they went offline — and can be deleted.
• A button next to an online player (live map player list, online accounts page) navigates to the analyzer page with that connection preselected.

The WinForms tool stays. This is an addition, not a replacement — both use the same analysis code after the split described below.

---

1. Starting point

1.1 The analyzer tool

Only a few of its 15 files are actually WinForms:

File	UI-bound?	Notes
MainForm.*, PacketSenderForm.*, Program.cs	yes	the app itself
LiveConnection.cs, LiveConnectionListener.cs, ClientConnectedEventArgs.cs	proxy only	only needed for the MITM mode
PacketAnalyzer.cs	no	resolves a packet to its definition and extracts the fields
Packet.cs, ICapturedConnection.cs, SavedConnection.cs, CapturedConnectionExtensions.cs, FieldExtensions.cs	no	model + .mucap save/load

1.2 The servers

All three server types create their client connections through the very same code path — MUnique.OpenMU.Network.Listener.CreateConnection returns a MUnique.OpenMU.Network.Connection:

• GameServer/DefaultTcpGameServerListener.cs:81
• ConnectServer/ClientListener.cs:67
• ChatServer/ChatServerListener.cs:52

That is the lever for this feature: one capture hook in Connection covers all server types, and it sees the traffic decrypted:

• incoming: Connection.ReadPacketAsync is called with one complete, already decrypted packet (the decryptor sits in front of it as Source);
• outgoing: Connection.Output is an ExtendedPipeWriter which wraps the encryptor's writer, so everything written to it is still plaintext.

---

2. Design

2.1 Capture hook in MUnique.OpenMU.Network — done in #897

/// Sink for captured, decrypted packets of a connection.
public interface IPacketCaptureSink
{
    void PacketCaptured(ReadOnlySpan<byte> packet, bool sent);
}

• IConnection got a Guid Id — a stable handle for the UI routes and useful for log correlation.
• Sinks are registered with IConnection.AddCaptureSink and removed with RemoveCaptureSink. They are kept in an ImmutableArray updated with ImmutableInterlocked, so there is no lock and iterating them per packet stays allocation free. As long as none is registered, a connection captures nothing.
• Incoming packets are handed to the sinks in Connection.ReadPacketAsync — they arrive as complete packets already.
• Outgoing data doesn't: a write to the pipe writer is not necessarily one packet. It is therefore copied into an own pipe, which is read by a CapturedPacketReader : PacketPipeReaderBase — reusing the packet splitting of the network layer instead of a bespoke assembler.
• A capture may only start or end at a packet boundary. The ExtendedPipeWriter knows where those are, so it applies a requested change when nothing is written to the target yet or right after a flush.
• The capture never blocks or breaks the connection: no writer backpressure on its pipe, failing capture writes are swallowed, and malformed captured data stops the capture (not the connection).

2.2 Analyzer library split — done in #896

• src/Network/Analyzer/MUnique.OpenMU.Network.Analyzer.csproj — net10.0, platform neutral: Packet, PacketAnalyzer, FieldExtensions, ICapturedConnection, SavedConnection, CapturedConnectionExtensions, plus the archive types of phase 5.
• src/Network/Analyzer.WinForms/MUnique.OpenMU.Network.Analyzer.WinForms.csproj — the WinForms app, net10.0-windows, referencing the library.

Three changes inside PacketAnalyzer (done in #897):

1. ClientVersion is a parameter of ExtractInformation / ExtractShortInformation instead of instance state, so one shared instance can serve connections with different client versions concurrently. This is what makes the automatic version handling below possible.
2. Definitions are keyed by a "definition set" instead of "one file per direction", and the direction is taken from PacketDefinition.Direction. That enables reusing ConnectServer/ConnectServerPackets.xml and ChatServer/ChatServerPackets.xml for the other two server types.
3. The FileSystemWatcher-based hot reload is opt-in via a ctor flag and stays off in the server hosts — editing packet definitions at runtime is a rare development activity, so it remains WinForms-only.

The XML files have to be copied to the output of the hosts which render the admin panel (src/Startup, src/Dapr/AdminPanel.Host); the MUnique.OpenMU.Network.Packets project already publishes them as content.

2.3 Capture service, server adapters and the automatic protocol version

public interface ICapturedConnectionInfo   // one entry in the left list
{
    Guid Id { get; }
    ServerType ServerType { get; }
    int ServerId { get; }
    string? AccountName { get; }
    string? CharacterName { get; }
    EndPoint? RemoteEndPoint { get; }
    ClientVersion ClientVersion { get; }   // kept up to date, see below
    DateTime ConnectedAt { get; }
    bool IsCapturing { get; }
    bool IsObserved { get; }
    string DisplayName { get; }            // character ?? account ?? endpoint
}

public interface IPacketCaptureService     // singleton
{
    IReadOnlyList<ICapturedConnectionInfo> Connections { get; }
    event EventHandler? ConnectionsChanged;

    ValueTask<ILiveCapturedConnection> StartCaptureAsync(Guid connectionId);
    ValueTask StopCaptureAsync(Guid connectionId);
    ValueTask DisconnectAsync(Guid connectionId);
    ValueTask SetObservationAsync(string accountName, bool active);
    ICapturedConnectionInfo? Find(int serverId, string playerOrAccountName);
}

ILiveCapturedConnection : ICapturedConnection holds the captured packets in a capped buffer (size from SystemConfiguration, default 5,000 packets, oldest dropped) and raises a change event. It is a sink of the connection.

The service aggregates IConnectionSource implementations registered in DI — one per server type, living in the respective server project so that no internals have to be made public to the web layer:

Server	Source of the connections	Name resolution	Protocol version
Game server	IGameContext.GetPlayersAsync() — players are added on connect (GameServer.OnPlayerConnectedAsync), so connections at the login screen are included	player.Account?.LoginName, player.SelectedCharacter?.Name, live-updated	RemotePlayer implements IClientVersionProvider: initial value from the endpoint's GameClientDefinition, updated in LogInHandlerPlugIn (ClientVersionChanged event)
Connect server	ClientListener.Clients (needs a small accessor on IConnectServer)	none — endpoint only	ConnectServer.ClientVersion from its settings
Chat server	ChatServer._connectedClients (needs an accessor on IChatServer)	IChatClient.Nickname, room id	version independent — the chat definition set

So the version is never asked for in the UI: the connection knows it, the source publishes it, the analyzer gets it per call. When it changes mid-session (0.75 default → the real version detected at login), the source raises a change and the already displayed rows are re-analyzed.

GameContext currently has no event when a player is added/removed (GameMapCreated/GameMapRemoved exist, players don't). Add PlayerAdded/PlayerRemoved events in GameContext.AddPlayerAsync / RemovePlayerAsync so the connection list can update push-based instead of polling.

No backfill for ad-hoc captures. Capture starts when a connection is selected; earlier packets of that connection are gone. The observation feature below is the answer when the full picture is needed.

2.4 Account observation and the session archive

The flag. MUnique.OpenMU.DataModel.Entities.Account gets

public bool IsNetworkObservationActive { get; set; }

modelled exactly like the existing IsBot / IsVaultExtended. The EF model class is generated from the data model, so this needs one new migration (see Persistence/EntityFramework/Migrations/Cheatsheet.md). EditAccount renders it automatically through AutoForm.

Toggling it from the analyzer page. The connection header gets a toggle which calls IPacketCaptureService.SetObservationAsync. That persists the flag on the account and applies it to the live player object, so it takes effect immediately without a reconnect.

When archiving starts. The game server connection source subscribes to player.PlayerState.StateChanged. When a player reaches the logged-in state, player.Account.IsNetworkObservationActive is checked and, if set, an archive writer is registered as a second sink of the connection. Consequence worth stating: the few packets exchanged before login (version check, handshake) are not part of the archive. Capturing those would require buffering every connection's first packets unconditionally, which contradicts the "no always-on capture" rule.

Archiving is a game server feature: the connect server has no account, the chat server only a nickname. Both are still listed live.

Where it runs. The archive writer belongs to the game server, not to the admin panel: it is registered in the all-in-one host and in Dapr/GameServer.Host, so an observed account is archived even when no admin panel is running (see the deployment note below).

Format. Reuse the existing .mucap CSV (timestampTicks;toServer;size;hexBytes) so files stay loadable by the WinForms tool, with two additions:

• a monotonically increasing per-session sequence number appended as a fifth field — the existing loader reads only fields 0..3 and ignores the rest, so this stays backwards compatible. It makes merging "archived so far" with the live tail exact (no gap, no duplicate).
• a sidecar .json with the metadata: account, character(s), server type/id, remote endpoint, start/end time, packet count, and the client version including its change points (sequence number → version), so a replay analyzes the early packets with the version that was actually in effect.

The writer is a streaming, append-only, periodically flushed writer (not save-at-the-end), so a session is readable while it runs and survives a crash. Readers open with FileShare.ReadWrite | FileShare.Delete.

Layout and housekeeping. <archivePath>/<accountName>/<yyyyMMdd-HHmmss>_<serverType><serverId>_<connectionId>.mucap plus the sidecar. Archive path, rotation size, quota and retention come from SystemConfiguration. The archive directory must not be inside the statically served logs folder.

2.5 The Blazor UI

New folder src/Web/AdminPanel/Components/NetworkAnalyzer/:

Component	Job
Pages/NetworkAnalyzer.razor	routes /network-analyzer, /network-analyzer/{ConnectionId:guid} and /network-analyzer/archive/{SessionId}; two-column layout
ConnectionList.razor	left column: Bootstrap accordion; "Live connections" grouped by server type → server instance, and "Archived sessions" grouped by account → session; search box; per live entry: display name, endpoint, packet counter, record toggle, disconnect button; per archived entry: date, duration, packet count, size, open/download/delete
PacketGrid.razor	QuickGrid with Virtualize; columns: timestamp, direction (C→S / S→C), size, code (hex), short message
PacketDetail.razor	extracted fields (PacketAnalyzer.ExtractInformation) plus a hex dump of the selected packet
AnalyzerToolbar.razor	pause/resume, autoscroll, clear, direction filter, text filter on code/name, observation toggle, download .mucap
NetworkAnalyzerViewModel.cs	scoped (per circuit) state: subscribes to the service, throttles the render via the existing Debouncer (e.g. max 10 updates/s), holds filter and selection, merges archive + live tail by sequence number

Details worth deciding early:

• Capture lifetime. An ad-hoc capture starts when a connection is selected and stops when the last viewer deselects it or the circuit is disposed (reference counted in the service). An archive capture is independent of any viewer and lives as long as the session.
• Opening an observed player shows the archived packets of the running session first, then continues live.
• Opening an archived session of an offline player is a pure file read; large files are read paged/tail-first with a cap, and the grid is virtualized.
• Live update. Packets arrive on network threads, so the view model snapshots under a lock and marshals via InvokeAsync(StateHasChanged).
• Deep link. Pages/LoggedIn.razor and Web/Map/Components/MapPlayerList.razor get a button navigating to network-analyzer/{id}; the id is resolved via IPacketCaptureService.Find(serverId, name) so the web layer never touches a Connection. MapPlayerList lives in the Map project, so the route is passed in via a cascading parameter like LiveMapRoute already does, keeping the Map project standalone-buildable.
• Menu. New entry in Components/Layout/NavMenu.razor, only rendered when an IPacketCaptureService is registered — the same graceful degradation the global message card in Servers.razor uses.
• Localization. All new strings go into Web/AdminPanel/Properties/Resources.resx.

2.6 Settings in the database

New properties on DataModel/Configuration/SystemConfiguration.cs (with Display attributes and resource entries, like the existing ones), covered by the same migration as the account flag:

Property	Default	Purpose
NetworkAnalyzerLiveBufferSize	5000	packets kept per live capture
NetworkObservationArchivePath	captures	archive root, relative to the app directory
NetworkObservationMaxSessionSizeMb	50	rotation threshold per session file
NetworkObservationMaxTotalSizeMb	1000	quota; oldest sessions removed first
NetworkObservationRetentionDays	30	age-based cleanup

They are editable in the admin panel's system configuration form for free.

---

3. Suggested phases

Each phase is meant to be a separate, self-contained PR.

• Phase 0 — analyzer library split. Done in Split the network analyzer into a library and the WinForms tool #896.
• Phase 1 — capture infrastructure. Done in Add packet capturing to the connections and make the analyzer reusable #897, including the definition-set, client-version and file-watcher changes of PacketAnalyzer.
• Phase 2 — capture service + server adapters. §2.3, including the PlayerAdded/PlayerRemoved events, the accessors on IConnectServer / IChatServer, and the automatic client version.
• Phase 3 — the page (live). §2.5 for live connections only.
• Phase 4 — deep links. Buttons in LoggedIn.razor and MapPlayerList.razor.
• Phase 5 — observation and archive. §2.4 and §2.6: account flag + migration, SystemConfiguration entries, streaming writer, sidecar metadata, rotation/quota/retention, registration in the all-in-one host and in Dapr/GameServer.Host. Tests for the writer/reader round-trip (including a file which is being written while read) and for the housekeeping.
• Phase 6 — archive browser. The "Archived sessions" part of the sidebar: list, open (offline and running), download, delete; observation toggle in the page.

Describe alternatives you've considered

• Keeping the MITM proxy as the only option. Rejected as the sole solution because an admin typically can't deploy it on the server; the WinForms tool stays for development.
• Wrapping the whole IConnection in a capturing decorator at listener level. Rejected because capture must be switchable at runtime for an already-connected player, which a decorator installed at accept time can't do without paying the cost for every connection.
• Always buffering the first packets of every connection so that the pre-login traffic of an observed account can be archived retroactively. Rejected — it contradicts "capture only what's asked for" for a gain of two or three handshake packets.
• A new capture file format. Rejected in favour of extending .mucap with a fifth CSV field plus a sidecar .json, which keeps the files loadable by the existing WinForms tool.
• A hand written packet assembler for the outgoing data. Replaced during phase 1 by an own pipe plus a PacketPipeReaderBase, which reuses the packet splitting we already have.

Additional context

Decisions:

1. Offline/AFK-trade players are not captured — they have no connection.
2. Observation is an account setting, archived per session on the file system, browsable online and offline, deletable.
3. Analyzer settings live in SystemConfiguration in the database and are editable in the admin panel.
4. The WinForms tool stays as a development tool; this is an addition.
5. The protocol version is derived from the connection, never chosen manually.
6. No backfill for ad-hoc captures; the observation archive covers that need.
7. No packet definition file watcher in the server hosts.
8. The chat server traffic of an observed account is not archived for now. It would need a mapping from the chat authentication token back to the account; the chat connections are still listed live.
9. An observed session is archived independently of the admin panel: the writer is registered in the game server host, so a distributed deployment archives the traffic of observed accounts as well.
10. The packet sender is not ported to the page. It stays a WinForms-only development tool.

Data protection. Admin panel authentication is tracked separately, but two properties of this feature outlive that issue: captures and archives contain the plaintext login packet, so the archive directory must not be served statically (unlike logs) and archived files should be created with restrictive permissions; and starting/stopping a capture, toggling the observation flag and deleting an archived session should be logged at information level, so observing a player leaves a trace.

Deployment. Like the live map, the page works in the all-in-one deployment only, because the capture service needs the server objects in-process. In the Dapr deployment the service is simply not registered and the menu entry is hidden. The archive is not restricted that way (decision 9): it lives in the game server, so observed accounts are archived in a distributed deployment too — the sessions are then browsable once a page can reach that host. A later iteration could stream live captures from the server hosts to the admin panel host (SignalR or the existing Dapr channels) without changing the interfaces above, since they are already async and id-based.

[sven-n]

Status: phases 2 and 3 are done

• Phase 0 — analyzer library split → Split the network analyzer into a library and the WinForms tool #896
• Phase 1 — capture infrastructure → Add packet capturing to the connections and make the analyzer reusable #897
• Phase 2 — capture service + server adapters → Add the packet capture service and the connection sources #901
• Phase 3 — the page (live) → Add the network analyzer page to the admin panel #907
• Phase 4 — deep links ← next
• Phase 5 — observation and archive
• Phase 6 — archive browser

(The body above is left as it was written — the API mangles generic type arguments when an issue body is rewritten, so the status lives here instead.)

What deviated from §2.3 and §2.5

• No PlayerAdded / PlayerRemoved events on GameContext. The connection list asks the sources on demand (every 5 s, and on request), so the push-based update wasn't needed. Adding unused public events felt speculative — they're easy to add if the polling ever shows.
• The list is only replaced when it actually differs. The servers build their ICapturedConnectionInfo objects on every request, so a naive refresh re-rendered the whole sidebar every five seconds and made the packet grid flicker. The refresh now compares id and display name and returns whether anything changed; the page only calls StateHasChanged when something did.
• SetObservationAsync / IsObserved are not in the service yet — they belong to the account flag of phase 5.
• No AnalyzerToolbar.razor and no NetworkAnalyzerViewModel.cs. The toolbar is a handful of controls in the page header, and the state is small enough to live in the page itself. Splitting it out would have been structure without a second consumer.
• The grid is not virtualized. QuickGrid's Virtualize has to shift its content whenever the item count changes — which is on every arriving packet — so the view jumped and flickered while traffic came in. It renders plain rows now and keeps the newest 500; appending a row below the viewport doesn't move the scroll position at all.
• Pause was replaced by a single "Follow" toggle. A small JS module reports the scroll state of the grid: scrolling up switches Follow off, which stops the view from taking new packets (the capture keeps running), and scrolling back to the bottom takes over everything that arrived in the meantime. With that, a separate pause button had no meaning left.
• Details are shown inside the selected row instead of below the table, so they're next to the packet they belong to.
• bUnit was added as a test dependency; the page and its three components are covered by component tests.

Two things which turned up by running it

• PacketAnalyzer resolved its definition files relative to the working directory, so it threw a FileNotFoundException whenever the server wasn't started from its own folder — that took the Blazor circuit down. It uses AppContext.BaseDirectory now and leaves the definitions empty when a file is missing. This affected the deployed server in general, not just the page.
• ICapturedConnectionInfo gained ServerDescription, so the groups read "Connect Server (Season 6 Episode 3 GMO Client)" instead of "Connect Server 65536".

Phase 4, as it's being implemented

The plan had the deep link resolve the connection id via IPacketCaptureService.Find(serverId, name) in the page which renders the button. Instead, the analyzer page gets a second route which takes the name:

/network-analyzer/player/{ServerId:int}/{PlayerName}

The lookup then happens in the analyzer page at the moment the link is opened, rather than in the linking page at render time — which also means the link doesn't go stale while a list is on screen. For MapPlayerList it has the additional benefit that the Map project needs no reference to the analyzer at all: the route prefix arrives as a cascading parameter, exactly like LiveMapRoute, and the button is simply not rendered when no prefix is given (which is the case in Dapr/GameServer.Host).

---

Generated by Claude Code

[sven-n]

Status update

• Phase 0 — analyzer library split → Split the network analyzer into a library and the WinForms tool #896
• Phase 1 — capture infrastructure → Add packet capturing to the connections and make the analyzer reusable #897
• Phase 2 — capture service + server adapters → Add the packet capture service and the connection sources #901
• Phase 3 — the page (live) → Add the network analyzer page to the admin panel #907
• Phase 4 — deep links → Link online players to their network traffic #916 (open)
• Phase 5 — observation and archive → Archive the traffic of observed accounts #917 (open, independent of Link online players to their network traffic #916)
• Phase 6 — archive browser ← what's left

Deviations of phase 5 from §2.4

• A session is a directory, not a single file plus a sidecar: <archive>/<account>/<date>_<serverId>/ with part-000.mucap, part-001.mucap, … and session.json. With rotation a session has more than one file, and a directory keeps opening, copying, deleting and measuring it simple. Each part is still a .mucap with its own header, so the WinForms tool can open it.
• No client version change points in the metadata. They were meant for a capture which starts before the version is known — this archive starts at login, where the version is already final, so there is exactly one.
• The migration's defaultValue for the new columns was set by hand to the documented defaults, so an existing database doesn't end up with 0 (which reads as "no quota, no retention").

Phase 6 gets the "Archived sessions" part of the sidebar (list, open — offline and running —, download, delete) plus the observation toggle in the page header; the archive API it needs (GetSessionsAsync, GetSessionAsync, DeleteSessionAsync, ArchivedSession.LoadAsync) already ships with #917.

---

Generated by Claude Code

[sven-n]

All phases are implemented

• Phase 0 — analyzer library split → Split the network analyzer into a library and the WinForms tool #896
• Phase 1 — capture infrastructure → Add packet capturing to the connections and make the analyzer reusable #897
• Phase 2 — capture service + server adapters → Add the packet capture service and the connection sources #901
• Phase 3 — the page (live) → Add the network analyzer page to the admin panel #907
• Phase 4 — deep links → Link online players to their network traffic #916 (open, independent)
• Phase 5 — observation and archive → Archive the traffic of observed accounts #917 (open)
• Phase 6 — archive browser → Browse the archived sessions in the network analyzer page #922 (open, stacked on Archive the traffic of observed accounts #917)

Deviations of phase 6 from §2.5

• No merging of the archive with the live tail by sequence number. It isn't needed: the archived file is the live tail, because the writer flushes as soon as its queue drains. An opened running session is simply re-read, so there is one source and no chance of a gap or a duplicate. The sequence numbers stay in the files as the record of dropped packets.
• The download goes through a controller (api/network-archive/{**sessionId}) rather than a link to the file: the archive must not be served statically. It concatenates the parts of a session behind a single header line, so a rotated session opens in the WinForms tool as one capture.
• IsObserved / SetObservationAsync are default interface members on ICapturedConnectionInfo. Only a game server connection knows an account, so the connect and chat server implementations stay untouched.
• The observation toggle persists the flag through the player, whose context owns the account object. Saving it from the panel's own context would race with the player's own save when it logs out.

---

Generated by Claude Code