Report command completion timeouts and recover the transport

[hangzqcom]

Summary

Device commands could report success even when the command never completed. This happened when the communication port was not actually open, or when a device did not acknowledge a queued command within the completion timeout.

This change makes command completion status observable, and adds an automatic recovery attempt so that a device which has stopped responding can be brought back without restarting the application.

Issue

The completion wait detected a timeout, logged it, and then cleared the same internal state used to represent successful completion. The wait function returned no status, so callers could not distinguish a completed command from a timed-out command.

The result was discarded at every layer of the command path:

• The completion wait returned no status.
• Device transport command handlers returned no status.
• The device command layer assumed success once a command name was resolved.
• The exported API therefore returned a success code for commands that never reached the hardware.

Queued multi-step command sequences had the same problem. They were treated as successful as soon as they were queued, without waiting for or validating completion.

A common trigger is a port that could not be opened, for example because another process already holds it. Opening the device correctly reported a failure, but if an application continued issuing commands anyway, those commands were queued against a port that was never usable and still returned success. An automation script could receive a success result for an operation such as a power or boot-mode request while the device state never changed, and the problem would only surface later through unrelated symptoms.

A second trigger is a debug board that stops acknowledging mid-session, for example after an unexpected USB re-enumeration. In that state every subsequent command times out, and reopening the device handle does not help because the underlying transport object is still the same unusable instance.

Fix Direction

Completion status is propagated through the command stack rather than discarded:

• Completion waits return true for completion and false for timeout.
• All device transport implementations propagate the result for pin-state commands.
• The device abstraction converts a timeout into a structured command-timeout exception.
• Queued command sequences now wait for completion and report a timeout failure.
• A missing transport path now reports an explicit device-inactive error instead of silently returning.
• The exported C API converts the timeout into a documented result code instead of allowing an exception to escape the ABI boundary.
• The Python binding exposes the new result code and message.

A timed-out command now triggers one transport recovery attempt before failing:

• The transport is torn down and reinitialized the same way it is set up at startup: stale protocol state is discarded, the device is given time to settle, the port is reopened, and the board identification handshake is completed before the transport is considered usable again.
• Recovery is reported as successful only when the device has actually responded. A reopened but silent port is reported as a failure, so the result reflects the device rather than the file handle.
• The command is then retried once. A persistent failure surfaces to the caller as a timeout rather than looping or stalling.
• Recovery is attempted again on a later command if the device stops responding again.

Recovery runs on the thread that owns the transport. QSerialPort and the FTDI handles must only be used from their owning thread, and the drive thread reads and writes them continuously, so the requesting thread posts a request and waits for the drive thread to perform the work. The wait is bounded, so a wedged drive thread reports failure instead of blocking the caller indefinitely.

Diagnostics for this class of failure were improved:

• A timeout now records the elapsed time, the port, the transport type and the command that stalled, instead of a bare message with no context.
• A command that completes but takes unusually long is reported as a warning, so a degrading transport is visible before it fails outright.
• The timeout, the recovery attempt and the retry outcome are summarised on a single line.

API Changes

• Added TACDEV_COMMAND_TIMEOUT = 7 to the public C API.
• Added TACDEV_COMMAND_TIMEOUT = 7 and its error text to the Python API.
• Added an internal timeout exception code used to carry the condition to the API layer.
• No existing result codes were renumbered.
• No exported function signatures were changed.

Compatibility

Binary and source compatibility are preserved. The new value is additive, existing codes keep their values, and no exported signature changed. Existing binaries do not need to be rebuilt to keep working.

What applications should expect now

• Commands that complete successfully continue to return 0.
• Commands that time out now return 7 (TACDEV_COMMAND_TIMEOUT) instead of 0.
• GetLastTACError() returns a human-readable reason for the failure.
• Python callers receive a RuntimeError for timed-out commands, consistent with other failures.

A command that previously appeared to succeed while doing nothing may now take a few seconds longer before returning, because a recovery attempt is made first. Applications that already check result codes need no changes. Applications that ignored result codes may begin to see failures reported for operations that were silently failing before.

Port open failures

Device open behavior is not changed by this PR:

• OpenHandleByDescription() already reported failure by returning a bad handle (0) and setting an error message, and it still behaves that way.
• The returned error text still distinguishes causes such as a port held by another process versus a port that does not exist.
• No new result code was introduced for open failures.

What changes is the behavior after a failed or unusable open. Commands issued in that state previously returned success; they now return 7 with an error message. Checking the result of the open call remains the recommended first step, because it reports the failure immediately rather than after a command timeout.

Validation

• Builds clean for x64 Debug and Release.
• Exported symbol set was compared before and after: no symbol was removed or re-signed, so existing binaries continue to link and load.
• Exercised against a PIC32CX debug board. Normal operation showed no regression across a 72-minute session with 61 commands and several boot-mode sequences: no spurious timeouts and no unnecessary recovery attempts.
• Verified recovering a board that had stopped acknowledging. The transport was reinitialized, the board completed its identification handshake, the retried command succeeded, and the subsequent boot-mode sequence ran normally.
• Verified that an invalid handle is still rejected cleanly and that no exception escapes the C API boundary.

Known limitations

• Recovery has been verified against one reproduced failure. An earlier revision of this change that reopened the port without completing the identification handshake did not recover, which is why the handshake is now required before reporting success.
• Recovery takes a few seconds. An application with a short external timeout may still exceed it even though recovery eventually succeeds.
• Recovery addresses the host side only. It does not prevent the underlying cause of a device dropping off the bus; unstable power or an unpowered USB hub remains an environmental issue to resolve separately.