Skip to content

Review v1/v0#31

Open
bertrand-marquis wants to merge 55 commits intoLinaro:virtio-msg-patch1from
bertrand-marquis:review-v1/v0
Open

Review v1/v0#31
bertrand-marquis wants to merge 55 commits intoLinaro:virtio-msg-patch1from
bertrand-marquis:review-v1/v0

Conversation

@bertrand-marquis
Copy link
Collaborator

Hi Everyone,

This pull request include a number of fixes or changes to handle the review comment we got on virtio mailing list.

Some points are not handled because they need a decision between us first (following summary was generated by chatgpt):

Ordering / Out‑of‑Order Feature Discussion

  • Parav Pandit proposes relaxing the strict request/response ordering because it is too constraining for high‑performance data paths. He suggests that the bus may need to deliver responses out of
    order, and the spec should support that for performance.
  • Demi Marie Obenour raises reset and out‑of‑order race risks if ordering is relaxed. She suggests a generation/stream number or bus‑level reset semantics to avoid stale or mis‑matched responses.
  • Michael S. Tsirkin recommends keeping v1 simple and introducing performance features via optional feature bits. This aligns with making relaxed ordering a negotiated capability rather than a
    baseline requirement.

Configuration Generation Optional Feature Discussion

  • Peter Hilber suggests making the SET_CONFIG generation count optional or dropping it, because Linux drivers assume config writes are not rejected and do not retry. His rationale is avoiding
    incompatibility with existing driver behavior.

VQ Enabled Flag Discussion

  • Peter Hilber suggests adding an explicit “queue enabled” indication in SET_VQUEUE (and also reporting it in GET_VQUEUE) so drivers can avoid an extra GET_VQUEUE round‑trip just to confirm
    enablement. This is aimed at reducing overhead in the initialization flow.

Device Number Size Discussion

  • Parav Pandit argues the 16‑bit device number is too small; he suggests 32‑bit to cover >64k devices and PCI BDF domain usage.
  • Demi Marie Obenour argues device identifiers should be 64‑bit or even 128‑bit to ensure long‑term uniqueness.
  • Peter Hilber highlights that reuse of device numbers after removal can cause races, and suggests adding a reuse‑limiting policy. He ties this to the broader question of expanding the device number
    space.

EVENT_CONFIG Data Mandatory Discussion

  • Demi Marie Obenour proposes making EVENT_CONFIG always include configuration data (MUST), to avoid a round trip. If data is mandatory, she notes that offset/length‑only signaling becomes
    unnecessary.
  • There is a noted contradiction with earlier guidance (Peter Hilber) and the current spec change ( I already updated the spec to require offset/length even when data is omitted, keeping data optional as I think this makes sense and i am not quite sure we can mandate the data without breaking compatibility with other transports or existing devices). This needs a clear decision.

Peter noted inconsistent use of “device number” vs “device ID”. I did not
find other ambiguous occurrences in transport-msg, so I added a sentence
explicitly distinguishing the bus-assigned device number from the virtio
device ID returned by GET_DEVICE_INFO.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Peter noted minor editorial issues (“Implement”/“respond”).
Update the Device bullet to “Implements” and “responds”.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Peter noted inconsistent naming. Update the virtio-msg revision wording
and paragraph title to consistently use “transport revision”.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Peter requested active-voice wording for the reserved header bits
requirement. Rephrase to make the bus implementation the subject.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Peter noted this requirement was already stated in the Initialization
Flow driver requirements. Remove the duplicate from the Device
Information driver requirements to avoid repetition.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Peter questioned “reset” on DEVICE_BUS_STATE_REMOVED. Rephrase the
intended-usage text to avoid implying a device reset after removal and
to focus on driver-side teardown.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Peter noted the spec implied “one or more” devices. Update the device
numbering section to allow zero devices, matching the earlier bus-layer
statement and hotplug use cases.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Demi suggested a non-normative note discouraging configuration space for
bulk data when a virtqueue can be used. Add a short explanatory note in
Device Configuration.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Parav requested symmetric naming for device hotplug events. Rename the
READY state to ADDED throughout the EVENT_DEVICE definition and related
text; values remain unchanged.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Peter asked whether transport feature bits are negotiated. Clarify that
the bus presents only the common subset and add a bus requirement to
advertise only mutually supported bits.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Peter asked whether devices may reset the configuration generation count.
Allow reset (including on device reset) and require drivers to avoid
assuming monotonic or persistent values.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Rephrase driver and device notification text to focus on which messages
are used, avoiding timing-specific requirements for EVENT_AVAIL,
EVENT_CONFIG, and EVENT_USED.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Strengthen EVENT_USED requirements to MUST, with explicit exceptions
for polling or other agreed-upon completion mechanisms.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Remove the runtime requirement to issue RESET_VQUEUE before
reprogramming a queue, since it does not avoid races.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Use le32 for admin_vq_start and admin_vq_count to match max_virtqueues.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
PROPOSAL: Clarify that echoed request parameters in responses are
informational and do not change the ordering-based correlation model.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Make device requirements explicit: if features are unsupported, the
device MUST clear FEATURES_OK in the status response.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Use le64 for shared memory length and address, and add a reserved
padding field aligned with existing message layouts.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Require EVENT_CONFIG to carry the affected offset/length even when
data is omitted, and update related guidance.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Clarify that DEVICE_NEEDS_RESET is reported via status updates without
repeating EVENT_CONFIG-specific requirements.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Require the bus to discard invalid bus msg_ids and leave transport
msg_id validation to the transport endpoints.

NOTE: I am unsure whether the explicit "MUST NOT validate transport msg_id"
line is necessary; please comment if a softer requirement is preferred.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Remove the redundant sentence in SET_DRIVER_FEATURES device requirements.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Add a non-normative description of bus-level rejection for forbidden
requests and how it can be surfaced to the driver.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Rephrase the Message Ordering requirements so the device, not the bus,
MUST send exactly one response per request. The bus ordering guarantee
remains unchanged.

Signed-off-by: Bertrand Marquis <Bertrand.Marquis@arm.com>
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Add an explicit note that virtqueue descriptors and buffer data are
exchanged via shared memory or other DMA-accessible memory. Transport
messages remain control-plane only.

Signed-off-by: Bertrand Marquis <Bertrand.Marquis@arm.com>
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Clarify that event notifications are independent and may be delivered
out of order with respect to other events, while request/response
ordering remains unchanged.

Signed-off-by: Bertrand Marquis <Bertrand.Marquis@arm.com>
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Align virtqueue and shared-memory address wording with mmio/pci by
consistently referring to physical addresses throughout the virtio-msg
transport text and message definitions.

Signed-off-by: Bertrand Marquis <Bertrand.Marquis@arm.com>
Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Mandatory messages such as GET_VQUEUE/SET_VQUEUE are 40-byte payloads
plus the 8-byte header. Update the revision table and the normative
minimum to 48 bytes so the minimum advertised size can carry all
mandatory messages.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
The bus is not responsible for validating device numbers. Drop the
"MUST NOT forward" clause from device-number assignment and clarify
that routing failures may be dropped or surfaced as a transport error.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
GET_DEVICE_INFO does not report shared memory segments. Update the
Device Information description to mention admin virtqueues only and
remove the shared-memory reference.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
The revision table shows the protocol-defined maximum message size,
while the text recommends advertising no more than 264 bytes. Add a
sentence to distinguish the protocol maximum from the recommended
advertised maximum.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Update the label path for "Bus Specific Messages" to match the Bus
Messages section hierarchy, keeping labels consistent with the document
structure.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
…our)

Clarify that buses may signal runtime notifications out-of-band but MUST
still relay or synthesize the corresponding EVENT_* transport message so
the transport always receives events uniformly.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Proposal: extend GET_DEVICE_INFO with a 16-byte device_uuid field so
a device can provide a unique identifier when available.

If UUID reporting is not supported, or the device has no identifier,
the device returns the nil UUID (all-zero value). Drivers interpret
the nil UUID as "no UUID provided".

Because this increases the fixed GET_DEVICE_INFO response size, raise
the transport minimum advertised message size from 48 to 52 bytes and
update the revision size table accordingly.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Proposal: update the common virtio-msg header to use `le32 msg_size`
and `le64 dev_num`, and move `dev_num` to the end of the header so the
new 16-byte layout keeps fields naturally aligned.

This updates header semantics and size-related limits accordingly, and
propagates 64-bit device numbers into bus message payloads that carry
device numbers (GET_DEVICES and EVENT_DEVICE). GET_DEVICES response
field order is adjusted to keep 64-bit fields aligned.

Related review discussion in mail threads:
- Parav Pandit (device-number width and msg_size concerns)
- Demi Marie Obenour (larger unique device identifiers)
- Peter Hilber (device-number reuse/race concerns)

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Add a bus-level recommendation in Device Number Assignment to prevent
immediate reuse of a device number after device removal.

This is specified as SHOULD (not MUST) to avoid hard policy
requirements while still reducing race conditions with delayed messages
associated with the removed device.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
… Hilber)

Extend GET_VQUEUE and SET_VQUEUE to carry queue state and update control
via a flags field.

For GET_VQUEUE, replace the reserved field with flags bit0 to report the
queue enabled state (bits[31:1] reserved zero).

For SET_VQUEUE, replace reserved0 with flags:
- bits[1:0]: queue state operation (disable, enable, keep, reserved)
- bits[5:2]: per-field ignore bits for size/desc_addr/driver_addr/device_addr
- bits[31:6]: reserved zero

Define flags==0 as legacy behavior (disable + apply all queue fields).
Add normative rules for valid flag values, atomic apply-or-reject
processing, and driver-side verification through GET_VQUEUE when
confirmation is required (SET_VQUEUE response has no payload).

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
@bertrand-marquis
Copy link
Collaborator Author

I add various patches for some specific proposals/requests in the review that we must discuss:

  • add a UUID field in get device info
  • make msg_size 32bit and device number 64bit
  • some attempt to say something about not reusing device numbers
  • modify set vqueue to add flags (enable or not, which field to update/modify)

…ter Hilber)

Clarify reset semantics so SET_DEVICE_STATUS(0) supports both synchronous
and asynchronous completion without ambiguity.

Define that a SET_DEVICE_STATUS response with status==0 indicates reset
completion, while a non-zero status may mean completion is still pending.
Require drivers that need confirmation to poll GET_DEVICE_STATUS until
status==0 before reinitialization, and require devices to report status 0
only after reset completion.

Align the Status Information and Device Reset/Shutdown sections with the
same completion model to avoid contradictory immediate-reset wording.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Proposal: gate strict configuration generation semantics behind a
transport feature bit (VIRTIO_MSG_F_STRICT_CONFIG_GENERATION).

When the feature is negotiated, keep strict generation behavior for
GET_CONFIG/SET_CONFIG/EVENT_CONFIG. When it is not negotiated,
generation fields are reserved: senders set them to 0 and receivers
ignore them, and SET_CONFIG generation mismatch must not cause
rejection.

Keep EVENT_CONFIG as the required configuration/status change
notification path, while allowing a simplified MMIO-like mode when
strict generation is not negotiated (including generic
offset/length=0 notifications).

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Proposal: describe baseline and strict configuration-generation behavior
in one dedicated "Configuration Semantics Profiles" subsection and
reference it from GET_CONFIG/SET_CONFIG/EVENT_CONFIG and related runtime
sections.

This keeps the non-strict (baseline) behavior explicit, reduces repeated
conditional wording across message definitions, and makes it clearer why
the baseline model matches MMIO-like config-change notification flow.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
…nour)

Allow EVENT_CONFIG to use offset=0, length=0, and no data as a generic
configuration-change notification when the device cannot express a bounded
affected range.

Rationale: this keeps an interoperable notification path for changes where
only an equivalent out-of-band notification is available without affected-range
details, while making driver behavior explicit through GET_CONFIG refresh.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
@bertrand-marquis
Copy link
Collaborator Author

I added some proposals for the points to be discussed among which:

  • make config generation count optional (default is MMIO compliant)
  • move extended generation count to its own chapter (to simplify the rest of the spec)
  • extend header size to 16bytes to have wider device number and max msg size
  • add a flag to set_vqueue

@wmamills
Copy link
Collaborator

wmamills commented Feb 26, 2026 via email

@wmamills
Copy link
Collaborator

wmamills commented Feb 26, 2026 via email

HVAC group discussion requested reverting the common header size field
change that moved msg_size to 32 bits.

Restore the common header to use a 16-bit msg_size plus a 16-bit
reserved field, and update common-header normative text to define
reserved-field transmit/receive behavior. This supersedes the earlier
32-bit msg_size direction and aligns the protocol maximum-size value
with a 16-bit size field.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
…group discussion)

HVAC group discussion identified that wrong generation count in
SET_CONFIG should not cause write refusal unless strict profile
semantics are negotiated.

Update configuration profile and message-specific normative text so
generation-mismatch rejection is strict-profile-only, baseline accepts
otherwise-valid writes, and baseline does not emit mismatch-only
EVENT_CONFIG notifications.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
…AC group discussion)

HVAC group discussion kept baseline ordering strict for request/response
handling while preserving asynchronous events.

Define per-device in-order forwarding for requests and in-order delivery
for corresponding responses, and state that events may be delivered at
any time relative to request/response traffic without bus- imposed delay
to enforce response-first ordering.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
…up discussion)

HVAC group discussion requested explicit virtqueue state-transition
rules so reconfiguration happens only while a queue is disabled.

Update SET_VQUEUE and RESET_VQUEUE semantics to prevent
enabled-to-disabled/reset transitions through SET_VQUEUE, require
RESET_VQUEUE before reconfiguring enabled queues when
VIRTIO_F_RING_RESET is negotiated, and tighten corresponding
driver/device normative behavior using explicit enabled/disabled
queue-state wording.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Andrei Homescu noted that feature capacity should use 32-bit block
semantics rather than bit-count semantics for consistency.

Rename the GET_DEVICE_INFO feature count field to num_feature_blocks and
align related GET_DEVICE_INFO/feature-negotiation prose to describe
32-bit block units consistently.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
…scu)

Andrei Homescu requested explicit rejected-response behavior for
SET_CONFIG to avoid ambiguity about returned payload bytes.

Specify that rejected SET_CONFIG responses MUST use length 0 and MUST
NOT include response data bytes, and align the message
description/comments with that requirement.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Andrei Homescu requested replacing bit-multiple wording in GET_DEVICES
with direct slot/byte semantics for simpler validation.

Update GET_DEVICES request/response descriptions and normative text to
define count in slots, bitmap size in bytes as (count + 7) / 8, and
explicit handling of unused bits in the final bitmap byte.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
… Tsirkin)

On-list review discussion requested including FF-A as a concrete
reference point for virtio-msg bus implementations.

Add a non-normative "Known Bus Implementation Examples" subsection with
references for virtio message bus over FF-A and PCI-for-AMP PoC, and
note Xen grant-table/event based bus work as under discussion.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
Andrei Homescu asked how drivers should handle GET_DEVICES responses
that return more entries than requested.

Define that GET_DEVICES responders MUST NOT return a count greater than
the request count. Keep smaller responses allowed when constrained by
message size or when requested ranges contain no present devices,
including count=0 with an empty bitmap when no devices are present in
the requested window.

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
- Replace old "Changes since RFC2" section with "Changes since v1
  (virtio-msg-patch1)"
- Summarize key v2 spec deltas with rationale tied to review feedback
- Add open-topics sectionfor ongoing discussions
- Move virtio-msg over virtqueues/admin-vq topic out of anticipated bus
  implementations into ongoing discussion
- Update FF-A reference to DEN0153 1.0 ALP1 link and mention it as the
  second draft aligned with the virtio-msg v1 proposal
- Reword series status text to reflect v2

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
@bertrand-marquis
Copy link
Collaborator Author

bertrand-marquis commented Mar 6, 2026

I added changes corresponding to the decisions we made during the last meeting and a proposal for the v2 cover letter and changes related to Andrei mail

@bertrand-marquis
Copy link
Collaborator Author

@wmamills , @edgarigl, @stsquad, @arnopo Could you give the result a look so that we could try to push this as v2 next week ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants