docs: point the changeme error at cvs man, add browse-config-parameters guide

README.md

@@ -347,15 +347,21 @@ The `exec` command supports the following options:
 
 ```bash
 $ cvs --help
-usage: cvs [-h] [--version] {run,list,generate,monitor,exec} ...
+usage: cvs [-h] [--version]
+           {copy-config,generate,list,man,run,scp,monitor,exec} ...
 
 Cluster Validation Suite (CVS)
 
 positional arguments:
-  {run,list,generate,monitor,exec}
-    run                 Run a specific test (wrapper over pytest)
-    list                List available tests
+  {copy-config,generate,list,man,run,scp,monitor,exec}
+                        Available commands
+    copy-config         List or copy config files from CVS package. Lists
+                        configs if --output not specified.
     generate            Generate configuration files or templates
+    list                List available tests
+    man                 Explain the config parameters for a test
+    run                 Run a specific test (wrapper over pytest)
+    scp                 Copy files to all nodes in the cluster
     monitor             Run cluster monitoring scripts
     exec                Execute a command on all nodes in the cluster
 
@@ -364,6 +370,28 @@ options:
   --version             show program's version number and exit
 ```
 
+Config parameters are documented by `cvs man` rather than by comments inside
+the sample config files:
+
+```bash
+$ cvs man rccl_perf nic_model
+
+cvs man rccl_perf nic_model
+================================================================================
+
+Sweeps RCCL collectives across the cluster and checks bandwidth and latency.
+
+  rccl.cvs_params
+  ------------------------------------------------------------------------------
+
+  • nic_model  [string, default 'ainic']
+      NIC family used to select model-specific validations.
+      example: 'thor', 'ainic', 'connectx'
+
+================================================================================
+Total: 1 parameter
+```
+
 ```bash
 $ cvs run --help
 usage: cvs run [-h] --cluster_file CLUSTER_FILE --config_file

cvs/lib/utils_lib.py

@@ -311,7 +311,8 @@ def replace_in_string(value, path=""):
             error_msg += f"{'=' * 70}\n"
             error_msg += "ACTION REQUIRED:\n"
             error_msg += "Please edit your configuration file and replace all the '<changeme>' placeholders\n"
-            error_msg += "with an appropriate value before running the tests.\n"
+            error_msg += "with an appropriate value before running the tests.\n\n"
+            error_msg += "Run 'cvs man <test>' to see what each config parameter expects.\n"
             error_msg += f"{'=' * 70}\n"
 
             log.error("%s", error_msg)

docs/how-to/browse-config-parameters.rst

@@ -0,0 +1,126 @@
+.. meta::
+  :description: Look up what each CVS test config parameter means using cvs man
+  :keywords: CVS, man, config, configuration, parameters, defaults, documentation
+
+*****************************************
+Look up test configuration parameters
+*****************************************
+
+CVS provides a ``man`` command that explains every configuration parameter a
+test accepts: its type, its real default, any constraints, and an example
+value. Use it instead of reading the sample config files, which deliberately
+carry no inline documentation.
+
+The reference is generated by introspecting the pydantic schemas in
+``cvs/parsers/``, so it reflects what the code actually reads. In particular
+the default shown is the value that takes effect when you omit the key, which
+is not always the value that happens to appear in the sample config.
+
+List what is documented
+=======================
+
+Run ``cvs man`` with no arguments to see which tests have a parameter
+reference:
+
+.. code:: bash
+
+  cvs man
+
+.. code:: text
+
+  Config parameter references
+  ================================================================================
+
+    • rccl_perf
+        Sweeps RCCL collectives across the cluster and checks bandwidth and latency.
+
+Explain a test
+==============
+
+Pass a test name to see every parameter it accepts, grouped by config section:
+
+.. code:: bash
+
+  cvs man rccl_perf
+
+.. code:: text
+
+    rccl.cvs_params
+    ------------------------------------------------------------------------------
+
+    • nic_model  [string, default 'ainic']
+        NIC family used to select model-specific validations.
+        example: 'thor', 'ainic', 'connectx'
+
+Explain a single parameter
+==========================
+
+Add a parameter name to narrow the output. The name is matched exactly first,
+then as a substring of the dotted path, so a partial name also works:
+
+.. code:: bash
+
+  cvs man rccl_perf nic_model
+  cvs man preflight_checks rdma
+
+Machine-readable output
+=======================
+
+``--json`` emits the same reference as JSON, for tooling that generates
+configs or validates them:
+
+.. code:: bash
+
+  cvs man rccl_perf --json
+
+.. code:: json
+
+  {
+    "parameters": [
+      {
+        "path": "rccl.cvs_params.nic_model",
+        "type": "string",
+        "required": false,
+        "description": "NIC family used to select model-specific validations.",
+        "default": "ainic",
+        "examples": ["thor", "ainic", "connectx"]
+      }
+    ],
+    "test": "rccl_perf",
+    "config_files": ["/path/to/cvs/input/config_file/rccl/rccl_config.json"]
+  }
+
+``config_files`` is the real filesystem path to the shipped sample, resolved
+against wherever ``cvs`` is installed — not a path relative to your current
+directory.
+
+Reading the output
+==================
+
+.. list-table::
+   :widths: 3 7
+   :header-rows: 1
+
+   * - Notation
+     - Meaning
+   * - ``required``
+     - The test fails if the key is missing; there is no usable default.
+   * - ``default <value>``
+     - The value used when the key is omitted, taken from the code.
+   * - ``constraints:``
+     - Validation bounds, such as ``>= 2, <= 512``.
+   * - ``<key>`` in a path
+     - A mapping level whose keys you choose, as in
+       ``model_params.single_node.<key>.<key>.precision``.
+   * - ``[]`` in a path
+     - A list level, as in ``rvs.tests[].name``.
+
+.. note::
+
+  Not every suite has a reference yet. ``cvs man`` reports how many test
+  suites are still undocumented; those are being migrated one suite at a time.
+
+.. tip::
+
+  When a run aborts because a ``<changeme>`` placeholder was left in your
+  config, ``cvs man <test>`` tells you what that parameter expects.

docs/index.rst

@@ -25,6 +25,7 @@ The component public repository is located at `https://github.com/ROCm/cvs <http
   .. grid-item-card:: How to
 
     * :doc:`Run tests <how-to/run-cvs-tests>`
+    * :doc:`Look up test configuration parameters <how-to/browse-config-parameters>`
     * :doc:`Run ad-hoc cluster-wide commands <how-to/execute-cluster-commands>`
     * :doc:`Copy files and directories to cluster nodes <how-to/copy-to-cluster>`
     * :doc:`Monitor the health of GPU clusters <how-to/run-cluster>`

docs/sphinx/_toc.yml.in

@@ -17,6 +17,8 @@ subtrees:
   entries:
   - file: how-to/run-cvs-tests
     title: Run tests
+  - file: how-to/browse-config-parameters
+    title: Look up test configuration parameters
   - file: how-to/run-with-containers
     title: Run tests with the container backend
   - file: how-to/run-vllm-benchmarks

test_cli.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 # Test script for CVS package
-# This script tests the cvs list, generate, copy-config, monitor, and exec commands
+# This script tests the cvs list, generate, copy-config, man, monitor, and exec commands
 
 # Use the CVS environment variable if set, otherwise default to 'cvs'
 CVS="${CVS:-cvs}"
@@ -85,6 +85,17 @@ done
 # Test: cvs copy-config --list
 run_test "cvs copy-config --list" "$CVS copy-config --list"
 
+# Test: cvs man (list tests with a config parameter reference)
+run_test "cvs man" "$CVS man"
+
+# Test: cvs man <test> for each documented test, in both output formats
+echo "Testing: cvs man <test> for each documented test"
+echo "===================="
+for test in $($CVS man | grep "•" | awk '{print $2}'); do
+    run_test "cvs man $test" "$CVS man $test"
+    run_test "cvs man $test --json" "$CVS man $test --json"
+done
+
 # Test: cvs monitor (list all monitors)
 run_test "cvs monitor" "$CVS monitor"