vda5050-lab — Evidence-Bound Diagnostics for VDA 5050 Temporal Failures

VDA 5050 commissioning failures are not always detectable with JSON Schema validation. An order or state message may be valid in isolation while the failure depends on cross-topic ordering, a connection transition, the publishing participant, or an incomplete observation point.

vda5050-lab is an open-source diagnostic tool that separates observed facts, protocol expectations, missing evidence, and the next component to investigate.

Problem definition#

The project addresses four related problems:

1. Single-message validation cannot evaluate state transitions. Reconnects, order updates, cancel flows, and action lifecycles depend on relationships among messages.
2. A passive log cannot always prove that a message was absent. Capture start time, QoS 0 loss, and observation vantage must be represented.
3. Multi-robot diagnosis requires strict identity separation. Topic, serialNumber, MQTT ClientId, participant, and connection epoch must not be joined across robots.
4. The rendered demo must use the same trace that the diagnostic engine evaluates. Otherwise the visualization does not validate the diagnostic path.

v0.2.0 implements one bounded reconnect diagnosis and validates it under the same contract with 1, 2, 5, 10, 50, and 100 robots.

Solution and implementation#

Inside an isolated Docker network, one run performs the entire evidence path:

1. Start a disposable Eclipse Mosquitto broker
2. Concurrently connect N virtual mobile robots, one fleet-control actor, and one recorder
3. Give every robot a unique VDA serialNumber, topic, MQTT ClientId, order ID, participant identity, and route
4. Abnormally disconnect only demo-001, causing the broker to publish its CONNECTION_BROKEN Last Will
5. Reconnect the same participant in a new connection epoch, intentionally omitting retained ONLINE
6. Record broker-egress observations as a canonical trace
7. Diagnose that trace in a Doctor container with networking disabled
8. Verify the manifest, trace, evidence, and report before tearing everything down

Scenario runner
  └─ Docker internal network
      ├─ Mosquitto
      ├─ Virtual robots × N
      ├─ Fleet actor
      └─ Recorder
            │
            ├─ run-manifest.json
            ├─ trace.canonical.jsonl
            └─ synthetic-evidence.json
                         │
                         ▼ network disabled
                   vda5050-doctor
                         │
                         └─ doctor-report.json

The broker has no host-published port, and Doctor runs with network_mode: none. Because the scenario uses retained connection messages, every scale and every fault/control case receives a fresh broker and network. Retained state from an earlier run cannot leak into a later verdict.

Validating identity separation at multiple scales#

All N robot actors connect to the same broker concurrently, and each carries a different identity and route. The contracts verify the count of unique identities in the run manifest and canonical trace in addition to the number of rendered robots.

The connection budget is N robots plus fleet control and the recorder. Before any connection, the run manifest fixes the message budget, 60-second duration limit, actor count, and MQTT connection ceiling. Exceeding a bound invalidates the run instead of producing a conveniently incomplete success.

Positions are protocol x/y coordinates#

Each route is generated as project-specific map coordinates carried in VDA messages. The synthetic map is warehouse-demo; x and y are meters, and the lower-left origin is only a presentation convention for this demo.

For a zero-based robot index i:

column       = floor(i / 10)
row          = i mod 10
start        = (6 × column,     2 × row)
released_end = (6 × column + 4, 2 × row)
horizon_end  = (6 × column + 4, 2 × row + 1)

This places demo-001 on (0, 0) → (4, 0) → (4, 1) and demo-100 on (54, 18) → (58, 18) → (58, 19). The renderer does not trust a decorative layout from the manifest: it reads x/y values from the order, state, and visualization payloads that were actually recorded in the trace.

The spacing makes routes visually distinct. It says nothing about robot footprint, braking distance, allowed deviation, collision avoidance, or clearance in a real warehouse.

Changing the verdict with the strength of evidence#

The LAB-D4-RECONNECT-STATE rule examines the VDA 5050 3.0.0 connection lifecycle. After an abnormal disconnect, the broker publishes the robot's CONNECTION_BROKEN LWT; after reconnecting, the robot is expected to publish retained ONLINE.

Doctor deliberately reaches different conclusions for the same fault scenario as the strength of evidence changes:

Turning “I did not see it” into “they did not send it” is an easy way to blame the wrong vendor. Important VDA topics can use QoS 0, so a passive subscriber cannot always prove absence. The lab therefore separates raw observations from external assertions and binds its evidence manifest to the exact trace digest.

“No finding” is not general PASS or conformance certification. It only says that this one rule's fault condition did not hold in the synthetic control.

Usage#

The prerequisites are Docker Compose, jq, and a repository checkout. Start with ten robots:

git clone https://github.com/ekusiadadus/vda5050-lab.git
cd vda5050-lab
make demo-fleet DEMO_ROBOTS=10

Run all six published scales with:

make demo-fleet-e2e

Each successful scale produces run-manifest.json, trace.canonical.jsonl, synthetic-evidence.json, and doctor-report.json. The README GIFs and MP4 files are rendered after checking those artifacts and the report digest. Their coordinates and identities come from the recorded trace payloads.

Binaries, all six MP4/GIF pairs, the overview media, and SHA256SUMS are available from the v0.2.0 release.

What the 100-robot result does not prove#

This is a bounded functional result: the isolated run handled one hundred virtual MQTT actors without confusing their identities while preserving the same diagnostic contract. It is not evidence that:

• one hundred physical robots were connected;
• one hundred vendors were tested for interoperability;
• production capacity, latency, or real-time behavior is guaranteed;
• collision avoidance, functional safety, or a site network was validated; or
• the implementation is certified conformant with all of VDA 5050.

Keeping demos, benchmarks, interoperability tests, and certification claims separate is a project-level design rule.

Remaining work and scope#

v0.2.0 verifies the evidence model end to end with synthetic traces. The next step is to collect real, safely shareable incident traces and add behavioral diagnoses for order updates, base/horizon stitching, newBaseRequest, cancel flows, action lifecycles, and 2.1-to-3.0 migration.

The scope is limited to VDA 5050 communication and state-transition diagnosis rather than a general-purpose viewer. The intended output for a commissioning engineer is:

1. what happened;
2. which messages support that statement;
3. which specification clause applies;
4. what was not observed and therefore cannot be concluded; and
5. whether to investigate the robot, fleet control, or communication layer next.

If you can share a real interoperability failure trace, open a GitHub issue. For traces that cannot be public, pseudonymization must reject unknown fields and unparseable payloads by default; those requirements should be defined with the teams that hold the source data.

Links#

• vda5050-lab repository
• v0.2.0 release
• Demo architecture and evidence contract
• Official VDA 5050 3.0.0 specification