h100 only has 70GB on bus

[poptree]

log is attached, any adviced?

I am testing on 2*8 h100 machines, DOCA-HOST installed, with 400GB * 8 IB.

log.txt

[AddyLaddy]

This is probably more a question for the main NCCL library project.

Anyway, I can't see anything wrong in your logs.
NCCL_COLLNET_ENABLE=1 is only required if you have an IB SHARP enabled network, but I can't see it would affect this test.

I'd measure what you achieve with each NIC/HCA individually and then test that the BW increases as you combine them.

But my suspicion would be that you have ACS still enabled on these nodes.