On Tue, 2016-10-04 at 12:52 -0600, Alex Williamson wrote:
I'ts all just idle number games, but what I was thinking of was the
difference between plugging a bunch of root-port+upstream+downstreamxN
combos directly into pcie-root (flat), vs. plugging the first into
pcie-root, and then subsequent ones into e.g. the last downstream port
of the previous set. Take the simplest case of needing 63 hotpluggable
slots. In the "flat" case, you have:
2 x pcie-root-port
2 x pcie-switch-upstream-port
63 x pcie-switch-downstream-port
In the "nested" or "chained" case you have:
1 x pcie-root-port
1 x pcie-switch-upstream-port
32 x pcie-downstream-port
1 x pcie-switch-upstream-port
32 x pcie-switch-downstream-port
You're not thinking in enough dimensions. A single root port can host
multiple sub-hierarchies on it's own. We can have a multi-function
upstream switch, so you can have 8 upstream ports (00.{0-7}). If we
implemented ARI on the upstream ports, we could have 256 upstream ports
attached to a single root port, but of course then we've run out of
bus numbers before we've even gotten to actual devices buses.
Another option, look at the downstream ports, why do they each need to
be in separate slots? We have the address space of an entire bus to
work with, so we can also create multi-function downstream ports, which
gives us 256 downstream ports per upstream port. Oops, we just ran out
of bus numbers again, but at least actual devices can be attached.
What's the advantage in using ARI to stuff more than eight
of anything that's not Endpoint Devices in a single slot?
I mean, if we just fill up all 32 slots in a PCIe Root Bus
with 8 PCIe Root Ports each we already end up having 256
hotpluggable slots[1]. Why would it be preferable to use
ARI, or even PCIe Switches, instead?