An Intro to NVIDIA Cluster Networking

Three ways to network a HPC Cluster and how to work out the best approach

There are generally three ways of networking your HPU cluster in an HPC environment:

1. Fat-tree (2 and 3-tier):
2. Spine-leaf (Clos):
3. Dragonfly+ :

Fat-tree (2 and 3-tier):

A typical three-layer fat-tree consists of your edge layer, the switches that connect toyour GPU servers; the spine layer (aggregation), the intermediate switches that connect the edge switches to each other; and the core layer, the top-tier switches that provide your inter-pod connectivity

In a pure fat-tree, every leaf switch connects to every spine switch, creating a blocking fabric. You are usually looking at four to six hops depending ontraffic moving leaf to spine to spine to leaf. In a three-tier design you can have 100,000-plus endpoints, generally.

These are usually for workloads where you need guaranteed training, and cost isn't so much of an issue. bandwidth, such as LLM

Spine-leaf (Clos):

A spine and leaf arhitecture has the leaf layer, the top of rack switches (although in and NVIDIA installation the switches are in a seperate rack), and the spine layer, which provides the inter-connectivity between the leaf switches. Every leaf connects to every spine, but the design allows for asymmetric desisns

Spine-leaf is excellent for moving large amounts of data quickly. There are generally only 2 hops between nodes; it allows for oversubscription and it can scale efficiently.Generally it is used for inference clusters, or where cost is an issue

NVIDIA DGX SuperPODs use spine-leaf with InfiniBand (usually)

Dragonfly+ :

Dragonfly+ is for huge-scale systems and organises nodes into groups called "super pods". Intra group, all the switches are fully connected; inter group, the switches are connected in a specific pattern. It can also be extended to multiple levels.

You’re looking at a diameter of 3 hops but you need to manage the traffic, so you’re looking at routing algorithms (adaptive) and you need to really think about where you are placing your workloads.

Choosing the right topology

1. Small to Medium Clusters: Spine and leaf in a two tier Clos works well: ideal for training and easy to deploy
2. Larger Clusters: Fat-free or Spine-leaf with minimal oversubscription, is what your're after. You get predictable performance and that justifies the higher costs.
3. Huge Clusters: Dragonfly+ or a multi tier Clos is what you'll need. It gets a lot cheaper than the fat-free equivalents, but at the expense of the increased complexity.

You can, of course, hybridise all of this for your own particular use case.