The PCIe lanes conundrum
WHEN PCI EXPRESS first started to replace AGP and PCI slots in 2003, most motherboards only included a single x16 slot, and maybe one or two x1 slots. Back then, it really didn’t matter, because most PCs only needed the new interface for the graphics card
The problem is, many PCs are severely lacking in the PCIe lanes department—and it’s largely Intel’s fault.
Starting in 2009, with socket LGA1156, Intel moved the fastest PCIe lanes directly on to the CPU package. Eight years ago, we had 16 PCIe 2.0 lanes on Intel’s Lynnfield (first-gen Core) processors, and for the mainstream platform, that’s where we’ve stayed. Doubling the bandwidth per lane going from PCIe 2.0 to PCIe 3.0 helped curb our need for more lanes, but we’re fast running into bottlenecks.
Consider a reasonably high-end PC, with a fast GPU and NVMe storage. The graphics card still gets the 16 PCIe lanes from the CPU, but the NVMe SSD shares bandwidth with networking, audio, USB, and more. Multiple NVMe drives can sit off the chipset, which provides 20/24 PCIe lanes for Z170/Z270, but the chipset still talks to the CPU over a DMI 3.0 link (equivalent to x4 PCIe Gen3). And the fastest NVMe drives can saturate the DMI link with a single drive.
You want faster PCIe-based storage? Intel has its enthusiast X99 and X299 platforms, where a single CPU can provide up to 40/44 PCIe lanes. But that “up to” just got a lot messier with Skylake-X and Kaby Lake-X, where the latter is basically just a Kaby Lake die slapped into a larger package. So, Kaby Lake-X gets you the same 16 PCIe lanes and dual-channel memory controller—and some of the PCIe slots on X299 mobos may not work with Kaby Lake-X. Ugh.
Here’s where I start to get angry. We already had the arbitrary limitation of 28 PCIe lanes on the entry-level enthusiast processors, such as the i7- 5820K and i7-6800K, which are still potent six-core processors. Pay an extra $150 for the i7-5930K or i76850K, and you get the fully enabled 40 PCIe lanes. Skylake-X bumps that up to 44—but only if you spring for a Core i9 part. The six-core i77800X and eight-core i7-7820X are both 28 PCIe lane parts. That means you can’t do dual x16 graphics cards in SLI or CrossFire unless you buy a $1,000-plus processor, at least on the Intel side of the fence.
Thank goodness for AMD, where not only do we have $300 eight-core processors that don’t suck, but we’ll soon get Ryzen Threadripper with up to 16 cores. The socket TR4 for Threadripper is insanely huge, as it’s a repurposed socket SP3 that supports up to 32core Epyc processors. The socket supports eight-channel memory, and 128 PCIe lanes with Epyc, but Threadripper will have to get by with “only” half the lanes and memory channels. The good news is that AMD says all the Threadripper parts will include a full 64 PCIe lanes.
Of course, both Threadripper and Skylake-X are geared for enthusiasts and professionals. If you’re running a mainstream Ryzen or Intel Core CPU, you’re stuck with 16 lanes on Intel and 24 on AMD, plus additional PCIe lanes from the chipset. Why does that matter? For a lot of us, it doesn’t, but multiple GPUs and/or M.2 SSDs can hit bottlenecks. There are also professional workloads (such as machine learning) where having the additional bandwidth is beneficial.
For typical consumers, you can ignore all the hubbub about PCIe lanes, and run your single graphics card and maybe an M.2 SSD just fine. But if you’re looking to build a personal Dream Machine? Don’t get caught with insufficient PCIe lanes. Or wait for PCI Express 4.0, which doubles the bandwidth per lane of PCIe 3.0—and preliminary specs for PCIe 5.0 are also out, penciled in for 2019 with up to 4GB/s per lane.
You can’t do dual x16 graphics cards in SLI or CrossFire unless you buy a $1,000-plus processor.
Jarred Walton has been a PC and gaming enthusiast for over 30 years.