AMD unveiled the first generation of Epyc, its new range of server processors built around its Zen architecture. Processors will range from the Epyc 7251—an eight-core, 16-thread chip running at 2.1 to 2.9GHz in a 120W power envelope—up to the Epyc 7601: a 32-core, 64-thread monster running at 2.2 to 3.2GHz, with a 180W design power.
AMD initially revealed its server chips, codenamed "Naples," earlier this year. Since then, we've known the basics of the new chips: they'll have 128 PCIe lanes and eight DDR4 memory controllers and will support one or two socket configurations. With today's announcement, we now know much more about how the processors are put together and what features they'll offer.
The basic building block of all of AMD's Zen processors, both Ryzen on the desktop and Epyc in the server, is the eight-core, 16-thread chip. Ryzen processors use one of these; the Threadripper high-end desktop chips use two; and Epyc uses four. Each chip includes two memory controllers, a bunch of PCIe lanes, power management, and, most important of all, Infinity Fabric, AMD's high-speed interconnect that is derived from coherent HyperTransport.
From our look at Ryzen, we already know that Infinity Fabric (IF) is used to connect two blocks of four cores (called "core complexes," CCXes) within each eight-core chip. IF is also used both to connect the chips within the multi-chip module (MCM), and, in two processor configurations, to connect the two sockets.
Within the processor, each chip has three IF links, one to each of the other three chips. Each link runs at up to 42GB/s in each direction. The speed of these links matches the 42GB/s of memory bandwidth offered by the two channels of 2,667MHz DDR4 memory that each individual chip supports, and what this means is that any one chip within the Epyc MCM can use the full memory bandwidth of the entire processor without bottlenecks. Accessing memory that's connected to a different chip will incur somewhat higher latency than accessing memory that's directly connected, but it comes at no bandwidth penalty.
In two socket configurations, there are four IF links between the sockets. Each chip in one socket is paired with a chip in the other socket, for four pairs total, with one IF link between each pair. This design means that accessing remote memory has, at most, a two-hop penalty and that there are multiple routes that data can use to move from a chip on one socket to a chip on the other. The cross-socket IF links are slightly slower than the internal ones, operating at 38GB/s bidirectional. This is because these links have higher error-checking overhead, which uses up some of their bandwidth.
Both the internal and external IF connections are power managed. If not much traffic is going across the links, the processor will cut back its performance and hence energy usage. Power not used on the links can instead be used for the cores themselves, with AMD saying that this power management can provide as much as an eight-percent improvement in performance per watt.
In total, each processor offers 128 I/O channels. In two socket configurations, 64 channels from each processor are used for Infinity Fabric connectivity, leaving an aggregate of 128 I/O channels still available. As such, both one socket and two socket configurations offer nearly identical I/O options. The main thing the I/O channels can be used for is PCIe connectivity, with up to eight PCIe 3.0x16 connections per system.
These can be subdivided all the way down to 128 PCIe 3.0 x1 links, and there's a good degree of flexibility to the possible configurations of PCIe lanes. Each chip can use eight of its links as SATA connections, too. This is one of the few areas where a two-socket system will give you more I/O capabilities; with two sockets, the chips would support a total of 16 SATA connections.
Epyc is designed as a system-on-chip. Many features that would typically need additional components on the motherboard have been integrated into what AMD calls the Server Controller Hub (SCH) within the Epyc processor itself. This includes four USB 3.0 controllers, serial port controllers, clock generation, and low-speed interfaces such as I2C. The one notable I/O component not in the processor is Ethernet; for that, you'll need a PCIe card or motherboard-integrated interface.