Building the EPYC box

I’ve been wanting to build an AI homelab for some time. The reason is that, like many others, I am becoming increasingly dependent on LLMs. However, I do not love sending all my data/ thoughts/ ideas/ doubts to servers all over the world. At the same time, small local models of 8 billion params, I found frustratingly bad. Then in early 2025 I saw this Youtube video. By around March 2026 I was ready to start building. This involved a lot of scouring ebay for deals.

At some point I saw a large amount of ram available for a not insane amount of money. This was my go-signal. The key to the CPU-based AI homelab is having a ton of RAM. So I bought 512GB. This got the clock ticking on the rest of the components, because Ebay has a 30 day return window after delivery. So I had to have a computer ready to test the RAM - the single biggest investment. Aside: PC components, especially AI related, fluctuate so much in price, so it doesn’t make sense to share all the prices.

Based on DigitalSpaceports’s build I had planned to go with a Gigabyte MZ32-AR0 motherboard. The awesome part of this is that it has 16 RAM slots. This means you can get to 512GB of RAM using 32GB sticks, which is much cheaper than getting to the same amount using 64GB sticks.

Takeaway - be very careful on the choice of RAM: RDIMM vs DIMM vs LRDIMM. LLMs were helpful here as this was new for me.

Then I needed a CPU. I went for the EPYC 64 core Rome edition. There is a quirk with the MZ32-AR0 which is that there are a few hardware versions rev1 to rev3. Rev1 can be upgraded in software to rev3, but CPUs that fit on rev3 will not work on rev1s. It’s complex and was one of the more stressful aspects of the build. In the end, I went for the oldest version. It was also cheaper.

A key takeaway from choosing a CPU for this work is you want one with 8 channels rather than 4. This can double the speed of Tokens Per Second (TPS).

With RAM, CPU, MOBO you have the core of the build. But there are still quite a few bits and bobs to get it going.

Case: I diverged from the digital spaceport’s open rack frame, because I wanted something neat and contained. Also because I have kids and openly accessible electronics are risky.

Power supply: went with the digital spaceport recommendation. I am very happy with this.

Cooler: went my own way - the Noctua Cooler. I have mixed feelings about this. First as an Austrian was keen to support the fam. Unlike a liquid cooler it really takes up a ton of space on the MOBO. This seemed to dash dreams I had of having a stack of GPUs. It’s turning out that because of the placement of the RAM on the MOBO, having multiple GPUs directly on the MOBO is not possible anyway. So in the end I may as well be happy with the cooler.

All of the lights. I went for 6 or so LGB fans. Yes, lights make the experience better. One gotcha in the fan specs is that "F" is for flow, which is for open frames. "P" is for pressure, for closed cases. I got "P" when I should have gotten "F", but it seems to not be significant.

SSD. Definitely keep the models on SSD. If they are on harddrive it takes ages to load large models into memory.

TLDR; the specs

• CPUAMD EPYC 7742 64c 2.25GHz 225W PROCESSOR
• MotherboardGigabyte mz32-AR0
• RAM16x MICRON 32GB DDR4-2400 RDIMM Server Ram
• CasePHANTEKS Enthoo Pro 2 Server Full Tower, XL-EEB, Tempered Glass
• CoolerNoctua NH-U14S TR4-SP3
• StorageSeagate IronWolf 4TB, Internal Hard Drive, for NAS RAID, NAS, 3.5", SATA 6GB/s, 7200RPM
• SSDSANDISK NVMe SSD 1TB
• Power Supply UnitAdapter ASROCK PG-1600G, PSU, 1600 W, 80 Plus Gold
• FansARCTIC P14 Pro A-RGB