CPU Benchmark Performance: AI and Inferencing

As technology progresses at a breakneck pace, so too do the demands of modern applications and workloads. With artificial intelligence (AI) and machine learning (ML) becoming increasingly intertwined with our daily computational tasks, it's paramount that our reviews evolve in tandem. Recognizing this, we have AI and inferencing benchmarks in our CPU test suite for 2024. 

Traditionally, CPU benchmarks have focused on various tasks, from arithmetic calculations to multimedia processing. However, with AI algorithms now driving features within some applications, from voice recognition to real-time data analysis, it's crucial to understand how modern processors handle these specific workloads. This is where our newly incorporated benchmarks come into play.

As chip makers such as AMD with Ryzen AI and Intel with their Meteor Lake mobile platform feature AI-driven hardware within the silicon, it seems in 2024, and we're going to see many applications using AI-based technologies coming to market.

We are using DDR5 memory on the Core i9-14900KS, as well as the other Intel 14th Gen Core series processors including the Core i9-14900K, the Core i7-14700K, Core i5-14600K, and Intel's 13th Gen at the relative JEDEC settings. The same methodology is also used for the AMD Ryzen 7000 series and Intel's 12th Gen (Alder Lake) processors. Below are the settings we have used for each platform:

  • DDR5-5600B CL46 - Intel 14th & 13th Gen
  • DDR5-5200 CL44 - Ryzen 7000
  • DDR5-4800 (B) CL40 - Intel 12th Gen

(6-1) ONNX Runtime 1.14: CaffeNet 12-int8 (CPU Only)

(6-1b) ONNX Runtime 1.14: CaffeNet 12-int8 (CPU Only)

(6-1c) ONNX Runtime 1.14: Super-Res-10 (CPU Only)

(6-1d) ONNX Runtime 1.14: Super-Res-10 (CPU Only)

(6-2) DeepSpeech 0.6: Acceleration CPU

(6-3) TensorFlow 2.12: VGG-16, Batch Size 16 (CPU)

(6-3b) TensorFlow 2.12: VGG-16, Batch Size 64 (CPU)

(6-3d) TensorFlow 2.12: GoogLeNet, Batch Size 16 (CPU)

(6-3e) TensorFlow 2.12: GoogLeNet, Batch Size 64 (CPU)

(6-3f) TensorFlow 2.12: GoogLeNet, Batch Size 256 (CPU)

(6-4) UL Procyon Windows AI Inference: MobileNet V3 (float32)

(6-4b) UL Procyon Windows AI Inference: ResNet 50 (float32)

(6-4c) UL Procyon Windows AI Inference: Inception V4 (float32)

Regarding AI and inferencing workloads, there is virtually no difference or benefit from going for the Core i9-14900KS over the Core i9-14900K. While Intel takes the win in our TensorFlow-based benchmark, the AMD Ryzen 9 7950X3D, and 7950X both seem to better grasp the type of AI workloads we've tested.

CPU Benchmark Performance: Science And Simulation Gaming Performance: 720p And Lower
POST A COMMENT

54 Comments

View All Comments

  • PeachNCream - Tuesday, May 14, 2024 - link

    They take a light approach to comment removal. Things would have to devolve into petty squabbling or delve into topics like AT's loss of web traffic or credibility so its not like they censor with a heavy hand. It's just to protect Future LLC's financial interests and how it appears to readers. Calm down please. Reply
  • Oxford Guy - Thursday, May 16, 2024 - link

    Are you an employee? Reply
  • nandnandnand - Friday, May 10, 2024 - link

    Instable means unstable?! NoooOOOoo! Reply
  • erotomania - Friday, May 10, 2024 - link

    Suffering instability while being unstable...yikes Reply
  • erotomania - Friday, May 10, 2024 - link

    Thanks Gavin. Enjoyable read. Pretty steep CPU+cooler investment, considering a $20ish Thermalright air cooler can cool an i5-14600K just fine.

    Autocorrect may have nipped this phrase: "...decide the IHS from the chip..."
    Reply
  • Gavin Bonshor - Friday, May 10, 2024 - link

    I appreciate your comment.

    Also, thank you for spotting that. My apologies! It has now been fixed :)
    Reply
  • NextGen_Gamer - Friday, May 10, 2024 - link

    I would so love for Anandtech to do another of their IPC-focused articles at this moment between AMD and Intel. Take the Core i9 14900K (or KS, don't matter), disable all the E-cores, and set all the P-cores at the same 4.0GHz. From AMD, use both the Ryzen 7 7800X and 7800X3D, and do the same: set all cores to 4.0GHz. Then take those three setups (Zen 4, Zen 4+X3D, Raptor Lake Refresh) through the full benchmark runs. Everyone on the same amount of cores, same frequency. Reply
  • Hulk - Friday, May 10, 2024 - link

    Can we see the v/f curve from the VID of the chip you have for testing?
    Specifically what are the VID voltages at 6.2 and 5.9? At 6.2 they can vary from just below 1.4V (golden sample) to over 1.5V (really bad).
    Reply
  • HideOut - Friday, May 10, 2024 - link

    Why is it Anandtech is posting reviews of a CPU thats been out a while now... Reply
  • Thunder 57 - Friday, May 10, 2024 - link

    They probably finally got enough ad revenue to buy one since no one samples them anything anymore it seems. Reply

Log in

Don't have an account? Sign up now