CPU Benchmark Performance: AI and Inferencing

As technology progresses at a breakneck pace, so too do the demands of modern applications and workloads. With artificial intelligence (AI) and machine learning (ML) becoming increasingly intertwined with our daily computational tasks, it's paramount that our reviews evolve in tandem. Recognizing this, we have AI and inferencing benchmarks in our CPU test suite for 2024. 

Traditionally, CPU benchmarks have focused on various tasks, from arithmetic calculations to multimedia processing. However, with AI algorithms now driving features within some applications, from voice recognition to real-time data analysis, it's crucial to understand how modern processors handle these specific workloads. This is where our newly incorporated benchmarks come into play.

As chip makers such as AMD with Ryzen AI and Intel with their Meteor Lake mobile platform feature AI-driven hardware within the silicon, it seems in 2024, and we're going to see many applications using AI-based technologies coming to market.

We are using DDR5 memory on the Core i9-14900KS, as well as the other Intel 14th Gen Core series processors including the Core i9-14900K, the Core i7-14700K, Core i5-14600K, and Intel's 13th Gen at the relative JEDEC settings. The same methodology is also used for the AMD Ryzen 7000 series and Intel's 12th Gen (Alder Lake) processors. Below are the settings we have used for each platform:

  • DDR5-5600B CL46 - Intel 14th & 13th Gen
  • DDR5-5200 CL44 - Ryzen 7000
  • DDR5-4800 (B) CL40 - Intel 12th Gen

(6-1) ONNX Runtime 1.14: CaffeNet 12-int8 (CPU Only)

(6-1b) ONNX Runtime 1.14: CaffeNet 12-int8 (CPU Only)

(6-1c) ONNX Runtime 1.14: Super-Res-10 (CPU Only)

(6-1d) ONNX Runtime 1.14: Super-Res-10 (CPU Only)

(6-2) DeepSpeech 0.6: Acceleration CPU

(6-3) TensorFlow 2.12: VGG-16, Batch Size 16 (CPU)

(6-3b) TensorFlow 2.12: VGG-16, Batch Size 64 (CPU)

(6-3d) TensorFlow 2.12: GoogLeNet, Batch Size 16 (CPU)

(6-3e) TensorFlow 2.12: GoogLeNet, Batch Size 64 (CPU)

(6-3f) TensorFlow 2.12: GoogLeNet, Batch Size 256 (CPU)

(6-4) UL Procyon Windows AI Inference: MobileNet V3 (float32)

(6-4b) UL Procyon Windows AI Inference: ResNet 50 (float32)

(6-4c) UL Procyon Windows AI Inference: Inception V4 (float32)

Regarding AI and inferencing workloads, there is virtually no difference or benefit from going for the Core i9-14900KS over the Core i9-14900K. While Intel takes the win in our TensorFlow-based benchmark, the AMD Ryzen 9 7950X3D, and 7950X both seem to better grasp the type of AI workloads we've tested.

CPU Benchmark Performance: Science And Simulation Gaming Performance: 720p And Lower
POST A COMMENT

54 Comments

View All Comments

  • PeachNCream - Sunday, May 12, 2024 - link

    Reasonably, yes readers shouldn't be concerned with AT funding and staffing problems, but I think it's out of an abundance of kindness that we try our best to overlook the inaccuracies, incompleteness, and typos that currently plagues content here on an on-going basis in the hopes of being encouraging for the future health of the site. Assuming it gets a little time and effort from both employees and leadership, AT can once again become a useful site that does more than just repost company press releases and add grammatical errors during ill-informed fluff commentary. Reply
  • Iketh - Monday, May 13, 2024 - link

    Is there a donate/patreon link? Gotta get with the times. Get funding for complete reviews instead of releasing data which isn't even recommended by Intel. I know this site can't make enough money to keep people around even with all these ads. If your base readership gave $5 per year, I bet you could afford some full-time staff. Reply
  • goatfajitas - Sunday, May 12, 2024 - link

    It went downhill even before Anand left. At some point he sort of "checked out". Reply
  • GeoffreyA - Monday, May 13, 2024 - link

    He caught the Apple bug... Reply
  • PeachNCream - Tuesday, May 14, 2024 - link

    Didn't Apple eventually turn Anand into an employee? If the half-eaten fruit company bribed me with a lorry load of money, I'd have departed as well and at least played lip service to their overpriced popularity contest products. Running this site has to be more than a bit draining for someone that might have cared. Reply
  • AndrewJacksonZA - Saturday, May 11, 2024 - link

    Thanks Gavin. The Intel BIOS issue is something that I've been wondering if the press would re-test CPUs or not for. Reply
  • mode_13h - Sunday, May 12, 2024 - link

    > Don't worry; I will be testing Intel Default settings, too.

    Thank you. I'm looking forward to it.
    Reply
  • Oxford Guy - Monday, May 20, 2024 - link

    As we all are... ... ... Reply
  • shabby - Monday, May 13, 2024 - link

    Just use an air cooler, 400w power usage problem solved. Reply
  • powerarmour - Sunday, May 12, 2024 - link

    Absolutely, considering comments were removed on the last article this smells of bias otherwise. Reply

Log in

Don't have an account? Sign up now