CPU Benchmark Performance: AI and Inferencing

As technology progresses at a breakneck pace, so too do the demands of modern applications and workloads. With artificial intelligence (AI) and machine learning (ML) becoming increasingly intertwined with our daily computational tasks, it's paramount that our reviews evolve in tandem. Recognizing this, we have AI and inferencing benchmarks in our CPU test suite for 2024. 

Traditionally, CPU benchmarks have focused on various tasks, from arithmetic calculations to multimedia processing. However, with AI algorithms now driving features within some applications, from voice recognition to real-time data analysis, it's crucial to understand how modern processors handle these specific workloads. This is where our newly incorporated benchmarks come into play.

As chip makers such as AMD with Ryzen AI and Intel with their Meteor Lake mobile platform feature AI-driven hardware within the silicon, it seems in 2024, and we're going to see many applications using AI-based technologies coming to market.

We are using DDR5 memory on the Core i9-14900KS, as well as the other Intel 14th Gen Core series processors including the Core i9-14900K, the Core i7-14700K, Core i5-14600K, and Intel's 13th Gen at the relative JEDEC settings. The same methodology is also used for the AMD Ryzen 7000 series and Intel's 12th Gen (Alder Lake) processors. Below are the settings we have used for each platform:

  • DDR5-5600B CL46 - Intel 14th & 13th Gen
  • DDR5-5200 CL44 - Ryzen 7000
  • DDR5-4800 (B) CL40 - Intel 12th Gen

(6-1) ONNX Runtime 1.14: CaffeNet 12-int8 (CPU Only)

(6-1b) ONNX Runtime 1.14: CaffeNet 12-int8 (CPU Only)

(6-1c) ONNX Runtime 1.14: Super-Res-10 (CPU Only)

(6-1d) ONNX Runtime 1.14: Super-Res-10 (CPU Only)

(6-2) DeepSpeech 0.6: Acceleration CPU

(6-3) TensorFlow 2.12: VGG-16, Batch Size 16 (CPU)

(6-3b) TensorFlow 2.12: VGG-16, Batch Size 64 (CPU)

(6-3d) TensorFlow 2.12: GoogLeNet, Batch Size 16 (CPU)

(6-3e) TensorFlow 2.12: GoogLeNet, Batch Size 64 (CPU)

(6-3f) TensorFlow 2.12: GoogLeNet, Batch Size 256 (CPU)

(6-4) UL Procyon Windows AI Inference: MobileNet V3 (float32)

(6-4b) UL Procyon Windows AI Inference: ResNet 50 (float32)

(6-4c) UL Procyon Windows AI Inference: Inception V4 (float32)

Regarding AI and inferencing workloads, there is virtually no difference or benefit from going for the Core i9-14900KS over the Core i9-14900K. While Intel takes the win in our TensorFlow-based benchmark, the AMD Ryzen 9 7950X3D, and 7950X both seem to better grasp the type of AI workloads we've tested.

CPU Benchmark Performance: Science And Simulation Gaming Performance: 720p And Lower
POST A COMMENT

54 Comments

View All Comments

  • Thunder 57 - Friday, May 10, 2024 - link

    Agree 100%. My guess is the benchmarks were already done and rather than throw them in the trash where they belong they published them. I will take Gavin at his word that he will update this, but IMHO they should've held back until all of the benchmakrs were done. Reply
  • BushLin - Thursday, May 16, 2024 - link

    So... Doesn't look like the review has been updated, coming up to a week later Reply
  • Oxford Guy - Monday, May 20, 2024 - link

    Is anyone surprised? Reply
  • kkilobyte - Friday, May 10, 2024 - link

    What's the point of publishing the article without those extra tests in the first place? And since this can change your conclusions a bit, this is confusing and misleading.

    "Don't worry"? Let me remind you the article on the i5-14600k, which stilks says after a month that you are currently re-running SPEC2017 tests and "will update the results". Have those tests been completely re-run? Were the graphs fully updated? When was it updated? You didn't care to update the text accordingly, making the review confusing at best.

    So please, stop publishing articles where half of the job isn't even (properly) done. If not all tests are finished, then by all means finish them before publishing the article.
    Reply
  • Thunder 57 - Friday, May 10, 2024 - link

    Exactly. I have little faith in anything they say here as this website has fallen off a cliff. They just completely gave up on reviewing video cards. CPU reviews seem half assed since Ian Cuttress left. And you are right, the conclusion could (and should) change based on the next set of results. Will we see an updated conclusion? I sure hope so. Reply
  • PeachNCream - Saturday, May 11, 2024 - link

    Please be kind to AT. They're short staffed, underfunded, and cannot hang onto english native speaking talent so they're doing the best with what they have available. Besides, this is the first non-PSU, watercooler, or external storage device review they've published in a while. It's better we encourage them than insult them for leaving it half finished. 50% is better than 0% isn't it? Reply
  • Oxford Guy - Saturday, May 11, 2024 - link

    Given the situation with Intel's CPUs, it's unreasonable to publish a new article without testing according to the so-called power defaults.

    Utterly unreasonable.
    Reply
  • powerarmour - Sunday, May 12, 2024 - link

    Unbelievable Intel shilling now here, a waste of space this site. Reply
  • GeoffreyA - Saturday, May 11, 2024 - link

    I agree with PeachNCream. The insults are not helping anybody but probably making them feel more despondent and driving them away. Reply
  • kkilobyte - Saturday, May 11, 2024 - link

    I don't see any 'insulting' here. What I'm saying is that the job was not properly done. And no, a half-finished review article with incomplete data and a possibly not proper conclusion is not "better than nothing".

    That AT is understaffed is none of my business, and it doesn't in any case justify publishing half-finished content. And let's remember that this website is not a charity either.
    Reply

Log in

Don't have an account? Sign up now