Deci delivers breakthrough inference performance on Intel’s 4th Gen Sapphire Rapids CPU

Print Friendly, PDF & Email

Deci, the deep learning company building the next generation of AI, announced a breakthrough performance on Intel’s newly released 4th Gen Intel® Xeon® Scalable processors, code-named Sapphire Rapids. By optimizing the AI models which run on Intel’s new hardware, Deci enables AI developers to achieve GPU-like inference performance on CPUs in production for both Computer Vision and Natural Language Processing (NLP) tasks.

Deci utilized its proprietary AutoNAC (Automated Neural Architecture Construction) technology to generate custom hardware-aware model architectures that deliver unparalleled accuracy and inference speed on the Intel Sapphire Rapids CPU. For computer vision, Deci delivered a 3.35x throughput increase, as well as a 1% accuracy boost, when compared to an INT8 version of a ResNet50 running on Intel Sapphire Rapids. For NLP, Deci delivered a 3.5x acceleration compared to the INT8 version of the BERT model on Intel Sapphire Rapids, as well as a +0.1 increase in accuray. All models were compiled and quantized to INT8 with Intel® Advanced Matrix Extensions (AMX) and Intel extension for PyTorch.

Figure 1: Chart illustrating the improved throughput by running an AutoNAC generated model for Intel Sapphire Rapids

“This performance breakthrough marks another chapter in the Deci-Intel partnership which empowers AI developers to achieve unparalleled accuracy and inference performance with hardware-aware model architectures powered by NAS,” said Yonatan Geifman, CEO and Co-Founder of Deci. “We are thrilled to enable our joint customers to achieve scalable, production grade performance, within days”.    

Figure 2: Chart illustrating the boost in NLP throughput performance by running an AutoNAC generated model onIntel Lake Sapphire Rapids.

Deci and Intel have maintained broad strategic business and technology collaborations since 2019, most recently announcing the acceleration of deep learning models using Intel Chips with Deci’s AutoNAC technology . Deci is a member of the Intel Disruptor program and has collaborated with Intel on multiple MLPerf submissions. Together, the two are enabling new deep learning based applications to run at scale on Intel CPUs, while reducing development costs and time to market. 

If you are using CPUs for deep learning inference or planning to do so, talk with Deci’s experts to learn how you can quickly obtain better performance and ensure maximum hardware utilization. To learn more about the Deci-Intel collaboration, go to 

Sign up for the free insideAI News newsletter.

Join us on Twitter:

Join us on LinkedIn:

Join us on Facebook: NewsNOW

Speak Your Mind