Intel AI Solutions in 2023: Xeon, FPGA, Movidius, OpenVINO and oneAPI

As of August 2023, when this post was written, Intel's AI offer combined hardware across the range, from Xeon Scalable server processors and FPGAs to Movidius vision processors for the edge, with a common software layer built on oneAPI and the OpenVINO toolkit, plus cloud access for testing through Intel DevCloud. The idea was that a developer could build a model once and deploy it on whichever Intel device suited the cost, power and latency target. Several parts of that line-up have since changed, and an update section at the end says how.

Intel AI solutions are aimed at business, science and society uses. Whether it is accelerating drug discovery, enhancing customer service or optimising supply chains, Intel positions its AI solutions as a way for organisations to achieve more with less time and resources. In this post we look at the key parts of the Intel AI portfolio as it stood in 2023, what each is for, and how they fit together.

Key takeaways

• Hardware (2023): Xeon Scalable CPUs, Optane persistent memory, Intel FPGAs and Movidius VPUs, with Intel DevCloud for trying them remotely.
• Software: oneAPI for cross-architecture programming, OpenVINO for optimised inference, and the oneAPI AI Analytics Toolkit (now called AI Tools) for data science and ML in Python.
• Intel's pitch is breadth: one software stack across CPU, GPU, FPGA and VPU.
• Optane was already being wound down when this post was written (announced July 2022), and the FPGA business has since become Altera, now majority owned by Silver Lake.

The hardware portfolio (as of 2023)

Intel AI solutions are built on a portfolio of hardware, software and services intended to deliver performance, scalability and ease of use.

Product	Where it fits	Typical AI role	Status since 2023
Xeon Scalable processors	Data centre servers	Training of smaller models, inference, analytics	Continuing product line
Optane persistent memory	Servers with large datasets	Large, fast memory tier	Wind-down announced July 2022
Intel FPGA	Edge, network, data centre	Custom low-latency inference	Now Altera, majority owned by Silver Lake since Sept 2025
Movidius VPU	Cameras, drones, robots	Low-power vision inference	Neural Compute Stick 2 discontinued
DevCloud for the Edge	Cloud	Remote testing on Intel hardware	Intel's developer cloud offers have since been reorganised

Intel Xeon Scalable processors

The foundation of Intel AI solutions, these processors offer high-performance computing for diverse workloads, including deep learning, machine learning and analytics. Xeon Scalable processors support frameworks and libraries such as TensorFlow, PyTorch and the Intel oneAPI AI Analytics Toolkit, to enable faster development and deployment of AI applications. The 4th Gen Xeon Scalable processors launched in January 2023 added Intel Advanced Matrix Extensions (AMX), on-chip matrix units for BF16 and INT8 arithmetic, which Intel positioned as the main AI feature of that generation. A CPU is often the cheapest place to run inference when the server already exists and the model is modest in size.

Intel Optane persistent memory

This technology provides large-capacity, high-speed and persistent memory for data-intensive workloads. Optane persistent memory enables faster data access and analysis than storage, lower latency and, by Intel's account, reduced power consumption. It can also be used as a cache or tiered storage to improve the performance of SSDs and HDDs. A correction for readers: Intel announced on 28 July 2022, with its second-quarter results, that it was winding down the Optane business and would stop future development, while continuing to supply and support existing customers for a period. So by the time this post was first published Optane was no longer a growing part of the AI line-up. CXL-attached memory is the direction the industry has taken for large memory tiers.

Intel FPGA

These programmable devices offer flexibility and customisation for specific AI tasks, such as image processing, natural language processing and computer vision. Intel FPGAs can accelerate inference, and in some cases training, by offloading work from the CPU or GPU. They can also be reconfigured to adapt to changing requirements and algorithms. FPGAs are most competitive for low-latency inference with reduced precision; see FPGA based machine learning inference and FPGAs and AI.

Intel Movidius VPU

These low-power devices are designed for edge computing and embedded applications, such as drones, cameras and robots. A Movidius VPU (vision processing unit) can perform vision tasks such as object detection, face recognition and semantic segmentation with high efficiency and low latency, and can run more than one neural network on a single device. The Neural Compute Stick 2, a USB device built on the Movidius Myriad X VPU, reached its last shipment date on 30 June 2022, according to Intel support. Intel's later client processors include an on-chip NPU, which was widely reported to build on the Movidius design team's work.

Intel DevCloud for the Edge

This cloud platform let developers test and prototype AI applications on various Intel hardware, such as CPU, GPU, FPGA and VPU, without buying it. It provided pre-installed frameworks and tools such as the Intel Distribution of OpenVINO toolkit, and hardware targets including the Intel Neural Compute Stick 2. A small correction: the original text listed the Neural Compute Stick 2 as a tool; it is a hardware device that DevCloud users could run their code on. DevCloud also offered tutorials, samples and documentation to help developers get started quickly.

The software stack

Intel's argument is that its AI solutions are not only capable but also easy to use and integrate, through a unified software stack that simplifies development and deployment across different hardware.

Intel oneAPI

oneAPI is a cross-architecture programming model intended to let developers write code once and run it on different processor types. It supports C++ and Python, with Data Parallel C++ (DPC++), Intel's implementation of the Khronos SYCL standard, as the main language for accelerators. oneAPI is an open specification, and with third-party plug-ins it can also target some non-Intel GPUs. It includes domain-specific toolkits for AI, HPC, IoT, rendering and media.

Intel Distribution of OpenVINO toolkit

This toolkit optimises the performance of inference, originally for computer vision and now for a wider range of models, on Intel hardware. It accepts models from frameworks and formats such as TensorFlow, TensorFlow Lite, PyTorch and ONNX, can be used from ONNX Runtime through its OpenVINO execution provider, works alongside OpenCV for image handling, and offered pre-trained models through the Open Model Zoo. It provides tools for model conversion, optimisation (including quantisation to INT8), validation and deployment.

Intel oneAPI AI Analytics Toolkit (now AI Tools)

This toolkit accelerates the end-to-end workflow of data science and machine learning on Intel hardware, with Intel-optimised versions of popular Python tools. Its well-documented components include Intel-optimised TensorFlow and PyTorch, the Intel Extension for Scikit-learn, an optimised XGBoost, Modin as a faster drop-in for pandas, and the Intel Distribution for Python with NumPy, SciPy, Numba and daal4py. The original post also listed Dask, Spark MLlib and daal-tk; these are related ecosystem tools that Intel has worked with, but they are not clearly part of the toolkit download, so treat that part of the list with care. A correction: RAPIDS, cuDF, cuML and cuGraph, which were also listed, are NVIDIA's GPU data science libraries and are not part of Intel's toolkit. Intel has since renamed the bundle AI Tools from Intel. It provides tools for data ingestion, preprocessing, feature engineering, model training, evaluation and deployment.

How the pieces fit: an illustrative example

Consider a retailer building shelf monitoring with cameras in stores. The numbers here are illustrative, not Intel benchmarks.

1. Data scientists clean images and train a detection model in PyTorch on Xeon servers, using the AI Tools packages.
2. The trained model is converted with OpenVINO and quantised to INT8, shrinking it to roughly a quarter of its FP32 size.
3. Candidate edge devices are compared on DevCloud: a small Core CPU, an integrated GPU and a VPU, measuring frames per second and power.
4. If the VPU delivers, say, 30 frames per second at about 2 W and the CPU 40 frames per second at 15 W, the VPU wins for a battery or fanless camera and the CPU for a store server handling several cameras.

The value of a common stack is that steps 2 to 4 reuse the same model and tools, rather than a separate flow per device.

Since this post: what has changed

• Optane: wind-down announced July 2022, as noted above.
• FPGAs: Intel announced in October 2023 that it would run its Programmable Solutions Group as a standalone business, relaunched it as Altera in 2024, and sold a 51 percent stake to Silver Lake in a deal that closed in September 2025. See Intel PSG spinoff.
• AI accelerators: Intel's Gaudi accelerators (from its Habana Labs acquisition) and data centre GPUs became the main parts of its data centre AI offer, alongside Xeon.
• Client AI: Core Ultra processors, launched in December 2023, added an on-chip NPU for AI PCs.

FAQ

Can I run deep learning on a Xeon CPU without a GPU?

Yes, for inference and for training smaller models. Newer Xeons with AMX are much faster at this than older ones, though large-model training still favours GPUs and accelerators.

What is OpenVINO used for?

Converting and optimising trained models so they run fast on Intel CPUs, GPUs and NPUs, mainly for inference.

Is Intel Optane still available?

Intel stopped developing Optane after its July 2022 announcement; check current supply status with Intel or resellers.

Related reading: Top 10 things to know about Intel data centre products (March 2023).