*OpenVINO™ Model Hub

*Select a popular AI model below to begin comparing performance benchmarks

*OpenVINO Verified Models ↗

*Filter by:

In Preview OpenVINO™ Model Server With Benchmarks Benchmark data available

*Sort by:

***

*Model List

*This page presents benchmark results for Intel® Distribution of OpenVINO™ toolkit for a representative selection of AI models on Intel® devices. The results may help you decide which hardware to use in your applications or plan AI workload for the hardware you already implemented in your solutions.

***

*Show Benchmark Settings

*Hide Benchmark Settings

*Choose by :

*Platform & SKUs *Inference hardware

*Platform & SKUs: :

*Inference hardware :

*CPUs *GPUs *NPUs

*Metrics :

*Throughput
(by batch size)

1
8
Auto

*Latency *First Token Latency *Second Token Latency

*Precision :

*INT4 *INT8 *FP16 *FP32 *BF16

*Click 'Show Benchmark Settings' above to change the default selections.

*The charts displayed below reflect pre-selected hardware and benchmark settings for this model.

*Hardware Platform details
*Configuration details

CSS Check

This feature is not yet implemented.

***

An unexpected error has occurred while initializing the application. Click the 'Reload' button below to try again.

Choose which input is prioritized when making secondary input selections available.

If 'Platform & SKUs' is chosen, then the hardware platform selections are the first priority. After selecting one or more platforms and SKUs, only the inference hardware associated with those platform selections will be available to be selected.

If 'Inference hardware' is chosen, then the inference hardware selections are the first priority. After selecting one or more inference hardware, only the platforms and SKUs associated with those inference hardware selections will be available to be selected.

Filter by Model Type: Use the Category filter to select your desired model type or deployment scenario.
Refine Your Search: Choose a Subcategory to narrow results based on your specific use case.
Select Hardware: Use the Inference Hardware filter to find models optimized for CPU, GPU, or NPU.

To see all the latest AI models optimized for OpenVINO, select the "All" option in each of the dropdown filters and uncheck any checkbox filters.

Platform

FPS

Tokens per sec.

higher is better

Throughput

(by batch size)

Latency

First Token Latency

Second Token Latency

seconds

lower is better

Chart data may not be available for some combinations of platforms, inference hardware, metrics, and precision.

CPU

iGPU

dGPU

NPU

Build charts

INT4

INT8

FP16

FP32

BF16

INT4

INT8

FP16

FP32

BF16

INT4 (MS)

INT8 (MS)

FP16 (MS)

FP32 (MS)

BF16 (MS)

CPUs

GPUs

NPUs

Inference hardware

Metrics

Precision

Platform & SKUs

Choose by

Show Benchmark Settings

Hide Benchmark Settings

OpenVINO™ performance benchmarks for a representative selection of Intel® platforms.

Model List

Download Model

OpenVINO Test Drive

Access Notebook

Discover AI models supported by OpenVINO. Select a popular AI model to access Jupyter notebooks and OpenVINO performance benchmarks.

OpenVINO™ Model Hub for AI Inference Benchmarks

Filter by

Sort by

Hardware platform details

Configuration details

Embedded Platforms

Client Platforms

Server Platforms

Accelerator Platforms

None

Featured models

Traditional AI models

Generative AI models

AI PCs

A to Z

Z to A

Newest first

Oldest first

Click 'Show Benchmark Settings' above to see the current selections and available options.

The charts displayed below reflect pre-selected hardware and benchmark settings for this model.

Disclaimers

Intel® Distribution of OpenVINO™ toolkit performance results are based on release {0} as of {1}.

The results may not reflect all publicly available updates. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software, or service activation. Learn more at intel.com, the OEM, or retailer.

See configuration disclosure for details. No product can be absolutely secure. Performance varies by use, configuration and other factors. Learn more at www.intel.com/PerformanceIndex. Intel optimizations, for Intel compilers or other products, may not optimize to the same degree for non-Intel products.

Results may vary. For more information, see F.A.Q. See Legal Information.

Details

JSON data invalid ({1}, file '{0}')

Model data invalid ({1}, file '{0}')

No models match filter selections.

View All

View Less

OpenVINO Verified Models ↗

This is a preview version validated on: {0}

Preview Version

Auto

In Preview