Overview

ONNX Runtime is a powerful, cross-platform accelerator designed for both machine learning inference and training. It's built to optimize the performance of ML models across diverse hardware and operating systems, enabling faster experiences and reduced costs.

Key Features

• High-Performance Inference: Accelerates inference for models from popular frameworks like PyTorch, TensorFlow/Keras, scikit-learn, LightGBM, and XGBoost.
• Broad Compatibility: Supports a wide range of hardware, drivers, and operating systems.
• Hardware Acceleration: Leverages available hardware accelerators to maximize performance.
• Graph Optimizations: Implements sophisticated graph optimizations and transforms for efficient execution.
• Accelerated Training: Significantly speeds up multi-node NVIDIA GPU training for transformer models with minimal code changes in PyTorch scripts.

Typical Use Cases

• Deploying Machine Learning Models: Integrate and run trained ML models efficiently in production environments, across various platforms.
• Accelerating Deep Learning Inference: Power real-time applications, recommendation engines, and computer vision tasks with low latency and high throughput.
• Optimizing Classical ML Performance: Enhance the speed of scikit-learn, LightGBM, and XGBoost models for faster predictions.
• Streamlining Model Training: Reduce training times for large transformer models on distributed GPU infrastructure.