ONNX Runtime: ML inference and training accelerator
Cross-platform engine for optimized ONNX model execution.
Learn more about ONNX Runtime
ONNX Runtime is a runtime engine that executes machine learning models in the ONNX format across different platforms and hardware configurations. It accepts models from frameworks like PyTorch, TensorFlow, and scikit-learn, converting them to an intermediate representation for optimized execution. The runtime applies graph transformations, operator fusions, and hardware-specific optimizations to improve performance. Common deployment scenarios include inference serving in production environments and distributed training on GPU clusters.
Multi-framework model support
Accepts trained models from PyTorch, TensorFlow, scikit-learn, LightGBM, XGBoost, and other libraries through the ONNX format, allowing a single runtime to handle models from different ecosystems.
Hardware abstraction layer
Provides consistent inference and training APIs across CPUs, GPUs, and specialized accelerators while handling platform-specific optimizations internally, reducing the need for framework-specific deployment code.
Graph-level optimization
Performs operator fusion, constant folding, and other graph transformations at the ONNX level before execution, independent of the original training framework used to create the model.
import onnxruntime as ort
import numpy as np
session = ort.InferenceSession("model.onnx")
input_name = session.get_inputs()[0].name
input_data = np.random.randn(1, 3, 224, 224).astype(np.float32)
outputs = session.run(None, {input_name: input_data})
print(outputs[0])Related Repositories
Discover similar tools and frameworks used by developers
DALL-E
Official PyTorch package implementing the discrete VAE component for image tokenization used in OpenAI's DALL-E system.
X Recommendation Algorithm
Open source implementation of X's recommendation algorithm for timeline and notification ranking.
Stable Diffusion WebUI
Web UI for Stable Diffusion enabling AI image generation and editing in browser.
Whisper
Speech recognition system supporting multilingual transcription, translation, and language ID.
OpenAI Python
Type-safe Python client for OpenAI's REST API.