ONNX Runtime: ML inference and training accelerator
Cross-platform engine for optimized ONNX model execution.
Learn more about ONNX Runtime
ONNX Runtime is a runtime engine that executes machine learning models in the ONNX format across different platforms and hardware configurations. It accepts models from frameworks like PyTorch, TensorFlow, and scikit-learn, converting them to an intermediate representation for optimized execution. The runtime applies graph transformations, operator fusions, and hardware-specific optimizations to improve performance. Common deployment scenarios include inference serving in production environments and distributed training on GPU clusters.
Multi-framework model support
Accepts trained models from PyTorch, TensorFlow, scikit-learn, LightGBM, XGBoost, and other libraries through the ONNX format, allowing a single runtime to handle models from different ecosystems.
Hardware abstraction layer
Provides consistent inference and training APIs across CPUs, GPUs, and specialized accelerators while handling platform-specific optimizations internally, reducing the need for framework-specific deployment code.
Graph-level optimization
Performs operator fusion, constant folding, and other graph transformations at the ONNX level before execution, independent of the original training framework used to create the model.
import onnxruntime as ort
import numpy as np
session = ort.InferenceSession("model.onnx")
input_name = session.get_inputs()[0].name
input_data = np.random.randn(1, 3, 224, 224).astype(np.float32)
outputs = session.run(None, {input_name: input_data})
print(outputs[0])Top in AI & ML
Related Repositories
Discover similar tools and frameworks used by developers
DALL-E
Official PyTorch package implementing the discrete VAE component for image tokenization used in OpenAI's DALL-E system.
X Recommendation Algorithm
Open source implementation of X's recommendation algorithm for timeline and notification ranking.
Stable Diffusion WebUI
Web UI for Stable Diffusion enabling AI image generation and editing in browser.
Whisper
Speech recognition system supporting multilingual transcription, translation, and language ID.
OpenAI Python
Type-safe Python client for OpenAI's REST API.
Related Reading
Guides and comparisons from the Greptile content library
Greptile Ranks #1 on Martian's AI Code Review Benchmark
Greptile ranks #1 on Martian's independent online AI code review benchmark, leading all evaluated tools in F1 score and precision with near-leading recall.
How to ACTUALLY Think About Code Reviews: The 3-Layer Checklist
A code review checklist and best practices organized around three layers — Mechanical, Structural, and Narrative — that help any team improve code quality without slowing reviews down.
What is agentic coding? Benefits, risks, and best tools to consider
What engineering leaders need to know about agentic coding: how it differs from AI coding and vibe coding, how it works, the benefits and risks, and the tools worth evaluating.
Choosing the Right Code Review Tool: Better Alternatives to Graphite AI
A breakdown of the best alternatives to Graphite for code reviews. Compare AI tools like Trag, open-source solutions like Gerrit, and database-focused platforms like Visual Expert.