usls

usls is a cross-platform Rust library powered by ONNX Runtime for efficient inference of SOTA vision and multi-modal models(typically under 1B parameters).

📚 Documentation

• API Documentation
• Examples

🚀 Quick Start

# CPU
cargo run -r --example yolo -- --task detect --ver 8 --scale n --dtype fp16  # q8, q4, q4f16

# NVIDIA CUDA
cargo run -r -F cuda --example yolo -- --device cuda:0   # YOLOv8-n detect by default

# NVIDIA TensorRT
cargo run -r -F tensorrt --example yolo -- --device tensorrt:0

# Apple Silicon CoreML
cargo run -r -F coreml --example yolo -- --device coreml

# Intel OpenVINO
cargo run -r -F openvino -F ort-load-dynamic --example yolo -- --device openvino:CPU

# And other EPs...

⚙️ Installation

Add the following to your Cargo.toml:

[dependencies]
# Recommended: Use GitHub version
usls = { git = "https://github.com/jamjamjon/usls", features = [ "cuda" ] }

# Alternative: Use crates.io version
usls = "latest-version"

⚡ Supported Models

Click to expand

Model	Task / Description	Example
BEiT	Image Classification	demo
ConvNeXt	Image Classification	demo
FastViT	Image Classification	demo
MobileOne	Image Classification	demo
DeiT	Image Classification	demo
DINOv2	Vision Embedding	demo
DINOv3	Vision Embedding	demo
YOLOv5	Image Classification Object Detection Instance Segmentation	demo
YOLOv6	Object Detection	demo
YOLOv7	Object Detection	demo
YOLOv8 YOLO11	Object Detection Instance Segmentation Image Classification Oriented Object Detection Keypoint Detection	demo
YOLOv9	Object Detection	demo
YOLOv10	Object Detection	demo
YOLOv12	Image Classification Object Detection Instance Segmentation	demo
YOLOv13	Object Detection	demo
RT-DETR	Object Detection	demo
RF-DETR	Object Detection	demo
PP-PicoDet	Object Detection	demo
DocLayout-YOLO	Object Detection	demo
D-FINE	Object Detection	demo
DEIM	Object Detection	demo
DEIMv2	Object Detection	demo
RTMPose	Keypoint Detection	demo
DWPose	Keypoint Detection	demo
RTMW	Keypoint Detection	demo
RTMO	Keypoint Detection	demo
SAM	Segment Anything	demo
SAM2	Segment Anything	demo
MobileSAM	Segment Anything	demo
EdgeSAM	Segment Anything	demo
SAM-HQ	Segment Anything	demo
FastSAM	Instance Segmentation	demo
YOLO-World	Open-Set Detection With Language	demo
YOLOE	Open-Set Detection And Segmentation	demo-prompt-free demo-prompt(visual & textual)
GroundingDINO	Open-Set Detection With Language	demo
CLIP	Vision-Language Embedding	demo
jina-clip-v1	Vision-Language Embedding	demo
jina-clip-v2	Vision-Language Embedding	demo
mobileclip & mobileclip2	Vision-Language Embedding	demo
BLIP	Image Captioning	demo
DB(PaddleOCR-Det)	Text Detection	demo
FAST	Text Detection	demo
LinkNet	Text Detection	demo
SVTR(PaddleOCR-Rec)	Text Recognition	demo
SLANet	Tabel Recognition	demo
TrOCR	Text Recognition	demo
YOLOPv2	Panoptic Driving Perception	demo
DepthAnything v1 DepthAnything v2	Monocular Depth Estimation	demo
DepthPro	Monocular Depth Estimation	demo
MODNet	Image Matting	demo
Sapiens	Foundation for Human Vision Models	demo
Florence2	a Variety of Vision Tasks	demo
Moondream2	Open-Set Object Detection Open-Set Keypoints Detection Image Caption Visual Question Answering	demo
OWLv2	Open-Set Object Detection	demo
SmolVLM(256M, 500M)	Visual Question Answering	demo
FastVLM(0.5B)	Vision Language Models	demo
RMBG(1.4, 2.0)	Image Segmentation Background Removal	demo
BEN2	Image Segmentation Background Removal	demo
MediaPipe: Selfie-segmentation	Image Segmentation	demo
Swin2SR	Image Super-Resolution and Restoration	demo
APISR	Real-World Anime Super-Resolution	demo

📦 Cargo Features

• ort-download-binaries (default): Automatically downloads prebuilt ONNXRuntime binaries for supported platforms
• ort-load-dynamic: Dynamic linking to ONNXRuntime libraries (Guide)
• video: Enable video stream reading and writing (via video-rs)
• viewer: Enable image and video stream visualization (via minifb)
• cuda: NVIDIA CUDA GPU acceleration support
• tensorrt: NVIDIA TensorRT optimization for inference acceleration
• coreml: Apple CoreML acceleration for macOS/iOS devices
• openvino: Intel OpenVINO toolkit for CPU/GPU/VPU acceleration
• onednn: Intel oneDNN (formerly MKL-DNN) for CPU optimization
• directml: Microsoft DirectML for Windows GPU acceleration
• xnnpack: Google XNNPACK for mobile and edge device optimization
• rocm: AMD ROCm platform for GPU acceleration
• cann: Huawei CANN (Compute Architecture for Neural Networks) support
• rknpu: Rockchip NPU acceleration
• acl: Arm Compute Library for Arm processors
• nnapi: Android Neural Networks API support
• armnn: Arm NN inference engine
• tvm: Apache TVM tensor compiler stack
• qnn: Qualcomm Neural Network SDK
• migraphx: AMD MIGraphX for GPU acceleration
• vitis: Xilinx Vitis AI for FPGA acceleration
• azure: Azure Machine Learning integration

❓ FAQ

See issues or open a new discussion.

🤝 Contributing

Contributions are welcome! If you have suggestions, bug reports, or want to add new features or models, feel free to open an issue or submit a pull request.

📜 License

This project is licensed under LICENSE.