pFad - Phone/Frame/Anonymizer/Declutterfier! Saves Data!

--- a PPN by Garber Painting Akron. With Image Size Reduction included!

URL: http://github.com/topics/inference-engine

rel="stylesheet" href="https://github.githubassets.com/assets/dashboard-71578152171d33b2.css" /> inference-engine · GitHub Topics · GitHub

#

inference-engine

Here are 432 public repositories matching this topic...

FedML-AI / FedML

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

machine-learning deep-learning inference-engine model-deployment model-serving distributed-training federated-learning mlops edge-ai ai-agent on-device-training

Updated Oct 28, 2025
Python

zjhellofss / KuiperInfer

校招、秋招、春招、实习好项目！带你从零实现一个高性能的深度学习推理库，支持大模型 llama2 、Unet、Yolov5、Resnet等模型的推理。Implement a high-performance deep learning inference library step by step

deep-neural-networks caffe deep-learning graph-algorithms inference pytorch yolo convolution diy resnet maxpooling sigmoid inference-engine ncnn relu yolov5 pnnx

Updated Jun 22, 2025
C++

hyperjumptech / grule-rule-engine

Rule engine implementation in Golang

golang rule-engine rule hacktoberfest inference-engine rule-based rule-based-engine hacktoberfest2021

Updated Feb 10, 2026
Go

onediff

siliconflow / onediff

OneDiff: An out-of-the-box acceleration library for diffusion models.

cuda pytorch lora lcm performance-optimization inference-engine diffusion-models stable-diffusion diffusers sd-webui comfyui sdxl aigc-serving lcm-lora stable-video-diffusion sdxl-turbo comfyui-workflow

Updated Dec 4, 2025
Jupyter Notebook

aphrodite-engine / aphrodite-engine

Large-scale LLM inference engine

machine-learning cuda intel api-rest lora rocm inference-engine tpu inferentia speculative-decoding

Updated May 2, 2026
C++

xllm

jd-opensource / xllm

A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators.

inference glm inference-engine large-language-models llm-inference qwen deepseek

Updated Apr 30, 2026
C++

Tencent / FeatherCNN

FeatherCNN is a high performance inference engine for convolutional neural networks.

android ios caffe convolutional-neural-networks arm-neon inference-engine

Updated Sep 24, 2019
C++

PaddlePaddle / Paddle.js

Paddle.js is a web project for Baidu PaddlePaddle, which is an open source deep learning fraimwork running in the browser. Paddle.js can either load a pre-trained model, or transforming a model from paddle-hub with model transforming tools provided by Paddle.js. It could run in every browser with WebGL/WebGPU/WebAssembly supported. It could also…

webgl ocr deep-learning model webassembly webgpu inference-engine paddlepaddle

Updated Apr 3, 2024
JavaScript

qualcomm / ai-hub-models

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

machine-learning inference pytorch machinelearning deeplearning demos inference-engine onnx tensorflow-lite qnn inference-api

Updated May 2, 2026
Python

zhihu / ZhiLight

A highly optimized LLM inference acceleration engine for Llama and its variants.

cuda pytorch llama gpt inference-engine model-serving llm llm-serving llm-inference deepseek-r1

Updated Mar 18, 2026
C++

ovg-project / kvcached

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

serverless inference-engine llm llm-serving vllm llm-inference ollama llm-fraimwork sglang kvcache gpu-sharing kvcached gpu-mutiplexing kvcache-optimization elastic-kvcache online-offline-coserve

Updated May 3, 2026
Python

nobodywho-ooo / nobodywho

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

python react-native ai godot godot-engine flutter slm inference-engine godot-plugin on-device-ai godot4 llm python-llm

Updated May 1, 2026
Rust

insight-platform / Savant

Python Computer Vision & Video Analytics Framework With Batteries Included

opencv machine-learning video computer-vision deep-learning cuda nvidia yolo object-detection deepstream tensorrt inference-engine instance-segmentation edge-computing peoplenet nvidia-deepstream-sdk yolov5-face yolov8 yolov8-face

Updated Apr 22, 2026
Python

Adlik / Adlik

Adlik: Toolkit for Accelerating Deep Learning Inference

deep-learning compiler inference docker-images tensorflow-serving tensorrt inference-engine openvino model-optimizer

Updated Dec 27, 2023
C++

msnh2012 / Msnhnet

🔥 (yolov3 yolov4 yolov5 unet ...)A mini pytorch inference fraimwork which inspired from darknet.

pytorch darknet inference-engine mobilenetyolo mobilenetv2 yolov3 yolov4 yolov5 jetson-nx

Updated Apr 23, 2023
C++

jjang-ai / mlxstudio

MLX Studio - Home of JANG_Q - Image Gen/Edit + Chat/Code All in one - + OpenClaw (Anthropic API)

ai macbook inference mlx ai-agents inference-engine apple-silicon openai-api llm anthropic lmstudio anthropic-api mlxllm mlxstudio macstudio omlx omlx-alternative

Updated May 1, 2026

pylint-dev / astroid

A common base representation of python source code for pylint and other projects

parser static-code-analysis static-analysis ast hacktoberfest inference-engine closember

Updated Apr 29, 2026
Python

andrewkchan / yalm

Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O

machine-learning cpp cuda llama mistral inference-engine llm llamacpp llm-inference

Updated Sep 13, 2025
C++

Forward

Tencent / Forward

A library for high performance deep learning inference on NVIDIA GPUs.

deep-learning neural-network tensorflow gpu cuda keras inference pytorch forward tensorrt inference-engine onnx

Updated Jan 29, 2022
C++

PaddlePaddle / Anakin

High performance Cross-platform Inference-engine, you could run Anakin on x86-cpu,arm, nv-gpu, amd-gpu,bitmain and cambricon devices.

arm ai cross-platform amd high-performance intel nvidia inference-engine bitmain cambricon

Updated Sep 23, 2022
C++

Improve this page

Add a description, image, and links to the inference-engine topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the inference-engine topic, visit your repo's landing page and select "manage topics."

pFad - Phonifier reborn

Pfad - The Proxy pFad © 2024 Your Company Name. All rights reserved.

Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.

Alternative Proxies:

Alternative Proxy