Repositories list

• vllm

  Public
  A high-throughput and memory-efficient inference and serving engine for LLMs

  amdcudainference+ 17pytorchtransformerllamagptrocmmodel-servingtpu+ 10

  Python

  •

  Apache License 2.0

  •6.5k•43k•1.5k•539•Updated Mar 30, 2025Mar 30, 2025

• vllm-ascend

  Public
  Community maintained hardware plugin for vLLM on Ascend

  inferencetransformermodel-serving+ 6mlopsascendllmllmopsllm-servingvllm

  Python

  •

  Apache License 2.0

  •70•394•89•25•Updated Mar 30, 2025Mar 30, 2025

• llm-compressor

  Public
  Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

  sparsitycompressionquantization

  Python

  •

  Apache License 2.0

  •105•1.1k•30•42•Updated Mar 30, 2025Mar 30, 2025

• production-stack

  Public
  vLLM’s reference system for K8S-native cluster-wide deployment with community-driven performance optimization

  Python

  •

  Apache License 2.0

  •124•932•41•20•Updated Mar 29, 2025Mar 29, 2025

• aibrix

  Public
  Cost-efficient and pluggable Infrastructure components for GenAI inference

  Jupyter Notebook

  •

  Apache License 2.0

  •311•3.3k•147•9•Updated Mar 29, 2025Mar 29, 2025

• vllm-spyre

  Public
  Community maintained hardware plugin for vLLM on Spyre

  Python

  •

  Apache License 2.0

  •9•16•16•5•Updated Mar 28, 2025Mar 28, 2025

• buildkite-ci

  Public
  HCL

  •20•8•0•5•Updated Mar 28, 2025Mar 28, 2025

• flash-attention

  Public
  Fast and memory-efficient exact attention

  Python

  •

  BSD 3-Clause "New" or "Revised" License

  •1.6k•57•0•9•Updated Mar 27, 2025Mar 27, 2025

• vllm-project.github.io

  Public
  HTML

  •14•6•0•1•Updated Mar 22, 2025Mar 22, 2025

• vllm-openvino

  Public
  Apache License 2.0

  •0•2•1•0•Updated Mar 19, 2025Mar 19, 2025

• FlashMLA

  Public
  C++

  •

  MIT License

  •812•4•0•0•Updated Feb 25, 2025Feb 25, 2025

• vllm-project.github.io-static

  Public archive
  HTML

  •

  MIT License

  •7•8•0•1•Updated Feb 7, 2025Feb 7, 2025

• vllm_allocator_adaptor

  Public
  An adaptor to allow Python allocator for PyTorch pluggable allocator

  C++

  •

  Apache License 2.0

  •1•2•0•0•Updated Jan 5, 2025Jan 5, 2025

• media-kit

  Public
  vLLM Logo Assets

  1•2•0•0•Updated Dec 12, 2024Dec 12, 2024

• vllm-nccl

  Public archive
  Manages vllm-nccl dependency

  Python

  •

  Apache License 2.0

  •3•17•2•0•Updated Jun 3, 2024Jun 3, 2024

• dashboard

  Public
  vLLM performance dashboard

  Python

  •

  Apache License 2.0

  •6•23•0•0•Updated Apr 26, 2024Apr 26, 2024