Skip to content

Efficient, Flexible and Portable Structured Generation

License

Notifications You must be signed in to change notification settings

zanderjiang/xgrammar

This branch is 5 commits ahead of, 9 commits behind mlc-ai/xgrammar:main.

Folders and files

NameName
Last commit message
Last commit date

Latest commit

c978553 · Mar 17, 2025
Mar 17, 2025
Jan 30, 2025
Jan 30, 2025
Mar 5, 2025
Feb 25, 2025
Mar 13, 2025
Mar 5, 2025
Mar 7, 2025
Feb 25, 2025
Dec 5, 2024
Mar 5, 2025
Mar 5, 2025
Jul 3, 2024
Aug 5, 2024
Oct 24, 2024
Jan 30, 2025
Feb 13, 2025
Dec 5, 2024
Feb 25, 2025
Dec 5, 2024
Nov 20, 2024
Mar 5, 2025
Feb 25, 2025

Repository files navigation

XGrammar

Documentation License PyPI

Efficient, Flexible and Portable Structured Generation

Get Started | Documentation | Blogpost | Technical Report

News

  • [2025/02] XGrammar has been officially integrated into Modular's MAX
  • [2025/01] XGrammar has been officially integrated into TensorRT-LLM.
  • [2024/12] XGrammar has been officially integrated into vLLM.
  • [2024/12] We presented research talks on XGrammar at CMU Catalyst, Berkeley SkyLab, MIT HANLAB, THU IIIS, SJTU, Ant Group, SGLang Meetup, Qingke AI, Camel AI. The slides can be found here.
  • [2024/11] XGrammar has been officially integrated into SGLang.
  • [2024/11] XGrammar has been officially integrated into MLC-LLM.
  • [2024/11] We officially released XGrammar v0.1.0!

Overview

XGrammar is an open-source library for efficient, flexible, and portable structured generation. It supports general context-free grammar to enable a broad range of structures while bringing careful system optimizations to enable fast executions. XGrammar features a minimal and portable C++ backend that can be easily integrated into multiple environments and frameworks, and is co-designed with the LLM inference engine and enables zero-overhead structured generation in LLM inference.

Get Started

Please visit our documentation to get started with XGrammar.

About

Efficient, Flexible and Portable Structured Generation

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C++ 54.7%
  • Python 33.4%
  • TypeScript 7.2%
  • Cuda 1.4%
  • Shell 1.2%
  • SCSS 0.8%
  • Other 1.3%