A tiny yet powerful LLM inference system tailored for researching purpose. vLLM-equivalent performance with only 2k lines of code (2% of vLLM).

Course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

C++ implementation of Qwen-LM

A Datacenter Scale Distributed Inference Serving Framework