JSON for Modern C++

PyTorch Quantization Aware Training Example

CUTLASS and CuTe Examples

ONNX Runtime Inference C++ Example

ONNX Python Examples

Quick and Self-Contained TensorRT Custom Plugin Implementation and Integration

CUDA Matrix Multiplication Optimization

A General-purpose Task-parallel Programming System using Modern C++

✍🏻 这里是写博客的地方 —— Halfrost-Field 冰霜之地

Python bindings for llama.cpp

A high-throughput and memory-efficient inference and serving engine for LLMs

Accessible large language models via k-bit quantization for PyTorch.

Get down and dirty with FlashAttention2.0 in pytorch, plug in and play no complex CUDA kernels