Easy to Use

Built for Node.js and easily accessible via npm. Support Typescript.

Open Source Language Model

Load large language model LLaMA, RWKV and LLaMA's derived models.

Cross platforms

Supports Windows, Linux, and macOS. Allow full accelerations on CPU inference (SIMD powered by llama.cpp/llm-rs/rwkv.cpp).