tiktoken_core

An experimental port of OpenAI's Tokenizer to lua

$ luarocks install tiktoken_core

The Lua module written in Rust that provides Tiktoken support for Lua.

Versions

0.2.1-1102 days ago1,266 downloads
0.2.0-1106 days ago11 downloads
0.1.0-1106 days ago4 downloads

Dependencies

lua >= 5.1

Manifests