tiktoken_core
An experimental port of OpenAI's Tokenizer to lua
$ luarocks install tiktoken_core The Lua module written in Rust that provides Tiktoken support for Lua.
Versions
Dependencies
lua >= 5.1
An experimental port of OpenAI's Tokenizer to lua
$ luarocks install tiktoken_core The Lua module written in Rust that provides Tiktoken support for Lua.