Ruby bindings to the gigatoken core crate: BPE and SentencePiece tokenization at GB/s.
Required Ruby Version
>= 3.3.0
Authors
Eric Jacobs
Versions
- 0.1.1 July 24, 2026 (304 KB)
- 0.1.1 July 24, 2026 x86_64-linux (11.8 MB)
- 0.1.1 July 24, 2026 arm64-darwin (8.73 MB)
- 0.1.1 July 24, 2026 aarch64-linux (9.54 MB)
- 0.1.0 July 24, 2026 (304 KB)
Pushed by
