Ruby bindings to the gigatoken core crate: BPE and SentencePiece tokenization at GB/s.

Required Ruby Version

>= 3.3.0

Authors

Eric Jacobs

Versions

  1. 0.5.0

    1. source

      1. 0.5.0 source October 07, 2026 (14 MB)

    2. aarch64-linux

      1. 0.5.0 multi-abi October 07, 2026 (23.2 MB)

    3. arm64-darwin

      1. 0.5.0 multi-abi October 07, 2026 (22.4 MB)

    4. x86_64-linux

      1. 0.5.0 multi-abi October 07, 2026 (25.6 MB)
  2. 0.4.0

    1. source

      1. 0.4.0 source September 19, 2026 (3.04 MB)

    2. aarch64-linux

      1. 0.4.0 multi-abi September 19, 2026 (12.2 MB)

    3. arm64-darwin

      1. 0.4.0 multi-abi September 19, 2026 (11.5 MB)

    4. x86_64-linux

      1. 0.4.0 multi-abi September 19, 2026 (14.6 MB)
  3. 0.3.0

    1. source

      1. 0.3.0 source September 19, 2026 (3.02 MB)

    2. aarch64-linux

      1. 0.3.0 multi-abi September 19, 2026 (12.2 MB)

    3. arm64-darwin

      1. 0.3.0 multi-abi September 19, 2026 (11.4 MB)

    4. x86_64-linux

      1. 0.3.0 multi-abi September 19, 2026 (14.6 MB)
  4. 0.2.2

    1. source

      1. 0.2.2 source September 08, 2026 (3.02 MB)

    2. aarch64-linux

      1. 0.2.2 multi-abi September 08, 2026 (12.2 MB)

    3. arm64-darwin

      1. 0.2.2 multi-abi September 08, 2026 (11.4 MB)

    4. x86_64-linux

      1. 0.2.2 multi-abi September 08, 2026 (14.6 MB)
  5. 0.2.1

    1. source

      1. 0.2.1 source August 11, 2026 (3.02 MB)

    2. aarch64-linux

      1. 0.2.1 multi-abi August 11, 2026 (12.3 MB)

    3. arm64-darwin

      1. 0.2.1 multi-abi August 11, 2026 (11.5 MB)

    4. x86_64-linux

      1. 0.2.1 multi-abi August 11, 2026 (14.6 MB)
Show all versions (32 total)

Pushed by

GitHub

SHA 256 checksum

← Previous version