JRRToken is a Ruby gem that wraps the tiktoken Rust library, enabling fast and efficient tokenization for OpenAI models. It supports multiple models including o200k_base, cl100k_base, p50k_base, p50k_edit, and r50k_base.

Required Ruby Version

>= 3.0.0

Required Rubygems Version

>= 3.0.0

Authors

Logan Bresnahan

Versions

  1. 1.2.1

    1. source

      1. 1.2.1 source May 13, 2026 (14.5 KB)

    2. aarch64-linux

      1. 1.2.1 multi-abi May 13, 2026 (24.2 MB)

    3. aarch64-linux-musl

      1. 1.2.1 multi-abi May 13, 2026 (24.2 MB)

    4. arm64-darwin

      1. 1.2.1 multi-abi May 13, 2026 (23.8 MB)

    5. arm-linux

      1. 1.2.1 multi-abi May 13, 2026 (23.9 MB)

    6. x86_64-darwin

      1. 1.2.1 multi-abi May 13, 2026 (23.8 MB)

    7. x86_64-linux

      1. 1.2.1 multi-abi May 13, 2026 (24.5 MB)

    8. x86_64-linux-musl

      1. 1.2.1 multi-abi May 13, 2026 (24.5 MB)
  2. 1.1.0

    1. source

      1. 1.1.0 source August 27, 2025 (13.5 KB)

    2. aarch64-linux

      1. 1.1.0 multi-abi August 27, 2025 (24.3 MB)

    3. aarch64-linux-musl

      1. 1.1.0 multi-abi August 27, 2025 (24.3 MB)

    4. arm64-darwin

      1. 1.1.0 multi-abi August 27, 2025 (23.9 MB)

    5. arm-linux

      1. 1.1.0 multi-abi August 27, 2025 (24.2 MB)

    6. x86_64-darwin

      1. 1.1.0 multi-abi August 27, 2025 (23.9 MB)

    7. x86_64-linux

      1. 1.1.0 multi-abi August 27, 2025 (24.6 MB)

    8. x86_64-linux-musl

      1. 1.1.0 multi-abi August 27, 2025 (24.6 MB)
  3. 1.0.4

    1. source

      1. 1.0.4 source August 02, 2025 (13.5 KB)

    2. aarch64-linux

      1. 1.0.4 multi-abi August 02, 2025 (24.3 MB)

    3. aarch64-linux-musl

      1. 1.0.4 multi-abi August 02, 2025 (24.3 MB)

    4. arm64-darwin

      1. 1.0.4 multi-abi August 02, 2025 (23.9 MB)

    5. arm-linux

      1. 1.0.4 multi-abi August 02, 2025 (24.2 MB)

    6. x86_64-darwin

      1. 1.0.4 multi-abi August 02, 2025 (23.9 MB)

    7. x86_64-linux

      1. 1.0.4 multi-abi August 02, 2025 (24.6 MB)

    8. x86_64-linux-musl

      1. 1.0.4 multi-abi August 02, 2025 (24.6 MB)
  4. 1.0.3

    1. aarch64-linux

      1. 1.0.3 multi-abi August 02, 2025 (24.3 MB)

    2. aarch64-linux-musl

      1. 1.0.3 multi-abi August 02, 2025 (24.3 MB)

    3. arm-linux

      1. 1.0.3 multi-abi August 02, 2025 (24.2 MB)

    4. x86_64-darwin

      1. 1.0.3 multi-abi August 02, 2025 (23.9 MB)

    5. x86_64-linux

      1. 1.0.3 multi-abi August 02, 2025 (24.6 MB)
  5. 1.0.0

    1. source

      1. 1.0.0 source August 01, 2025 (19 KB)

    2. aarch64-linux

      1. 1.0.0 multi-abi August 01, 2025 (24.3 MB)

    3. aarch64-linux-musl

      1. 1.0.0 multi-abi August 01, 2025 (24.3 MB)

    4. arm64-darwin

      1. 1.0.0 multi-abi August 01, 2025 (23.9 MB)

    5. arm-linux

      1. 1.0.0 multi-abi August 01, 2025 (24.2 MB)

    6. x86_64-darwin

      1. 1.0.0 multi-abi August 01, 2025 (23.9 MB)

    7. x86_64-linux

      1. 1.0.0 multi-abi August 01, 2025 (24.6 MB)

    8. x86_64-linux-musl

      1. 1.0.0 multi-abi August 01, 2025 (24.6 MB)
Show all versions (41 total)

SHA 256 checksum

← Previous version