discourse_ai-tokenizers 0.2.0
A Ruby gem providing a consistent interface for various AI/ML tokenizers including OpenAI GPT, Anthropic Claude, Google Gemini, Meta Llama, Mistral, Qwen, and embedding models like BERT, BGE, and multilingual-E5. Features caching, truncation, token counting, and error handling across different tokenization libraries.
Gemfile:
=
安裝:
=
Runtime 相依性套件 (3):
Development 相依性套件 (2):
rubocop-discourse
= 3.8.1
syntax_tree
~> 6.2.0
擁有者:
推送者:
作者:
-
Rafael Silva
SHA 256 總和檢查碼:
=