RubyGems Navigation menu

tokenizer 0.3.0

A simple multilingual tokenizer for NLP tasks. This tool provides a CLI and a library for linguistic tokenization which is an anavoidable step for many HLT (human language technology) tasks in the preprocessing phase for further syntactic, semantic and other higher level processing goals. Use it for tokenization of German, English and French texts.

版本列表:

  1. 0.3.0 - January 20, 2016 (10.0 KB)
  2. 0.2.0 - January 11, 2016 (10.0 KB)
  3. 0.1.2 - September 03, 2015 (9.5 KB)
  4. 0.1.1 - August 25, 2011 (8.0 KB)
  5. 0.1.0 - May 19, 2011 (5.5 KB)
显示所有 (6 个版本)

所有者:

作者:

  • Andrei Beliankou

SHA 256 checksum:

f3fea578e7a2f9a9ae22de432ea1254e22ed1fc2e58e0ee670a0273582f51520

下载总次数 105,197

这个版本 16,369

Gemfile:
= 复制到剪贴板 已复制

安装:
=

许可:

MIT

需要的 Ruby 版本: >= 1.9.3

相关链接: