RubyGems Navigation menu

tokenizer 0.1.2

A simple multilingual tokenizer for NLP tasks. This tool provides a CLI and a library for linguistic tokenization which is an anavoidable step for many HLT (human language technology) tasks in the preprocessing phase for further syntactic, semantic and other higher level processing goals. Use it for tokenization of German, English and French texts.

版本列表:

  1. 0.3.0 - January 20, 2016 (10.0 KB)
  2. 0.2.0 - January 11, 2016 (10.0 KB)
  3. 0.1.2 - September 03, 2015 (9.5 KB)
  4. 0.1.1 - August 25, 2011 (8.0 KB)
  5. 0.1.0 - May 19, 2011 (5.5 KB)
顯示所有版本(共 6)

擁有者:

作者:

  • Andrei Beliankou

SHA 256 checksum:

8ac5293741168112e228f6667bf3bbc132f816f42f3e3f2ccd8cad295b8b39fd

總下載次數 106,093

這個版本 5,144

Gemfile:
= 複製 已複製

安裝:
=

Licenses:

Ruby 版本需求: >= 1.9.3

相關連結: