RubyGems Navigation menu

TokenizerProjectUT 0.0.1

A simple multilingual tokenizer for NLP tasks. This tool provides a CLI and a library for linguistic tokenization which is an anavoidable step for many HLT (human language technology) tasks in the preprocessing phase for further syntactic, semantic and other higher level processing goals. Use it for tokenization of German, English and French texts.

Gemfile:
= Copy to clipboard Copied!

install:
=

Versions:

  1. 0.0.1 - November 29, 2011 (6.5 KB)

Owners:

Authors:

  • David Alfter

SHA 256 checksum:

406a66a42085fcb5a448cd202a4f6334ae5fd49758b2558c8a630558a96f5636

Total downloads 4,097

For this version 4,097

Licenses:

N/A

Required Ruby Version: None

Links: