RubyGems.org

gtokenizer

1.0.0

GTokenizer recreates the closed-source tokenization library used by Google for their Google NGrams app (http://ngrams.googlelabs.com/), based on the information in the associated Science paper (http://www.sciencemag.org/content/suppl/2010/12/16/science.1199644.DC1/Michel.SOM.revision.2.pdf)

installgem install gtokenizer
Authors

Alex Peattie

1,179 total downloads 1,179 for this version
Owners

716521667807b542d722f51281f97e72

Gemfile
gem 'gtokenizer', '~> 1.0.0'
Versions
  1. 1.0.0 July 1, 2011 (6.5 KB)
Development Dependencies
  1. rspec >= 0