+ make-kn-counts no_max_order=1 max_per_file=10000000 order=3 kndiscount1=1 kndiscount2=1 kndiscount3=1 kndiscount4=1 kndiscount5=1 kndiscount6=1 kndiscount7=1 kndiscount8=1 kndiscount9=1 output=biglm.kndir/kncounts
+ merge-batch-counts biglm.kndir
final counts in biglm.kndir/./kncounts-1.ngrams.gz
+ get-gt-counts out=biglm max=20 maxorder=3
+ ngram-count -read - -read-with-mincounts -order 3 -kn1 biglm.kn1 -kn2 biglm.kn2 -kn3 biglm.kn3 -debug 1 -interpolate -gt3min 2 -vocab ../ngram-count-gt/eval2001.vocab -lm swbd.3bo.gz -meta-tag __meta__ -kn-counts-modified
using ModKneserNey for 1-grams
using ModKneserNey for 2-grams
using ModKneserNey for 3-grams
discarded 1 1-gram probs predicting pseudo-events
warning: distributing 0.0130397 left-over probability mass over 6550 zeroton words
discarded 2 2-gram contexts containing pseudo-events
discarded 11 2-gram probs predicting pseudo-events
discarded 10919 3-gram contexts containing pseudo-events
discarded 55089 3-gram probs predicting pseudo-events
writing 33110 1-grams
writing 425750 2-grams
writing 268962 3-grams
using existing biglm.kncounts.gz
using existing gtcounts
+ ngram-count -read - -read-with-mincounts -order 3 -kn1 biglm.kn1 -kn2 biglm.kn2 -kn3 biglm.kn3 -debug 1 -interpolate -gt3min 2 -vocab ../ngram-count-gt/eval2001.vocab -lm swbd.3bo.gz -meta-tag __meta__ -kn-counts-modified
using ModKneserNey for 1-grams
using ModKneserNey for 2-grams
using ModKneserNey for 3-grams
discarded 1 1-gram probs predicting pseudo-events
warning: distributing 0.00673829 left-over probability mass over 6550 zeroton words
discarded 2 2-gram contexts containing pseudo-events
discarded 11 2-gram probs predicting pseudo-events
discarded 10919 3-gram contexts containing pseudo-events
discarded 55089 3-gram probs predicting pseudo-events
writing 33110 1-grams
writing 425750 2-grams
writing 268962 3-grams