203 lines
6.1 KiB
Groff
203 lines
6.1 KiB
Groff
|
.\" $Id: segment-nbest.1,v 1.10 2019/09/09 22:35:37 stolcke Exp $
|
||
|
|
.TH segment-nbest 1 "$Date: 2019/09/09 22:35:37 $" "SRILM Tools"
|
||
|
|
.SH NAME
|
||
|
|
segment-nbest \- rescore and segment N-best lists using hidden segment N-gram model
|
||
|
|
.SH SYNOPSIS
|
||
|
|
.nf
|
||
|
|
\fsegment-nbest\fP [ \fB\-help\fP ] \fIoption\fP ... \fInbest-file-list\fP ...
|
||
|
|
.fi
|
||
|
|
.SH DESCRIPTION
|
||
|
|
.B segment-nbest
|
||
|
|
processes a series of consecutive N-best lists from a speech
|
||
|
|
recognizer
|
||
|
|
and applies a hidden segment N-gram language model to them.
|
||
|
|
The language model is a standard backoff N-gram model in ARPA
|
||
|
|
.BR ngram-format (5)
|
||
|
|
modeling sentence segmentation using the boundary tags <s> and </s>.
|
||
|
|
The program reads in all N-best lists and outputs the
|
||
|
|
hypotheses that have the highest aggregate (combined acoustic
|
||
|
|
and language model) score.
|
||
|
|
Hypothesized sentence boundaries are marked by <s> tags.
|
||
|
|
.SH OPTIONS
|
||
|
|
.PP
|
||
|
|
Each filename argument can be an ASCII file, or a
|
||
|
|
compressed file (name ending in .Z or .gz), or ``-'' to indicate
|
||
|
|
stdin/stdout.
|
||
|
|
.TP
|
||
|
|
.B \-help
|
||
|
|
Print option summary.
|
||
|
|
.TP
|
||
|
|
.B \-version
|
||
|
|
Print version information.
|
||
|
|
.TP
|
||
|
|
.BI \-order " n"
|
||
|
|
Set the maximal N-gram order to be used, by default 3.
|
||
|
|
NOTE: The order of the model is not set automatically when a model
|
||
|
|
file is read, so the same file can be used at various orders.
|
||
|
|
.TP
|
||
|
|
.BI \-debug " level"
|
||
|
|
Set the debugging output level (0 means no debugging output).
|
||
|
|
Debugging messages are sent to stderr.
|
||
|
|
.TP
|
||
|
|
.BI \-lm " file"
|
||
|
|
Read the N-gram model from
|
||
|
|
.IR file .
|
||
|
|
.TP
|
||
|
|
.B \-tolower
|
||
|
|
Map all vocabulary to lowercase.
|
||
|
|
Useful if case conventions for N-best lists and language model differ.
|
||
|
|
.TP
|
||
|
|
.BI \-mix-lm " file"
|
||
|
|
Read a second, standard N-gram model for interpolation purposes.
|
||
|
|
.TP
|
||
|
|
.BI \-lambda " weight"
|
||
|
|
Set the weight of the main model when interpolating with
|
||
|
|
.BR \-mix-lm .
|
||
|
|
Default value is 0.5.
|
||
|
|
.TP
|
||
|
|
.BI \-bayes " length"
|
||
|
|
Interpolate the second and the main model using posterior probabilities
|
||
|
|
for local N-gram-contexts of length
|
||
|
|
.IR length .
|
||
|
|
The
|
||
|
|
.B \-lambda
|
||
|
|
value is used as a prior mixture weight in this case.
|
||
|
|
.TP
|
||
|
|
.BI \-bayes-scale " scale"
|
||
|
|
Set the exponential scale factor on the context likelihood in conjunction
|
||
|
|
with the
|
||
|
|
.B \-bayes
|
||
|
|
function.
|
||
|
|
Default value is 1.0.
|
||
|
|
.TP
|
||
|
|
.BI \-nbest-files " list"
|
||
|
|
Specifies a list of N-best files.
|
||
|
|
The file
|
||
|
|
.I list
|
||
|
|
should contain a list of filenames, one per line,
|
||
|
|
each corresponding to an N-best file in one of the formats
|
||
|
|
described in
|
||
|
|
.BR nbest-format (5).
|
||
|
|
The N-best files should correspond to consecutive speech waveforms
|
||
|
|
in the order listed.
|
||
|
|
.TP
|
||
|
|
.B \-fb-rescore
|
||
|
|
Perform Forward-backward rescoring.
|
||
|
|
This generates new N-best lists
|
||
|
|
as output whose LM scores reflect the posterior probability of each
|
||
|
|
hypothesis.
|
||
|
|
The default is to perform Viterbi rescoring and output only the
|
||
|
|
best combined hypothesis.
|
||
|
|
.TP
|
||
|
|
.BI \-write-nbest-dir " dir"
|
||
|
|
Write rescored N-best lists to directory
|
||
|
|
.I dir
|
||
|
|
instead of to stdout.
|
||
|
|
The filenames from the input are preserved.
|
||
|
|
.TP
|
||
|
|
.BI \-max-nbest " n"
|
||
|
|
Limits the number of hypotheses read from each N-best list to the first
|
||
|
|
.IR n .
|
||
|
|
.TP
|
||
|
|
.BI \-max-rescore " m"
|
||
|
|
Only choose among the top
|
||
|
|
.I m
|
||
|
|
hypotheses of each list (after reordering hypotheses, see below).
|
||
|
|
This is an effective way to limit the quadratic computation
|
||
|
|
of the Viterbi or forward/backward dynamic programming.
|
||
|
|
.TP
|
||
|
|
.B \-no-reorder
|
||
|
|
Do not reorder the hypotheses before limiting the computation to
|
||
|
|
the top
|
||
|
|
.IR m .
|
||
|
|
By default the hypotheses will first be sorted according to the
|
||
|
|
acoustic and language model scores recorded in the N-best lists.
|
||
|
|
.TP
|
||
|
|
.BI \-rescore-lmw " weight"
|
||
|
|
Specifies the language model weight to be use in combining
|
||
|
|
acoustic and language model scores to select the best hypotheses.
|
||
|
|
.TP
|
||
|
|
.BI \-rescore-wtw " weight"
|
||
|
|
Specifies the word transition weight to be used in selecting the
|
||
|
|
best hypotheses.
|
||
|
|
.TP
|
||
|
|
.BI \-noise " noise-tag"
|
||
|
|
Designate
|
||
|
|
.I noise-tag
|
||
|
|
as a vocabulary item that is to be ignored by the LM.
|
||
|
|
(This is typically used to identify a noise marker.)
|
||
|
|
.TP
|
||
|
|
.BI \-noise-vocab " file"
|
||
|
|
Read several noise tags from
|
||
|
|
.IR file ,
|
||
|
|
instead of, or in addition to, the single noise tag specified by
|
||
|
|
.BR \-noise .
|
||
|
|
.TP
|
||
|
|
.BI \-decipher-lm " model-file"
|
||
|
|
Designates the N-gram backoff model (typically a bigram) that was used by the
|
||
|
|
Decipher(TM) recognizer in computing composite scores.
|
||
|
|
Used to compute acoustic scores from the composite scores if the
|
||
|
|
N-best lists are in "NBestList1.0" format.
|
||
|
|
.TP
|
||
|
|
.BI \-decipher-lmw " weight"
|
||
|
|
Specifies the language model weight used by the recognizer.
|
||
|
|
Used to compute acoustic scores from the composite scores.
|
||
|
|
.TP
|
||
|
|
.BI \-decipher-wtw " weight"
|
||
|
|
Specifies the word transition weight used by the recognizer.
|
||
|
|
Used to compute acoustic scores from the composite scores.
|
||
|
|
.TP
|
||
|
|
.BI \-stag " string"
|
||
|
|
Use
|
||
|
|
.I string
|
||
|
|
to mark segment boundaries in the output.
|
||
|
|
Default is the start-of-sentence symbol defined in the language model (<s>).
|
||
|
|
.TP
|
||
|
|
.BI \-bias " b"
|
||
|
|
Make a segment boundary a priori more likely by a factor of
|
||
|
|
.IR b .
|
||
|
|
If
|
||
|
|
.I b
|
||
|
|
is 0, the dynamic program algorithm is restricted to never consider
|
||
|
|
hidden sentence boundaries; this is useful when
|
||
|
|
.B segment-nbest
|
||
|
|
is used merely for its ability to apply the LM across N-best boundaries.
|
||
|
|
.TP
|
||
|
|
.BI \-start-tag " string"
|
||
|
|
Insert a tag
|
||
|
|
.I string
|
||
|
|
at the front of every N-best hypothesis read in.
|
||
|
|
.TP
|
||
|
|
.BI \-end-tag " string"
|
||
|
|
Insert a tag
|
||
|
|
.I string
|
||
|
|
at the end of every N-best hypothesis read in.
|
||
|
|
This and the previous option are useful if the LM marks acoustic
|
||
|
|
waveform boundaries with a special tag.
|
||
|
|
.PP
|
||
|
|
.B segment-nbest
|
||
|
|
will also process any command line arguments following the options
|
||
|
|
as lists of N-best lists, as with the
|
||
|
|
.B \-nbest-files
|
||
|
|
option.
|
||
|
|
Each
|
||
|
|
.I nbest-file-list
|
||
|
|
will be processed in turn,
|
||
|
|
with individual output delimited by a line of the form
|
||
|
|
.nf
|
||
|
|
<nbestfile \fInbest-file-list\fP>
|
||
|
|
.fi
|
||
|
|
.SH "SEE ALSO"
|
||
|
|
ngram-count(1), segment(1), ngram-format(5), nbest-format(5).
|
||
|
|
.br
|
||
|
|
A. Stolcke, ``Modeling Linguistic Segment and Turn Boundaries for N-best
|
||
|
|
Rescoring of Spontaneous Speech,'' \fIProc. Eurospeech\fP, 2779\-2782, 1997.
|
||
|
|
.SH BUGS
|
||
|
|
N-gram models of arbitrary order can be used, but the context at the
|
||
|
|
beginning of a hypothesis never extends beyond the words from the preceding
|
||
|
|
N-best list.
|
||
|
|
.SH AUTHOR
|
||
|
|
Andreas Stolcke <stolcke@icsi.berkeley.edu>
|
||
|
|
.br
|
||
|
|
Copyright (c) 1997\-2004 SRI International
|