-- Two NIST System descriptions concatented below -- ;; 2009 NIST Open Machine Translation (MT09) ;; System Description ;; ;; Participants are to, as accurately as possible, fill in the sections ;; below and return to NIST by the submission deadline stated in the ;; evaluation plan. ;; ;; System descriptions must be in one of the following formats: ;; * ASCII TEXT ;; * Microsoft Word ;; * PDF ;; ;; System descriptions must be submitted separately for each source ;; language and follow this naming convention: ;; __sysdesc. ;; where SITE identifies your affiliation and language identifies the ;; language pair processed, e.g. NIST_a2e_sysdesc.txt. ;; ------------------------------------------------------------------------- NIST OPEN MACHINE TRANSLATION 2009 EVALUATION PARTICIPANT SYSTEM DESCRIPTION ------------------------------------------------------------------------- 1.0 SITE AFFILIATION National Research Council Canada (NRC) 2.0 CONTACT INFORMATION George Foster george.foster@nrc.gc.ca Roland Kuhn roland.kuhn@nrc.gc.ca 3.0 SUBMISSIONS: NRC_c2e_cn_primary 4.0 PRIMARY SYSTEM SPECS: 4.1 SINGLE SYSTEM VS. SYSTEM COMBINATION Single System. 4.2 CORE MT ENGINE ALGORITHMIC APPROACH The NRC system uses a standard two-pass phrase-based approach. Major features in the first-pass loglinear model include phrase tables derived from symmetrized IBM2 alignments and symmetrized word-based HMM alignments (He 2007), a static 6-gram LM trained on the Gigaword corpus, an adapted 5-gram LM derived from the parallel corpus, and a lexicalized distortion model. Decoding uses the cube-pruning algorithm of (Huang and Chiang, 2007) with an 8-word distortion limit. Loglinear parameter tuning is performed using Och's max-BLEU algorithm over lattices (Macherey et al, 2008). The rescoring pass uses 1000-best lists, with additional features including various LM and IBM-model probabilities; ngram, length, and reordering posterior probabilities and frequencies; and quote and parenthesis mismatch indicators. 4.3 CRITICAL ADDITIONAL FEATURES AND TOOLS USED Domain-specific lexical probability estimates. Phrase pairs are extracted from HMM and IBM2 symmetrized alignments over all available training material using standard algorithms. However, "lexical" estimates for conditional phrase probabilties are assigned using the method of (Zens and Ney 2004) from the ttable of an HMM model trained on only a subset of the parallel corpus that excludes the UN, Hong-Kong Laws and Hong-Kong Hansard material. Dynamic LMs with discriminative linear weight assignment. Adapted LMs are linear mixtures of ngram models trained on parallel sub-corpora (FBIS, UN, etc), with weights set to minimize perplexity of the current source text as described in (Foster and Kuhn, 2007). The sub-corpora are augmented with a set of closest-match sentences for the current source text retrieved by the Lemur toolkit. To avoid a strong bias in favour of the closest-match corpus under perplexity weighting, the weight on this corpus is tuned discriminatively on a development set. MAP smoothing for lexicalized distortion counts. Distortion probabilities are conditioned on phrase pairs in a model similar to that used by Moses. A MAP-based backoff smoothing scheme is used to combat data sparseness when estimating probabilities. Feature selection for the rescoring pass. To improve the quality of the maxima found by MERT when using large sets of partially-overlapping rescoring features, we use greedy feature selection, first expanding from a baseline set, then pruning. 4.4 SIGNIFICANT DATA PRE/POST-PROCESSING Preprocessing uses a modified version of the LDC segmenter, along with regular-expression rules for number and date translation. For training, all English text is mapped to lowercase. Truecasing uses an HMM approach, with lexical probabilities derived from the parallel corpus and transition probabilities from a 4gram language model trained on the truecase Gigaword corpus. A subsequent rule-based pass capitalizes sentence-initial words, and content words in document titles. A final detokenization step aims to make output text consistent with the tokenization performed by the official scoring tool. 4.5 OTHER DATA USED (OUTSIDE THE PRESCRIBED LDC TRAINING DATA) None 5.0 KEY DIFFERENCE IN CONTRASTIVE SYSTEMS 6.0 REFERENCES (if applicable): George Foster and Roland Kuhn. Mixture-Model Adaptation for SMT. WMT 2007. Xiaodong He. Using Word-Dependent Transition Models in HMM based Word Alignment for Statistical Machine Translation. WMT 2007. Liang Huang and David Chiang. Forest Rescoring: Faster Decoding with Integrated Language Models. ACL 2007. Wolfgang Macherey, Franz Josef Och, Ignacio Thayer, and Jakob Uszkoreit. Lattice-based Minimum Error Rate Training for Statistical Machine Translation. EMNLP 2008. Richard Zens and Hermann Ney. Improvements in Phrase-Based Statistical Machine Translation. NAACL 2004. ;; 2009 NIST Open Machine Translation (MT09) ;; System Description ;; ;; Participants are to, as accurately as possible, fill in the sections ;; below and return to NIST by the submission deadline stated in the ;; evaluation plan. ;; ;; System descriptions must be in one of the following formats: ;; * ASCII TEXT ;; * Microsoft Word ;; * PDF ;; ;; System descriptions must be submitted separately for each source ;; language and follow this naming convention: ;; __sysdesc. ;; where SITE identifies your affiliation and language identifies the ;; language pair processed, e.g. NIST_a2e_sysdesc.txt. ;; ------------------------------------------------------------------------- NIST OPEN MACHINE TRANSLATION 2009 EVALUATION PARTICIPANT SYSTEM DESCRIPTION ------------------------------------------------------------------------- 1.0 SITE AFFILIATION National Research Council Canada (NRC) SYSTRAN Software Inc. 2.0 CONTACT INFORMATION George Foster george.foster@nrc.gc.ca Jens Stephan stephan@systransoft.com 3.0 SUBMISSIONS: SystranNRC_c2e_un_primary 4.0 PRIMARY SYSTEM SPECS: 4.1 SINGLE SYSTEM VS. SYSTEM COMBINATION Single system. 4.2 CORE MT ENGINE ALGORITHMIC APPROACH Serial combination: initial translation is performed by Systran; Systran output is mapped to final output by NRC's phrase-based system. The Systran System is based on Systran's Chinese to English rule based MT system. The source is normalized to Simplified Chinese and annotated in XML with entity and expression information from the source analysis. The translation result is also annotated with the same information. Both of these sets are passed on to the NRC system. The NRC system uses a standard two-pass phrase-based approach. Major features in the first-pass loglinear model include phrase tables derived from symmetrized IBM2 and HMM word alignments, a static 6-gram LM trained on the Gigaword corpus using the SRILM toolkit, and an adapted 5-gram LM derived from the parallel corpus. Decoding uses cube-pruning with a 7-word distortion limit. Loglinear parameter tuning is performed using Och's max-BLEU algorithm. The rescoring pass uses 1000-best lists, with additional features including various LM and IBM-model probabilities; ngram and length posterior probabilities and frequencies; and quote and parenthesis mismatch indicators. 4.3 CRITICAL ADDITIONAL FEATURES AND TOOLS USED Systran output is annotated with confidence scores. Regions of high confidence are preserved verbatim in NRC output. 4.4 SIGNIFICANT DATA PRE/POST-PROCESSING Data is mapped to lowercase during training, then truecased after translation. Truecasing uses an HMM approach, with lexical probabilities derived from the parallel corpus and transition probabilities from a 4gram language model trained on the truecase Gigaword corpus. A subsequent rule-based pass capitalizes sentence-initial words, and content words in document titles. A final detokenization step aims to make output text consistent with the tokenization performed by the official scoring tool. 4.5 OTHER DATA USED (OUTSIDE THE PRESCRIBED LDC TRAINING DATA) Systran's dictionary has been used as part of the rule-based translation step. The version used in this experiment exceeded 1 million entries. 5.0 KEY DIFFERENCE IN CONTRASTIVE SYSTEMS 6.0 REFERENCES (if applicable):