Conference Program

We propose a novel source-side dependency tree reordering model for statistical machine translation, in which subtree movements and constraints are represented as reordering events associated with the widely used lexicalized reordering models. This model allows us to efficiently capture the subtree-to-subtree transitions observed not only in the source of word-aligned training data but also at decoding time. Using subtree movements and constraints as features in a log-linear model, we are able to help the reordering models make better selection. It also allows the subtle importance of monolingual syntactic movements to be learned alongside other reordering features. We show improvements in translation quality on English-Spanish and English-Iraqi translation tasks.

The EDEAL project seeks to identify, collect, evaluate, and enhance resources relevant to processing collected material in African languages. Its priority languages are Swahili, Hausa, Oromo, and Yoruba. Resources of interest include software for OCR, Machine Translation (MT), and Named Entity Extraction (NEE), as well as data resources for developing and evaluating tools for these languages, and approaches-whether automated or manual-for developing capabilities for languages that lack significant data resources and reference material. We have surveyed the available resources, finding useful existing solutions for OCR on Swahili, Hausa, and Oromo, and for MT on Swahili and Hausa. The project is now in its first execution phase, focused on providing end-to-end capabilities and solid data coverage for a single language; we have chosen Swahili since it has the best existing coverage to build on. In future phases, we will address the additional languages. We will describe our findings on tools, data, and approaches, and we will present our present work, including the development of a significant data collection to support OCR, NEE, and MT, which will be freely available to the U.S. Government community.

In this paper we present an extension of a phrase-based decoder that dynamically chunks, reorders, and applies phrase translations in tandem. A maximum entropy classifier is trained based on the word alignments to find the best positions to chunk the source sentence. No language specific or syntactic information is used to build the chunking classifier. Words inside the chunks are moved together to enable the decoder to make long-distance reorderings to capture the word order differences between languages with different sentence structures. To keep the search space manageable, phrases inside the chunks are monotonically translated.

In this paper, we present a reordering model based on Maximum Entropy. This model is extended from a hierarchical reordering model with PBSMT (Galley and Manning, 2008), which integrate syntactic information directly in decoder as features of MaxEnt model. The advantages of this model are (1) maintaining the strength of phrase based approach with a hierarchical reordering model, (2) many kinds of linguistic information integrated in PBSMT as arbitrary features of MaxEntropy model. The experiment results with English-Vietnamese pair showed that our approach achieves improvements over the system which use a lexical hierarchical reordering model (Galley and Manning, 2008).

The Government of India established the National Translation Mission (NTM) (http://www.ntm.org.in/ntm) to address the formidable challenges of multilingual information sharing in India. The primary goal of NTM is .to make knowledge-based texts accessible in all Indian languages through translation. The idea of NTM stemmed from a statement of the Prime Minister of India stressing how vital is the access to translated material, for increasing access to knowledge in many critical areas.. In this presentation, I present a framework aimed at integrating MT in the human translation process for enhancing translation throughput, bootstrapping MT capability, translators. training, tool deployment, employment generation, and finally providing access to knowledge in native language(s) to socially deprived masses.

This presentation focuses on emerging language ecosystems with new collaborative approaches and business models based on cloud computing technology platforms, smart marketplaces with aggregation, recombination and hyperdistribution. We talk about ecosystems because this term describes adaequately the evolutionary character of the emerging online landscape with various niche markets and their demands and requirements besides the industrial main stream developments.

Our discussion also includes succint case studies on how a big German automotive manufacturer and an SME in the field of serious gaming are employing such services through a flexible Translation Management Framework within a three-dimensional translingual webservice-based information space. In this scenario, the first dimension represents the entire workflow. The second dimension depicts the different information sources. The third dimension reflects the management of the information's life cycle. The framework's components and modules work together in an adaptable and emergent fashion thus allowing for additional savings, revenues and even new market gains.

Evidence is presented to support the hypothesis that variation and inflection in Arabic has a negative impact on scores from automated measures of speech translation (e.g., WER, BLEU). Normalization operations improve correlation between BLEU scores and Likert-type judgments of semantic adequacy -- as well as between BLEU scores and human judgments of successful transfer of the meaning of individual content words from English to Arabic.

We presents an introduction to the CWMT2008 evaluation and focus on its two new metrics: BLEU-SBP (Chiang et al., 2008) and linguistic check-point method (Zhou et al., 2008). Our experiments validated BLEU-SBP's effectivity in resolving the nondecomposability problem of both NIST-BLEU and IBM-BLEU at sentence level. Our evaluation indicates linguistic check-point method is a valid metric to evaluate the capability of an MT system in translating various linguistic phenomena. With the aid of these metrics, we disclosed some performance differences between statistical MT systems and rule-based MT systems. We suggests the high BLEU score doesn't necessarily mean high translation adequacy.

The reader of a machine-translated text is actually forming hypotheses about the original text, based on multiple sources of evidence, of which the actual machine translation output is only one. Among other things, these sources of evidence include the original text, the reader's knowledge of the language of the original text, the reader's knowledge about the language and culture of the original text, the reader's understanding of how machine translation works, the reader's recourse to dictionaries and other linguistic resources, and the reader's world and domain knowledge. We will illustrate the influence of these sources of evidence in the reading of some machine-translated examples from Chinese web pages.

A method for evaluating MT performance embedded in Cross-Language Instant Messaging (CLIM) systems is presented. A web interface that provided concurrent real-time translation for instant messaging from multiple MT services was developed and used by paid participants to collaborate on a photo identification task. The method showed a task performance benefit due to the availability of multiple translation alternatives. The method also provides a new evaluation metric for MT systems based on user's task motivated choices. This method was used to compare two English-Japanese online translation systems, one from Google, and one from Excite/Japan.

The Japan Patent Office (JPO) has two types of systems in which machine translations are utilized: 1) IPDL (Industrial Property Digital Library) which refers to a system that offers IP information in English to the public users all over the world free of charge; 2) AIPN (Advanced Industrial Property Network) which refers to a system that provides examination-related information in Japan, e.g., legal status information, cited document information, etc. to the examiners of foreign IP offices in order to contribute to work-sharing among IPOs.

The presentation will consist of the following subjects: 1) an explanation on how the machine translations are utilized in the systems; 2) an introduction to the JPO.s current status and future plan, as well as to the JPO.s approach to improve the accuracy of machine translation.

The Director of the Foreign Language Program Office, Dan Scott, will present his viewpoints on users, machine translation needs, and measurement of success. He will present a range of user scenarios which present challenges to the MT community to create valid and replicable measures of success. These metrics must be meaningful within the context of various workflows and use cases. If there is no single "MT solution" nor a single metric, then the riddle for the MT community to answer is: For users, what exactly is the meaning of 94% success?

This talk will discuss machine translation evaluation in the context of multilingual speech and text processing applications, including the following highlights: (1) creation of new machine translation evaluation methods linked with standard Defense Language Institute measures of human foreign language skills, to measure the utility of machine translation for government applications; (2) development of a testbed for machine translation algorithms, with a special focus on less commonly taught languages. The key idea is to use human translation practices as the control condition for a baseline, and to contrast test scores on that condition with conditions involving human interaction with machine translation systems. Higher standardized test scores means better translation.

Hybrid Spoken Language Translation Using Sentence Splitting Based on Syntax Structure|Satoshi Kamatani, Tetsuro Chino and Kazuo Sumita|In this paper, we propose a hybrid spoken language translation method utilizing sentence segmentation. By portioning the sentence using the result of syntax analysis, we can utilize rule-based control of the integration of subtranslations translated by a suitable method for each segment. We also report a preliminary experiment on translation quality of our prototype Japanese-to-English translation system. We confirmed that our method achieved a 13.4% advantage in NIST score for the individual RBMT method, and a 6.0% advantage for the individual EBMT method.

Statistical MT is limited by reliance on large parallel corpora. We propose Lemmatic MT, a new paradigm that extends MT to a far broader set of languages, but requires manual encoding effort. The author encodes each sentence as a sequence of words drawn from a translation dictionary. We report on an experimental investigation of LEMUEL, a prototype Lemmatic MT system that outperforms Google Translate and also has high translation adequacy on language pairs not handled by Google Translate.

Since 2005, the US Army and the Joint Staff have validated requirements for machine foreign language translation capability. In the effort to develop a comprehensive machine foreign translation capability, the Army not only needs to enable software to handle one of the most complex systems that humans deal with, but also the architecture and processes to routinely produce and maintain this capability. The Army has made the initial effort, funding a machine foreign language translation program known as the Sequoyah Foreign Language Translation Program. It is intended to be the overarching Army Program with Department of Defense interest to provide machine foreign language translation capabilities that meet language translation gaps.

We introduce the first machine translation system for Yiddish-English and English-Yiddish. We discuss challenges presented by this language and their solutions, including an algorithm for cognate extraction.

We built 462 machine translation systems for all language pairs of the Acquis Communautaire corpus. We report and analyse the performance of these system, and compare them against pivot translation and a number of system combination methods (multi-pivot, multi-source) that are possible due to the available systems.

We investigate novel types of assistance for human translators, based on statistical machine translation methods. We developed a tool that makes suggestions for sentence completion, shows word and phrase translation options, and allows post-editing of machine translation output. A user study validates the types of assistance and provides insight into the human translation process.

As suited as Statistical Machine Translation is to large volume translation of news corpora, manuals, and other texts for which there are large parallel corpora from which to train, optimizing the use and value of MT within the Intelligence Community (IC) remains a significant challenge. Customizations methods, in general, include the development of custom dictionaries or incremental re-training of a Translation Model. While suitable source texts and their existing translations may be available to government organizations for MT training and customization, alignment of these text pairs at the sentence level is the crucial first step toward producing usable training data. Extracted terms may also used to develop dictionary resources. But what is the relative cost and value of customization? How do we develop and sustain a customization program given the ebb and flow of interest across languages? How do we manage customization resources? We present these issues and propose a general roadmap for addressing them.

A method is presented which integrates target language automatic speech recognition (ASR) models with source language statistical machine translation (SMT) and named entity recognition (NER) information at the phonetic level. Information extracted from a source language document including translation model probabilities and translated named entities are combined with acoustic-phonetic information obtained from phone lattices produced by the ASR system. Phone-level integration allows the combined MAHT system to correctly decode words that are either not in the ASR vocabulary or would have been incorrectly decoded by the ASR system. It is shown that the combined MAHT system results in a decrease in word error rate on the dictated translations of up to 32% relative to a stand alone baseline ASR system.

Commercial off-the-shelf (COTS) machine translation engines and translation support tools such as translation memory (TM) have been developed primarily for use in translating grammatically well-formed, edited text. The real-world, foreign language (FL) document collections that our users work with consist instead of noisy and complex image files. We are currently conducting experiments that involve building and evaluating the effectiveness of different multi-component workflows for the automated processing and translation of these FL images into English text. The evaluation side of our experiments requires "ground truth" (GT) character files of the FL text and "reference translations" (RT) of the English text. Thus the purpose of the RTs is for evaluation of the individual MT engines, as well as their output at the end of different image processing workflows.

In observing our team translator translate previously-produced GT texts into RTs, we noticed the range of translation-support resources and methods that he applies to the task. Given his experience with semi-automated and fully automated MT metrics and his involvement in our design of MT user-support and evaluation tools, we are now working ---as we have in the past--- in an extreme programming paradigm to develop a software framework that (i) streamlines access to and later add-ons to different resources he already uses (ex. MT, TM, dictionaries, morphological analyzers, LM), (ii) provides persistent data objects for later re-use, and (iii) enables him to set the tools’ options via a configuration page that in turn impacts the translation canvas page. In this paper, we present results of this development paradigm with rapid turnaround in software modifications and iterative testing, where the translator serves on the software design team and as the software’s main user and tester, while others develop the code base and test datasets, and analyze experimental results.

We investigate the problem of estimating the quality of the output of machine translation systems at the sentence level when reference translations are not available. The focus is on automatically identifying a threshold to map a continuous predicted score into ``good'' / ``bad'' categories for filtering out bad-quality cases in a translation post-edition task. We use the theory of Inductive Confidence Machines (ICM) to identify this threshold according to a confidence level that is expected for a given task. Experiments show that this approach gives improved estimates when compared to those based on classification or regression algorithms without ICM.