Abstract
In emergency situations it is crucial that instructions are straightforward to understand. For this reason a controlled language for crisis management (CLCM), based on psycholinguistic studies of human comprehension under stress, was developed. In order to test the impact of CLCM machine translatability of this particular kind of sub-language text, a previous experiment involving machine translation and human post-editing has been conducted. Employing two automatic evaluation metrics, a previous evaluation of the experiment has proved that instructions written according to this CL can improve machine translation (MT) performance. This paper presents a new cognitive evaluation approach for MT post-editing, which is tested on the previous controlled and uncontrolled textual data. The presented evaluation approach allows a deeper look into the post-editing process and specifically how much effort posteditors put into correcting the different kinds of MT errors. The method is based on existing MT error classification, which is enriched with a new error ranking motivated by the cognitive effort involved in the detection and correction of these MT errors. The preliminary results of applying this approach to a subset of the original data confirmed once again the positive impact of CLCM on emergency instructions' machine translatability and thus the validity of the approach.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010 |
| Publisher | European Language Resources Association (ELRA) |
| Pages | 3485-3490 |
| Number of pages | 6 |
| ISBN (Electronic) | 2951740867, 9782951740860 |
| Publication status | Published - 1 Jan 2010 |
| Externally published | Yes |
| Event | 7th International Conference on Language Resources and Evaluation, LREC 2010 - Valletta, Malta Duration: 17 May 2010 → 23 May 2010 |
Other
| Other | 7th International Conference on Language Resources and Evaluation, LREC 2010 |
|---|---|
| Country | Malta |
| City | Valletta |
| Period | 17/5/10 → 23/5/10 |
Fingerprint
ASJC Scopus subject areas
- Education
- Library and Information Sciences
- Linguistics and Language
- Language and Linguistics
Cite this
Cognitive evaluation approach for a controlled language post-editing experiment. / Temnikova, Irina.
Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010. European Language Resources Association (ELRA), 2010. p. 3485-3490.Research output: Chapter in Book/Report/Conference proceeding › Conference contribution
}
TY - GEN
T1 - Cognitive evaluation approach for a controlled language post-editing experiment
AU - Temnikova, Irina
PY - 2010/1/1
Y1 - 2010/1/1
N2 - In emergency situations it is crucial that instructions are straightforward to understand. For this reason a controlled language for crisis management (CLCM), based on psycholinguistic studies of human comprehension under stress, was developed. In order to test the impact of CLCM machine translatability of this particular kind of sub-language text, a previous experiment involving machine translation and human post-editing has been conducted. Employing two automatic evaluation metrics, a previous evaluation of the experiment has proved that instructions written according to this CL can improve machine translation (MT) performance. This paper presents a new cognitive evaluation approach for MT post-editing, which is tested on the previous controlled and uncontrolled textual data. The presented evaluation approach allows a deeper look into the post-editing process and specifically how much effort posteditors put into correcting the different kinds of MT errors. The method is based on existing MT error classification, which is enriched with a new error ranking motivated by the cognitive effort involved in the detection and correction of these MT errors. The preliminary results of applying this approach to a subset of the original data confirmed once again the positive impact of CLCM on emergency instructions' machine translatability and thus the validity of the approach.
AB - In emergency situations it is crucial that instructions are straightforward to understand. For this reason a controlled language for crisis management (CLCM), based on psycholinguistic studies of human comprehension under stress, was developed. In order to test the impact of CLCM machine translatability of this particular kind of sub-language text, a previous experiment involving machine translation and human post-editing has been conducted. Employing two automatic evaluation metrics, a previous evaluation of the experiment has proved that instructions written according to this CL can improve machine translation (MT) performance. This paper presents a new cognitive evaluation approach for MT post-editing, which is tested on the previous controlled and uncontrolled textual data. The presented evaluation approach allows a deeper look into the post-editing process and specifically how much effort posteditors put into correcting the different kinds of MT errors. The method is based on existing MT error classification, which is enriched with a new error ranking motivated by the cognitive effort involved in the detection and correction of these MT errors. The preliminary results of applying this approach to a subset of the original data confirmed once again the positive impact of CLCM on emergency instructions' machine translatability and thus the validity of the approach.
UR - http://www.scopus.com/inward/record.url?scp=85037109770&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85037109770&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85037109770
SP - 3485
EP - 3490
BT - Proceedings of the 7th International Conference on Language Resources and Evaluation, LREC 2010
PB - European Language Resources Association (ELRA)
ER -