Deep Learning-Based Machine Translation Evaluation Metric
摘要
Automatic MT evaluation has been the focus of the study for quite long. To date, human evaluation is considered as the golden metric for the MT evaluation. Any automatic evaluation metric has to have a good correlation with human evaluation, only then it is considered as a reliable metric. Thus, in this paper, we proposed a reference-based MT evaluation metric METAL (MT EvaluaTion using Applied Learning) which has been trained using deep neural networks. For this we constructed a dataset of around 1.30 million source sentences, their target translations and their human references for correlation. Further, we have created a metric based on the scores using different aspects of DNN with 10 optimisers, 3 activation functions, hidden layers, 100 epochs, use of dropouts and batch normalisation. Those MT metric outputs were trained with 1 human reference translation. Experimental results demonstrate that METAL proposed metric aligns closely with human evaluation, establishing its reliability and effectiveness in real-world MT evaluation tasks.