Evaluating the Effectiveness of Machine Translation Tools in Translating China-specific Discourse

Abstract

The accuracy of China-specific discourse translation has emerged as a critical factor in conveying China’s voice and shaping its global image with China’s comprehensive national strength steadily rising and international exchanges becoming increasingly frequent. This study conducts a comparative analysis between the official English translations of ten representative Chinese sentences from a selected China-specific discourse and the outputs of four mainstream international and domestic machine translation tools: Google Translate, ChatGPT, Wenxin Yiyan (Ernie Bot), and DeepSeek. The objective is to assess the effectiveness of these machine translation tools in the context of China-specific document translation. The evaluation combines automatic metrics—BLEU and TER—to quantify translation quality, and is further supplemented by human judgment to offer a more comprehensive understanding of each system’s performance. The results indicate that ChatGPT is the most robust overall, excelling in both machine and human metrics. ERNIE Bot shows strong potential in human-preferred translation of idiomatic and China-specific language, despite poor automatic scores. Finally, this study explores future trends in translation development in the era of artificial intelligence.