JAPANEDU: Jurnal Pendidikan dan Pengajaran Bahasa Jepang Vol.
No.
June 2024, pp.
JAPANEDU:
Jurnal Pendidikan dan Pengajaran Bahasa Jepang http://ejournal.
edu/index.
php/japanedu/index A Comparative Analysis of Japanese-English Machine Translation Outputs Using Neural and Statistical Systems Google Translate vs.
Systran Muhammad Nadzif Bin Ramlan Institute of Social Sciences.
Istanbul University.
Tyrkiye ramlan@ogr.
ABSTRACT
This study explores the effectiveness and accuracy of Google Translate and Systran in translating Japanese to English, focusing on syntactic and semantic error patterns.
It evaluates the translation quality of the text uAEIEEyAUAUOoEEa AyoAaOAAo AuOngaku no tenpo ga keizaiteki ishi kettei ni oyobosu eikyouAy (The Effect of Music Tempo on Economic Decision-Makin.
by Kobayashi.
Fujikawa, and Foo .
The methodology employs a qualitative approach, categorizing errors based on syntactic and semantic criteria.
Despite advancements in Neural Machine Translation (NMT), challenges remain in achieving accurate and contextually appropriate translations, particularly for complex language pairs like Japanese-English.
The study highlights the persistent issues in maintaining syntactic and semantic accuracy in translations produced by Google Translate and Systran.
It underscores the importance of Machine Translation Post-Editing (MTPE) to enhance translation quality.
The findings reveal that while MT systems have significantly improved, human intervention remains essential to address nuanced linguistic and cultural elements.
The research emphasizes the relevance of MachineAided Human Translation (MAHT) as a balanced approach, combining the efficiency of MT with human expertise to ensure high-quality translations.
This approach is crucial for fostering better cross-cultural communication and understanding in the translation industry.
KEYWORDS
Machine translation.
Neural.
Post-editing.
Syntax.
Semantics.
First received: 13 September 2023
ARTICLE INFO
Revised: 13 June 2024
Available online: 25 June 2024
INTRODUCTION
The English word AotranslationAo emerged around 1340, derived from the Old French word AutranslationAy or the Latin AutranslationAy .
eaning transportin.
, according to Munday .
According to the Cambridge Dictionary.
Final proof accepted: 20 June 2024 translation refers to the process of converting something from one language to another while retaining the same meaning.
Combining these views, translation can be defined as the process of transferring input from the source language (SL) to the target language (TL), aiming to preserve the .
Page e- ISSN 2528-5548 | p-ISSN 27764478 Muhammad Nadzif Bin Ramlan A Comparative Analysis of Japanese-English Machine Translation Outputs Using Neural and Statistical Systems:
Google Translate vs.
Systran original meaning, form, and style of the SL as much as possible.
The translation process necessitates extensive knowledge of both SL and TL, including cultural adaptations, local idioms, and more.
However, achieving a top-notch translation product is challenging even with such knowledge.
This is due to the requirement for translators to possess practical skills and apply correct theoretical approaches as guidance to produce satisfactory One significant advancement in the translation industry that aids in meeting these demands is the application of machine translation (MT).
LITERATURE REVIEW
Historical Overview Machine translation (MT) research began in the 1950s, following significant advancements in cryptography during World War II.
The Georgetown experiment in 1954 marked a milestone, translating sixty Russian sentences into English (Hutchins, 2005.
Norwati, 2.
Initially, researchers anticipated rapid progress, expecting fully automated systems within a few years.
However, the complexity of natural language posed significant challenges, leading to a shift from rule-based models to statistical approaches in the late 1980s and 1990s (Kay, 1.
Despite early setbacks.
MT systems evolved, leveraging statistical models to process large bilingual corpora.
SMT systems, introduced by IBMAos research team, marked a significant advancement by using statistical learning methodologies (Brown et al.
, 1.
These systems relied on vast multilingual data to build models, enhancing translation quality and efficiency.
In recent years, the advent of Neural Machine Translation (NMT) has revolutionized the field.
GoogleAos introduction of NMT in 2016 replaced earlier statistical methods, offering more nuanced and accurate translations by leveraging deep learning techniques (Sutskever.
Vinyals, & Le.
Despite these advancements, challenges remain, particularly in maintaining syntactic and semantic accuracy across diverse language pairs (Yamagishi.
Kanouchi.
Sato, & Komachi, 2.
Translation Software Background The rapid advancement of technology and globalization has increased the demand for translation tools.
Various MT applications, such as Citcat.
Google Translate, and Systran, offer solutions for different language pairs.
Google Translate, supporting over 100 languages, and Systran, one of the oldest MT companies, are widely used for their accessibility and comprehensive language support.
This study evaluates their effectiveness in translating Japanese-English text, focusing on their syntactic and semantic accuracy (Li, 2015.
Zhu, 2.
Recent Advances in Japanese-English Machine Translation Recent studies have highlighted the progress and challenges in Japanese-English MT.
Yamagishi.
Kanouchi.
Sato.
Komachi .
demonstrated the effectiveness of controlling the voice in Japanese-English NMT, achieving a 0.
73point improvement in BLEU score by This advancement underscores the importance of context in translation, as tonal and formality nuances significantly affect the translated output.
Similarly.
Dabre .
explored the use of multilingualism and transfer learning to enhance translation quality for low-resource languages, showing significant improvements in translation accuracy through leveraging related languages and transfer learning techniques.
Li .
discussed the impact of globalization and information technology on translation, emphasizing the role of computer-aided translation (CAT) technologies in modern translation This study provides a comprehensive overview of CAT tools, including translation memory and terminology management, and their applications in improving translation efficiency and quality.
Evaluating the effectiveness of different MT systems.
Zhu .
conducted a detailed analysis of Japanese-English translation, comparing the performance of various NMT models.
Their findings indicated that while NMT systems offer improved fluency and coherence, challenges in maintaining syntactic and semantic accuracy persist, particularly for complex language pairs like Japanese-English.
Yanase and Lees .
categorized errors in academic essays translated from Japanese (L.
to English (L.
, highlighting the common issues faced by MT systems in maintaining grammatical accuracy and contextual relevance.
Their findings underscore the importance of context-aware translation models for improving overall translation quality.
Page e- ISSN 2528-5548 | p-ISSN 27764478 JAPANEDU: Jurnal Pendidikan dan Pengajaran Bahasa Jepang Vol.
No.
June 2024, pp.
Nakazawa .
discussed the paradigm shift brought about by NMT, comparing it with SMT and highlighting the significant improvements in translation accuracy.
NakazawaAos study also identified unique challenges posed by NMT, such as handling out-of-domain translations and ensuring complete translations without omissions.
Feely.
Hasler, and de Gispert .
focused on controlling Japanese honorifics in English-toJapanese NMT, presenting methods to adjust the level of formality in translations.
This study addresses the complexity of honorific speech and provides insights into improving the cultural and contextual appropriateness of NMT outputs.
Okita and Kurokawa .
investigated the use of MT among graduate students in Japan, emphasizing the ethical implications and the impact on academic writing.
Their study revealed a high reliance on MT tools, highlighting both the benefits and challenges of integrating MT into academic workflows.
These studies collectively highlight the significant advancements in Japanese-English MT and emphasize the importance of contextual and cultural factors in achieving high-quality The integration of voice control, honorific management, and the handling of academic writing nuances demonstrates the evolving sophistication of MT systems.
The Concept of Machine Translation Post-Editing (MTPE) Machine Translation Post-Editing (MTPE) involves human editors refining machinegenerated translations to ensure quality.
This process is critical in the localization industry, where AI and machine learning advancements necessitate continuous improvement in translation accuracy (Gouadec, 2.
Despite the potential of MT systems, human intervention remains essential to correct errors and enhance the final output.
Although machine translation systems might have improved over time, human translation will always be relevant despite concerns about the possibility of full automation in translation.
One of the reasons is that human languages carry polysemous words and are thus susceptible to multiple interpretations, often more than not depending on the interlocutors themselves.
This is especially true for literary texts such as idioms and haiku/poems as shown in Table 1 and Table 2 respectively Ae in which certain advanced translation procedures such as adaptation and transposition would be applied.
Based on Table 1 and Table 2, proves that MT is not viable for these sociocultural and literary phrases, making human translation as valid as ever.
Table 1: Machine Translation Errors for Kotowaza (Proverb.
Input
Output* Correct
Translation UAaO
Oval for gold coin to Neko ni a cat
The frog A frogAos uo
child is offspring is Kaeru no ko a frog
a frog
wa kaeru
AAUA
I canAot people, love AUCOAcCU can miss Nakute zo .
hito wa koi *MT outputs from Google Translate Malay Adaptation (Ramlan.
Bagai kera diberi kaca Bapa borek, anak rintik Jauh di mata, dekat di hati Table 2: Machine Translation Errors for Haiku (Poe.
Input MT Output* From Sakura AiAaCOCOCO Sakura yori I made it a Eas Momo ni Yasushi Koie Koie nari *MT outputs from Google Translate Phrasing (Ramlan, 2021.
comparing the cherry blossoms peach blossoms .
ould fi.
this small house I wonder
METHODOLOGY
This study employs a qualitative methodology to assess the effectiveness and accuracy of Google Translate and Systran in translating texts, specifically focusing on error patterns in their The chosen text sample was uAEIE EyAUAUOoEEaAyoAaOAAo AuOngaku no tenpo ga keizaiteki ishi kettei ni oyobosu eikyouAy (The Effect of Music Tempo on Economic DecisionMakin.
by Kobayashi.
Fujikawa, and Foo .
, translated from Japanese .
ource language.
SL) to English .
arget language.
TL).
The text was specifically used as a sample for gauging the adaptability of machine translation to its equal mix of social science and arts .
Errors .
Page e- ISSN 2528-5548 | p-ISSN 27764478 Muhammad Nadzif Bin Ramlan A Comparative Analysis of Japanese-English Machine Translation Outputs Using Neural and Statistical Systems:
Google Translate vs.
Systran were categorized based on syntactic and semantic criteria, allowing us to evaluate the quality of translations produced by these tools.
The article investigates the intriguing observation that rhythmic sound can influence behavioral economics, specifically decisionmaking.
It also considers the reactions and states of subjects as they listen to music while buying or choosing products.
The specific reason for choosing the Japanese-English pair is that, according to Norwati .
Japanese-Malay translation development is still underdeveloped compared to other language pairs like Korean and Arabic.
Our evaluation framework incorporates insights from PopoviN .
, who underscores the resource-intensive nature of manual error classification and the potential for automatic tools to supplement human evaluators by estimating error distributions and facilitating pre-annotation.
We utilized Machine Translation Post-Editing (MTPE) and detailed tables to present our findings.
The relevance of using qualitative methods for evaluating MT systems is supported by recent For instance.
PopoviN .
emphasizes the importance of qualitative analysis in understanding the nuances of MT errors.
Freitag.
Foster.
Grangier.
Ratnakar.
Tan, & Macherey .
highlight the necessity of robust evaluation methods, illustrating that inadequate procedures can lead to erroneous conclusions about the quality of MT systems.
They recommend using the Multidimensional Quality Metrics (MQM) framework for explicit error analysis, which provides a more reliable assessment of translation quality by involving professional translators and detailed error categorization.
Additionally.
Li et al.
stressed the importance of qualitative analysis in handling the noisy and informal text common on social media, further underscoring the need for thorough error categorization in MT Through this qualitative lens, our study categorizes and examines error patterns in Google Translate and Systran outputs, providing a comprehensive assessment of their translation accuracy and identifying areas for improvement in MT systems.
RESULTS AND DISCUSSION
Errors in Translation Due to Inaccurate Choice of Semantic Correspondence For this error acquired, we have found out that the translation output for this language pair has an emphasis on its output orientation.
This error may have been caused by this application that is no longer being able to identify the meaning of the word the user wants to translate.
This is because there are several words in the source language that have a lot of synonyms in the target language as shown in Table 3.
For the first one, the word C E E C E E .
directly means AucommercialAy and this is due to its Katakana orthography Ae a Japanese writing system that explicitly indicates foreign However, the output for the word from Google is also AucommercialsAy which is redundant to the following word in the input y .
should be AuadvertisementAy as in SystranAos output.
This redundancy highlights a common issue in neural machine translation systems where contextual appropriateness is not adequately addressed (Koehn & Knowles, 2.
The second one involves two parallel inputs AECEUCE .
hssaina mekanizum.
and N aO u i .
ukugs say.
which mean Aucomplex mechanismAy and Audetailed mechanismsAy for Systran and Aucompound actionAy on the Audetailed mechanismAy for Google.
As it is, both agree on the output Audetailed mechanismAy for the input A ECEUCE.
Despite that, the second input has the 1=> feature in which u i .
is also a AumechanismAy redundant to the previous word E CEUCE .
due to its similar nature as the first case - its Katakana orthography explicitly indicates foreign or loaned words and thus should be highly restrictive to the sole meaning of the original language.
This issue is also discussed in studies exploring the translation of semantically redundant phrases and the impact of languagespecific orthographic conventions (Dabre.
Chu & Kunchukuttan, 2.
The third one shows two synonyms, y .
and OIU .
By default, two different Kanji characters mean two different words.
Therefore.
SystranAos outputs of AuconditionAy and AustateAy for the respective inputs are correct, yet Google conceives both as AuconditionAy.
This difficulty in differentiating synonyms and .
Page e- ISSN 2528-5548 | p-ISSN 27764478 JAPANEDU: Jurnal Pendidikan dan Pengajaran Bahasa Jepang Vol.
No.
June 2024, pp.
producing contextually accurate translations is a known challenge for NMT systems (Toral & Way.
The fourth is related to the position of the word in the sentence according to TL.
The input ACAyA AA .
oshiteA) is actually a discourse marker in the Japanese Language and thus has two meanings AuandAy as well as AuthenAy.
But since the word is after a full stop, the output AuthenAy would be more appropriate considering English as TL, not the output AuandAy put forth by both translation The importance of context-aware translation models is critical for accurately translating discourse markers (Lyubli.
Sennrich, & Volk, 2.
The fifth one is the wrong choice of lexis for the input ECCA u .
isuku senk.
and oC neA u .
ikan senk.
Since the context of the article revolves around economic matters, the output for Au .
should be AupreferenceAy as in Systran but Google wrongly chose AuappetiteAy as its output This highlights the challenges of translating domain-specific terminology and the importance of context in choosing the correct lexical equivalents (Arcan & Buitelaar, 2.
Table 3: Errors in Translation Due to Inaccurate Choice of Semantic Correspondence.
Input SMT (Systra.
NMT (Googl.
CEECEECeAAoCay CEcea komsharu wo hajime to shita senden ya aikyattchi, such as advertisements, the eye catches, such as commercials and commercial, eyecatching.
AECEUCEaEAANaO
uiaEACCOUAEIAoCUIA
AUACCUAC
Shssaina mekanizumu ni tsuite wa fukugs says ni tsuite mo kongo ksryo suru hitsuys ga aru and it is necessary to consider the mechanism for detailed mechanism in the future.
There and it is necessary to consider the compound action in the detailed mechanism in the In the experiment, one subject is measured under only one condition and is measured Throughout the experiment, that state is maintained.
The experiment is carried out independently by measuring only one condition per During the experiment, the condition is In microeconomics, individual economic decisions are often explained by a unified indicator of AndA economic decisions are often explained by a unified measure of utility.
AndA Risk appetite and time appetite are events that are often treated in the same line.
A The risk preference and the time preference are often treated in the same class.
A eaeAIasa yaOaUUAAO CeUAIAC Jikken wa hitori no hikensha ni tsuki hitotsu no jsken nomi de keisoku shi, dokuritsu shite keisoku wo okonau.
eAAoayAOIUCeUA a Jikkench wa tsshite sono jstai o iji shi.
ECEAUOaUAI AAUOoEEaAyoCeOiAAEAIA AAiCUAUNoACOAAoaAoCUAC Aya Mikuro keizaigaku de wa shibashiba kokojin no keizaiteki ishi kettei wo ksys to iu tsitsu sa reta shihys ni yotte setsumei suru.
SoshiteA ECCAuAoCneAua aUOAOCaCUCUUAACCOa Risuku senks to jikan senks wa shibashiba dsretsu ni atsukawareru jishs de ari.
A MTPE Process CEECEE komsharu Literally means AocommercialAo -should be Auand advertisementAy but comes out as Auand commercialsAy (Systra.
, redundant to the previous Katakana word.
AECEUCE
Shssaina mekanizumu NaOui Fukugs says Both words in the noun contribute to the 1=> y Ae jsken OIU Ae jstai By default, two different Kanji characters mean different words.
But Systran depicts OIU jstai as a AoconditionAo as well.
AyAA Ae soshite And/then But more appropriate to be AothenAo, since it is after the full Au Ae senks Appetite/preference -Should be a preference based on the context .
Page e- ISSN 2528-5548 | p-ISSN 27764478 Muhammad Nadzif Bin Ramlan A Comparative Analysis of Japanese-English Machine Translation Outputs Using Neural and Statistical Systems:
Google Translate vs.
Systran Errors in Phrasal/Syntactic Structures Several outputs in the article from both translation software have spotted discrepancies with the supposed syntax of the respective SLs, as shown in Table 4.
Their syntax is usually reciprocal to the actual order or maybe premature/incomplete Since English syntax vastly differs from Malay as in Japanese syntax to English, phrase structure errors done by Google Translate and Systran in the translation process would be The output will be difficult to read and understand when the translation of the translation sentence is inaccurate and riddled.
This in turn causes the users to misinterpret the exact meaning of the translated sentence resulting in a misunderstanding of the context of the translated This problem may be due to the lack of word storage data for this application which results in incompletely translated sentences in terms of sentence structure.
This problem of incorrect sentence arrangement is also due to the less systematic translation process by this application.
Therefore, the quality of the output produced by Table 4: Errors in Phrasal/Syntactic Structures.
No.
Input
AECEUCEaEAANaO
uiaEACCOUAEIAoCUIA
AUACCUAC
Shssai na mekanizumu ni tsuite wa fukugs says ni tsuite mo kongo ksryo suruhitsuys ga aru.
AauaoCneAuAOiaNACO
CUECCAOAAoCUAEAOnAAC
CUAC
Kekka to shite jikan senks wa waribikiritsu ni yoru risuku to tska to suru kangaekata de aru.
soACUOAAUAACCAAUA OIuAuAACCUAC Tan naru oto shigeki ga eikys shita mono ka wa bunri ga konnan de aru.
unAACeIEEyAe CCEEyeCAUAACUCEAoAECOAI
aUeoEaEOIAcCOCUAAE
CUAC
Goraku shisetsu dewa apputenpona rokku ya poppusu ga nagare yasui ys ni, keiken-teki ni wa tsukaiwakerarete SMT (Systra.
NMT (Googl.
and it is necessary to consider the compound action in the detailed mechanism in the and it is necessary to consider the complex mechanisms for detailed mechanisms in the future.
There the time preference is considered to be equivalent to the risk by the discount rate.
time appetite is equivalent to risk by the discount rate.
is there.
but it is difficult to separate whether this is essentially music or just sound or simple sound stimulation has an It is difficult to separate.
and in entertainment facilities, there is a lot of up-tempo rock and pop music that are used differently empirically.
and entertainment facilities have been used differently to make it easier to play up-tempo rock and For the first and second inputs.
AUACCUAC.
a ar.
and AACCUAC.
e ar.
A both respective outputs of Authere is.
Ay and Auis there.
Ay at the end of the sentences are unconventional and instead should be embedded in the early part of the sentence in the proper order .
there isA).
The third example, uAuAACCUAC.
anare ga konAonandearu.
) has the correct output for Systran Aubut it is difficult to separateAAy but the wrong syntax for Google TranslateAos Auit is difficult to separate.
Ay since that AuseparateAy is a transitive verb and thus the sentence structure is unsuitable.
The fourth input.
AUeoEA A (Keiken-teki ni w.
has the output for Systran Audifferently empiricallyAo which is incorrect since there are two adverbials and adverbials cannot end a sentence.
Thus, the correct output syntactically would be Auempirically different.
Ay.
These issues signify the challenges MT systems face in maintaining syntactic accuracy across different language pairs (Bentivogli.
Bisazza.
Cettolo, & Federico, 2016.
Sennrich, 2.
Page e- ISSN 2528-5548 | p-ISSN 27764478 JAPANEDU: Jurnal Pendidikan dan Pengajaran Bahasa Jepang Vol.
No.
June 2024, pp.
Technical Errors It is observed that MT systems produced erroneous outputs that differ conventionally from those of human translators Ae incorrect prepositions, articles, pronouns, verb tenses, etc.
(Hutchins.
Therefore, this section focuses solely on this claim and we would verify them as well through our findings from the article using the translation software as shown in Table 5.
Table 5: Errors in Technicalities.
No.
Input Outputs eaeAIasAA AyaOaUUAA OCeUAIAC Jikken wa hitori no hikensha ni tsuki hitotsu no jsken nomi de keisoku shi, dokuritsu shite keisoku wo okonau.
The experiment is carried out independently by measuring only one condition per one (Systra.
uAUAAnACEEICCEOA ANaOACOauiAoCUAe AAUAEAOCOCUCUAC Ongaku no motsu ta no kontekisuto to no fukugs ni yotte tsuyoku says suru koto ga kangaerareru.
ECCAuAaOAEAoCneAu AaOAEaEaCUAC Risuku senks no doai to jikan senks no doai ni tsuite nomi shiraberu.
EIEEyunAECECEICECE NayAnAoaUAo yAACUCOAIaC Tenpo igai no veroshit ya onrys, sonohoka no yoin nado wa onaji jsken ni naru ys ni shita.
auCeuUACOAIAOaC AAoAA.
Genkyoku o ika no ys ni seigyo shita.
Mazu.
A It is thought that it acts strongly by the combination (Google Translat.
and it seems to act strongly by the combination with other context of the music.
(Systra.
the degree of risk and time preference for economic decision making.
Velocity other than tempo, volume, and other reverberations are the same.
(Google Translat.
The verocity, volume, and other aftertones other than the tempo were made to have the same conditions.
(Systra.
we controlled the original as First.
CONCLUSION
The quality of machine translation for JapaneseEnglish pairs remains a significant concern.
Despite the advancements in MT technologies, such as Google Translate and Systran, there are still notable challenges in achieving accurate and contextually appropriate translations.
These systems often struggle with linguistic nuances.
MTPE Process eAIasAA - Hikensha ni tsuki hitotsu per one subject -Per subject .
ne is omitted or vice vers.
since AoperAo already means Aofor every singleAo uAUAAnACEEICCEOA - Ongaku no motsu ta no kontekisuto Google Translate omits the whole phrase, and thus the correct output should be like SystranAos.
One full-stop suffices ECECEICE - Veroshiti Systran has the wrong spelling for the output AuverocityAy.
The correct one should be like GoogleAos AuvelocityAy aC - shita.
English convention would best be colon ( : ) rather than full stop including syntactic and semantic accuracy, which are critical for producing high-quality translations.
Systran, with its long history and experience in machine translation, has shown fewer errors and better handling of technical and academic texts compared to Google Translate.
This is likely due to its extensive development and refinement over the decades.
Conversely.
Google Translate, although relatively newer, has made significant .
Page e- ISSN 2528-5548 | p-ISSN 27764478 Muhammad Nadzif Bin Ramlan A Comparative Analysis of Japanese-English Machine Translation Outputs Using Neural and Statistical Systems:
Google Translate vs.
Systran improvements in lexical selection and sentence structure consistency.
Nonetheless, both tools have substantial room for improvement, especially in handling the intricacies of the Japanese language and its translation into English.
The concept of Machine-Aided Human Translation (MAHT) is particularly relevant in this MAHT leverages the strengths of machine translation for speed and efficiency while relying on human expertise to ensure accuracy and contextual appropriateness.
This approach is essential for translating texts that require a deep understanding of both languagesAo cultural and linguistic subtleties.
In conclusion, while machine translation has undoubtedly enhanced the process of translating Japanese to English, it is not yet a substitute for human translation.
Continued development and refinement of MT systems are crucial to improving their accuracy and reliability.
By combining the capabilities of MT with human expertise, we can achieve translations that are both efficient and of high quality, thereby fostering better cross-cultural communication and understanding.
REFERENCES