BioNMT: A Biomedical Neural Machine Translation System

Authors

  • Hongtao Liu
  • Yanchun Liang
  • Liupu Wang
  • Xiaoyue Feng
  • Renchu Guan

Keywords:

neural machine translation, Transformer, self-attention, semantic disambiguation

Abstract

To solve the problem of translation of professional vocabulary in the biomedical field and help biological researchers to translate and understand foreign language documents, we proposed a semantic disambiguation model and external dictionaries to build a novel translation model for biomedical texts based on the transformer model. The proposed biomedical neural machine translation system (BioNMT) adopts the sequence-to-sequence translation framework, which is based on deep neural networks. To construct the specialized vocabulary of biology and medicine, a hybrid corpus was obtained using a crawler system extracting from universal corpus and biomedical corpus. The experimental results showed that BioNMT which composed by professional biological dictionary and Transformer model increased the bilingual evaluation understudy (BLEU) value by 14.14%, and the perplexity was reduced by 40%. And compared with Google Translation System and Baidu Translation System, BioNMT achieved better translations about paragraphs and resolve the ambiguity of biomedical name entities to greatly improved.

References

[1] Bahdanau, D.; Cho, K.; Bengio, Y. (2015). Neural Machine Translation by Jointly Learning to Align and Translate. 3rd International Conference on Learning Representations, San Diego, 2015.

[2] Bazzi, I.; Glass, J.R. (2000). Modeling Out-of-vocabulary Words for Robust Speech Recognition. Proc. of ISCA ASR2000, 401-404, 2000.

[3] Blei, D.M.; Ng, A.Y.; Jordan, M.I. (2003). Latent Dirichlet Allocation. Journal of Machine Learning Research, 3, 993-1022, 2003.

[4] Bo, Q.; Xiong, N.; Zou J. et al. (2007). Internationally agreed medical terminology: Medical Dictionary for Regulatory Activities. Chinese Journal of Clinical Pharmacology and Therapeutics, 2007.

[5] Brazill, S. (2016). Chinese to English Translation: Identifying Problems and Providing Solutions. Graduate Theses & Non-Theses. 71, 2016.

[6] Bu, F.; Gharajeh, M.S.(2019). Intelligent and Vision-based Fire Detection Systems: a Survey. Image and Vision Computing, 91, 2019. https://doi.org/10.1016/j.imavis.2019.08.007

[7] Chen, H.B.; Hsen, H.H.; Chang, H.A. (2017). A Simplification Translation Restoration Framework for Domain Adaptation in Statistical Machine Translation: A Case Study in Medical Record Translation. Computer Speech & Language, 42:59-80, 2017. https://doi.org/10.1016/j.csl.2016.08.003

[8] Cheng, J.; Dong, L.; Lapata, M. (2016). Long Short-Term Memory-Networks for Machine Reading. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 551-561, 2016. https://doi.org/10.18653/v1/D16-1053

[9] Cho, K.; van Merrienboer, B.; Gulcehre, C. et al. (2014). Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 1724-1734, 2014. https://doi.org/10.3115/v1/D14-1179

[10] Devlin, J.; Chang, M.W.; Lee, K. et al. (2019). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of NAACL-HLT 2019, 1, 4171-4186, 2019.

[11] Garcia-Castillo, D.;Fetters, M.D. (2007). Quality in Medical Translations: A Review. Journal of Health Care for the Poor and Underserved, 18(1): 2007. https://doi.org/10.1353/hpu.2007.0009

[12] Guan, R.;Zhang, H.; Liang, Y. et al. (2020). Deep feature-based text clustering and its explanation. IEEE Transaction on Knowledge and Data Engineering, 2020. https://doi.org/10.1109/TKDE.2020.3028943

[13] Gulcehre, C.; Ahn, S.; Nallapati, R. et al. (2016). Pointing the Unknown Words. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2016. https://doi.org/10.18653/v1/P16-1014

[14] Khanmohammadi, S.; Gharajeh, M.S. (2017). A Routing Protocol for Data Transferring in Wireless Sensor Networks Using Predictive Fuzzy Inference System and Neural Node, Ad Hoc & Sensor Wireless Networks, 38(1-4), 103-124, 2017.

[15] Kuzmina, O. D., Fominykh, A.D; Abrosimova, N.A. (2015). Problems of the English Abbreviations in Medical Translation. Procedia-Social and Behavioral Sciences, 199, 548-554, 2015. https://doi.org/10.1016/j.sbspro.2015.07.545

[16] Luong, M.T.; Pham, H.; Manning, C.D. (2015). Effective Approaches to Attention-based Neural Machine Translation. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 1412-1421, 2015. https://doi.org/10.18653/v1/D15-1166

[17] Manning, C.D.; Raghavan, P.; Schutze, H. (2008). Introduction to information retrieval, vol. 1, Cambridge University Press, Cambridge, 2008. https://doi.org/10.1017/CBO9780511809071

[18] Meng, X.; Liu, X.Z. Li, Y.Y. et al. (2020). Correlation between Genotype and Phenotype in 69 Chinese Patients with USH2A Mutations: A comparative study of the patients with Usher Syndrome and Nonsyndromic Retinitis Pigmentosa. Acta Ophthalmologica, 2020. https://doi.org/10.1111/aos.14626

[19] Mercy, O.E. (2006). English-Edo Medical Translation. Perspectives: Studies in Translatology, 13(4), 268-277, 2006. https://doi.org/10.1080/09076760608668997

[20] Mi, H.; Wang, Z.; Ittycheriah, A. (2016). Supervised Attentions for Neural Machine Translation. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, 2283- 2288, 2016. https://doi.org/10.18653/v1/D16-1249

[21] Mikolov, T.; Karafiát, M.; Burget, L. et al. (2010). Recurrent Neural Network based Language Model. INTERSPEECH, 1045-1048, 2010.

[22] Mitka, M. (2001). Tearing down the Tower of Babel: Medical Translation in today's world. Journal of the American Medical Association, 285(6), 722-3, 2001. https://doi.org/10.1001/jama.285.6.722-JMN0214-2-1

[23] Neubig, G. (2017). Neural Machine Translation and Sequence-to-sequence Models: A Tutorial. arXiv:1703.01619 [cs.CL], 2017.

[24] Papineni, K.; Roukos, S.; Ward, T. et al. (2002). BLEU: a Method for Automatic Evaluation of Machine Translation. Computational Linguistics, 40, 311-318, 2002. https://doi.org/10.3115/1073083.1073135

[25] Pascanu, R.; Mikolov, T.; Bengio, Y. (2013). On the Difficulty of Training Recurrent Neural Networks. Proceedings of the 30 th International Conference on Machine Learning, Atlanta, Georgia, USA, 2013, (3), 1310-1318, 2013.

[26] Peters, P., Qian, Y. Ding, J. (2018). Translating medical terminology and bilingual terminography. Lexicography ASIALEX, 3, 99-113, 2018. https://doi.org/10.1007/s40607-018-0037-y

[27] Rocktí¤schel, T.; Grefenstette, E.; Hermann, K.M. et al. (2015). Reasoning about Entailment with Neural Attention. Proceedings of ICLR2015, 2015.

[28] Sennrich, R.; Haddow, B.; Birch, A. (2016). Neural Machine Translation of Rate Words with Subword Units. The 54th Annual Meeting of the Association for Computational Linguistics, 1715- 1725, 2016. https://doi.org/10.18653/v1/P16-1162

[29] Shen, L.; Chen, H.; Yu, Z. et al. (2016). Evolving support vector machines using fruit fly optimization for medical data classification. Knowledge-Based Systems,96, 61-75. 2016. https://doi.org/10.1016/j.knosys.2016.01.002

[30] Slimani, T.(2013). Description and Evaluation of Semantic Similarity Measures Approaches International Journal of Computer Applications, 80(10), 25-33, 2013. https://doi.org/10.5120/13897-1851

[31] Vaswani, A.; Shazeer, N.; Parmar, N. et al. (2017). Attention is All You Need. 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA, 5998-6008, 2017.

[32] Wu, Y.; Schuster, M.; Chen, Z. et al. (2016). Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv:1609.08144, 2016.

[33] Yandell, M.D.; Majoros, W.H. (2002). Genomics and Natural Language Processing. Nature Reviews Genetics, 601-610, 2002. https://doi.org/10.1038/nrg861

[34] Ziemski, M.; Junczys-Dowmunt, M.; Pouliquen, B. (2016). The United Nations Parallel Corpus. Language Resources and Evaluation (LREC'16), 2016.

[35] [Online]. Science Fund Shared Service Network. Available: https://output.nsfc.gov.cn/

[36] [Online]. Chinese Medical Journal Network. Available: https://medjournals.cn/index.do

Additional Files

Published

2020-11-20

Most read articles by the same author(s)

Obs.: This plugin requires at least one statistics/report plugin to be enabled. If your statistics plugins provide more than one metric then please also select a main metric on the admin's site settings page and/or on the journal manager's settings pages.