The paper, titled "Improving sentence representation for Vietnamese natural language understanding using Optimal Transport," was the result of collaborative efforts with guidance from:
- Associate Professor Nguyen Luu Thuy Ngan
- MSc. Nguyen Van Kiet
Abstract of the paper:
Understanding Natural Language (NLU) poses a challenge in Natural Language Processing (NLP), focusing on techniques to help computers understand human language and perform tasks related to language content. In recent years, with the emergence and development of Transformer networks and pre-trained language models, significant advancements have been made in the performance of NLU tasks. Research efforts have primarily concentrated on building pre-trained models for languages with abundant resources and large datasets. However, for languages with limited resources, pre-trained models may not be optimized well due to challenges in data collection. The introduction of multilingual models has addressed this issue by extending the model's representational capabilities to various languages, leading to significant improvements in NLU tasks for languages with limited resources.
Upon reviewing existing research results on multilingual models, the student identified a drawback: performance on tasks decreases when adding more languages during pre-training. Despite having a large number of parameters, multilingual models are often not well-optimized, especially for languages with limited resources due to the large number of languages that need to be represented. In this thesis, the student aims to enhance the performance of multilingual models on Vietnamese for NLU tasks, specifically text comprehension and natural language inference. These tasks are commonly applied in practical scenarios, especially in automated question-answering systems or information extraction. However, building models to solve these tasks still presents some challenges. Vietnamese is considered a language with limited resources due to a lack of labeled datasets. Additionally, processing Vietnamese faces difficulties, especially in vocabulary and syntax ambiguity, requiring models to possess inference and information synthesis capabilities. Therefore, the student recognizes the need to propose models that improve results on these tasks.
The student expresses deep gratitude to Associate Professor Nguyen Luu Thuy Ngan, Mr. Nguyen Van Kiet, and co-author Nguyen Hoang Thien Thu for their invaluable guidance, knowledge sharing, and precious experiences throughout the paper's development.
The Intelligent & Fuzzy Systems journal focuses on applications in Advanced Engineering and Technology, promoting the application and dissemination of research results related to fuzzy logic, intelligent systems, and web-based applications across various fields. The journal is dedicated to computer science, electrical engineering, manufacturing engineering, industrial engineering, chemical engineering, mechanical engineering, civil engineering, engineering management, biotechnology, and biomedical engineering. The scope of the journal also includes the development of technologies in mathematics, research activities, technology management, technical issues, society, and the environment. The Intelligent & Fuzzy Systems journal is listed in prestigious journal databases such as SCOPUS and ISI Web of Science.
Further information: https://www.facebook.com/UIT.Fanpage/posts/pfbid0kpEmtmbWNkFxnWJyFDXUMrYXFfsa79cvhgbXSpkWGAyUUgGWd9qzLWnSmSGmAtYfl
Written by: Hai Bang
Translated: Ngoc Diem
