Transformer-based Machine Translation

Transformer-based Machine Translation from English to Indonesian

Project Objective

The objective of this project is to re-implement the Transformer architecture as described in the paper “Attention is All You Need” and use it to train a machine translation model for translating English to Indonesian on the Opus-100 dataset from Helsinki-NLP, which consists of 1 million sentence pairs.

Data

The project utilizes the Opus-100 dataset from Helsinki-NLP, containing 1 million parallel sentence pairs in English and Indonesian. This extensive dataset enables the development of a robust machine translation model.

Benefits

  • Improved Translation Quality: Enhanced translation accuracy and fluency between English and Indonesian.
  • Scalable Training: Distributed training setup allows efficient use of multiple GPUs for faster training times.
  • Research Contribution: Provides a high-quality implementation of the Transformer architecture for the research community.
  • Resource Optimization: Efficient utilization of computational resources through distributed training.
  • Enhanced NLP Capabilities: Contributes to the development of NLP tools and applications for the Indonesian language.

Project Scope

The project focuses on:

  • Re-implementing the Transformer architecture as described in the seminal paper.
  • Training the model on the Opus-100 dataset for English-Indonesian translation.
  • Setting up distributed training using PyTorch to leverage multiple GPUs for efficient training.
  • Evaluating the model’s performance and comparing it with existing benchmarks.

GitHub Repository