Project Blog

Week 11 - Apr 12- Apr 18

• Presented Chinese to English translation model results

Things to do this week

• Finish report first draft

Week 10 - Apr 5 - Apr 11

Training Chinese to English translation model - no meeting

Week 9 - Mar 29 - Apr 4

• Implemented the Big Bird attention mechanism (TensorFlow)

Things to do this week

• Fine tune HAN English to Chinese translation model
• Clean up HAN code
• Change the attention layers to the Big Bird pattern

Week 8 - Mar 22 - Mar 28

Spring recess - no meeting

• Implemented the Big Bird attention mechanism (NumPy)
• Fine-tune the en-zh HAN model

Week 7 - Mar 15 - Mar 21

• Finished training the HAN model
• Presented the result of the HAN model

Things to do this week

• Fine tune HAN English to Chinese translation model
• Clean up HAN code
• Change the attention layers to the Big Bird pattern

Week 6 - Mar 8 - Mar 14

• Configure large scale word vectorizer
  • Vectorizing in the HAN model ran into OOM
    • 2^32 tokens into vectors ([num_tokens_in_sentence, 1] => [num_tokens_in_sentence, embedding_size])
• Adjusted HAN design to reduce hardware workload
  • Original design used Dense layer to "summarize" sentence information
  • Instead, an auto encoder (implemented with conv) can first summarize the document into chunks of sentence embeddings, and the normal attention model can extract information from the encoded information instead.

Things to do this week

• Fine tune HAN English to Chinese translation model
• Clean up HAN code
• Change the attention layers to compatible with Big Bird pattern
• Come up with a 2-week plan for spring recess

Week 5 - Mar 1 - Mar 7

• Discussion on HAN design

Things to do this week

• Implement HAN

Week 4 - Feb 22 - Feb 28

• Discussion on problems / solutions for implementing HAN

Things to do this week

• Implement HAN independently
• Keep trying to run [5] published code

Week 3 - Feb 15 - Feb 21

• Skipped

Week 2 - Feb 8 - Feb 14

• Presented proposal

Things to do this week

• CS 298 documents
• [1], [5] study

Week 1 - Jan 31 - Feb 7

Things to do this week

• Meeting with advisor
• Finish proposal

CS 297 completed

Week 15 - Nov 30 2022

• Presented report draft
• Presented deliverable 4

Things to do this week

• Complete CS 297 report
  • Complete abstract
  • Change of plan for CS 298?

Week 14 - Nov 23 2022

• Presented attention model results

Things to do this week

• Experiment with 1 row / column global attention
• Complete deliverable 4
• CS 297 report draft

Week 13 - Nov 15 2022

• Skipped - attention model still training

Week 12 - Nov 8 2022

• Presented NMT deliverable
• Presented attention introduction

Things to do this week

• Develop attention model
• Experiment on
  • Random attention
  • Window attention
  • Global attention

Week 11 - Nov 1 2022

• Presented NMT results

Things to do this week

• Make slides about the seq2seq results
• Finish deliverable 3
• Attention readings

Week 10 - Oct 25 2022

• Skipped -- model training still in process

Week 9 - Oct 18 2022

• Presented NMT initial implementation

Things to do this week

• Continue on the seq2seq model

Week 8 - Oct 11 2022

• Presented NMT readings

Things to do this week

• Start working on a seq2seq model
• [Optional] hash2vec MT solution

Week 7 - Oct 4 2022

• Presented SMT deliverable
• Discussed SMT improvement
  • Translate purely based on target language usage
  • Randomly translate source language to NULL

Things to do this week

• NMT readings
  • Charniak's book - Introduction to deep learning
  • Seq2seq papers

Week 6 - Sep 27 2022

• Presented SMT prototype
• Discussed SMT improvement

Things to do this week

• SMT improvement
• SMT deliverable

Week 5 - Sep 20 2022

• Presented SMT slides

Things to do this week

• Greedy SMT implementation

Week 4 - Sep 13 2022

• Presented RBMT prototype implemented with URBANS
• Discussed about data sources.

Things to do this week

• Finish RBMT deliverable
• SMT slides
• Define SMT implementation scope

Week 3 - Sep 6 2022

• Presented RBMT and general MT history in late 90s
• Discussed about data sources.

Things to do this week

• Present RBMT prototype with URBANS
• SMT slides
• Survey for data sources

Week 2 - August 29 2022

Created account for accessing the project site. Discussed about the proposal

Things to do this week

• Refine proposal description
• List out references
• Make summarization slides about rule-based machine translation paper(s)