Project Blog
Week 11 - Apr 12- Apr 18
- Presented Chinese to English translation model results
Things to do this week
- Finish report first draft
Week 10 - Apr 5 - Apr 11
Training Chinese to English translation model - no meeting
Week 9 - Mar 29 - Apr 4
- Implemented the Big Bird attention mechanism (TensorFlow)
Things to do this week
- Fine tune HAN English to Chinese translation model
- Clean up HAN code
- Change the attention layers to the Big Bird pattern
Week 8 - Mar 22 - Mar 28
Spring recess - no meeting
- Implemented the Big Bird attention mechanism (NumPy)
- Fine-tune the en-zh HAN model
Week 7 - Mar 15 - Mar 21
- Finished training the HAN model
- Presented the result of the HAN model
Things to do this week
- Fine tune HAN English to Chinese translation model
- Clean up HAN code
- Change the attention layers to the Big Bird pattern
Week 6 - Mar 8 - Mar 14
- Configure large scale word vectorizer
- Vectorizing in the HAN model ran into OOM
- 2^32 tokens into vectors ([num_tokens_in_sentence, 1] => [num_tokens_in_sentence, embedding_size])
- Adjusted HAN design to reduce hardware workload
- Original design used Dense layer to "summarize" sentence information
- Instead, an auto encoder (implemented with conv) can first summarize the document into chunks of sentence embeddings, and the normal attention model can extract information from the encoded information instead.
Things to do this week
- Fine tune HAN English to Chinese translation model
- Clean up HAN code
- Change the attention layers to compatible with Big Bird pattern
- Come up with a 2-week plan for spring recess
Week 5 - Mar 1 - Mar 7
Things to do this week
Week 4 - Feb 22 - Feb 28
- Discussion on problems / solutions for implementing HAN
Things to do this week
- Implement HAN independently
- Keep trying to run [5] published code
Week 3 - Feb 15 - Feb 21
Week 2 - Feb 8 - Feb 14
Things to do this week
- CS 298 documents
- [1], [5] study
Week 1 - Jan 31 - Feb 7
Things to do this week
- Meeting with advisor
- Finish proposal
CS 297 completed
Week 15 - Nov 30 2022
- Presented report draft
- Presented deliverable 4
Things to do this week
- Complete CS 297 report
- Complete abstract
- Change of plan for CS 298?
Week 14 - Nov 23 2022
- Presented attention model results
Things to do this week
- Experiment with 1 row / column global attention
- Complete deliverable 4
- CS 297 report draft
Week 13 - Nov 15 2022
- Skipped - attention model still training
Week 12 - Nov 8 2022
- Presented NMT deliverable
- Presented attention introduction
Things to do this week
- Develop attention model
- Experiment on
- Random attention
- Window attention
- Global attention
Week 11 - Nov 1 2022
Things to do this week
- Make slides about the seq2seq results
- Finish deliverable 3
- Attention readings
Week 10 - Oct 25 2022
- Skipped -- model training still in process
Week 9 - Oct 18 2022
- Presented NMT initial implementation
Things to do this week
- Continue on the seq2seq model
Week 8 - Oct 11 2022
Things to do this week
- Start working on a seq2seq model
- [Optional] hash2vec MT solution
Week 7 - Oct 4 2022
- Presented SMT deliverable
- Discussed SMT improvement
- Translate purely based on target language usage
- Randomly translate source language to NULL
Things to do this week
- NMT readings
- Charniak's book - Introduction to deep learning
- Seq2seq papers
Week 6 - Sep 27 2022
- Presented SMT prototype
- Discussed SMT improvement
Things to do this week
- SMT improvement
- SMT deliverable
Week 5 - Sep 20 2022
Things to do this week
- Greedy SMT implementation
Week 4 - Sep 13 2022
- Presented RBMT prototype implemented with URBANS
- Discussed about data sources.
Things to do this week
- Finish RBMT deliverable
- SMT slides
- Define SMT implementation scope
Week 3 - Sep 6 2022
- Presented RBMT and general MT history in late 90s
- Discussed about data sources.
Things to do this week
- Present RBMT prototype with URBANS
- SMT slides
- Survey for data sources
Week 2 - August 29 2022
Created account for accessing the project site. Discussed about the proposal
Things to do this week
- Refine proposal description
- List out references
- Make summarization slides about rule-based machine translation paper(s)
|