A reimplementation of GPT architecture incorporating Curriculum Learning and Memory Augmentation by integrating external memory components
- Write script for EDA, cleaning, and formating of OpenWebText datasset
- Experiment with SentencePiece tokenization
- Experiment with Byte Pair Encoding (BPE)