GPT-Genesis

A reimplementation of GPT architecture incorporating Curriculum Learning and Memory Augmentation by integrating external memory components

To-Do

• Write script for EDA, cleaning, and formating of OpenWebText datasset
• Experiment with SentencePiece tokenization
• Experiment with Byte Pair Encoding (BPE)