I Built My Own LLM Completely From Scratch (for pirates)

openai spent millions. i spent 0.70 euros for training. it was a really fun probject, hope you learned something. It was interesting to see the entire pipeline from data to a workable llm, in really just a part of my weekend. Honestly, editing this video almost took longer than the entire project. Nevertheless, here is the HF space if you want to try it out: huggingface.co/spaces/younissk/nanoBeard-playground And here github: github.com/younissk/pirate_llm 00:00 The Goal 00:52 How to build a LLM 02:54 Finding the data 04:30 Tokenizer And BPE Vocab 06:52 GPT Architecture Explained 14:01 GPU Training Run 15:53 Sampling The Base Model 17:41 Instruction Tuning Chatbot 20:13 Branding And Demo Space 22:55 Final Wrap Up