 |
| One user recommended this comment. What they said: |
|
xianshou Dec 14, 2023 | on: Ask HN: Daily practices for building AI/ML skills? | context
Not a complete answer, but here are the most helpful resources for understanding transformer basics in particular: Original transformer paper: https://arxiv.org/abs/1706.03762 Illustrated transformer: http://jalammar.github.io/illustrated-transformer/ Transformer visualization: https://bbycroft.net/llm minGPT (Karpathy): https://github.com/karpathy/minGPT --- Next, some foundational textbooks for general ML and deep learning: Elements of Statistical Learning (aka the bible): https://hastie.su.domains/ElemStatLearn/ Probabilistic ML: https://probml.github.io/pml-book/book2.html Deep Learning Book (Goodfellow/Bengio): https://www.deeplearningbook.org/ Understanding Deep Learning: https://udlbook.github.io/udlbook/ --- Finally, assorted tutorials/resources/intro courses: Beyond the Illustrated Transformer: https://news.ycombinator.com/item?id=35712334 AI Zero to Hero: https://karpathy.ai/zero-to-hero.html AI Canon: https://a16z.com/2023/05/25/ai-canon/ LLM University by Cohere: https://llm.university/ Practical Guide to LLMs: https://github.com/Mooler0410/LLMsPracticalGuide Practical Deep Learning for Coders: https://course.fast.ai/Lessons/part2.html --- Hope that helps! |
The Illustrated Transformer is fantastic, but I would suggest that those going into it really should read the previous articles in the series to get a foundation to understand it more, plus later articles that go into GPT and BERT, here's the list:
A Visual and Interactive Guide to the Basics of Neural Networks - https://jalammar.github.io/visual-interactive-guide-basics-n...
A Visual And Interactive Look at Basic Neural Network Math - https://jalammar.github.io/feedforward-neural-networks-visua...
Visualizing A Neural Machine Translation Model (Mechanics of Seq2seq Models With Attention) - https://jalammar.github.io/visualizing-neural-machine-transl...
The Illustrated Transformer - https://jalammar.github.io/illustrated-transformer/
The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning) - https://jalammar.github.io/illustrated-bert/
The Illustrated GPT-2 (Visualizing Transformer Language Models) - https://jalammar.github.io/illustrated-gpt2/
How GPT3 Works - Visualizations and Animations - https://jalammar.github.io/how-gpt3-works-visualizations-ani...
The Illustrated Retrieval Transformer - https://jalammar.github.io/illustrated-retrieval-transformer...
The Illustrated Stable Diffusion - https://jalammar.github.io/illustrated-stable-diffusion/
If you want to learn how to code them, this book is great: https://d2l.ai/chapter_attention-mechanisms-and-transformers...