إنشاء حساب
الرئيسية
الكورسات
Learn AI
Building Large Language Models (LLMs) from Scratch – Full Series
Lecture 16: Causal Self Attention Mechanism | Coded from scratch in Python

Lecture 16: Causal Self Attention Mechanism | Coded from scratch in Python

Building Large Language Models (LLMs) from Scratch – Full Series

الدرس السابق الدرس التالي

محتوى الكورس

جميع الدروس
43 درس
16:08 Lecture 1: Building LLMs from scratch: Series introduction
33:42 Lecture 2: Large Language Models (LLM) Basics
28:12 Lecture 3: Pretraining LLMs vs Finetuning LLMs
40:39 Lecture 4: What are transformers?
48:05 Lecture 5: How does GPT-3 really work?
20:15 Lecture 6: Stages of building an LLM from Scratch
09:44 Lecture 7: Code an LLM Tokenizer from Scratch in Python
53:35 Lecture 8: The GPT Tokenizer: Byte Pair Encoding
55:45 Lecture 9: Creating Input-Target data pairs using Python DataLoader
00:52 Lecture 10: What are token embeddings?
48:52 Lecture 11: The importance of Positional Embeddings
34:15 Lecture 12: The entire Data Preprocessing Pipeline of Large Language Models (LLMs)
51:25 Lecture 13: Introduction to the Attention Mechanism in Large Language Models (LLMs)
19:22 Lecture 14: Simplified Attention Mechanism - Coded from scratch in Python | No trainable weights
19:08 Lecture 15: Coding the self attention mechanism with key, query and value matrices
55:55 Lecture 16: Causal Self Attention Mechanism | Coded from scratch in Python
32:19 Lecture 17: Multi Head Attention Part 1 - Basics and Python code
01:13 Lecture 18: Multi Head Attention Part 2 - Entire mathematics explained
48:51 Lecture 19: Birds Eye View of the LLM Architecture
38:57 Lecture 20: Layer Normalization in the LLM Architecture
27:57 GELU Activation Function in the LLM Architecture
32:46 Shortcut connections in the LLM Architecture
45:06 Coding the entire LLM Transformer Block
01:33 Coding the 124 million parameter GPT-2 model
40:59 Coding GPT-2 to predict the next token
56:14 Measuring the LLM loss function
58:36 Evaluating LLM performance on real dataset | Hands on project | Book data
43:21 Coding the entire LLM Pre-training Loop
26:32 Temperature Scaling in Large Language Models (LLMs)
23:34 Top-k sampling in Large Language Models
12:26 Saving and loading LLM model weights using PyTorch
50:21 Loading pre-trained weights from OpenAI GPT-2
27:24 Introduction to LLM Finetuning | Python Coding with hands-on-example
31:03 Dataloaders in LLM Classification Finetuning | Python Coding | Hands on LLM project
34:44 Coding the model architecture for LLM classification fine-tuning
49:39 Coding a fine-tuned LLM spam classification model | From Scratch
25:32 Introduction to LLM Instruction Fine-tuning | Loading Dataset | Alpaca Prompt format
52:02 Data Batching in LLM instruction fine-tuning | Hands on project | Live Python coding
24:25 Dataloaders in Instruction Fine-tuning
19:29 Instruction fine-tuning: Loading pre-trained LLM weights
23:47 LLM fine-tuning training loop | Coded from scratch
52:58 Evaluating fine-tuned LLM using Ollama
19:18 Build LLMs from scratch 20 minutes summary