| Bonus Resources.txt | 102.4 B | ||
| Get Bonus Downloads Here.url | 204.8 B | ||
| ~Get Your Files Here ! | |||
| 1 - Section 1 The Architecture Landscape and Your Bench | |||
| 1 - Course Documents and How to Use Them.mp4 | 29 MB | ||
| 1 - D01_Reading_an_Architecture_Papers_Claim_Checklist_and_Worked_Examples.docx | 26.4 KB | ||
| 1 - D02_Fair_Architecture_Comparison_Protocol_Rules_bench_py_Harness_and_Pre_Run_Checklist.docx | 25.5 KB | ||
| 1 - D03_CPU_Lab_Bench_Setup_and_Reproducibility_Checklist.docx | 25.1 KB | ||
| 1 - D04_Recursion_in_Neural_Networks_Field_Guide_Tree_Networks_Looped_Depth_and_RLMs.docx | 24.2 KB | ||
| 1 - D05_Recursive_Language_Models_in_Practice_Minimal_Loop_Trace_Reading_Safe_REPL.docx | 24.3 KB | ||
| 1 - D06_Attention_Cost_and_KV_Cache_Sizing_Worksheet.docx | 21.8 KB | ||
| 1 - D07_State_Space_Linear_Recurrent_and_Hybrid_Models_Reference_Card.docx | 22.6 KB | ||
| 1 - D08_Mixture_of_Experts_Design_and_Router_Health_Checklist.docx | 23.8 KB | ||
| 1 - D09_RAG_Pipeline_Evaluation_Template_Labelled_Set_Retrieval_Metrics_Faithfulness.docx | 22.9 KB | ||
| 1 - D10_Results_Table_and_Experiment_Log_Template.docx | 21.3 KB | ||
| 1 - D11_Architecture_Selection_Report_Template_with_the_Capstone_Worked_Example.docx | 22.6 KB | ||
| 10 - Section 1 Wrap The Map, the Bench and the Rules of Fair Measurement.mp4 | 11.7 MB | ||
| 10 - Section 10 RAG Patterns That Work in Production | |||
| 11 - Section 11 Benchmarking and Evaluating New Architectures | |||
| 100 - Fair Comparison Rules Same Data, Tokens, Parameters, Seeds — and Bits per Byte.mp4 | 32.9 MB | ||
| 101 - Perplexity Pitfalls Tokenizers, Context Windows and Strided Evaluation.mp4 | 26.7 MB | ||
| 102 - [LAB] Throughput, Time-to-First-Token and Memory on CPU Warm-Up and Spread.mp4 | 28 MB | ||
| 103 - [LAB] Long-Context Tests Needle Sweeps and Position Effects on a Small Model.mp4 | 27.2 MB | ||
| 104 - [LAB] Synthetic Probes Selective Copying, Induction Heads, Multi-Query Recall.mp4 | 29.6 MB | ||
| 105 - [LAB] lm-evaluation-harness on CPU One Small Task, Two Architectures.mp4 | 27 MB | ||
| 106 - Scaling Laws and What a Toy Experiment Can and Cannot Tell You.mp4 | 28.4 MB | ||
| 107 - Section 11 Wrap The Results Table, Complete and Honest.mp4 | 6.7 MB | ||
| 12 - Section 12 Decisions, Trade-Offs and the Capstone | |||
| 108 - Section 12 Intro From Experiments to an Architecture Decision.mp4 | 6.9 MB | ||
| 109 - The Trade-Off Axes Training Cost, Latency, Memory, Context Length, Maturity.mp4 | 28.4 MB | ||
| 110 - Ecosystem Reality Check What vLLM, llama.cpp and transformers Support Today.mp4 | 13.9 MB | ||
| 111 - [LAB] Capstone Part 1 One Task, Three Architectures, One Harness.mp4 | 33.8 MB | ||
| 112 - [LAB] Capstone Part 2 Running the Comparison and Reading the Spread.mp4 | 32 MB | ||
| 113 - [LAB] Capstone Part 3 Writing the Architecture Selection Report From Results.mp4 | 27.6 MB | ||
| 114 - Where the Field Is Moving Hybrids, Recursive Inference and Test-Time Memory.mp4 | 27.5 MB | ||
| 115 - Section 12 Wrap What You Can Now Build, Measure and Decide.mp4 | 16 MB | ||
| 2 - Section 2 A Transformer Baseline You Built Yourself | |||
| 11 - Section 2 Intro Every Comparison Needs a Baseline.mp4 | 14.6 MB | ||
| 12 - [LAB] Data and Tokenisation A Character-Level Corpus and Its Train Val Split.mp4 | 25.8 MB | ||
| 13 - [LAB] Self-Attention From Scratch in PyTorch, Checked Against the Built-In SDPA.mp4 | 27.8 MB | ||
| 14 - [LAB] A Tiny Decoder-Only Transformer Blocks, Norms, Residuals, Causal Mask.mp4 | 26.9 MB | ||
| 15 - [LAB] Training It on CPU Loss Curves, Steps per Second and Where the Time Goes.mp4 | 34.5 MB | ||
| 16 - [LAB] Loss, Perplexity and Bits per Character — Computed, Not Quoted.mp4 | 20.4 MB | ||
| 17 - [LAB] The Bench Harness One Tool for Speed, Memory and Quality Across Models.mp4 | 23.2 MB | ||
| 18 - [LAB] Sampling From the Baseline and Recording Its Row in the Results Table.mp4 | 22.3 MB | ||
| 19 - Section 2 Wrap A Measured Baseline and the Three Measurement Mistakes.mp4 | 12.6 MB | ||
| 3 - Section 3 Recursive Neural Networks — From Trees to Looped Depth | |||
| 20 - Section 3 Intro Reusing the Same Weights, Again and Again.mp4 | 15.2 MB | ||
| 21 - Recursive vs Recurrent From RAAM (1990) to the Recursive Neural Tensor Network.mp4 | 29.5 MB | ||
| 22 - Tree-LSTM (2015) Gates Over a Parse Tree Instead of a Chain.mp4 | 13.3 MB | ||
| 23 - [LAB] A Recursive Network Over Binary Trees in PyTorch, Step by Step.mp4 | 26.6 MB | ||
| 24 - [LAB] Tree-LSTM on Tiny Arithmetic Expressions Training and Checking Answers.mp4 | 19.8 MB | ||
| 25 - Universal Transformers and Adaptive Depth The Halting Idea.mp4 | 25.6 MB | ||
| 26 - Looped Transformers 2024-2025 Relaxed Recursion and Mixture-of-Recursions.mp4 | 30.3 MB | ||
| 27 - HRM and the Tiny Recursive Model (TRM) Small Networks That Iterate.mp4 | 27.6 MB | ||
| 28 - [LAB] A Weight-Tied Looped Transformer vs the Baseline at Equal Parameters.mp4 | 31.1 MB | ||
| 29 - Section 3 Wrap What Recursion in Depth Buys You, and What It Costs.mp4 | 14 MB | ||
| 4 - Section 4 Recursive Language Models (RLM) | |||
| 30 - Section 4 Intro When the Prompt Is Too Long to Read.mp4 | 13.3 MB | ||
| 31 - Context Rot What NIAH, RULER, Lost in the Middle and OOLONG Show.mp4 | 16.3 MB | ||
| 32 - The RLM Idea The Prompt as a Variable in a REPL, Sub-Calls as a Function.mp4 | 27.7 MB | ||
| 33 - RLM vs RAG vs ReAct vs CodeAct Where the Long Input Lives, Who Decides Access.mp4 | 26.1 MB | ||
| 34 - What the RLM Paper Reports Benchmarks, Depth Ablations, Costs and Versions.mp4 | 29.7 MB | ||
| 35 - [LAB] A Minimal RLM Loop by Hand Root Model, Python REPL, llm_query and FINAL.mp4 | 40.5 MB | ||
| 36 - [LAB] Serving a Local Model for the Loop Ollama's OpenAI-Compatible Endpoint.mp4 | 14.6 MB | ||
| 37 - [LAB] Running the Hand-Built RLM on a Needle Task and Reading the Full Trace.mp4 | 19.3 MB | ||
| 38 - [LAB] The Official rlms Library on a Local Model, and Why a Small Root Fails.mp4 | 31.1 MB | ||
| 39 - [LAB] Aggregating a Long Document Chunked Sub-Calls vs One Long Prompt.mp4 | 29 MB | ||
| 40 - Stated Limits Cost Blow-Ups, Sequential Calls, Brittle FINAL, REPL Sandboxing.mp4 | 25.7 MB | ||
| 41 - Section 4 Wrap When an RLM Is the Right Tool.mp4 | 12 MB | ||
| 5 - Section 5 Efficient Attention and Linear Transformers | |||
| 42 - Section 5 Intro Making Attention Cheaper Without Losing the Plot.mp4 | 7.2 MB | ||
| 43 - Sparse and Local Patterns Longformer, BigBird and Sliding-Window Attention.mp4 | 27.1 MB | ||
| 44 - Low-Rank and Kernel Tricks Linformer, Performer and Transformers Are RNNs.mp4 | 28.8 MB | ||
| 45 - FlashAttention 1 to 4 Exact Attention, IO-Aware — and Why It Is a GPU Story.mp4 | 27.2 MB | ||
| 46 - [LAB] Sliding-Window Attention on the Baseline Memory and Quality vs Window.mp4 | 31.3 MB | ||
| 47 - [LAB] A Linear-Attention Layer From Scratch and Its Recurrent Form.mp4 | 30.9 MB | ||
| 48 - [LAB] Timing Attention on CPU Length Sweeps, Math vs Flash Backend in PyTorch.mp4 | 21.1 MB | ||
| 49 - MQA, GQA and Multi-Head Latent Attention Shrinking the Cache, Not the Compute.mp4 | 27.8 MB | ||
| 50 - Section 5 Wrap Which Attention Trick for Which Bottleneck.mp4 | 11.8 MB | ||
| 6 - Section 6 State Space Models — S4 and Mamba | |||
| 51 - Section 6 Intro A Recurrent Model That Trains Like a Convolution.mp4 | 13.4 MB | ||
| 52 - From Continuous Systems to Discrete Recurrence The SSM Equations Made Concrete.mp4 | 26.7 MB | ||
| 53 - S4, H3 and Hyena Long Convolutions and the Long Range Arena.mp4 | 26.7 MB | ||
| 54 - Mamba's Selective Scan Input-Dependent Parameters, Hardware-Aware Algorithm.mp4 | 26.7 MB | ||
| 55 - [LAB] A Selective SSM Layer From Scratch Discretise, Scan, Gate.mp4 | 34.4 MB | ||
| 56 - [LAB] Sequential vs Chunked Scan on CPU Speed, and Keeping the Numbers Stable.mp4 | 34.5 MB | ||
| 57 - [LAB] Training a Tiny Mamba-Style Model and Adding Its Row to the Results Table.mp4 | 26.4 MB | ||
| 58 - [LAB] Pretrained Mamba-130m vs Pythia-160m on CPU Same Data and Tokenizer.mp4 | 33.6 MB | ||
| 59 - Mamba-2 and SSD, Mamba-3 What Changed and What Needs a GPU.mp4 | 26.3 MB | ||
| 60 - Section 6 Wrap Where SSMs Win, and Where Your CPU Disagrees.mp4 | 13.8 MB | ||
| 7 - Section 7 Hybrids and Modern Linear Recurrences | |||
| 61 - Section 7 Intro Why the 2025 Models Mix Layer Types.mp4 | 14.1 MB | ||
| 62 - RWKV and RetNet RNN Inference, Parallel Training.mp4 | 24.5 MB | ||
| 63 - Gated DeltaNet and Kimi Delta Attention The Delta Rule Returns.mp4 | 24.2 MB | ||
| 64 - Hybrid Stacks in Production Jamba, Zamba, Falcon-H1, Nemotron-H, Qwen3-Next.mp4 | 28.7 MB | ||
| 65 - [LAB] Associative Recall The Task That Separates Attention From Pure SSMs.mp4 | 36.4 MB | ||
| 66 - [LAB] A Hybrid Model SSM Layers Plus One Attention Layer, Trained and Compared.mp4 | 36.8 MB | ||
| 67 - Section 7 Wrap Choosing a Mix, Not a Religion.mp4 | 11.9 MB | ||
| 8 - Section 8 Mixture of Experts | |||
| 68 - Section 8 Intro More Parameters, Same Compute per Token.mp4 | 12.6 MB | ||
| 69 - Sparsely-Gated MoE to Switch Transformer Top-k Routing, Capacity, Token Drops.mp4 | 31.9 MB | ||
| 70 - Load Balancing Auxiliary Loss, Router z-Loss, Expert Choice, Aux-Loss-Free Bias.mp4 | 28.6 MB | ||
| 71 - Fine-Grained and Shared Experts DeepSeekMoE, Mixtral, OLMoE and Qwen MoE.mp4 | 28.9 MB | ||
| 72 - [LAB] An MoE Feed-Forward Layer From Scratch Router, Top-2 and Weighted Combine.mp4 | 31.2 MB | ||
| 73 - [LAB] Training the MoE and Watching Expert Load With and Without Balance Loss.mp4 | 41.9 MB | ||
| 74 - [LAB] Collapse on Purpose What an Unbalanced Router Looks Like in the Logs.mp4 | 32 MB | ||
| 75 - [LAB] A Real MoE on CPU Granite 3.1 1B-A400M Router Choices Token by Token.mp4 | 37.6 MB | ||
| 76 - MoE in Practice Memory, Serving, Expert Parallelism and When Dense Is Better.mp4 | 25.8 MB | ||
| 77 - Section 8 Wrap Sparse Compute, Full Memory.mp4 | 12.3 MB | ||
| 9 - Section 9 Memory-Augmented Networks and the KV Cache | |||
| 78 - Section 9 Intro Where Does a Model Keep What It Has Read.mp4 | 6.9 MB | ||
| 79 - External Memory Neural Turing Machine, Memory Networks and the DNC.mp4 | 32.4 MB | ||
| 80 - Memory for Transformers Transformer-XL, Memorizing Transformers, RMT and Titans.mp4 | 29.8 MB | ||
| 81 - [LAB] A Content-Addressed Memory Module Trained on Associative Recall.mp4 | 32.5 MB | ||
| 82 - [LAB] The KV Cache Measured With and Without It, and Its Size Formula.mp4 | 28.3 MB | ||
| 83 - [LAB] Cache Management in transformers Dynamic, Static, Quantized, Cropping.mp4 | 28.1 MB | ||
| 84 - PagedAttention, Attention Sinks and KV Quantization The Serving View.mp4 | 29.1 MB | ||
| 85 - [LAB] Sliding Window Plus Sinks Keeping Generation Stable Past the Window.mp4 | 31.1 MB | ||
| 86 - Section 9 Wrap Memory Inside, Beside and Outside the Model.mp4 | 13.2 MB | ||
| 99 - Section 11 Intro Numbers You Would Bet Your Architecture On.mp4 | 7.1 MB | ||
| 87 - Section 10 Intro Retrieval as External Memory You Can Update.mp4 | 14.2 MB | ||
| 88 - RAG From Lewis et al. (2020) to Today The Moving Parts and Their Failure Points.mp4 | 26.9 MB | ||
| 89 - [LAB] Chunking Strategies on One Corpus Fixed, Recursive, Sentence-Aware.mp4 | 32.3 MB | ||
| 90 - [LAB] Embeddings on CPU Choosing a Small Model, Encoding, Checking Its Licence.mp4 | 26.6 MB | ||
| 91 - [LAB] Vector Indexes With FAISS Flat vs HNSW vs IVF, Recall and Latency.mp4 | 28.9 MB | ||
| 92 - [LAB] BM25 Plus Dense Hybrid Search With Reciprocal Rank Fusion.mp4 | 27.2 MB | ||
| 93 - [LAB] Re-Ranking With a Cross-Encoder What It Adds, What It Costs.mp4 | 25.4 MB | ||
| 94 - [LAB] Retrieval Evaluation A Labelled Set Scored With Recall@k, MRR and nDCG.mp4 | 30.9 MB | ||
| 95 - [LAB] End-to-End Local RAG Answers With Citations and a Faithfulness Check.mp4 | 36.3 MB | ||
| 96 - Contextual Retrieval, GraphRAG, Self-RAG and CRAG When the Extra Machinery Pays.mp4 | 25 MB | ||
| 97 - [LAB] RAG vs RLM vs Long Prompt on the Same Question Set.mp4 | 37.2 MB | ||
| 98 - Section 10 Wrap A RAG Pipeline You Can Defend With Numbers.mp4 | 6.8 MB | ||
| 2 - Section 1 Intro Why Look Past the Plain Transformer.mp4 | 7.4 MB | ||
| 3 - What Advanced Architecture Means in 2026 Attention, Recurrence, Sparsity.mp4 | 33.3 MB | ||
| 4 - Three Meanings of Recursive Tree Networks, Looped Depth and RLMs.mp4 | 14.5 MB | ||
| 5 - The Quadratic Wall Where Attention's Cost and the KV Cache Actually Bite.mp4 | 22.5 MB | ||
| 6 - Reading a Paper's Claim Correctly Hardware, Scale, Setup and Version.mp4 | 25.1 MB | ||
| 7 - [LAB] The Bench 8 CPUs, No GPU — Measuring What This Machine Can Do.mp4 | 17.5 MB | ||
| 8 - [LAB] Building the Lab Environment CPU PyTorch, Transformers, Pinned Versions.mp4 | 18.7 MB | ||
| 9 - [LAB] Opening archlab The Course Repository and Its Empty Results Table.mp4 | 15.5 MB |
Recursive Language Models and Advanced LLM Architectures
https://WebToolTip.com
Published 10/2026
MP4 | Video: h264, 1920x1080 | Audio: AAC, 44.1 KHz, 2 Ch
Language: English | Duration: 11h 12m | Size: 2.74 GB
Build and measure RLMs, looped transformers, efficient attention, Mamba, hybrids, MoE, memory and RAG on a CPU
What you'll learn
Explain the three meanings of recursive in deep learning: tree networks, looped depth and Recursive Language Models
Build a small decoder-only transformer from scratch and use it as a measured baseline for every other architecture
Implement recursive networks, Tree-LSTM and a weight-tied looped transformer, and compare them at equal parameters
Build a Recursive Language Model loop by hand with a Python REPL and a local model, and read every step of its trace
Add sliding-window and linear attention to the baseline and measure memory, speed and quality against sequence length
Implement a selective state space layer, train a tiny Mamba-style model and compare pretrained Mamba with Pythia on CPU
Build mixture-of-experts layers, watch expert load with and without balancing, and inspect a real MoE router
Measure the KV cache, manage it with the transformers cache classes and keep generation stable with attention sinks
Build and evaluate a production-style RAG pipeline with chunking, FAISS, hybrid search, re-ranking and faithfulness checks
Run fair architecture comparisons and write an architecture selection report backed by a complete results table
Requirements
Working Python and basic PyTorch: you can read a training loop and run a script from the command line
A computer with a multi-core CPU. No GPU is needed: the course bench is an 8-CPU machine and every lab is measured on it
All Comments