Description

Through 50 hands-on, guided projects solved in Python, you will investigate the internal mechanisms of large language models by treating their hidden states, attention patterns, and embeddings as data to analyze. Rather than accepting LLMs as black boxes, you will open them up, examine what’s inside, and run experiments to understand why they behave the way they do. All projects are based on Python (using libraries such as NumPy, PyTorch, statsmodels, scikit-learn, Matplotlib, Pandas, and Seaborn) and come with full solutions and partial solution notebook files, so you can practice and improve your skills in data science, deep learning, data visualization, and scientific and statistical coding.

What You Will Learn:

– Tokenization schemes and their statistical properties

– Embedding spaces: cosine similarity, semantic axes, and analogy vectors

– Output logits, softmax distributions, perplexity, and language biases

– Layer-by-layer transformer dynamics and dimensionality

– Attention mechanisms: QKV weights, attention scores, head ablation, and activation patching

– MLP subblocks: neuron tuning, mutual information, subspace analysis, and statistics-based causal manipulations

– Logit lens, indirect object identification, and causal tracing

Who this book is for:

This book is for data scientists, ML engineers, and researchers who want to go beyond surface-level understanding of LLMs. Prior Python experience is required. Familiarity with machine learning or deep learning is helpful but not required – techniques are introduced as they arise throughout the projects.

Table of Contents

– Introductions

– Tokenization

– Embeddings

– Output logits

– Transformer outputs

– Attention

– MLP