THE LIBRARY
Learn
Step-by-step lessons for studying an idea. Blog is the separate archive of essays and notes.
ATTENTION & REPRESENTATION
Inside the Attention Hull
See how an attention head combines value vectors, why their weighted sum stays inside the value hull, and how output projection and a residual write change the result.
Vectors and dot products → weighted averages → convex combinations → attention.
The lesson introduces convex combinations, softmax weights, and the geometric argument as you go.
IN THIS LESSON
Three values and their weighted average.
Coordinates and the convex-hull argument.
The same idea with four values in space.
Terms and sources used in the lesson.
Planned foundation chapters: vector spaces, linear maps, convex sets, and barycentric coordinates. For now, the attention lesson includes its own definitions and worked example.