Formal Algorithms for Transformers

Purpose

Transformers provide the basis to LLMs. Understand their inner workings.

Reading suggestions

• “Formal Algorithms for Transformers” by Phuong et. al.

• “An introduction to transformers” by Richard Turner

• “Attention Is All You Need” by Vaswani et al.

• A softer software oriented intro available

• Step-by-ste implementation

Date

April 16th at 11.30. @CUNEF

Facilitator

Roi Naveiro (CUNEF Universidad)

Discussion Topics

Implement or explore a basic transformer model for a text classification task, focusing on the self-attention mechanism. A deep dive into the algorithms that drive transformer models, including attention mechanisms and positional encoding.

Session recording

You can watch the session here.

Slides

Available here.

Code

Code available here.