Generative Pre-Training (GPT)

Purpose

Get acquainted with the formal foundations central to GPT and delineate its instrumental role in the evolution of LLMs.

Reading suggestions

• “Improving Language Understanding by Generative Pre-Training” by Radford et al.

• “Language Models are Few-Shot Learners” by Brown et al.

Date

April 23rd at 11.30. @ICMAT

Facilitator

Carlos García Meixide (ICMAT-CSIC)

Discussion Topics

Quick historical timeline of language AI, theoretical basis of GPT, practical Python demonstrations, the mathematics of in-context learning, future directions.

Session recording

You can watch the session here.

Slides

Available here.