Skip to main content

Ctrl+K

Pragmatic Natural Language Generation with Neural Language Models

00 organization

Course overview
Practicalities
Schedule
Slides from session 1

01 Probabilistic pragmatics

Probabilistic pragmatics
Sheet 1.1: Vanilla RSA
Sheet 1.2: RSA with politeness

02 PyTorch Basics

Basics of PyTorch
Sheet 2.1: PyTorch essentials
Sheet 2.2: ML-estimation

03 Optimization

Optimization in PyTorch
Sheet 3.1: Gradient descent by hand
Sheet 3.2: Optimizing an RSA model

04 ANNs in PyTorch

Artificial neural networks
Linear algebra
Sheet 4.1: Non-linear regression (custom MLP)
Sheet 4.2: Non-linear regression (MLP w/ PyTorch modules)

05 RNNs in PyTorch

Recurrent neural networks
Sheet 5.1: Character-level sequence modeling w/ RNNs

06 LSTMs in PyTorch

Long-Short Term Memory (LSTM) models
Sheet 6.1: Anatomy of a single LSTM forward pass
Sheet 6.2: Character-level sequence modeling w/ LSTMs
Sheet 6.3: Decoding strategies

07 Large language models

Large Language Models
Sheet 7.1: Using pretrained LLMs w/ the ‘transformers’ package

08 Grounded language models

Grounded Language Models
Sheet 8.1: An LSTM-based image captioner for the annotated 3D-Shapes data set
Sheet 8.2: Using 🤗’s pretrained models for image captioning

09 neural-pragmatic NLG

Neural pragmatic Natural Language Generation
Project ideas

Repository
Open issue

Index

By Michael Franke

© Copyright 2022.