Your Daily AI Research tl;dr — 2022–07–13 🧠

BLOOM is here: the largest open-access multilingual language model ever. A special Transformer iteration.

Welcome to your official daily AI research tl;dr (often with code and news) for AI enthusiasts where I share the most exciting papers I find daily, along with a one-liner summary to help you quickly determine if the article (and code) is worth investigating. I will also take this opportunity to share daily exciting news in the field.

Let’s get started with this iteration!

1️⃣ Embedding Recycling for Language Models

They study how to decrease computational cost when training and inferring with large neural models through re-using activations from previous model runs when performing training or inference (recycling models..). More specifically: they “propose caching an intermediate layer’s output from a pretrained model and finetuning the remaining layers for new tasks.”

Link to the paper: https://arxiv.org/pdf/2207.04993.pdf

Code: https://github.com/allenai/EmbeddingRecycling/tree/main/recycling-code-demo

2️⃣ Exploring Length Generalization in Large Language Models