Frontier Signal
ICL OOD Generalization: Subspace Theory for Transformers
New arXiv research models how In-Context Learning (ICL) in transformers generalizes out-of-distribution (OOD). It finds that task vectors in a union of low-dimensional subspaces enable OOD generalization.
Read the briefing