T5Gemma

A collection of open encoder-decoder models that improve deep contextual comprehension in Gemma

View tech report

Download

T5Gemma 2, based on Gemma 3, generally yields better post-training performance on coding, reasoning, and multilingual tasks than its decoder-only counterparts

T5Gemma 2 - Post-training Reasoning and Factuality — Results of minimal supervised fine-tuning without reinforcement learning. For illustrative purposes only.

T5Gemma 2 - Post-training Long Context — Results of minimal supervised fine-tuning without reinforcement learning. For illustrative purposes only.

T5Gemma 2 - Post-training STEM and Code — Results of minimal supervised fine-tuning without reinforcement learning. For illustrative purposes only.

T5Gemma 2 - Post-training Multimodal — Results of minimal supervised fine-tuning without reinforcement learning. For illustrative purposes only.

Superior quality-to-efficiency ratio without extensive compute requirements

T5Gemma 2

T5Gemma

T5Gemma 2

T5Gemma 2 ties the embeddings between the encoder and decoder and merges decoder self- and cross-attention, saving model parameters. This enables more active capabilities—including multimodal processing, multilingual support, and a 128k token context window—with the same memory footprint as Gemma 3.

Models are available in pre-trained versions ideal for rapid experimentation and deployment in on-device applications.

270M

270M-270M

~370M total memory footprint (excluding vision encoder)

1B-1B

~1.7B total memory footprint

4B-4B

~7B total memory footprint

T5Gemma

T5Gemma converts decoder-only Gemma 2 models to a T5 (Text-to-Text Transfer Transformer) encoder-decoder architecture.

Models are available in either instruction-tuned versions or pre-trained with PrefixLM for strong generative performance or UL2 for high-quality contextual representations.