Google DeepMind’s DiffusionGemma is challenging one of the assumptions behind modern AI development: that building a new type of language model requires training an entirely new foundation model from the ground up.
Instead, Google researchers took an existing Gemma 4 model and converted it into a text-diffusion model using less than 10% of the original model’s training-token budget. The resulting system can generate text at around 1,500 tokens per second on an Nvidia H100 accelerator while offering a fundamentally different way of producing text.
The development could make experimentation with diffusion-based language models considerably cheaper and faster, while also opening up new possibilities for applications where models need to revise their answers before producing a final result.
What is DiffusionGemma?
Most popular AI chatbots today use autoregressive language models. These systems generate responses sequentially, predicting one token at a time based on the tokens that have already been produced.
DiffusionGemma takes a different approach.
Instead of committing to the answer from left to right, it works on blocks of 256 tokens in parallel. The model starts with a noisy or uncertain representation and progressively refines it until a coherent piece of text emerges.
The concept is similar to image-generation diffusion models, which begin with noise and repeatedly remove that noise until a complete image is produced.
DiffusionGemma at a glance
| Metric | DiffusionGemma |
|---|---|
| Starting model | Gemma 4 26B-A4B |
| Generation approach | Text diffusion |
| Tokens refined per block | 256 |
| Reported speed | ~1,500 tokens/second |
| Hardware example | Nvidia H100 |
| Additional training budget | Less than 10% of original |
| Refinement for some structured tasks | 2-3 steps |
| Sudoku accuracy after minimal fine-tuning | Nearly 85% |
| License | Apache 2.0 |
| Status | Experimental |
Google says the model is designed primarily as a research foundation for text-diffusion experimentation and resource-efficient specialised models.
The biggest breakthrough: Google did not train it from scratch
The most important part of the DiffusionGemma research may not be its 1,500-token-per-second generation speed.
It is the fact that Google did not build the model from scratch.
Researchers started with Gemma 4 26B-A4B, an already-trained autoregressive model, and subsequently converted it into a diffusion model.
According to Google’s technical report, the conversion required less than 10% of the original training-token budget.
Why this matters
Training a large foundation model requires enormous amounts of computing power, training data and engineering resources.
If an existing model can instead be repurposed into a different generation architecture, researchers could experiment with alternative AI architectures without paying the full cost of another pretraining run.
Infographic: How Google converted Gemma into DiffusionGemma
EXISTING AI MODEL
│
▼
┌───────────────────┐
│ Gemma 4 │
│ 26B-A4B model │
└─────────┬─────────┘
│
│ Conversion
▼
┌───────────────────┐
│ Stage 1 Training │
│ │
│ Learn to rebuild │
│ noisy text blocks │
└─────────┬─────────┘
│
▼
┌───────────────────┐
│ SD·RL │
│ │
│ Reinforcement │
│ Learning + │
│ Sampler │
│ Distillation │
└─────────┬─────────┘
│
▼
┌───────────────────┐
│ DiffusionGemma │
│ │
│ Parallel text │
│ refinement │
└───────────────────┘
The approach could eventually make alternative language-model architectures much more accessible to researchers and smaller AI companies.
How text diffusion differs from normal AI generation
The key difference is how the model approaches the answer.
An autoregressive model generally works like this:
Token 1 → Token 2 → Token 3 → Token 4 → … → Final answer
Each new token depends on what came before it.
DiffusionGemma instead works more like:
Noisy text → Refinement → Refinement → Refinement → Final text
Multiple parts of the answer can be reconsidered simultaneously.
Autoregressive vs diffusion generation
| Feature | Autoregressive model | DiffusionGemma |
|---|---|---|
| Generation | Sequential | Parallel refinement |
| Basic unit | Individual tokens | Blocks of tokens |
| Direction | Usually left to right | Bidirectional refinement |
| Can revise earlier output? | Limited during generation | Yes |
| Parallelism | Lower | Higher |
| Strength | Mature quality and reasoning | Speed and revisability |
| Main weakness | Sequential decoding | Quality gap and refinement artefacts |
The ability to reconsider earlier output is particularly interesting for reasoning tasks.
DiffusionGemma can correct mistakes before finalising the answer
One of the most interesting characteristics of diffusion-based text generation is that the model does not have to permanently commit to its first answer.
Google’s report gives an example involving mathematical reasoning. An autoregressive Gemma model initially produces “-1”, later realises that the correct answer is “-25”, and then has to add a correction.
DiffusionGemma can revise the earlier portion during subsequent refinement steps before the final answer is produced.
This gives the model a form of bidirectional reasoning.
Infographic: The difference in reasoning
AUTOREGRESSIVE AI
Question
↓
First answer
↓
Reasoning
↓
Discovers mistake
↓
Correction added
↓
Final response
DIFFUSION AI
Question
↓
Initial rough answer
↓
┌─────────────────────┐
│ Refine answer │
│ Refine reasoning │
│ Reconsider earlier │
│ tokens │
└─────────┬───────────┘
↓
Corrected final answer
This could be useful for problems where different parts of the answer depend on each other.
Sudoku demonstrates the advantage of bidirectional reasoning
Sudoku is a particularly interesting test because every entry on the board depends on other entries.
An AI system that makes an early decision may later discover that the decision conflicts with another part of the puzzle.
DiffusionGemma can revisit earlier decisions while refining the complete output.
After minimal fine-tuning, Google reports that DiffusionGemma solved nearly 85% of Sudoku puzzles, whereas the base model failed at the task.
Selected performance indicators
| Area | Reported result |
|---|---|
| Text generation | ~1,500 tokens/sec on H100 |
| Sudoku | Nearly 85% solved after minimal fine-tuning |
| Structured-output refinement | 2-3 steps |
| Reasoning benchmark improvement from SD·RL | ~10 points average |
| Training conversion budget | <10% of original |
These numbers demonstrate why diffusion could be particularly interesting for constrained or structured tasks.
Structured outputs could be a major use case
DiffusionGemma does not necessarily need many refinement cycles for every task.
Google reports that some structured outputs, including JSON generation and code repairs, can be completed in just two or three refinement steps.
That is possible because much of the final output is already constrained by the input.
For example, when repairing a piece of code, the model may only need to change a relatively small portion of the existing program.
Instead of generating the entire response sequentially, a diffusion model can potentially refine the relevant sections simultaneously.
This could make the approach useful for:
- Code repair
- JSON generation
- Structured document creation
- Data transformation
- Form completion
- Editing
- Translation
- Constrained reasoning
- Interactive applications
Google’s two-stage training approach
Google did not simply convert Gemma 4 and immediately deploy the resulting model.
The researchers used two major training stages.
Stage 1: Learning to reconstruct noisy text
The model first learned how to reconstruct text that had been deliberately corrupted or made noisy.
This effectively teaches the model how to move from an uncertain representation toward meaningful language.
Stage 2: SD·RL
The second stage combines reinforcement learning and sampler distillation, which Google calls SD·RL.
Reinforcement learning helps improve the quality of the generated responses, while sampler distillation helps the model achieve useful results with fewer computational steps.
Google reports that this combined process improved reasoning benchmark performance by around 10 points on average while nearly quadrupling the number of tokens generated per compute step.
The resulting answers were also around 50% shorter, providing another source of speed improvement.
Training strategy
Gemma 4
│
▼
Noisy-text reconstruction
│
▼
SD·RL
├── Reinforcement learning
│ ↓
│ Better answers
│
└── Sampler distillation
↓
Fewer compute steps
│
▼
DiffusionGemma
│
├── Higher generation speed
├── Parallel refinement
└── Better reasoning quality
Speed is DiffusionGemma’s biggest advantage
Google reports around 1,500 tokens per second on an Nvidia H100.
That is a significant number because conventional autoregressive generation is inherently sequential.
The model must repeatedly predict the next token before moving to the following one.
Diffusion allows multiple tokens to be processed simultaneously, potentially making better use of the parallel computing capabilities of modern GPUs.
Speed comparison
| Factor | Conventional autoregressive model | DiffusionGemma |
|---|---|---|
| Token generation | Sequential | Parallel refinement |
| H100 reported speed | Lower | ~1,500 tokens/sec |
| Ability to revise output | Limited | Built into refinement |
| Best suited to | General-purpose generation | Fast/structured generation |
| Scaling challenge | Sequential decoding | Refinement and concurrency |
The speed advantage, however, does not apply equally to every workload.
DiffusionGemma loses its speed advantage at high concurrency
One of the biggest caveats in Google’s research is that the speed advantage is strongest in single-user scenarios.
When around 32 requests are running concurrently, conventional language models can catch up in throughput.
This matters for companies operating large AI services.
A model that is extremely fast for one user is not automatically the most efficient model for a service handling thousands of simultaneous requests.
Infographic: Where the advantage changes
LOW CONCURRENCY
1 user
│
▼
DiffusionGemma
████████████████████
Very strong speed advantage
MEDIUM CONCURRENCY
Several users
│
▼
Advantage begins narrowing
██████████████
HIGH CONCURRENCY
~32 requests
│
▼
Autoregressive models
begin catching up
████████████████
This means the commercial value of diffusion models will depend heavily on the exact workload and infrastructure configuration.
The model still trails Gemma 4 on quality
DiffusionGemma is not yet a universal replacement for conventional language models.
Google’s research shows that the diffusion version still trails the autoregressive Gemma 4 model on some quality benchmarks.
There are several reasons.
First, DiffusionGemma was retrofitted from an existing model rather than being designed specifically for diffusion from the beginning.
Second, the additional training was relatively short compared with the original foundation-model training.
Third, the SD·RL stage prioritised speed and efficiency rather than maximum benchmark performance.
Finally, the architecture, training data and other configuration choices were inherited from Gemma 4, even though they may not be optimal for diffusion.
Current trade-off
| Category | DiffusionGemma |
|---|---|
| Generation speed | 🟢 Strong |
| Parallel processing | 🟢 Strong |
| Revising earlier output | 🟢 Strong |
| Structured tasks | 🟢 Promising |
| Reasoning quality | 🟡 Below base Gemma 4 |
| General-purpose maturity | 🟡 Experimental |
| High-concurrency advantage | 🟡 Limited |
| Reliability | 🟡 Still being researched |
Repetition loops remain a problem
The model also has some technical weaknesses.
Google researchers observed cases where DiffusionGemma becomes trapped in repetition loops, producing the same words multiple times.
The problem appears to be connected partly to aggressively reducing the number of compute steps.
In other words, reducing the number of refinement steps can make the model faster, but pushing the process too far can hurt output quality.
This creates an important engineering trade-off:
More refinement steps → potentially better quality, but slower generation
Fewer refinement steps → higher speed, but greater risk of errors
Finding the optimal point will be one of the important areas of future research.
Multimodal limitations also remain
DiffusionGemma can also struggle with some multimodal tasks.
Google reported instances where the model failed to properly close its reasoning section, artificially hurting benchmark results.
This indicates that simply converting an existing language model into a diffusion architecture does not automatically preserve every capability perfectly.
More specialised training may be required for multimodal applications.
Why Google’s approach could change AI development
The biggest implication of DiffusionGemma is economic.
Training a foundation model from scratch can require enormous amounts of compute.
If an existing model can be converted using less than 10% of the original training-token budget, researchers could potentially explore new architectures at a fraction of the cost.
Traditional approach vs Google’s approach
| Development approach | Traditional | DiffusionGemma approach |
|---|---|---|
| Start with | New architecture | Existing Gemma 4 |
| Pretraining | From scratch | Reuse existing model |
| Additional training | Very large | <10% of original token budget |
| Main goal | Build foundation model | Convert generation architecture |
| Cost of experimentation | Very high | Potentially much lower |
| Research flexibility | Lower | Higher |
This could be particularly valuable for smaller research groups and startups that cannot afford to train huge foundation models.
Diffusion could create a new AI model-development path
The AI industry has historically focused heavily on scaling.
More parameters, more training data and more compute have driven much of the progress in large language models.
DiffusionGemma represents a different philosophy:
Instead of always making a new model, make better use of the models that already exist.
That could encourage researchers to experiment with alternative inference architectures without repeatedly paying the cost of foundation-model pretraining.
The approach could also allow specialised models to emerge around specific workloads.
Where DiffusionGemma could be useful
The technology is still experimental, but its characteristics point toward several potential applications.
| Application | Why diffusion could help |
|---|---|
| Code repair | Existing code constrains much of the output |
| JSON generation | Highly structured output |
| Editing | Earlier tokens can be reconsidered |
| Mathematical reasoning | Multiple parts of the answer interact |
| Sudoku and puzzles | Decisions depend on the complete solution |
| Real-time applications | High single-user generation speed |
| Speech applications | Fast text generation can reduce latency |
| Medical report generation | Structured outputs can benefit from refinement |
Google says DiffusionGemma is already being used by startup Interfaze for multilingual speech recognition and in research involving interactive radiology-report generation.
What this means for AI inference
The significance of DiffusionGemma extends beyond one model.
The AI industry is increasingly focused on inference costs because training costs are only one part of running modern AI systems.
Every time a user sends a prompt, the model must perform inference.
For AI companies operating at massive scale, improving inference efficiency can translate into substantial savings.
If diffusion models can deliver high-quality answers using fewer expensive sequential operations, they could eventually become an important alternative for selected workloads.
The challenge is proving that their speed advantage survives real-world workloads involving multiple users, long contexts and complex reasoning.
Google is positioning DiffusionGemma as an experiment, not a replacement
Google explicitly describes DiffusionGemma as an experimental model intended to accelerate research into text diffusion.
The company has made the model available under an Apache 2.0 licence on Hugging Face, allowing researchers and developers to experiment with the technology.
This open approach could be important because the next major improvements may come from researchers outside Google.
Developers can experiment with:
- Different sampling strategies
- More efficient refinement
- Larger diffusion models
- Domain-specific fine-tuning
- Code generation
- Multimodal diffusion
- Local inference
- Alternative hardware
The more experimentation the model receives, the easier it may become to identify where text diffusion actually provides a meaningful advantage.
The bigger picture: AI generation may become less sequential
The most important takeaway from DiffusionGemma is that text generation does not necessarily have to follow the same left-to-right process that has dominated language models.
Diffusion introduces another possibility:
Generate → evaluate → revise → generate again
rather than:
Generate token → generate token → generate token → finish
That distinction could become increasingly important as AI systems are asked to perform more complicated tasks.
For simple chat responses, autoregressive models may remain extremely effective.
For structured, constrained or highly interactive workloads, however, the ability to revise multiple parts of an answer simultaneously could prove valuable.
What happens next?
DiffusionGemma is still early-stage technology, and Google has not demonstrated that it can broadly outperform conventional language models.
The current evidence instead suggests that diffusion models have a promising combination of speed, parallel generation and built-in revision, but still face quality, reliability and scaling challenges.
The next generation of research will likely focus on closing the quality gap while preserving the speed advantage.
The biggest question will be whether future models designed specifically for diffusion can outperform retrofitted models such as DiffusionGemma.
If they can, the economics of AI inference could change considerably.
DiffusionGemma: key numbers
┌─────────────────────────────────────┐
│ DIFFUSIONGEMMA │
├─────────────────────────────────────┤
│ ~1,500 tokens/sec │
│ Nvidia H100 │
│ │
│ <10% │
│ of original training-token budget │
│ │
│ 256 tokens │
│ refined in parallel │
│ │
│ ~10 points │
│ reasoning benchmark improvement │
│ │
│ ~85% │
│ Sudoku accuracy after fine-tuning │
│ │
│ 2-3 steps │
│ for some structured outputs │
│ │
│ ~32 concurrent requests │
│ where conventional models catch up│
└─────────────────────────────────────┘
Conclusion
Google’s DiffusionGemma is important not because it has already replaced conventional language models, but because it demonstrates a potentially cheaper path toward a different kind of AI generation.
By converting an existing Gemma 4 model instead of training a diffusion model from scratch, Google showed that researchers can explore an alternative architecture with less than 10% of the original training-token budget.
The model’s roughly 1,500-token-per-second performance on an H100, ability to refine 256-token blocks in parallel and capacity to correct earlier decisions make text diffusion particularly interesting for structured and interactive applications.
However, the technology still has significant limitations. It trails the autoregressive Gemma 4 on some quality benchmarks, can enter repetition loops and loses much of its throughput advantage at higher concurrency.
The next major breakthrough will therefore not simply be making diffusion models faster. It will be making them as capable and reliable as conventional language models while retaining their parallel-generation advantage.
If researchers succeed, DiffusionGemma could mark the beginning of a broader shift in how AI models generate text — from systems that simply predict the next token to systems that can repeatedly rethink an entire answer before presenting it.
Get the day’s top stories in your inbox
One concise email. No spam, unsubscribe anytime.
