Dory 1.0: the small model that loops its layers

Transcript

The original narration, preserved as published. Relative dates refer to the episode’s coverage window.

A small model can take another pass without growing another set of weights. NVIDIA lists October sixteenth as the release date for Dory one point zero: a four billion parameter text model built around a reusable internal block.

Input layers prepare the information. A shared block runs repeatedly, then output layers produce the answer. Changing the loop count changes computation per token. The checkpoint stays the same; extra reasoning does not require downloading a larger model.

NVIDIA recommends starting with two loops. More loops can improve some tasks, but do not improve every score. Its results are vendor measurements, not our test. Fewer output tokens also do not automatically mean lower end to end latency.

The model offers two cache modes. The usual mode keeps a separate cache for each loop. Last loop caching keeps only the final pass, reducing that memory overhead. This trades how the model stores intermediate attention information.

Weights are downloadable under Open M D W one point one, including a quantized format. Runtime support still needs special builds: the llama dot C P P pull request is a draft. Test compatibility, accuracy and latency before deploying it.