D&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models

Try it
Marius Memmel1,2, Shawn Yin1,*, Urvi Rutia1,*, Angad Wadhwa1,*, Eldon Li1, Gavin Angeloff1, Ankit Goyal2, Jesse Zhang1, Dieter Fox1, Fabio Ramos2, Anqi Li2,†, Abhishek Gupta1,†
1University of Washington 2NVIDIA

*equal contribution   †equal advising

Paper arXiv Code D&A Bench Data

General Reward Model

D&A inherits the generality of frontier models.

Hover clip to play it · press and slide to scrub through the rollout · load in demo
⚠️ The above is representative of D&A performance (GPT-6 Astra). ⚠️
We used the first language prompt we could come up with. Zero tuning!

Easy to Use

No training, no local GPU, just two prompts, and an API.

from dna import DNA

dna = DNA(model="openrouter/openai/gpt-6-astra")
progress = dna.compute_progress(video, "assemble the car")

Code here.

Method Overview

Describe what happened, then assesses progress against explicit success criteria.

Method overview: a Describer narrates the video without seeing the instruction, then an Assessor scores the description against a success criterion to produce per-frame rewards and feedback.

    key frame 1 / 15 Open in demo

    1Problem

    Existing zero-shot VLM approaches (GVL, TOPReward) suffer from confirmation bias and hallucinated progress in failed runs when estimating rewards.
    Post-trained alternatives (Robometer, RoboReward, SOLE-R1) require expensive real-world data.

    2Method

    To address this, D&A decouples progress estimation into a Describe and Assess stage, first describing what happened in the video before assessing progress with respect to a success criterion.

    3Results

    D&A Enables Real-World Reinforcement Learning

    D&A Achieves SOTA on 3 Reward Benchmarks

    D&A Scales with 4 Model Families and 28 Models

    D&A Predicts Rich Language Feedback

    Visual Nuances and Failure Explanations

    D&A describes what happened, pinpoints the failure, and says what to do instead.

    “”
    Video
    key frame – / –
    Progress — All Models
    Current State — Grounded Description (D&A)
    D&A Analysis
    failure
    feedback
    success criterion

    D&A-Bench consists of 54 real rollouts across 14 tasks (27 success, 27 failure). Benchmark results here.

    Enables Real-World Reinforcement Learning

    D&A rewards drive online and offline real-world RL.

    Advantage-Filtered Behavior Cloning Evaluation

    “nudge out the book ‘Thinking, Fast and Slow’, grasp it, and place it on the table”bimanual YAM

    20×
    π0.5 behavior cloning 80% success. Competitive, but imitates suboptimal demonstrations, e.g., nudging and pulling out the wrong books.
    20×
    TOPReward 25% success. Assigns similar reward to optimal and suboptimal behavior, filtering action chunks from both.
    20×
    Robometer 50% success. Likewise assigns similar reward to optimal and suboptimal behavior.
    20×
    D&A 95% success. Filters out suboptimal chunks and keeps optimal ones. Higher success rate and execution speed.

    We fine-tune π0.5 on the top 35% of action chunks, ranked by D&A's progress advantage against a single dataset-wide threshold, using 50 optimal and 100 suboptimal demonstrations. On success, D&A finishes the task faster than the baseline (18.8 ± 1.8 s versus 28.5 ± 9.2 s), with markedly lower variance across rollouts.

    Offline RL Evaluation (RECAP)

    “move the red book from the top shelf to the gray box”bimanual YAM

    10×
    π0.5 behavior cloning 20% success. Clones suboptimal behaviors, e.g., misses grasps or drops the book.
    10×
    TOPReward 80% success. Noisy filtering of suboptimal data, worse than Robometer.
    10×
    Robometer 85% success. Noisy filtering of suboptimal data.
    10×
    D&A 100% success. Filters the data correctly.

    We condition π0.5 on advantages computed from D&A rewards (CFGRL/RECAP) and LoRA-fine-tune it for 50k steps on 50 optimal, 50 suboptimal and 50 failure demonstrations. Failures include using the wrong box or arm, or dropping the book mid-transfer.

    Online RL Evaluation (DSRL)

    “put the fruit in the bowl”DROID Franka

    10×
    π0 baseline Before RL — 10% success. Sometimes picks the banana, but usually grasps the ice cream cone.
    10×
    TOPReward 5% success. Assigns high reward to the ice cream cone (wrong object) and largely ignores the banana, which is underrepresented in its training buffer.
    10×
    D&A 100% success. Assigns zero reward to the ice cream cone, so the policy converges to consistently picking the banana.

    We apply DSRL to a π0 policy fine-tuned on DROID, using D&A's progress score directly as the dense reward. D&A raises the success rate from 10% to 100% in about 90 minutes of wallclock time, while TOPReward learns the opposite behavior.

    Online RL Training (DSRL)

    “put the fruit in the bowl”DROID Franka

    100×
    TOPReward Rewards both successful and failed trajectories alike
    100×
    D&A Zero reward for the wrong object (ice cream cone), dense reward for the correct one (banana)

    Better VLM, Better Reward Model

    Swap in a stronger VLM and D&A gets better with it.

      Reward quality of D&A (Kendall τα on RBM-EVAL-OOD, higher is better), same prompts for every model.

      How much does D&A gain over naive frontier models? Find out here.
      More models and scaling laws here.

      Predicts Rich Language Feedback

      D&A tells Code-as-Policies what went wrong and what to change.

      Allowing Code-as-Policies (CaP) to iterate over its solutions improves performance from 11.4% (no history) to 40.6% (history). Adding D&A's rich language feedback achieves 51.4%, showcasing how rich semantic feedback provides a more effective signal than scalar rewards for downstream refinement.

      11.4%
      No history
      40.6%
      History
      51.4%
      + D&A feedback

      Pick a case, then move the attempt slider: both planners update to the same attempt.

      Baseline — Code + Binary Success

      No feedback signal — the planner only sees its own past code and success/failure.

      + D&A Feedback

      analysis
      failure
      feedback
      attempt – / –

      More Results

      Benchmarks, confusion matrices, scaling laws, and frontier models.

      Generalization Across Tasks, Embodiments, and Benchmarks

      Scaling up the base VLM, D&A* (GPT-6 Astra) tops every benchmark.

      Click a model to hide or show it in all three charts.

      D&A-Bench Kendall τα · higher is better ↑
      RBM-EVAL-OOD Kendall τα · higher is better ↑
      RoboRewardBenchmark MAE · lower is better ↓

      D&A (Gemini 3 Flash) emerges as the strongest zero-shot reward model across three offline benchmarks, outperforming zero-shot baselines and matching or surpassing post-trained reward models on tasks, embodiments, and language it has never seen.

      Confusion Matrices

      D&A achieves clear separation indicated by a clean diagonal while GVL and TOPReward largely fail to distinguish the pairs.

      Video-language reward confusion matrices on RBM-EVAL-OOD for each reward model
      Video-language reward confusion matrices on D&A-Bench for each reward model
      Video–language reward confusion matrices over all trajectory/instruction pairings from RBM-EVAL-OOD (left) and D&A-Bench (right). A clear diagonal indicates assignment of high rewards to the correct pairings. While strong color on the off-diagonal indicates poor separation of video–language pairs. We further report column-normalized diagonal mean (fraction of reward on correct matches).

      D&A Scaling Laws

      Drag a box on the plot to zoom · hover a point for details · click a chip to hide or show a model family Tap a point for details · tap a chip to hide or show a model family

      View underlying data
      Model Family Cost ($ / 1M output tok) Kendall τα (final step)

      D&A Scales to Frontier Models

      D&A improves on D&A-Bench for every frontier model we tested, and on RBM-EVAL-OOD for two of three.

      RBM-EVAL-OOD

      Kendall τα · higher is better ↑

      Qwen 3.6 Plus 0.349 → 0.583 +0.234
      GPT-6 Astra 0.671 → 0.714 +0.043
      Gemini 4 Argon* 0.734 → 0.724 −0.010

      * suspected to train on RBM-EVAL-OOD

      D&A-Bench

      Kendall τα · higher is better ↑

      Qwen 3.6 Plus 0.597 → 0.753 +0.156
      GPT-6 Astra 0.768 → 0.790 +0.022
      Gemini 4 Argon 0.778 → 0.853 +0.075

      Kendall τα with the same VLM prompted naively (teal) versus with D&A (pink), for Qwen 3.6 Plus, GPT-6 Astra and Gemini 4 Argon. The number on the right of each row is the change from Naive to D&A. Gemini 4 Argon is the one case where D&A is slightly lower (−0.010 on RBM-EVAL-OOD*).

      BibTeX

      @misc{memmel2026dna,
  title={D\&A: Decoupling Description and Assessment Enables VLMs as Zero-Shot Robotic Reward Models},
  author={Memmel, Marius and Yin, Shawn and Rutia, Urvi and Wadhwa, Angad and Li, Eldon and Angeloff, Gavin and Goyal, Ankit and Zhang, Jesse and Fox, Dieter and Ramos, Fabio and Li, Anqi and Gupta, Abhishek},
  year={2026},
  note={Preprint}
}