Research
The question running through my research is: how should models learn from their own generated computation (rollouts, search traces, latent futures) while preserving distributional coverage and knowing when their internal representation is sufficient for the decision at hand? Underneath it is a formulation question. The choice of training formulation shapes what a model can represent, where it fails, and how detectable those failures are. I think of this as representational/model competence under distribution shift.
I am interested in this as one system with two layers: a diffusion world action model (WAM) that predicts futures and produces continuous actions, and an agentic planning layer that searches, verifies, and decides. The same loop runs at both layers:
- Search creates supervision. Rollouts, imagined futures, verifiers, and search traces produce behavior better than the model’s default.
- Distribution-aware correction internalizes it without collapse. The model trains on the states it actually induces, while mixing in teacher and reference states to preserve diversity and coverage.
- Competence gates it. Uncertainty estimates decide where to spend search, which states deserve a teacher query, and when to replan, query a simulator, or defer.
1. Diffusion Models and Sequential Decision Making
Origin: diffusion distillation as learning from rollouts. I treated diffusion distillation as an imitation-learning problem: the student is a policy, and its denoising path is a rollout. Seen this way, a distilled student has two separate problems. The first is exposure bias. Standard distillation trains the student only on teacher-induced states, but at inference it conditions on its own previous outputs, so errors compound across denoising steps from states it never learned to recover from. The DAgger-style answer is to train on student-induced states. Depending on trainign objective and distillation framework there could be coverage collapse: training narrows toward the modes the student already produces, intermediate marginals drift from the teacher’s, and diversity collapses.On-policy methods such as DMD2 run into this. DDIL solves both by making the training distribution itself distribution-aware. It trains on a mixture of student-induced and teacher-induced states, which corrects the states the student actually visits while preserving the teacher’s intermediate marginals and coverage of the data manifold.
Shipped: the world’s first & fastest on-device text-to-image generation. I first applied DDIL to progressive distillation, where it helped make sub-0.6s Stable Diffusion on Snapdragon possible. As ML lead in the cross-functional Qualcomm team, I built the training pipeline for that deployment, including the 50TB data pipeline and the manifold-thresholding fix for diversity collapse. I later extended DDIL to on-policy distillation/DMD2. The deployment was featured at MWC’23 and Snapdragon Summit and covered by The Verge and Engadget.
The same two problems reappear in world action models, compounded. A WAM is a diffusion model whose rollouts are not only denoising paths but action and latent-future trajectories. It conditions on its own predicted futures and on the consequences of its own actions, so exposure bias now compounds over two horizons. Post-training it on its own rollouts risks the same collapse, with a higher cost: for a robot, lost coverage means fewer recovery behaviors and imagined futures that miss the failure modes a planner needs to see. DDIL’s recipe carries over directly: correct the model on its own induced states, and keep the training mixture anchored to demonstration, teacher, and pretrained-prior states.
Diffusion RL and multi-task, multi-reward post-training. The open problems I’m focused on:
- Stability of RL over the denoising trajectory under sparse or partial task reward.
- Credit assignment across two horizons at once: denoising steps and control steps.
- Coverage across tasks, so that multi-task on-policy distillation and reward optimization don’t collapse the diversity of the pretrained prior that made it useful.
Efficiency as retention. On-device robots make compute budgets concrete. The metric I care about is how much trajectory quality (recoverability, diversity, self-correction) survives when the model is distilled or quantized. Compressing an iterative model changes the states it visits, which is itself a new distribution shift.
2. Agentic Post-Training: Planning & Search over World Models
From open-loop to feedback. Vision-language-action models are strong feedforward planners with native language reasoning, but they are largely open-loop. Video and diffusion world models act as model-based feedback controllers with physics-aware priors that feedforward reasoning lacks. The key open question is the interface between them: the representation, rollout-correction, and monitoring layer that connects high-level reasoning to closed-loop control.
Search over imagined futures, amortized. A world model makes search possible at the planning layer: sample candidate plans, roll them out in latent space, and score them. Post-training then pulls the results of that search back into the policy. That brings back the familiar risks of exposure bias, verifier overfitting, reward hacking, and coverage collapse, now paired with an action-level decision: replan, query a simulator, switch controllers, or defer.
LLM/VLM reasoning post-training as a testbed
Reasoning post-training is the faster-feedback version of the same loop. A model samples solutions, a verifier or process signal scores them, and on-policy distillation or RL pushes the model toward successful traces. I study the mechanism, not only the reward: how distribution-aware correction changes training-signal variance, teacher/student overlap, credit localization, and coverage, and whether the model has internalized useful search or is imitating scaffold artifacts. All of this is evaluated against matched controls across the OPD, RL, and post-training design space.
3. Competence Estimation & Uncertainty
What a representation preserves bounds what any monitor can know. My grad school work at Oregon State (advisors: Tom Dietterich, Alan Fern) studied open-set recognition and failure detectability under distribution shift. The core lesson: a confidence head, verifier, or selective-prediction rule cannot recover competence-relevant information the representation has already discarded. Pairing discriminative objectives with generative priors raised the detectability ceiling because it preserved more of the data manifold. For embodied agents, a value model or monitor is only as good as the world-model latent it reads from.
In robotics I treat uncertainty as a working part of the loop, with two concrete jobs:
UQ as an acquisition function for sample-efficient fine-tuning. Uncertainty decides which states or episodes are worth a teacher query, a human correction, or a gradient update, concentrating supervision at the frontier of failure (the DAgger view). The test: UQ-gated querying should reach a target success rate on a smaller query and rollout budget than uniform or outcome-gated selection.
UQ as the gate on agentic search. Uncertainty decides how much compute to spend, which branches to prune, and when to escalate from the world model to a simulator, the real system, or a human. The test: UQ-gated search should beat uniform search at matched compute.
Risk-sensitive RL and constrained MDPs. A concrete instance of competence as a gate: incorporating uncertainty into policy learning through constraint-based methods that penalize acting where the world model is unreliable. This is the policy-learning face of knowing when not to trust a rollout.
Online sufficiency. Can competence be assessed online, from intermediate computation (world-model latents, partial plans, feature trajectories), early enough to replan, resample, or abstain before the action fails? The connecting idea: competence is upstream of the head, and online sufficiency is the runtime instrument that closes the loop.
Search creates supervision; distribution-aware correction turns it into capability without collapse; competence estimates when that capability applies; control uses that estimate to act, replan, or defer reliably under shift.
Last updated: September 2026