Things I’m thinking about

1. How do we train many agents to work together over long horizons?

   Most reinforcement learning still rewards one agent for what happens in the next few steps. The problems I care about look different: many agents, some cooperating and some competing, working toward something that only pays off hours or weeks later. That’s where credit assignment breaks down (whose choice, made when, actually mattered?) and where reward hacking gets easiest, because the checks you can run are much narrower than the outcome you want. My work on rewarding long-horizon code agents is one corner of this, and training agents to resist adversarial policies is another. I want to know what it takes for a group of agents to stay honest, keep learning from each other, and still get the long job done.

   Where I’d bet right now Intermediate checkpoints and random audits will matter more than better final rewards.

2. How do we build tools so anyone can do anything they want?

   The best tools shrink the distance between wanting to do something and being able to: spreadsheets let anyone model, phones let anyone film. AI and robots could close that distance for almost anything, from writing software to running an experiment to making a physical thing. The hard parts are keeping people in charge of what they’re making, having the tool understand what they mean rather than only what they said, and making it work the first time for someone who isn’t an expert. A few of my projects are small tries at this, like a 3D viewer that takes any object apart, a course on how language models are trained, and a webcam coach for sport technique.

   Where I’d bet right now The winners will understand what people mean, not add more buttons.

3. How do we build tools that help people escape inequity?

   Some of the most useful technology lowers the cost of something that used to take money, connections or the right zip code: a tutor, a coach, a clinic, a second opinion. AI could widen that access or concentrate it further. I keep asking what it takes to build tools that reach the people who need them most: cheap enough to run, simple enough to trust, and designed with them rather than for them. And how to tell whether a tool actually changed someone’s odds, not just looked helpful.

   Where I’d bet right now Usability, accessibility and likeability, not model quality, will decide who these tools reach.

4. What can brains teach robots about learning to move?

   Animals pick up new movements from a handful of tries, keep perceiving while they act, and correct themselves mid-motion. Most robot learning still leans on huge amounts of data and separate stages for seeing, planning and moving. Sensorimotor AI asks what changes if perception and action are learned together, as one loop. It’s the thread running through the Sensorimotor AI journal club at Berkeley’s Redwood Center and through my own work building brain–computer interfaces, such as decoding hand movement from motor-cortex spikes. I want to know which of the brain’s tricks (prediction, fast feedback, learning from little) actually carry over to agents, and which are just biology.

   Where I’d bet right now Prediction and fast feedback will carry over; most of the rest is biology.

5. Where do vision-language-action models meet physical intelligence?

   VLAs bring internet-scale pretraining to robot control: one model that sees, reads an instruction and outputs actions. What I keep wondering is how much of the hard part that covers (contact, balance, recovering when the world doesn’t match the plan) and how much still needs reinforcement learning, simulation and real-world experience. Training a robot dog in simulation that then hit its target about 60% of the time on hardware made that gap concrete for me. I’m curious where the two approaches meet: VLAs for breadth, RL for the last stretch of skill.

   Where I’d bet right now VLAs for breadth, then RL in simulation and the real world for the last stretch of skill.

← Back to the profile