For residual learning that refines existing behavior, sample efficiency depends on two things: how much information each rollout returns, and how efficiently the learner uses that information. Reinforcement learning's standard scalar reward carries far less information than the directional task error that defines the task. Random exploration further discards whatever information each rollout returns.
Through residual learning with directional task-error supervision and a task error model that drives sample selection, we achieve stable three-, four-, and five-ball juggling on anthropomorphic Barrett WAM arms. Despite planning and controlling through a deliberately simple stack with idealized assumptions, the system converges from the second attempt: the first attempt drops, after which task error decreases monotonically without further failures. In comparison, five-ball juggling typically takes humans years of practice.
We compare residual learners across two ternary axes, the directional information in the learning feedback and the commitment of the analytic prior, spanning Newton-style Jacobian updates, Composite Bayesian Optimization, and stochastic search methods. Both axes prove necessary for fast convergence: neither directional feedback nor an informative prior alone matches their combination, and the simplest method that combines them, a fixed-Jacobian Newton update, is the fastest and fully reliable. The learned residual tolerates substantial prior misalignment and degraded joint tracking, affecting mainly convergence speed. The bottleneck for residual learning on real robots is therefore the information content of the supervision signal and how the learner uses it, not the accuracy of the surrounding stack.
Fast, reliable convergence needs both directional feedback and an informative prior. Evaluated in simulation at the five-ball cascade across the full feedback × prior-specificity matrix. Bars give attempts to the first success over 20 throws (solid) and to a ten-in-a-row streak (light; reached by X/20 seeds). Mean ± s.d. over 20 seeds spanning 10 randomized plants. The Fixed Jacobian Newton update is the simplest method of all and performs best.
Residual learning through Fixed Jacobian Newton updates on a directional task-error signal for three-, four-, and five-ball juggling. Three balls work on the first attempt. Four and five on the second. Each panel in the video shows ten consecutive juggling attempts of the same learning run. Each attempt is capped at 120 throws.
The deliberately simple control stack consists of a PD tracking controller with a CAD derived feedforward torque around a planned throw trajectory. The choice of control gains mostly affect convergence speed, not final performance. Five ball juggling succeeds at surprisingly low gains. Nominal values are Kp = [400, 400, 200, 200] and Kd = [40, 20, 15, 5]. Each attempt is capped at 10 throws.
The analytic Jacobian is rotated from 0° to 90° around four fixed random axes, one per seed. Rotations up to 30° leave convergence speed unchanged, and even 60° still converges. Each setting is run over four seeds of ten attempts each, with each attempt capped at 10 throws.
@unpublished{ploeger2026residual,
title = {Task-Error Residual Learning for Real-Robot Five-Ball Juggling},
author = {Ploeger, Kai and Kshirsagar, Alap and Peters, Jan},
year = {2026},
note = {Under submission},
}