Physical2RSI
GitHub

Toward Recursive Self‑Improvement
in Physical AI

Meng Luo1Zechen Bai1Shanqing Xu2Yanlin Li1Kaixin Li1Mengchen Qian2Weifeng Lu3Ran Zhang4Yingtian Zou3Ziyang Luo5Wei Chen2Hongzhan Lin1*

1NUS2HUST3SJTU4NTU5HKBU

* Corresponding author

Paper (arXiv) Read the summary Reference library BibTeX GitHub
A painting in the style of Van Gogh: under a swirling starry sky, a small robot practises grasping a mug, files the change it kept, rewrites its own learning recipe, and climbs a step holding the improved recipe, while a golden trail loops back to the start.

The survey

Can a robot improve the way it improves?

01Overview

From improving behavior to improving the improver

Recursive self-improvement (RSI) describes a system that improves not only its behavior but also the process by which it improves. Recent digital agents test this idea by rewriting their own code and checking each version with executable tests. Bringing these mechanisms into the physical world requires learning from partial observations through costly trials, while accounting for changing operating conditions and potentially irreversible consequences.

The survey positions RSI within robot learning and traces pathways from improving robot capabilities to improving the processes that generate them. It examines how physical experience can strengthen these improvement mechanisms, and when the resulting gains can support further self-improvement of the mechanisms themselves.

Figure 1. From digital to physical recursive self-improvement.
Figure 1. From digital to physical recursive self-improvement. Numbers mark where experience leaves a change: policy (1), world model (2), and external knowledge (3) improve what the robot does; the learning procedure (4) changes how it learns; and the improver, drawn as the loop around the procedure, revises it. RSI further requires that repair knowledge improve the improver and that the improver, once improved, help produce its own next improvement.
Figure 2. Improving what the robot does and how it learns.
Figure 2. Improving what the robot does and how it learns. On the left, the learner improves under a fixed recipe and retains the change in its policy, world model, or external knowledge. On the right, the improver revises the learning procedure using retained repair knowledge; the dashed return asks whether that knowledge improves how the next revision is made (R1), and whether the improver, once improved, then helps produce its own next improvement (R2). The bottom row lists the physical limits behind the four conditions that let gains carry forward.

02Research questions

What the survey asks

  1. RQ1What limits a robot's next improvement?

    Is useful experience missing, or is progress limited by supervision, optimization, action generation, interfaces, or retention?

  2. RQ2Which changes address that limitation?

    How does experience change behavior, reusable models or skills, and the collection, supervision, curriculum, or update procedures used in later learning?

  3. RQ3When can improvements build on one another?

    When do improvements carry forward into better learning procedures and improvers, and what evidence would establish recursive self-improvement?

  4. RQ4What makes self-improvement different in the physical world?

    How do sensing, embodiment, reset, safety, cost, and hardware changes constrain the process?

03The review at a glance

Four uses, four conditions, four claims

Figure 3. The review at a glance.
Figure 3. The review at a glance. Illustrations in (b) show the four grasp breakdowns from Section 1.

What current evidence shows. Existing studies, concentrated in manipulation, support behavioral improvement, later-learning benefits, and procedure improvement. Some systems also change their improver's knowledge or tools, but no physical study reviewed here has yet isolated a gain attributable to recursion, which requires a controlled comparison against a version of the improver whose acquired knowledge is frozen.

Figure 8. Does a revised component help the next learner?
Figure 8. Does a revised component help the next learner? Two matched learners differ only in the component under test, here an evaluator. Their common outcome check stays fixed and separate from that evaluator; comparing the learners' outcomes tests procedure improvement.

04Four uses of retained change

Where a retained change goes next

§3.1.1 Policy updates

Behavior

Retained change
Policy, value, residual
Immediate use
Execute or choose actions
Principal claim
Better task behavior

§3.1.2 World models

Prediction and practice

Retained change
Dynamics or world model
Immediate use
Plan or supply training experience
Principal claim
Better decisions or learning

§3.1.3 Memories, skills, code

Reuse and recovery

Retained change
Memory, skill, contract, program
Immediate use
Retrieve, recover, or synthesize
Principal claim
Reusable execution knowledge

§3.2 Learning procedures

Learning procedure

Retained change
Collector, evaluator, editor, updater
Immediate use
Choose experience, judge, or construct updates
Principal claim
Better subsequent learning
Figure 4. Where a retained change is kept and how it is used.
Figure 4. Where a retained change is kept and how it is used. Each row follows the grasp example from physical experience to the retained change and its immediate use; the italic line names the use. Dashed tags mark secondary handoffs into the learning procedure.
Figure 5. Four teaching signals from physical experience.
Figure 5. Four teaching signals from physical experience. Each column follows the grasp example: what was experienced, what the signal changes in the training data, and representative systems from this section.
Figure 7. Three forms of external knowledge in the grasp example.
Figure 7. Three forms of external knowledge in the grasp example. Each panel shows one saved form of the slip repair and representative systems from this section; reuse on another robot first checks the interface.

Four systems where models, policies, and physical data change

SILVR

Collect on the real robot, then revise the planner

RISE

Improve a policy inside fixed learned models

PlayWorld

Acquire play data, fit a model, then optimize a noise policy

VLAW

Update the world model, then the policy; repeat with new physical data

Figure 6. Where models, policies, and physical data change.
Figure 6. Where models, policies, and physical data change. Learning loops of SILVR, RISE, PlayWorld, and VLAW.

05Toward recursive self-improvement

Testing the two links of recursion

Figure 9. A proposed test of the two links of recursive self-improvement (R1 and R2).
Figure 9. A proposed test of the two links of recursive self-improvement (R1 and R2). In R2, the updated improver revises itself with frozen or acquired knowledge, producing the control or next improver, respectively. Each then revises a learning procedure, which is tested through fresh matched learners under a common, protected outcome check; the next improver must beat both the control and the updated improver; for the updated improver, its learner from R1 serves as the baseline. The R2 control assumes that the acquired knowledge can be withheld without changing the parent improver, as with a memory snapshot; the count of four hardware-trained learners per seed assumes that one candidate per improver survives screening.
  • The first three claims already have support. Existing studies show behavioral improvement, later-learning benefit, and procedure improvement; some systems also change their improver's knowledge or tools. For R1 and R2, the survey distinguishes mechanism only, uncontrolled results, and controlled results; evidence for an earlier claim does not establish a later one.
  • R1: repair knowledge must improve the improver. Give the improver, with its earlier and with its acquired repair knowledge, the same new failure histories, tools, objectives, and budget; apply the learning procedures each produces to matched learners and compare common physical outcomes.
  • R2: the improved improver must help improve itself. Given common new repair histories, let the updated improver propose a revision of itself and compare the result both with the unrevised improver and with a control revision of the same improver proposed with its earlier knowledge frozen; repeating the first two tests on the resulting improvers links further gains to what was learned.
  • Where to start. Reward construction and skill-library editing yield inspectable revisions that simulation can screen, leaving four hardware-trained learners per seed when one candidate per improver survives. Eureka (a reward improver whose knowledge does not change) and PRACTICE (a learned skill editor whose own improver is fixed) already supply baselines.

06Open problems

Six directions to start from

Keeping judges aligned

Robustness to distribution shift

Refresh the judge, keep the task

Starting benchmarkVary only the evaluator; protected outcomes

Surviving system changes

UsabilityRobustness to distribution shift

Reuse, adapt, revalidate, or retire?

Starting benchmarkDeclared changes plus gradual drift

Budgeting physical resources

Continued practice

Spending robot time, attention, compute, wear

Starting benchmarkTask sequences with upkeep; Pareto reports

Finding the bottleneck

Informative feedback

Which component limits improvement?

Starting benchmarkInjected faults; held-out natural failures

Choosing the next experiment

Informative feedback

Which probe yields usable evidence?

Starting benchmarkJoint vs fixed, information-gain, sequential

Transferring repair knowledge

UsabilityRobustness to distribution shift

Do repair lessons transfer to new failures?

Starting benchmarkTwo-round test (Figure 9)

Three near-term programs

  1. Combine cross-component diagnosis with joint experiment–update selection under matched multi-resource budgets.
  2. Test versioned evaluators and repair knowledge on held-out failures and declared hardware or software changes, using protected outcome checks to separate transfer from drift.
  3. Connect these tests across at least two causally isolated procedure-revision rounds and report their full acquisition, maintenance, and revalidation costs.
Physical recursive self-improvement will be demonstrated not when a robot changes many times, but when knowledge of why one repair worked causally improves how the next repair is made, and when that improved capability, applied to new evidence, helps improve itself again without losing the ability to keep learning.

07Citation

Cite the survey

@article{luo2026physicalrsi,
  title   = {Toward Recursive Self-Improvement in Physical AI},
  author  = {Luo, Meng and Bai, Zechen and Xu, Shanqing and Li, Yanlin and Li, Kaixin and Qian, Mengchen and Lu, Weifeng and Zhang, Ran and Zou, Yingtian and Luo, Ziyang and Chen, Wei and Lin, Hongzhan},
  journal = {arXiv preprint arXiv:2610.29677},
  year    = {2026},
  url     = {https://arxiv.org/abs/2610.29677}
}

The literature

Reference library.

229
references
1966 – 2026

Loading references…