Robots capable of rearranging objects in cluttered
and confined spaces have numerous real-world applications,
such as in retail, logistics, and household tasks. However,
environmental constraints and visual occlusions make it
difficult to predict the effects of robot-environment interactions,
presenting a significant challenge for rearrangement planning.
Existing solutions rely on simplified assumptions, such as
full observability, and typically use collision-free, single-object
prehensile manipulation strategies that are less effective in
partially observable settings. To address these limitations, this
paper proposes a data-driven approach that leverages deep
reinforcement learning to learn a rearrangement policy that
combines two actions: pushing and pick-and-place. Specifically,
the approach uses fully convolutional networks and Q-learning
to make dense pixel-wise predictions of expected rewards for the
two actions from side-views of the cluttered and confined space.
At each step, the learned policy executes the action with the
highest Q-value given the current observation. The proposed
method is evaluated in simulation, where a learned policy is
rolled out to rearrange up to 14 objects inside a confined
cabinet. Results show that the approach achieves an average
success rate improvement of 31.7% compared to baselines.
Overall, this work demonstrates the efficacy of a data-driven
approach to enable robots to effectively rearrange objects in
complex, cluttered environments.
