A Hierarchical Bin Packing Framework with Dual Manipulators via Heuristic Search and Deep Reinforcement Learning

We address the bin packing problem (BPP), which aims to maximize bin utilization when packing a variety of items. The offline problem, where the complete information about the item set and their sizes is known in advance, is proven to be NP-hard. The semi-online and online variants are even more challenging, as full information about incoming items is unavailable. While existing methods have tackled both 2D and 3D BPPs, the 2D BPP remains underexplored in terms of fully maximizing utilization. We propose a hierarchical approach for solving the 2D online and semi-online BPP by combining deep reinforcement learning (RL) with heuristic search. The heuristic search selects which item to pack or unpack, determines the packing order, and chooses the orientation of each item, while the RL agent decides the precise position within the bin. Our method is capable of handling diverse scenarios, including repacking, varying levels of item information, differing numbers of accessible items, and coordination of dual manipulators. Experimental results demonstrate that our approach achieves near-optimal utilization across various practical scenarios, largely due to its repacking capability. In addition, the algorithm is evaluated in a physics-based simulation environment, where execution time is measured to assess its real-world performance. Note to Practitioners—Bin packing is a key task in logistics and manufacturing, where items must be packed tightly into limited space within a short amount of time. Traditional systems typically use a single robot arm and pack items in the order they arrive, without the ability to rearrange previous placements. This limits space use and makes it hard to improve the result once early placements are made. This work introduces a packing system that incorporates a repacking strategy, allowing previously placed items to be moved when necessary to improve space usage. The system also supports dual-arm operation, where two robot arms work in parallel to reduce overall task time. Items are assigned based on reachability, and placement decisions are made using a combination of predefined rules and AI. The system is intended for environments such as automated warehouses and production lines, where items arrive one by one and must be packed accurately in real time. It was tested in simulation using UR5e robots and open-source planning tools, demonstrating effective use of bin space and faster task execution with the dual-arm setup compared to the single-arm setup.