Summary
The paper introduces an Exploratory Retrieval-Augmented Planning (ExRAP) framework, that tackles the problem of embodied planning in dynamic environments. The study focuses on continual instruction following, which is when the task consists of multiple conditional subtasks. The execution of each subtask is dependent on the environment state. This requires continual exploration to keep up-to-date information and integrated planning abilities for efficiency.
ExRAP combines Large Language Models (LLMs) with environmental context memory represented as a temporal embodied knowledge graph (TEKG), that captures the states of objects in the scene and their pairwise relationships. The TEKG is updated at each step using the most recent observation and an update function $\mu$. Each subtask is translated into a pair of a query about the scene and a corresponding execution command with an instruction interpreter $\Phi_l$. Next, the memory-augmented query evaluator $\Phi_{M}}$ estimates the likelihood of the query being satisfied, using quadruples retrieved by $\Phi_R$ as environmental context information for the LLM. Authors enforce a temporal-consistency constraint on the evaluator, resulting in a continual steady decrease of the entropy of the answer. The execution commands corresponding to the satisfied queries are then passed to the execution part of the model. This part consists of exploitation and exploration planners. The former assesses the effectiveness of the skill in accomplishing the execution task. The latter one does the same with regard to reducing the uncertainty of the query evaluator $\Phi_M$. The performed skill is chosen as the one providing the maximum weighted sum of the two scores, balancing exploration and exploitation.
In the experiments, the authors measure the Success Rate (SR) of the task completion and the Pending Step (PS), representing the average number of steps taken to complete the task. The evaluation is performed on the VirtualHome, ALFRED, and CARLA environments. Performance in various degrees of stationarity in the environment is assessed, showing the advantage of the method. The performance with respect to the number of instructions is also evaluated. The method seems to excel in integrated planning, effectively solving multiple tasks with fewer steps. Ablation studies demonstrate the importance of temporal consistency and exploration-integrated planning. Intrestingly, the study reveals a limited effect of choosing a larger LLM as a base model.
Strengths
- To the best of my knowledge, one of the first works to tackle conditional embodied planning in a dynamically changing environment.
- The results of the paper suggest that the method is clearly excels in producing more effective plans than the baselines, integrating multiple tasks in parallel. The plans are being adapted on the fly from the belief about the environment, and the belief is updated by neccessary exploration. This is a valuable contribution.
Weaknesses
- The paper shows that the method is quite robust to the size of the base LLM. It could be a strength if there would be some training procedure possible. However, ExRAP relies exclusively on the in-context learning abilities of the LLM, thus the experiment shows that it scales poorly. The paper would benefit from an ablation study on the size of the retrieval dataset, so it can be shown that the method can scale with the context length of the model by increasing the number of examples.
- The method excels at following multiple simple parallel conditional instructions, but it is not clear if it will be as good at the long-term tasks that require multiple steps to achieve, like the ones on which the LLM-Planner was tested. The advantage of having an adaptable planner may become a disadvantage against the baselines, as it may require more queries to the environment context memory, which will create a large computational overhead. It is unclear whether the method is better than the baselines in integrating multiple complex tasks, each of which requires multiple steps, into an optimal plan. Unfortunately, this is not investigated in the paper.
- The applicability to the real-world setting is questionable and not investigated. Would the method scale appropriately with the number of objects in the scene? Methods like ConceptGraphs do not show robustness in that case.
- The termin "continual embodied instruction following" is a little misleading, as in the paper, the instructions are passed all at once, and not at random times, which would require replanning. The "conditional embodied instruction following in the dynamic environment" could be a better name, though this is not a significant issue.
Questions
- Are the environment settings used in the in-context learning retrieval dataset the same or different from the ones used for the test?
- What does the observation $o_t$ consist of for ExRAP? Is it a list of quadruples? How would the setup be adapted for the real-world case?
- How does the system handles contradicting instructions? Like “If there is a storm outside, turn off the electricity in the house. If the TV says that the storm is over, turn the electricity on.”
- To me, it is not clear why the temporal consistency works. Would a simple baseline of reducing the temperature of prediction perform as good as the temporal consistency constraint in the paper?
- It would be interesting to investigate the exploration dynamics of the agent. Does the exploration scores on average decrease steadily throughout the experience or more rapidly in some particular times?
Limitations
- Runtime overhead not investigated, but mentioned.
- Applicability to the real world is not addressed.