Enhancing Robot Program Synthesis Through Environmental Context

Summary

This paper introduces the Environmental-context Validated lAtent Program Synthesis framework (EVAPS), which builds upon the SED approach by utilizing a trail-eval-repair loop to enhance program evolution and improve generalization capabilities. EVAPS leverages partial environmental observations by initially obtaining candidate programs from existing synthesizers. By executing these candidate programs and capturing the environmental context before and after each action, EVAPS models both the environmental and syntax contexts concurrently. Through iterative modifications to address semantic conflicts across program tokens, EVAPS aims to correct erroneous program fragments that do not produce the desired output. The proposed framework was evaluated in the partially observed Vizdoom domain, involving a robot navigating a 3D world and interacting with various elements, including objects and adversaries. The experiment results demonstrate the superiority of the approach over the baselines in accurately modeling program semantic subtleties and effectively resolving potential errors.

Strengths

+ EVAPS synthesizes robot programs operating in partially observed environments (surrounding RGBD image inputs). + EVAPS uses a trail-eval-repair loop to iteratively improve program generalizability by rectifying potentially erroneous code segments. + EVAPS is capable of handling noisy observations. A key novelty of EVAPS, when compared to previous program synthesis approaches in the Karel domain, is its ability to handle partial environmental observations. This capability is crucial for synthesizing robot programs, as relying on global environment information is often impractical in real-world robot navigation tasks. EVAPS consists of two essential modules: the partial observation leveraging module, which emphasizes the environmental context of specific program tokens, and the code symbol alignment module, which focuses on modeling the implicit connections between different code segments in semantic spaces. By integrating both syntax and semantic information, these modules facilitate program repair and enhance the effectiveness of EVAPS in generating accurate and meaningful robot programs.

Weaknesses

The title of the paper may appear to overstate the actual contribution made by the research. The choice of the Vizdoom environment as a benchmark for evaluating Robotic Program Synthesis in this paper raises some concerns. The evaluation of EVAPS in the navigation-focused Vizdoom environment does appear to have limitations. While efficient navigation is undoubtedly an important robotic task, it may not fully represent the diverse range of tasks in robotics, particularly those involving object manipulation in fully observable environments. It would be valuable to explore the applicability of EVAPS in synthesizing robot programs for other more realistic environments like Miniworld and AI2-Thor, which involve long-horizon vision-based tasks. Assessing the performance of EVAPS in such contexts would provide a more comprehensive evaluation of its capabilities. Given the current state of the paper, it remains uncertain whether EVAPS is adequately equipped to handle the challenges posed by complex robotic navigation tasks. To determine the true extent of EVAPS' applicability across a broader range of robotic tasks and environments, additional research and experimentation are necessary. Nevertheless, I do believe this paper contributes to the program synthesis community. The comparative evaluation of EVAPS against the baselines, when provided with partial observations, showcases its superior performance, thereby highlighting the significance of the observation leveraging module and the code symbol alignment module. However, there remains a lingering uncertainty regarding the practical implications of this demonstrated value in the context of robot control.

Questions

EVAPS seems incremental to SED. To gain a comprehensive understanding of EVAPS' contribution, it would be beneficial to apply the framework to the Karel domain, where the baselines were evaluated. This approach would effectively demonstrate the effectiveness of the observation leveraging module and the code symbol alignment module. Is there any evidence or indication from the authors regarding the performance of EVAPS specifically in the Karel domain?

Rating

3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The paper presents an approach for robot program synthesis. However, the evaluation falls short in providing a convincing argument on how EVAPS can be effectively applied to learn robot-control policies, as the experiments are confined to a relatively simple video game environment. Further investigation and experimentation in more complex and realistic robotic scenarios would be necessary to demonstrate the practical utility and effectiveness of EVAPS in learning robot-control policies.

Summary

The paper claims that global observation in robot program synthesis is not achievable, so it proposes to use partial observation. And it learns an observation embedding module and a semantic-grammatical alignment module to repair the candidate programs that can increase the accuracy and generalization of robot program synthesis.

Strengths

1. The problem this paper focused on is important to the application of PS as the perfect and complete observable environment is often not available in the real world. 2. The experiment setting and analysis are clear and comprehensive. The paper analyses the effect of multiple aspects factors in the in-perfect observation of the proposed method.

Weaknesses

[W1]. The key weakness is the choice of baseline methods. As a program repair method, it’s better to compare with methods that have a fair setting rather than program generation methods. There are other program repair methods that utilize trajectory to enhance performance. Moreover, the baseline methods are too old (most before 2020). [1] Execution-guided neural program synthesis [2] Write, execute, assess: Program synthesis with a REPL [2] Improving neural program synthesis with inferred execution traces [W2]. Needs more explanation about the difference between global environment information and partially observable environments. [W2.1] As using pre-observation and post-observation which are related to specific program segments to enhance the information contained in the embedding is a general trick. The PS methods which take global environment information as input can also learn such representation to enhance accuracy and generalization. What is the key connection between the proposed method and partially observable environments? [W2.2] Needs some insightful evaluation in the results section to demonstrate the relation between the main motivation (partial observable environment) and the proposed architecture. [W3]. There are some typos, like line 203, citation 41 is not the baseline

Questions

Please see weaknesses.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

2 fair

Presentation

1 poor

Contribution

2 fair

Limitations

N/A

Summary

This paper proposes EVAPS to enhance robotic program synthesis by integrating parietal environmental observations. Specifically, EVAPS utilizes both the environmental context leveraging module and the code symbol alignment module to improve its ability to rectify semantically erroneous program segments and generalize across various tasks. Comprehensive experiments on the partially observed Vizdoom benchmark demonstrate its superior performance over other baselines across various tasks. Overall, this is a well-written paper and its extensive experiments and ablation studies verify the effectiveness of the proposed method. I would be leaning to accept this paper.

Strengths

* The proposed method is technically sound and well motivated. * This paper is well written and well structured. It provides clear formulations and an overview figure to well explain the method. * The experimental evaluation is rather comprehensive and provides convincing results to demonstrate the effectiveness of the proposed method.

Weaknesses

I did not identify any weaknesses for this paper as I am not familiar with the task of robotic program synthesis.

Questions

Can you shed some light on the limitations of the proposed method?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

This paper just discussed the limitations very lightly. I would like to see more discussion on its limitations.

Summary

The paper proposes Environmental-context Validated lAtent Program Synthesis framework (EVAPS). a program synthesis model that generates executable program for robotic programming, evaluated in the vizdoom environment. It initially obtains candidate programs using other available synthesizers, then performs program repair by executing the candidate program and collect resulted partial environmental observations. It outperfoms a range of prior works for viszoom program synthesis, and demonstrated robustness again observation noises and task complexity.

Strengths

- using the aid of partially observed environments for program synthesis and repair is novel and reasonable - the proposed framework is sound, and the design choices for utilizing observations, and using a graph structure to aggregate environmental and syntactic information flow is convincing - extensive experiments

Weaknesses

- the writing of the paper can be improved, by reducing repetitive and inconsistent adjectives - do the baselines use program repair? Does any of them rely on environmental observations? - the assumption of executing candidate programs makes more sense when a privileged simulation environment is available, and when real-time control is not needed. Can the author provide further justification regarding this? How would you expect to transfer to real world? - the author claim that EVAPS shows is more robust agains noise in the conclusion, but there's no comparison with baseline on this matter in the experiment section.

Questions

see weeknesses.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

The major limitation i see in this framework is the assumption of aqcuiring partial observations and executing program candidates in the environment. These assumption is not valid in real-world real-time control. More justifications on this would be very helpful.

Summary

This paper proposes an approach for program synthesis in robotic domains where the environment context can provide valuable insight for what the correct program should be. The approach takes candidate programs, executes them to get a trace of observations, and then passes the program and trace through a combination of local and global feature extraction with neural networks. The output is a refined program. The approach is trained and tested on randomly generated Vizdoom programs and is substantially better than the alternatives. They also have an ablation study for the contribution of the two types of feature extraction and a study of the approach's noise tolerance.

Strengths

The results are good. The description of the approach is very clear and easy to understand. Some details are missing but overall it's clear. Vizdoom seems like a popular domain and their approach performs much better than the tested alternatives. The introduction does a great job of describing the intuition behind incorporating observation feedback into the program proposal. I don't have a lot to say about the paper. It is mostly a standard type of paper of applying a new, well-thought out approach to an existing domain. The approach seems sound and the results are good. The main insight is incorporating the environmental feedback to improve the program, which is similar to execution-guided synthesis.

Weaknesses

The paper doesn't have any glaring weaknesses. As usual, it could be strengthened by applying the approach to another benchmark. My main concern is that the gist of the idea seems very similar to work on execution-guided synthesis. But, the approach is shown to be much better than SED on the vizdoom benchmark, so there must be something more here. # Addressing rebuttal I have read the authors rebuttal. In particular, they satisfactorily address my main concern about similarity to other work, which I have no further concerns about.

Questions

Can SOTA approaches on Karel be applied to Vizdoom? If so, how do they compare? It seems like the intuitive reason this approach works is similar to the idea behind execution-guided synthesis work applied to Karel, so that could be a good comparison to include. Where do the program candidates come from before refinement? I don't recall the paper explaining this. This is a big confusion of mine. I would prefer to have the description of the Vizdoom domain in the evaluation section rather than the preliminary. A figure showing an example task would be helpful too.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

N/A

Thanks for your response

I appreciate the authors' response. However, I am still not convinced that the video game experiments sufficiently demonstrate the suitability of EVAPS for synthesizing robot programs in visually realistic navigation environments such as Miniworld and AI2-Thor, which entail navigating complex and partially observable long-horizon robot tasks. It would be advisable for the authors to assess the applicability of EVAPS across a wider spectrum of navigation tasks and environments. With that being mentioned, even if the title is narrowed down to navigation tasks, I would still argue that the evaluation does not provide support for it. Regarding program synthesis for robot navigation with partial observability, it looks like at least the benchmarks from the following papers are more challenging than the set of tasks explored in this paper. [1] Cao, Yushi, et al. "GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesis." Advances in neural information processing systems (2022). I would keep my score based on the above discussion.

Thanks for your rebuttal

1. Thanks for the authors' rebuttal, the concern about baselines is addressed. I suggest the author add these discussions to the paper. 2. However, I still think that the core design of the proposed method and the problem of the partially observed env lack some key and reasonable connection. I will keep my score. Thanks for the rebuttal.

Thanks

Thank you for taking the time to respond.

Thanks

Thank you for taking the time to respond.