NeuralSolver: Learning Algorithms For Consistent and Efficient Extrapolation Across General Tasks

Summary

The paper proposes a method called "NeuralThink", which is designed to improve the same-size task generalization and different-size task extrapolation performances. The proposed algorithm composes of three components: (1) A recurrent module utilizing a LSTM network to process inputs of different scale, (2) A processing module serving as the aggregation layer, and (3) A curriculum learning training scheme that gradually increases the dimensionality of the observation. Experiments on an algorithm learning benchmark showed that the proposed approach surpassed the baselines being compared, especially in terms of the different-size task extrapolation performance. It is also demonstrated that the proposed approach has a higher efficiency both in terms of training size and parameter size.

Strengths

- The paper addresses a meaningful problem space: Algorithm learning and extrapolation ability. The proposed method achieves oracle-level performance on different-size tasks, showcasing the improved generalization and extrapolation power. - The proposed approach outperforms the selected baselines on an algorithm learning benchmark, proving the effectiveness to a certain extent. Further experiments also indicates improved training size and parameter efficiency. The ablation study demonstrates that all components in the proposed approach contribute positively to the final result. - The intro section is well-written. Figure 1-3 are well-illustrated.

Weaknesses

- It's not immediately clear why the proposed approach achieves improved performance above prior arts. Explanation on (1) the fundamental difference compared to the previous approach, and (2) why this should work better would be helpful. - The result being presented in the experiment section is a bit counter-intuitive. Table 3 shows that the proposed approach achieves oracle-level performance on different-size task extrapolation, yet Table 1 shows that the performance on same-size tasks is imperfect. Intuitively one would imagine different-size tasks to be more challenging for the algorithm, yet the result suggests otherwise. An analysis on the intuition behind would be helpful. Besides, more case studies to compare the performance gap between the proposed approach v.s. prior baselines would help solidify the understanding. - The number of baselines being compared to is limited. It would be helpful to know how does the proposed approach compare to other recently published methods, e.g. [1], or if they are not directly comparable, what is the reason behind. - nit: The presentation of some figures need to be improved for better clarity. e.g. Figure 5 it's not immediately clear what does the notion "x/y" mean in the legend of the bar chart. [1] Adaptive recurrent vision performs zero-shot computation scaling to unseen difficulty levels. NeurIPS 2023.

Questions

See above.

Rating

5

Confidence

2

Soundness

2

Presentation

2

Contribution

3

Limitations

N/A

Summary

1. The paper introduces NeuralThink, a novel deep thinking architecture designed to efficiently and consistently extrapolate learned algorithms from smaller problems to larger ones. 2. Unlike previous deep thinking methods, NeuralThink can be applied to both same-size problems (where input and output sizes are the same) and different-size problems (where input and output sizes differ). 3. The architecture consists of three main components: - A recurrent module that iteratively processes input information at different scales - A processing module that aggregates previously processed information - A curriculum-based training scheme to improve extrapolation performance 4. NeuralThink outperforms prior state-of-the-art deep thinking approaches in: - Extrapolation capability: Consistently executing learned algorithms on larger problems - Training efficiency: Learning algorithms from smaller problems - Parameter efficiency: Requiring fewer parameters than other approaches 5. The authors introduce a set of novel different-size tasks to evaluate their method.

Strengths

The paper is easy to follow and experiments nicely presented. The proposed model seems to outperform (one) related work method and results seem quite impressive.

Weaknesses

Since the authors introduce novel benchmarks, the very positive experimental results need more ablations. Please provide hyperparameter scan and their results on baselines.

Questions

See Weaknesses. Can you provide a more detail comparison between DeepThink and your model. What happens generally for both of these models if you scale them? It would be nice to see when NeuralThink breaks? Afaiu DeepThink was going for the recurrent module to see if larger inputs could be solved by running the network longer in depth. Your model works quite differently afaik and I would again highlight these differences in detail.

Rating

6

Confidence

3

Soundness

3

Presentation

3

Contribution

3

Limitations

I would welcome a dedicated limitation section.

Summary

The authors propose a new architecture that improves on the Deep Thinking (DT) architecture by Bansal et al, 2022. It replaces the recurrent ResNet block with a convolutional LSTM+layernorm. They also use a curriculum-based learning schema. This improves the model's extrapolation capability significantly. The authors test their model on identical tasks to those of Bansal et al, and show that they outperform the DT model on all of them, often significantly.

Strengths

The authors propose a strong model that outperforms the baseline in all tasks they test for. Their model is more general than the baseline, allowing generalization also in tasks whose output grid is of a different size compared to the input. The model doesn't suffer from overthinking and does not require special regularization.

Weaknesses

The writing can be improved: the authors do not clearly state what is novel in their architecture in the main text. They should emphasize the exact differences to the literature more clearly: e.g by stating that the difference in their architecture and DT is replacing the ResNet by a convolutional LSTM. Figures with the double meaning for the color are confusing (Fig 5 and Fig. 6). Also, for Fig 6, there are 3 options but just 2 numbers for each color. The exact details of the architecture are not entirely clear. It would be better if the authors could describe their architecture with equations, at least in the appendix. The authors are using LSTMs but do not cite [1]. [1] Hochreiter et al, 1997: Long Short-Term Memory

Questions

It would be nice to see an ablation on the layernorms in Tab. 4. In line 168, the authors write "consider the best accuracy obtained by the models at any iteration". Is this also done for NeuralThink? It should not suffer from overthinking, right? Why is think maze more difficult than the thick one, especially for the baselines?

Rating

7

Confidence

3

Soundness

4

Presentation

3

Contribution

3

Limitations

It is not clear how these architectures can transfer from simple grid world problems to some more real-world applications. The authors could discuss this more in the paper.

Thank you!

Thank you for the additional data. I will raise my score to 6. Many thanks!