Response
Thank you for your recognition of the novelty and theoretical foundation of our work. Here we have included a detailed response with additional experimental results to address your concerns.
> whether it can achieve similar results on relatively simpler mathematical datasets(without causing adverse effects), such as Addsub, MultiArith, and SVAMP.
Thank you for your insightful question. Following your suggestion, we have added a comparative experiment on the MultiArith dataset. Please see Table 4 in Appendix G for the details.
The results show that TSMC outperforms the baselines under all scenarios without bringing any adverse effects. All the inference-time parameters are kept the same over the three datasets (MATH500, GSM8K, Multiarith).
> adding a comparison for w/o MV （zero-shot performance）in Table 1
Thank you for your suggestion, we have added the comparison in Table 1. The method is denoted as “Greedy” , using the greedy decoding strategy.
> In Figure 4, why does the solving rate via TSMC show continuous and significant improvement with an increasing number of solutions for the same batch size (e.g., M=10)
We would like to first clarify that the batch size here refers to the resampling batch size of the TSMC rather than the number of voting samples $N$, where the latter is kept as a constant as 240 in this paper. $M$ is a TSMC-specific parameter not present in other methods.
The resampling batch size of TSMC refers to the number of samples used to form the categorical distribution for resampling in Equation 5. For example, when M=10, we would have 24 distributions and each distribution contains 10 categories.
The optimal number we find in Figure 4 is actually $M=20$. We hypothesize this is related to the weight degeneracy issue of importance sampling. When having a large resampling batch size, one sample with a large probability would dominate the distribution and lead to a bad diversity after resampling. So there is a tradeoff between the diversity and the variance. We treat the understanding of this phenomenon as a part of our future work as stated in Section 6.
> In the algorithm pseudocode, is there any difference between 'concat' and 'cat' (Appendix B)?
Yes. ‘Cat’ is initially defined in Equation 5, whose meaning is the categorical distribution. ‘Concat’ simply means the concatenation. We have added one more comment in the pseudocode to clarify this.
> Could you provide a more detailed comparison with Zhao's work (line 524)
Both works follow the classical framework of TSMC, while Zhao’s work is more focused on a general alignment framework while our method is primarily an inference-time search algorithm.
The major difference lies in our assumption that the generator is not powerful enough and refining the distribution of the generator is expensive (intractable step-level guidance), this is in line with the motivation of recently popular inference-time searching algorithms like MCTS, backtracking or o1 model. With this constraint, we can only use a suboptimal generator $q=p$ and need to address the potential large variance via more informative intermediate targets (Section 3.2).
In Zhao’s work, they allow the refinement of the generator via a tractable token-level guidance, but this token-level guidance would be expensive in case of long outputs since it needs the estimation of the twist objective at each token. This makes our work more significant under this setting (long output), for example, the multi-step reasoning task.