Weaknesses
Motivation: The motivation is not presented well. The authors fail to adequately situate their approach within the existing body of literature. E.g. in line 14 "current methods insufficiently learn their representations" is not supported by examples or according literature. In line 50 it is claimed that "Exploring how to integrate and fully leverage performance across different scales while controlling complex learning dynamics is a promising area of research", however a reference or argumentation supporting the claim is missing.
In section 2, autoregressive and neural operators are presented as two different approaches, however the authors do not clearly position their method within both. In line 154, the authors claim “However, the aforementioned methods do not efficiently leverage frequency characteristics, and they lack a controllable mechanism for adjusting the learning process of partial differential equations.”, which is not supported by citations, experiments or further arguments. Please provide specific examples or citations from the literature that demonstrate the limitations of current methods in learning representations of multi-scale PDEs. Furthermore, references or a brief discussion of existing work on multi-scale modeling including connecntions to the proposed approach would better support their claim about the promise of this research direction and provide more structure in the literature review.
Results: The method does not increase the prediction quality for the Navier Stokes data set compared to the Fourier Neural Operator (FNO) (line 448: "Specifically, M2M outperforms the strong baseline of FNO128 in the performance a little."), which is not significant enough to support that this is achieved by the architecture design of the method. A more detailed discussion (especially on hyperparameter tuning) is missing.
The custom dataset: The custom dataset consists of fields of 4 squares with different dynamics. The proposed methodology seems to segment this in 16 squares, inducing an alignment of these squares. This is most probable not a fair comparison as 1) in physics, such quadratically structures field do not exist and 2) the alignment of dataset segments and segments that are distributed within the method induce an unfair bias to this method being superior. Experiments on publicly available datasets would provide a fairer evaluation. Usage of e.g. the PDEBench from the University of Stuttgart should be considered.
Unsupported claims: E.g. Contribution 2, Line 97 “By bridging the control theory-PID, this unified theory demonstrates its strong generalizability. It is a versatile and scalable method for the machine learning and science simulation community. “ The method is only validated for two different datasets, and a reason why PID controller integration unifies a theory is not given.
E.g. in line 442 "As shown in table 4.2, our M2M can allocate high FNO modes to the high-frequency region" - but table 4.2 does not show this. To provide evidence on the claim of generalizability more corresponding experiments (see comment w.r.t. to datasets above) should be added. Moreover, a more detailed discussion and analysis on how the PID controller integration contributes to the method's generalizability and versatility should be added, e.g. how do the distinct controller parts (P, I and D contribute).
Clarity: The explanation of expert nets in Line 191-193 is too short and thus hard to understandable in the current status. This section should be detailed to describe the function and concept of the expert nets. Moreover, the aggregation function in eq. 6 is should be specified. Also, a more detailed discussion on the routing should be added since it is hard to follow currently. The same holds for the explanation of PI control (252, 253). Algorithm 1 is described very imprecisely and uses not introduced variable names. According symbols and variables should be introduced and explained.
Distinction between the method presentation and its parametrization for the application is not given; this raises confusion about how the model is later used, e.g. starting in line 179, 4 patches are used, and then again n in equation 5, and then again 4 in line 219.
Graphics and tables with error metrics and terms that are not introduced/explained: e.g. caption of Figure 3 (“scale is set to 4”, how is the error metric defined), Figure 4 (“Prior [0000]”, “TOP_k”, how is the expert specified), Figure 5 (scale again, values plotted are not described, graphics quality is poor), Table 1 (what does the subsets mean).
Submitted source code is not usable: The required Python environment is not specified, there is no explanation how to use models and how to get datasets, all comments are Chinese such that they are not understandable to the worldwide community. Please provide a description or HowTo specifying the Python environment, including instructions for using the models and obtaining datasets, and translate comments to English to make the code accessible to a wider audience.
Mathematical notation unclear: use of introduced subsets (N, M, equation 6), in line 235 x misses the subscript, use of same variable for different things (e.g. x in line 107 and 222), not clear where equation 7 is used, meaning of variables in equation 10 is not clear.
Misleading introduction of PI control: The authors claim that they have a proof for the application of PI control in their method and link to the appendix. But what is claimed to be a proof is not a proof, but an introduction to PID control that can be found in many teaching books. Furthermore, intensified by the not defined error metric used in Figure 3, it remains unclear if the realized convergence increase is due to a different scaling of the components of the loss function, or a real benefit. Anyhow, to achieve the best result in Figure 3, both models still need to be trained for the same number of epochs. Additionally, the authors mismatch feedback and reference in line 297 and are not clear about what they control with PI (KL or loss?)
Paper quality: Grammar, sentence structure and spelling are frequently faulty. It seems that the quality of explanations, language and presentation gradually degrades to the end of the paper.
I think the paper introduces too many new elements of a model structure, such that the implications of the single changes cannot be separated. Here an ablation study should be added.
Important parts of results are in the appendix, as well as the limitations. This should be fixed.
Minor points:
The first two sentences in the abstract are missing arguments or citations which prove the statement “Learning the evolutionary dynamics of Partial Differential Equations (PDEs) is critical in understanding dynamic systems, yet current methods insufficiently learn their representations. This is largely due to the multi-scale nature of the solution, where certain regions exhibit rapid oscillations while others evolve more slowly.”
• The meaning of M2M is not explained.
• Contribution 3, line 99, is not a real contribution as validation is a must in machine learning papers “We validate the aforementioned method using the standard Navier-Stokes equations and a custom multi-scale dataset, …”
• FNO-3D allows for zero shot super resolution, while your approach does not. This should be mentioned.
• The title of 4.1 is not exact.
• The numbering in the appendix is wrong; it should start with letters.