PACE: Pacing Operator Learning to Accurate Optical Field Simulation for Complicated Photonic Devices

Electromagnetic field simulation is central to designing, optimizing, and validating photonic devices and circuits. However, costly computation associated with numerical simulation poses a significant bottleneck, hindering scalability and turnaround time in the photonic circuit design process. Neural operators offer a promising alternative, but existing SOTA approaches, NeurOLight, struggle with predicting high-fidelity fields for real-world complicated photonic devices, with the best reported 0.38 normalized mean absolute error in NeurOLight. The inter-plays of highly complex light-matter interaction, e.g., scattering and resonance, sensitivity to local structure details, non-uniform learning complexity for full-domain simulation, and rich frequency information, contribute to the failure of existing neural PDE solvers. In this work, we boost the prediction fidelity to an unprecedented level for simulating complex photonic devices with a novel operator design driven by the above challenges. We propose a novel cross-axis factorized PACE operator with a strong long-distance modeling capacity to connect the full-domain complex field pattern with local device structures. Inspired by human learning, we further divide and conquer the simulation task for extremely hard cases into two progressively easy tasks, with a first-stage model learning an initial solution refined by a second model. On various complicated photonic device benchmarks, we demonstrate one sole PACE model is capable of achieving 73% lower error with 50% fewer parameters compared with various recent ML for PDE solvers. The two-stage setup further advances high-fidelity simulation for even more intricate cases. In terms of runtime, PACE demonstrates 154-577x and 11.8-12x simulation speedup over numerical solver using scipy or highly-optimized pardiso solver, respectively. We open sourced the code and dataset.

Paper

Similar papers

Peer review

Reviewer weV66/10 · confidence 3/52024-07-03

Summary

This paper proposes a neural operator (PACE) with improved accuracy, efficiency, and speed for electromagnetic field simulation. PACE is a network block in the surrogate machine learning model.

Strengths

This paper conducted extensive experimental evaluations for the proposed method. Also, the accuracy and efficiency of machine learning surrogate models for physical simulation are of great importance, therefore I think even though a little improvement is valuable.

Weaknesses

This paper, including the writing and the experiments, is very similar to the main comparison NeurOLight, which has code available online. Therefore I think it is important to discuss if it is only an incremental work or in-depth research.

Questions

1. How does this paper reflect “complicated” photonic devices, especially when compared with NeurOLight? I found that the datasets of both papers are custom-generated with random configurations and have a similar data volume. I expect the authors to provide more detailed explanations about the differences, particularly regarding the use of the term “complicated”, to avoid misleading readers into interpreting this paper as a simple incremental study. 2. This paper mentions that the NeurOLight method fails for “real-world” photonic devices, but I did not see any evaluation experiments of the proposed method on real-world datasets in this paper. 3. Is the speed improvement compared against CPU algorithms (line 308)? If the proposed method uses a GPU while the baseline Angler algorithm uses only a CPU, I do not think this is a fair comparison, and the corresponding 577x time boost could be misleading. 4. Please avoid using exaggerated language, for example: “boost the prediction fidelity to an unprecedented level” (line 12), and “extremely challenging cases” (line 57). 5. Unexplained descriptions, for example: “inspired by human learning” (line 57). There is no explanation provided as to why this method is inspired by human learning. 6. Some of the paper copy-paste from the NeurOLight paper, for example, equations 1 and 2. Corresponding references and explanations should be added. 7. Figure 4 is difficult to understand, and the caption is too brief. 8. In Table 1, the phrase “Improvement over all baselines (geo-means)” is rarely used and may not be considered as a good evaluation in research.

Rating

6

Confidence

3

Soundness

3

Presentation

2

Contribution

2

Limitations

Limitations are discussed in this paper but some points may be missed. For example, this paper works on only simulated and customized datasets.

Reviewer TfAG7/10 · confidence 4/52024-07-13

Summary

The authors propose PACE operators, which perform much better than existing methods on predicting optical fields for real-world complicated optical devices. The work builds up on and is compared to the NeurOLight framework which represents the SOTA in this field. The benefits of physics-agnostic PACE operators stem from their cross-axis factorized structure. This enables modeling of complicated devices in a parameter-efficient manner. The authors also propose a two-stage learning process which further improves the performance of PACE operators. Finally, the authors open-source this dataset consisting of optical field simulations for complicated devices. The authors demonstrate strong improvements over SOTA in solving the Maxwell PDE for complicated devices through novel operator design and learning workflows.

Strengths

Strengths: 1. The paper is well-structured with clear background and motivation. The authors highlight the issues that make optical field prediction for complicated devices challenging for existing methods. 2. The authors introduce a novel PACE operator that effectively utilizes the identified challenges to mitigate the difficulties in optical field prediction for complex devices. 3. The authors demonstrate the strong performance of PACE operators compared to the existing NeurOLight framework as well as other commonly used ML PDE solvers such UNets, FNO variants and attention based methods. The proposed method achieves significantly lower errors with comparable or lower number of parameters, while also being faster than numerical methods. 4. The authors also validate all the proposed improvements individually with well-designed ablation studies. 5. The curation of the dataset for optical field simulations of complicated devices will also be helpful to the community for future benchmarking. Overall the paper serves as a good example of identifying a clear problem in a specific domain,  thoroughly analyzing the shortcomings of existing methods, and leveraging domain knowledge to enhance neural operators. The results are strong and showcase the applicability of this method for optical device simulation.

Weaknesses

1. While demonstrating strong results on a challenging problem, my main concern for this paper is that it might be too specific of an application for a broader ML conference like NeurIPS. The paper might be better suited to a photonics journal. This can be mitigated by demonstrating results on other standard PDE benchmarks and showing improvements over existing methods. The authors allude to this in line 362 Conclusion but concrete results would greatly improve the paper. 2. The interpretation of errors is not clear. First the normalized absolute error (relative error) of NeurOLight is repeatedly mentioned. Then the authors use the normalized mean squared error as the learning objective (which is rightly justified in A.6). However it is not clear what these errors mean. The paper would also be clearer if it is explicitly stated what error metrics are used in the various table (I assume it is N-MSE). What is the tolerance for acceptable errors for downstream tasks? What magnitude of error makes it comparable to existing numerical solvers used in practice? 3. It is common in ML-PDE papers to show speed ups over numerical methods while the ML models have less accuracy. If you were to achieve this same lower accuracy using the numerical methods, it might greatly reduce the runtime of numerical methods as well. What is the runtime of Angler/scipy/pardiso that achieves comparable accuracy to the PACE operator? 5. The authors mention multiple times about inspiration from "human learning". However there is no description of what this means. Section 4.2 on cascaded learning can be improved by expanding on how this related to human learning with appropriate citations. 6. It has been mentioned that the model preserves the spectrum of the solution field. The author show the spectrum of the reference field in Figure 1d and Figure 9 (for Darcy Flow). However there is no discussion on the spectrum of the predicted solution fields. The authors should visualize the spectrum of predicted fields and compare it to the reference. The paper would benefit from a thorough proof-reading. There are multiple typographical errors and some sentences are not clear. Examples are: - Line 5: space after comma ", NeurOLight," - Line 162: satisfactory instead of satisfied - Line 188: shortcomings instead of shortness - Line 196: O(nlogn) - Line 205: "is a must" is extra/ - Line 224: satisfactory instead of satisfying - Line 238: What does condensate mean here? - Line 252: "neural" - Line 255: Establish LSM acronym here - Line 269: I think the authors mean to say that there is a 73.85% reduction in error. The way this sentence is phrased right now implies that there is a 73.85% error. - Line 284: Fourier - Line 294: Sorely seems to not fit here. Maybe "just" or "only"? - Line 297: Appendix - Lines 314 and 315: It is a bit confusing. The sentence can be made clearer by avoiding the use of worse. - Line 319: Decomposition doesn't seem to fit here. - Line 336: Factorized - Line 362: Restricted might be a better alternative to dedicated. Some figures are hard to decipher: - Figure 1 a,b,c should be made larger so that the readers can clearly see the issues mentioned in the Challenges section. - Figures 5, 6, 7 show important comparisons between different methods and should be made larger. - Figure 9 should have subfigures. Also 9a (Darcy flow field) should have axis labels and colorbar. - Figures 10 and 11 should be made bigger so that the light fields and errors are clearly visible. The absolute error is not closed with "|" in the blue boxes in Figure 10. Figure 11 is missing colorbars that show the magnitudes of the optical fields and errors. Right now the dataset URL is behind a hyperref "URL". It might be helpful to mention the full address somewhere in the paper for accessibility.

Questions

1. The etymology of "PACE" is not clear. The all-caps spelling suggests it is an acronym but it is not expanded anywhere in the paper. If there is another contextual meaning of the word "pace" that is applicable to the model, it is not clear from the paper. Why is it called PACE? 2. The works cited in the paper (reference 25, 12) seem to use the word "complex" to describe non-trivial optical devices. Would this be a better word to use than "complicated" used in the title and throughout the paper? 3. In "Generalization to ood testing" (line 322), the results are only shown for interpolation, i.e., for wavelength ranges within the training domain. It is great that the model interpolates well in this range. However, for true ood generalization, the authors should also show results for extrapolation. How does the model behave for wavelengths outside the training range (outside 1.53-1.565 micrometers). 4. Related to point of weaknesses, what is the contribution of using the different N-MSE metric as the learning objective? Does the performance of NeurOLight also increase when trained using N-MSE? 5. In section 5.2.3 Speedup over numerical tools: FNOs are resolution invariant by design. Does PACE also exhibit this property? In the speed up section, were multiple PACE models trained for each discretization or are the results shown for one model with inference over different discretization? 6. The authors qualitatively state in section A.9 that the prediction results show "near-black error map" (line 582). However it seems that the colorbars have different scales across different models. There are no colorbars in Figure 11. Also, since in cases like these with very low errors, it would be interesting to see the absolute errors in log space. This might shed further light into structural sources of errors. How would this look in log space?

Rating

7

Confidence

4

Soundness

4

Presentation

3

Contribution

3

Limitations

The authors have a brief limitations section where they state that exploring operator learning on FDTD methods and general optimizations to FFT kernels on GPUs are interesting future directions. While this is already a parameter efficient model, it is a known limitation of Neural Operators that they don't scale well with increasing dimensions. It might be helpful to address this limitation here. How would the PACE operator scale in 3D and beyond? I don't think that there are any negative societal implications for this work.

Authorsrebuttal2024-08-07

References

[1] Liu, Xinliang, et.al. "Mitigating spectral bias for the multiscale operator learning with hierarchical attention." arXiv preprint arXiv:2210.10890 (2022). [2] Bo Xu and Lei Zhang. “Dilated convolution neural operator for multiscale partial differential equations.” arXiv preprintarXiv:2408.00775v1 (2024). [3] Timurdogan, Erman, et al. "APSUNY process design kit (PDKv3. 0): O, C and L band silicon photonics component libraries on 300mm wafers." Optical Fiber Communication Conference. Optica Publishing Group, 2019. [4] Github of Latent-Spectral-Models

Authorsrebuttal2024-08-12

Follow-Up on Rebuttal

Dear Reviewer TfAG, We sincerely appreciate your valuable comments and time. You provided a very high-quality discussion and made important suggestions to further improve our paper writing and figures. We have carefully addressed all your comments in our response above, with extra experiments as you suggested. We would like to point out that our paper **aligns with NeurIPS’s interests** and offers **novel ML contributions** to the AI for PDE community. We built **a new and stronger SOTA** compared to the previous SOTA (NeurOLight @ NeurIPS’22). As the discussion period is drawing to a close in 1 day, we would be very grateful if you could take a moment to review our responses. If our replies have satisfactorily addressed your concerns, we greatly appreciate it if you could acknowledge this in the discussion thread and consider raising the rating. If you have any remaining questions or concerns, please do not hesitate to contact us. We look forward to your feedback.

Authorsrebuttal2024-08-14

Dear Reviewer TfAG, Thank you once again for your review. As the deadline for the author-reviewer discussion approaches, we noticed that we haven't received any comment from you. We have addressed all your questions with additional experiments and clarifications. * Regarding your main concern about our scope, we would like to point out that our paper aligns with NeurIPS’s interests and offers novel ML contributions to AI for the PDE community (See global response Q.1), with **a new parameter-efficient cross-axis factorized neural operator** and **a new learning flow for solving challenging PDE cases**. * We would point out that no one model can rule all PDE cases. **Our work shows important insights into how to enhance neural operators for challenging PDE cases**. Moreover, we compare with one SOTA work for the challenging multi-scale PDE problem where previous neural operators also fail (similar to our challenge in that the PDE coefficient is highly contrasted). Because they didn't release the dataset, we compared our work with theirs on our benchmark. Our work shows significantly better accuracy than theirs, proving it can provide important insight into the multi-scale PDE problem. As the discussion period is ending, we would greatly appreciate any additional feedback you might have. If our responses have clarified your understanding of our paper, we sincerely hope you might consider raising the rating. Thank you again for your effort in reviewing our paper. Best regards, Authors of Paper 472

Reviewer TfAG2024-08-14

Response to Rebuttal

Dear authors, Thanks for the detailed point-by-point clarifications to my review. I went through the rebuttal and the attached pdf and believe that it addresses most of the points I raised. The following questions have been answered sufficiently: Q1, Q2, Q4, Q5, Q6, Q7, Q8, Q9, Q10, Q11, Q12. Please add the plots and explanations for Q5, Q7, Q9, Q10 in the camera-ready version. Your explanation for relevance to NeurIPS (and energy-efficient ML hardware in general) should also be incorporated in the papers as it can serve as a strong motivation for general ML readers. Also add the origin of PACE acronym somewhere early in the paper. I am not totally convinced by the speedup claims as the comparisons are not one-to-one wrt accuracy and computational time/hardware, but the authors made a good-faith effort to address this. Further exploration would be outside the scope of this paper at this point. For more information on assessing speed-up claims for ML-PDE models, please refer to this paper[1], specifically eq 1 on page 11. I also agree with the other reviewer that coarse-to-fine is a better description than divide-and-conquer. You can also frame it as an iterative model. In this context, using the term human learning feels a little bit hyperbolic. Finally, please proof-read the paper for typos that are mentioned in Q12 and also for those that might have been missed. I am hoping that the mentioned changes are incorporated in the camera-ready version of the paper, and raising the score assuming this. [1] McGreivy and Hakim, Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations. https://arxiv.org/abs/2407.07218

Reviewer 9WdN6/10 · confidence 4/52024-07-13

Summary

This paper propose a novel cross-axis factorized PACE operator with a strong long-distance modeling capacity to connect the full-domain complex field pattern with local device structures. Additionally, the authors proposed a two-stage model learning method for extremely hard cases, with a first-stage model learning an initial solution refined by a second model. Finally, experimental results show that the proposed PACE model can achieve good performance with fewer parameters.

Strengths

1. Major challenges for simulating real-world complicated optical devices is well analyzed. 2. Their proposed PACE model significantly boost prediction fidelity to an unprecedented level with much faster running speed than traditional numerical solver on a 20-core CPU. 3. A two-stage model learning method is used for extremely challenging cases. 4. Experimental results show that on the complicated device benchmarks, their proposed PACE model significantly outperforms previous baseline models. 5. A dataset on for the complicated optical device is released to facilitate AI for PDE community.

Weaknesses

1. I do not agree with the authors that the proposed two-stage learning method belongs to a divide-and-conquer approach. From my perspective, it is called coarse to fine. 2. The writing should be improved as some sentences are not precise. For example, at line 19&20, “one sole PACE model …fewer parameters” (compare with which method?) 3. It is common that deep learning based method runs much faster than traditional numerical solvers. The testing error of the traditional numerical solvers may plays as a groundtruth for the upper bound of precision. Thus, the authors should list the testing error of the traditional numerical solvers. 4. In Table 1, except for the number of Params, you should list other metrics like Flops and testing time to show the efficiency performance of the proposed PACE model.

Questions

1. Training details in Sec. 5.1 and A.2 should be integrated. 2. The effectiveness of pre-normalization and double skip is not well verified. In Table 6 of the appendix, you should list the results of PACE-12 layer without double skip and pre-norm. 3. At line 211, the authors claim that non-linear activation is known to help generate high-frequency features. Therefore, some visualization of the high-frequency features is needed to support this statement.

Rating

6

Confidence

4

Soundness

4

Presentation

3

Contribution

4

Limitations

NA

Authorsrebuttal2024-08-12

Follow-Up on Rebuttal Response

Dear Reviewer 9WdN, We sincerely appreciate your valuable comments and suggestions for our paper. We have carefully addressed all your comments in our response above. Regarding your questions, * We have added an experiment for the ablation study to test the effectiveness of pre-normalization and double skip. The results confirm that pre-normalization and double skip are not the main sources of improvement. **Our proposed new neural operator block with many essential model design considerations, such as cross-axis factorized operator and high-frequency projection path, is the key to unlocking the improvement.** * We have provided extra visualization of the feature map before/after nonlinear activation **in Rebuttal Figure 1**. It **validates our model design choice by clearly showing that the high-frequency feature is generated after the nonlinear**. * Runtime is provided as you suggested, and the tab. 1 will be updated in the revised version. (please check our response Q3) As the discussion period is drawing to a close in 1 day, we would be very grateful if you could take a moment to review our responses. If our replies have satisfactorily addressed your concerns, we would greatly appreciate it if you could acknowledge this and consider raising the rating. If you have any remaining questions or concerns, please do not hesitate to reach out to us. We look forward to your feedback.

Authorsrebuttal2024-08-14

Dear Reviewer 9WdN, Thank you once again for your review. As the deadline for the author-reviewer discussion approaches, we noticed that we haven't received any comment from you. We have addressed all your questions with additional experiments and clarifications: * We have added an experiment for the ablation study to test the effectiveness of pre-normalization and double skip. The results confirm that pre-normalization and double skip are not the main sources of improvement. **Our proposed new neural operator block with our carefully designed model components, such as cross-axis factorized operator and high-frequency projection path, is the key to unlocking the improvement.** * We have provided extra visualization of the feature map before/after nonlinear activation **in Rebuttal Figure 1**. It **validates our model design choice by clearly showing that the high-frequency feature is generated after the nonlinear**. * We provide the runtime (please check our response Q3) and the tab. 1 will be updated in the revised version. As the discussion period is coming to an end, we would greatly appreciate any additional feedback you might have. If our responses have clarified your understanding of our paper, we sincerely hope you might consider raising the rating. Thank you again for your effort in reviewing our paper. Best regards, Authors of Paper 472

Reviewer ZkVJ6/10 · confidence 3/52024-07-14

Summary

This paper develops a computationally efficient learning based PDE simulator for modeling optical fields within photonic devices. The proposed method builds upon NeurOLight [7] and adds a novel factorization of the integral kernel and a two-stage approach to prediction---produce a rough initial estimate that is then refined. The proposed method is noticeably more accurate than [7] and still considerably faster than conventional, non-learning based PDE solvers. An ablation study validates the importance of both the factorization adn the two-stage approach.

Strengths

Efficiently and accurately simulating photonic devices is an important problem with broad applications. THe proposed method is more accurate than existing learning based solvers and 12x faster than conventional methods

Weaknesses

The paper is written for a somewhat narrow audience and contains a number of typos. E.g., "boarder impact". The 577x comparison number is somewhat misleading and should be dropped from the abstract. The proposed method may be much faster than scipy, but its only 12x faster than paradiso.

Questions

Are the "real-world" devices actually manufactured and measured or merely simulated? I found section 5.1 unclear.

Rating

6

Confidence

3

Soundness

3

Presentation

2

Contribution

3

Limitations

Yes

Authorsrebuttal2024-08-12

Follow-Up on Rebuttal Response

Dear Reviewer ZkVJ, We sincerely appreciate your valuable time in reviewing our paper. We have carefully addressed all your comments in our response above. We would like to point out that our paper **aligns with NeurIPS’s interests** and offers **novel ML contributions** to AI for the PDE community. We built a **new and stronger SOTA** compared to the previous SOTA (NeurOLight @ NeurIPS’22). Moreover, we follow your suggestion to clarify the unclear description, and those parts will be merged into the revised version. Thank you again for your suggestion, which is crucial to improve our paper writing. As the discussion period draws to a close in 1 day, we would be very grateful if you could take a moment to review our responses. If our replies have satisfactorily addressed your concerns, we would greatly appreciate it if you could acknowledge this in the discussion thread. If you have any questions or concerns, please do not hesitate to contact us. We look forward to your feedback.

Authorsrebuttal2024-08-10

Follow-Up on Rebuttal Response

Dear Reviewer, Thank you very much for your thorough review and valuable comments on our paper. We have carefully considered all your feedback and have addressed your concerns and questions in our rebuttal. Especially regarding your question on whether we are doing incremental work over NeurOLight, we would like to emphasize that our work proposes **a new neural operator backbone** and **a new learning strategy** for ***complicated*** photonic device simulation problem, and we achieve > 39% lower error than NeurOLight (**a significant improvement**). We would greatly appreciate it if you could take some time to review our responses. We will be online over the next few days and are happy to address any further questions or concerns you may have.

Authorsrebuttal2024-08-10

Follow-Up on Rebuttal Response

Dear Reviewer, Thank you very much for your thorough review and valuable comments on our paper. We have carefully considered all your feedback and have addressed your concerns and questions in our rebuttal. We would greatly appreciate it if you could take some time to review our responses. We will be online over the next few days and are happy to address any further questions or concerns you may have.

Authorsrebuttal2024-08-10

Follow-Up on Rebuttal Response

Dear Reviewer, Thank you very much for your thorough review and valuable comments on our paper. We have carefully considered all your feedback and have addressed your concerns and questions in our rebuttal. We would greatly appreciate it if you could take some time to review our responses. We will be online over the next few days and are happy to address any further questions or concerns you may have. Thank you again for your time and consideration!

Authorsrebuttal2024-08-10

Follow-Up on Rebuttal Response

Dear Reviewer, Thank you very much for your thorough review and valuable comments on our paper. We have carefully considered all your feedback and have addressed your concerns and questions in our rebuttal. We would greatly appreciate it if you could take some time to review our responses. We will be online over the next few days and are happy to address any further questions or concerns you may have.

Authorsrebuttal2024-08-13

Follow-Up on Rebuttal Response

Dear Reviewer, We sincerely appreciate your valuable time in reviewing our paper. We have carefully addressed all your comments in our response and added extra experiments/visualization as you suggested (appended in the pdf in the global rebuttal). Our work tackles the challenging photonic Maxwell PDE simulation problem on ***real-world*** ***complicated devices*** with **a novel neural operator design** and **a new learning flow** and establishes **a strong SOTA** (>39% lower error) on AI for optics simulation compared to previous SOTA (NeurOLight @ NeurIPS’22). As the discussion period draws to a close in 1 day, we would be very grateful if you could take a moment to review our responses. If our replies have satisfactorily addressed your concerns, we would greatly appreciate it if you could acknowledge this and kindly consider raising the rating. We are always **open to further suggestions or feedback** that could help us improve the paper even more. If you have any questions or concerns, please do not hesitate to contact us. Thank you again for your valuable input. Best regards, Authors of Paper 472

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC