Prefix-Tree Decoding for Predicting Mass Spectra from Molecules

Computational predictions of mass spectra from molecules have enabled the discovery of clinically relevant metabolites. However, such predictive tools are still limited as they occupy one of two extremes, either operating (a) by fragmenting molecules combinatorially with overly rigid constraints on potential rearrangements and poor time complexity or (b) by decoding lossy and nonphysical discretized spectra vectors. In this work, we use a new intermediate strategy for predicting mass spectra from molecules by treating mass spectra as sets of molecular formulae, which are themselves multisets of atoms. After first encoding an input molecular graph, we decode a set of molecular subformulae, each of which specify a predicted peak in the mass spectrum, the intensities of which are predicted by a second model. Our key insight is to overcome the combinatorial possibilities for molecular subformulae by decoding the formula set using a prefix tree structure, atom-type by atom-type, representing a general method for ordered multiset decoding. We show promising empirical results on mass spectra prediction tasks.

Paper

References (65)

Scroll for more · 38 remaining

Similar papers

Peer review

Reviewer 8EJR8/10 · confidence 4/52023-06-28

Summary

This paper proposes a novel method for predicting a tandem mass spectrum from a given molecular graph. The key idea is to subdivide the problem into predicting fragmentation of the graph and then separately predicting intensities associated with each fragment. Relative to other recent methods that have adopted this approach, this method uses a prefix tree to make the identification of subgraphs more efficient. Experimental results show that the proposed method performs better than state-of-the-art methods.

Strengths

The paper addresses an important problem. The paper is extremely well written, with a nice intro to tandem mass spectrometry targeted to a NeurIPS audience. The treatment of related work is well organized and clear. The main idea is well supported by empirical results.

Weaknesses

On lines 37-42, I wasn't super convinced that "physically-inspired" (which, incidentally, doesn't need to be hyphenated) belongs in this list of desiderata. I think the main thing is that the method is fast and accurate. It seems more like a hypothesis to say that in order to achieve speed and accuracy, a method should be physically inspired. Perhaps the notion of "accurate" should be segregated into various dimensions: e.g., accurate in the sense that it doesn't just predict binned m/z values, or accurate in the sense that it doesn't predict extra peaks that can't be explained by the formula. I would argue that if someone came along with a method that was very accurate and fast, there is no scenario in which an end user would prefer a slightly slower or slightly less accurate method just because it comes with a story about how it is physically plausible. Figure 1A is the key idea, and I think it needs to be expanded. As drawn, it's not clear how, e.g., you decide that the root should have three children (C7, C12 and C14). Why not C1, C2, etc.? The answer presumably has to do with the molecular graph, which is the primary input but is not shown here. I would also like to hear a bit more about this prefix tree step in the text. E.g., does it matter what order the atom types are introduced? If so, how is the order chosen? Related to the first point above, the claim on lines 100-102 that binning methods are less interpretable was hard for me to understand. The reduced accuracy due to the lack of shared information across fragments is plausible, but that seems like an empirical claim that requires support. Maybe just say that you hypothesize that this may lead to reduced accuracy. I also found it strange that you didn't explicitly call out the most obvious source of inaccuracy, which is that the binning means that the peak m/z locations are inherently inaccurate. For the SCARF-Weave predictor, please indicate how large the sets can be come, both in principle and in practice. I wonder whether this is a potential problem for the method. When you present the various competing methods on lines 248-255, I think it would be helpful if you could introduce them in terms of the three groups from Section 2.2. The closest competing methods are references [24] and [45]. In your experimental comparisons, why did you use a method based on MS-GRAFF ([24]) rather than just using MS-GRAFF itself? And why didn't you use method [45] at all? On a related note, if FixedVocab falls into the third category of methods, then why aren't its predictions 100% valid in Table 2?

Questions

Is there a concrete sense in which binning methods are less interpretable? How large are the sets handled by SCARF-Weave? Why are MS-GRAFF and FormulaNet/SubsetNet not included in the empirical comparison?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

4 excellent

Presentation

4 excellent

Contribution

4 excellent

Limitations

N/A

Reviewer 188s7/10 · confidence 5/52023-07-06

Summary

The manuscript presents a method called SCARF for the prediction of tandem mass spectra from molecular structures. This allows for building up large in-silico libraries that, together with measured spectra, can be used in spectral library search workflows for metabolite identification. From a general machine learning perspective, the method is an interesting approach for the problem of predicting sets of multisets (in this case: sets of molecular formulas). The author note that autoregression is not well suited for this task, as it is not ordering invariant. Instead, they suggest an approach that is quite similar to the way molecular formulas are decomposed from masses: they step-by-step build a prefix-tree where each node represents a subset of the molecular formula space with certain elements are fixed to specific abundancies. The child nodes are then possible abundancies for the next chemical element, restricting the formula space further until the the leafs of the tree describe only a single molecule formula. The resulting set of molecular formulas is then transformed into a spectrum, by utilizing a second transformer based network for intensity prediction.

Strengths

- it is the first paper that predicts high resolution spectra without involving combinatorial fragmentation - use of prefix tree for the prediction of molecular formulas (or sets of multisets) is very innovative - use of random Fourier features for molecular formula embedding is innovative and seem to work much better than the common approaches of encoding formulas as frequency vectors or one-hot vectors/dictionaries - evaluation on the spectrum prediction part is done very well. In particular, certain methods are retrained (or even re-implemented) and parameter-optimized to ensure fair comparison - manuscript is clearly written, the method is described in detail with all hyper parameters given in the supplement - source code is available

Weaknesses

- there is already a successor method (ICEBERG) by the same authors for the same task. This is not a problem in general, as both methods have very different strategies to solve the problem. I still think it would have strengthen the manuscript if pros and cons of both methods and their synergistic potential is discussed - the manuscripts list three applications for the predicted spectra: molecule identification, learning about fragmentation, and training machine learning methods on spectra. However, in contrast to ICEBERG the method presented here cannot give any insights about the fragmentation process (as it is not a fragmenter). I also doubt that synthetic data (predicted machine learning) is well suited as training data for other machine learning methods. This is some kind of self-training or self-supervised learning which is a difficult and challenging problem on its own. Thus, the only convincing application for the spectrum prediction task is the molecule identification. Beside spectrum prediction, there are other methods that allow for molecule identification (including methods from the same authors such as MIST). Although the evaluation on metabolite identification contains many methods for spectrum prediction, it does not contain any of these alternative approaches. - the observation that identification rate is decoupled from database retrieval rate is concerning, as this is the main application for spectrum prediction - although all methods perform worse on the NPLIB1 data, it is strange that NEIMS(FFN) and SCARF perform so very different on the retrieval task. It would be interesting to find out why the fingerprint based method performs and the dictionary based method both perform so well here but not in NIST. - besides its application in metabolomics, the presented approach might be interesting for other areas with sets of multisets as target variable. Unfortunately, the baseline method (autoregression) is not evaluated against the more sophisticated prefix tree decoding method

Questions

- does the cosine similarity calculation takes the precursor ion into account? For many spectra, the precursor can have high intensity (and, thus, heavily effect cosine similarity) while it does not provide any new information. If so, I would suggest to recalculate cosine similarity and leave out the precursor - I do not understand why the spectrum retrieval task is performed on a subset of 49 isomers (although the most similar ones) instead of just all isomers

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

4 excellent

Presentation

4 excellent

Contribution

3 good

Limitations

The main limitation of the method is in my opinion its limited performance on the metabolite identification task. This and other limitations are discussed properly in the manuscript.

Reviewer sZMH8/10 · confidence 4/52023-07-12

Summary

The paper presents a new method for predicting mass spectra from molecules called SCARF, which stands for Subformulae Classification for Autoregressively Reconstructing Fragmentations. Mass spectra are sets of peaks that represent the masses and intensities of fragments of molecules after they are ionized and broken down in a mass spectrometer. Predicting mass spectra from molecules is useful for identifying unknown molecules from experimental data, as well as for understanding the fragmentation process and generating virtual spectra libraries. SCARF predicts mass spectra in two steps: first, it generates the set of chemical formulae for the fragments, which define the locations of the peaks on the mass-to-charge axis; second, it assigns intensities to these formulae, which define the heights of the peaks. The key innovation of SCARF is that it uses prefix trees to efficiently generate the set of formulae, overcoming the combinatorial challenge of enumerating all possible subformulae of a given molecule. SCARF also ensures that all predicted peaks are physically plausible, meaning that they correspond to valid subformulae of the original molecule. The paper evaluates SCARF on two datasets of molecules and their experimentally measured spectra, NIST20 and NPLIB1. The paper shows that SCARF outperforms existing methods based on fragmentation rules or binned prediction in terms of accuracy, physical sensibility, and speed. The paper also demonstrates that SCARF can improve the retrieval of unknown molecules from new spectra by comparing them to predicted spectra from a database of candidate molecules. The paper concludes by discussing the limitations and future directions of SCARF.

Strengths

The paper makes several contributions. First, the paper introduces a new method for predicting mass spectra from molecules, which is based on a novel combination of subformulae classification and autoregressive reconstruction. The method overcomes the limitations of existing methods based on fragmentation rules or binned prediction, which are either too restrictive or too coarse-grained. It achieves state-of-the-art performance in terms of accuracy, physical sensibility, and speed, as demonstrated on two datasets of molecules and their experimentally measured spectra. The method is based on a simple yet powerful idea of using prefix trees to generate the set of formulae for the fragments efficiently. It has the potential to revolutionize the field of mass spectrometry by enabling more accurate and efficient identification of unknown molecules from experimental data. Second, the paper provides a thorough evaluation of the proposed method on two datasets of molecules and their experimentally measured spectra, NIST20 and NPLIB1. It uses a new metric called physical sensibility, which measures how well the predicted spectra match the physical constraints of mass spectrometry. SCARF outperforms existing methods based on fragmentation rules or binned prediction in terms of accuracy, physical sensibility, and speed. The paper provides detailed descriptions of the datasets, metrics, baselines, and results. The evaluation is significant because it demonstrates the effectiveness and robustness of SCARF across different datasets and scenarios. Third, the paper discusses the limitations and future directions of SCARF. It identifies several open problems and challenges in mass spectrometry that SCARF or its variants can address and provides insightful analyses and suggestions for future research. Overall, the paper represents a significant advance in mass spectrometry by introducing a new method for predicting mass spectra from molecules.

Weaknesses

I list some weaknesses below: 1. The paper assumes that the fragmentation process is purely additive, meaning that each fragment is formed by adding one or more atoms to the previous fragment. This assumption may not hold for some molecules that undergo complex fragmentation pathways, such as rearrangement, elimination, or charge transfer. To address this weakness, future work could explore more general models of fragmentation that can capture these pathways, such as machine learning models or expert systems. 2. It assumes that the mass spectra are measured under ideal conditions, meaning there is no interference from other molecules or ions. This assumption may not hold for some real-world scenarios, such as complex mixtures or dirty samples. To address this weakness, future work could investigate how to incorporate prior knowledge or external data sources to improve the accuracy and robustness of mass spectra prediction. 3. The assumption is that the molecules are represented by their molecular formulae, which are discrete and symbolic. This representation may not capture the continuous and structural features of molecules that affect their fragmentation patterns and mass spectra. To address this weakness, future work could explore more expressive and flexible representations of molecules that can capture these features, such as molecular graphs or descriptors. 4. The fragmentation patterns are assumed independent of each other, meaning that each fragment is formed independently of the others. This assumption may not hold for some molecules that undergo correlated fragmentation pathways, such as cleavage of adjacent bonds or the formation of cyclic structures. To address this weakness, future work could investigate how to model these correlations explicitly or implicitly in the prediction process. 5. The paper assumes that the mass spectra are measured with high resolution and accuracy, meaning each peak is resolved and assigned a precise mass-to-charge ratio. This assumption may not hold for some low-quality or noisy spectra that have overlapping or shifted peaks. To address this weakness, future work could develop methods for denoising or deconvolving mass spectra before or after prediction. Overall, these weaknesses suggest several directions for future research in mass spectrometry and related fields.

Questions

- How does SCARF handle molecules that contain elements that are not in the predefined element set? How does it handle molecules that have unknown or ambiguous formulae? - Can SCARF handle spectra that have multiple precursor ions or multiple charge states? How does it handle spectra that have different adduct types or ionization modes? - Will SCARF handle molecules that have multiple conformers or stereoisomers? How does it handle molecules with different fragmentation patterns depending on their conformation or configuration? - How sensitive is SCARF to the choice of hyperparameters, such as the number of predicted peaks, the prefix tree depth, or the neural network architecture? How did the authors tune these hyperparameters, and what are the trade-offs involved? - How generalizable is SCARF to other datasets or domains, such as proteomics, metabolomics, or natural products? How would the authors adapt SCARF to these domains or datasets? - How interpretable is SCARF in terms of explaining its predictions or providing chemical insights? How would the authors improve the interpretability or visualization of SCARF?

Rating

8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

3 good

Contribution

4 excellent

Limitations

The authors have discussed some of the limitations of their work in Section 5, such as the data dependency, the quality of product formula annotation, and the assumptions and simplifications made by their model. However, they could also mention some of the other limitations I pointed out in my previous comments, such as the interference from other molecules or ions, the continuous and structural features of molecules, the correlated fragmentation pathways, and the low-quality or noisy spectra. They could also provide some empirical evidence or analysis to support their claims about the limitations and future directions of their work.

Reviewer ZFnW7/10 · confidence 5/52023-07-20

Summary

The authors propose a model, SCARF, that predicts Tandem-MS spectra in a two-step process: 1) predicting atomic subsets, 2) predicting peaks given the subsets. The model's performance is evaluated on two datasets (NIST20, Natural Product Library) using on spectral similarity, coverage, validity, and retrieval, and compared with relevant baseline models.

Strengths

+ Clear figures and descriptions of the modeling approach. + Clear explanation of background/previous work. The main difference between the SCARF model and other models which use subset prediction is that the SCARF model uses prefix trees to predict subgraphs, so this model considers all relevant subformulae and does not have a limit on the number of valid substructures considered. + Compares against relevant modeling baselines. Evaluates models on coverage and validity to judge the interpretability of the spectra.

Weaknesses

- It would be helpful to provide some more context for the contents of the two datasets, NIST20 and NPLIB. For example, what classes of molecules are present in each dataset (Classyfire might be a helpful tool for this purpose: http://classyfire.wishartlab.com/). What are the molecular weights of compounds of molecules in this dataset? - Retrieval task: If I understand correctly, the retrieval task here is set up such that the 'library' or search space is 50 candidates total, 1 target and 49 other decoys, which are selected from the set of chemical isomers with the highest Tanimoto similarity. Each model is used to predict the spectra for all 50 candidates. Why not perform the retrieval task on the entire NIST 20 library? Or if NIST20 is too large, why not use the set of all structures with the same chemical formula or precursor formula, rather than limiting the task to 50 isomers? I believe that by limiting the search library to a set of 50 molecules that are most chemically similar to the target, the retrieval task might become easier than the real world version. In particularly, typically when one wants to identify a compound, you have few assumptions about what your target molecule is, other than the chemical formula, so you would not be able to filter by chemical similarity to the target. Retrieval tasks in previous works such as Wei et al 2019 were performed on the entire NIST17 dataset.

Questions

- It's great that the SCARF algorithm, and subset prediction generation in general, allows for easy interpretation of peaks. How does the peak annotation produced by SCARF compare to the assignments given by NIST's MS Interpreter tool? - In comparing predicted spectra with ground truth spectra such as in figures 5B&C and Figure A.1: Do you notice any classes of molecules or other patterns where SCARF is particularly good at making predictions? Or particularly bad at making predictions? What is your guess for why the SCARF model generates some extra peaks for some of the molecules in Figure A.1? - How does the performance of SCARF on the retrieval task depend on the size of the molecule, and therefore the number of plausible isomers. - In the retrieval task section, it is mentioned that 'database and model biases can skew retrieval results'. Could elaborate more on the biases that are referred to here? - It could be helpful to provide what you perceive the use case of the SCARF model in real world applications. Would you expect experimental chemists to predict all candidate structures for their target molecule using SCARF, and then perform the retrieval task to identify the right molecule?

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

I do not identify any negative broader societal impacts associated with this work. I agree with the author's statement that the model's performance will be data dependent. If you plan to release your model (which I would highly encourage), it would be helpful to give an overview of the types of molecules that you expect to be in or out of distribution for the model.

Reviewer 8EJR2023-08-10

I have read the other reviews and the authors' rebuttal. I found the rebuttal clear and convincing, particularly with respect to the choice of competing methods.

Reviewer ZFnW2023-08-15

I have read the reviewers responses. Thank you to the authors for the detailed explanations and the additional analysis on the effect on library selection, and the analysis by compound class of cosine similarity performance. Overall it seems that SCARF has consistently high performance across compound classes, even when other methods show much lower performance. I'm guessing that some of the performance differences across classes are due to representation in the dataset, as the authors mention. I edit my rating to a 7

Area Chair YgUD2023-08-18

Rebuttal

Thank you for your rebuttal. We will take it into account in making the final recommendation.

Reviewer 188s2023-08-21

I have read the other reviews and the authors rebuttal. I thank the authors for clarifications and for performing additional experiments (i.e. leaving out precursor ion from cosine calculation).

Program Chairsdecision2023-09-21

Decision

Accept (spotlight)

© 2026 NYSGPT2525 LLC