PERFOGRAPH: A Numerical Aware Program Graph Representation for Performance Optimization and Program Analysis

The remarkable growth and significant success of machine learning have expanded its applications into programming languages and program analysis. However, a key challenge in adopting the latest machine learning methods is the representation of programming languages, which directly impacts the ability of machine learning methods to reason about programs. The absence of numerical awareness, aggregate data structure information, and improper way of presenting variables in previous representation works have limited their performances. To overcome the limitations and challenges of current program representations, we propose a graph-based program representation called PERFOGRAPH. PERFOGRAPH can capture numerical information and the aggregate data structure by introducing new nodes and edges. Furthermore, we propose an adapted embedding method to incorporate numerical awareness. These enhancements make PERFOGRAPH a highly flexible and scalable representation that effectively captures programs intricate dependencies and semantics. Consequently, it serves as a powerful tool for various applications such as program analysis, performance optimization, and parallelism discovery. Our experimental results demonstrate that PERFOGRAPH outperforms existing representations and sets new state-of-the-art results by reducing the error rate by 7.4% (AMD dataset) and 10% (NVIDIA dataset) in the well-known Device Mapping challenge. It also sets new state-of-the-art results in various performance optimization tasks like Parallelism Discovery and NUMA and Prefetchers Configuration prediction.

Paper

Similar papers

Peer review

Reviewer ubPp7/10 · confidence 5/52023-06-28

Summary

The paper proposed a novel graph-based program representation, PerfoGraph, which is based on the current state-of-the-art method PrograML and aims to address its limitation by providing numerical awareness, introducing new tactics for handling local variables, and supporting aggregate data types. By conducting experiments on various performance-optimizing oriented downstream tasks, it is shown that the proposed PerfoGraph representation is more effective compared to the existing methods.

Strengths

- The paper was clearly presented overall. - The paper showed great originality in proposing a novel program representation method, PerfoGraph, that enhances previous results by providing numerical awareness, introducing new tactics for handling local variables, and supporting aggregate data types. - Regarding numerical awareness, the authors introduced a novel idea to split the numbers into digits and positions before encoding, which could effectively represent numbers without a huge vocabulary. - Regarding local variables, the author took good consideration of GNN's mechanism and adjusted the graph representation accordingly.

Weaknesses

- Unconvincing experiment setup in comparison with PrograML. In the original PrograML paper, MPNN was introduced as an encoder. However, the proposed PerfoGraph chose RGCN for the same role. The reviewer believes that it is possible to also use RGCN as PrograML's encoder. Thus, it is unclear whether the performance improvements in the comparisons with PorgraML stem from the novel graph representation or from a more suitable encoder.

Questions

1. The authors claimed that the proposed PerfoGraph representation could better capture **composite** data structure information. However, the paper only introduced a novel method for incorporating list-like data structures, such as **arrays and vectors**, into the graph representation of the program. The reviewer's question is how PerfoGraph could support more general composite data types such as struct. From the current presentation, the reviewer's opinion is that the term **aggregate data types** suit better. 2. How does the choice of aggregation function impact the performance of PerfoGraph? Is it possible that the information compression introduced in the aggregation function confuses the deep learning model? 3. Do the authors consider moving the part of the ablation study from Supplementary Materials to the main text to better clarify the importance of numerical awareness in program representations? 4. Possible error in Figure 2a: In section 4.1, the authors used Figure 2a as an example of the PrograML representation to point out that PrograML assigns a position feature to the address edge of the store node. However, there is no corresponding label near the first store node.

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

The authors have addressed the limitations of their work in Section 6.

Reviewer vHZH7/10 · confidence 4/52023-07-02

Summary

The paper presents a graph-based representation for LLVM IR programs for processing with graph neural networks. The representation is based on ProGraML, an established technique for such graph representations, but adds several features to the graph: collapsing nodes that refer to the same variable, adding additional edges between memory stores and the variables they can affect, explicitly representing constants in the graph, and representing compositional data types (e.g., arrays of arrays) as multiple distinct nodes rather than single nodes. The authors evaluate across a range of LLVM IR learning tasks, and show that Perfograph outperforms ProGraML (and all other baselines in most contexts).

Strengths

* Finding and understanding good program representations is an important problem in the field of learning for code * The proposed approach identifies a novel set of features which qualitatively seem important, and which are validated to be useful in the ablation study * Assuming that the evaluation is indeed fair (see weaknesses below), the proposed approach significantly outperforms ProGraML, achieving a large increase in accuracy (seemingly solving many of these tasks near perfectly).

Weaknesses

* Fairness of comparison against baselines: * Perfograph is a version of ProGraML with some added and removed nodes and edges. Each added node/edge adds additional power for a graph neural network. For a fair comparison against baselines (ProGraML in particular), the authors must show that for a given compute budget (i.e., FLOPs or wall clock time), Perfograph outperforms ProGraML; this would involve either scaling up/down ProGraML or Perfograph. * Are the datasets used exactly the same as in prior work? For example, Section 5.4 reports using an 80/20 train/test split, which I cannot find discussed in [10] * Line 372 mentions that the authors "could not reproduce the results in PROGRAML paper" for the algorithm classification task. Does this lack of reproducibility also affect the evaluated benchmarks that were not in the ProGraML paper (seemingly, everything other than 5.2)? If so, this is a major point that must be discussed at a high level in the paper. * Given the advent of large-context Transformers, I would be curious how well such a model performs when applied to the raw IR (i.e., without an explicitly featurized graph representation). However, since this paper is explicitly building on the prior work of ProGraML, I do not consider this to be a condition for acceptance. * Overall, the clarity of the submission is moderate. * Section 4.1: without a deep understanding of the ProGraML representation, it is very hard to understand these graphs (and therefore the issues that the proposed approach fixes in them). Perhaps highlighting or otherwise identifying the specific nodes discussed could help. * Line 323: is this an arithmetic or geometric average? * Line 368: what are the units of these errors? * Several claims about novelty or related work are misleading: * Lines 72-73: "this token-based representation... fails to capture..." This claim is misleading. Sequence processing models are entirely capable of learning such graph-structured relationships; such complex relationships are also not unique to code (e.g., natural language has pronouns, nested structures, etc.) * Line 176: the proposed "Digit Embedding" is very similar to some digit representations approaches used in Transformers (for example, Geva et al. 2020) which embed each digit, add/concatenate with a positional encoding, and aggregate with attention. * I believe the paper would be significantly stronger with the ablation studies moved to the main body, but this is a stylistic choice and is not a factor in my score * If I'm understanding Appendix 2 correctly, the Perfograph GNN is evaluated with only 2 layers; ProGraML is evaluated with 6 (Cummins et al., Section 6.2). How was this parameter chosen? What is the average diameter of the evaluated graphs (for both Perfograph's graphs, and ProGraML's graphs)?

Questions

* Please see the questions in "clarity" in the discussion of weaknesses above (and other questions throuhgout). * Does Perfograph use more compute or have more capacity than ProGraML? Does it still outperform ProGraML when given equal power? * What steps have the authors taken to ensure that the implementation of ProGraML compared against in Sections 5.3-5.7 is correct (since as noted in Line 372, in one context the authors "could not reproduce the results in PROGRAML paper")? * Do all other baselines use the exact same training/test set split? * How were hyperparameters tuned for Perfograph? I would be willing to raise my score given evidence that for the same compute budget, Perfograph still outperforms ProGraML (or evidence that my assumption here, that Perfograph uses more compute than ProGraML, is incorrect).

Rating

7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

2 fair

Contribution

3 good

Limitations

The paper does not explicitly discuss limitations of the technique.

Reviewer inFa6/10 · confidence 3/52023-07-07

Summary

This work proposes Perfograph, a program graph representation based on LLVM-IR and an extension to ProGraML. This graph representation is designed for the purpose of performance optimization and program analysis applications based on graph neural networks (GNN). This work made three contributions to the existing representation. First, it tracks reused local identifiers and memory locations. Second, it uses decimal-based encoding to embed numerical constants into the graph. Last, it breaks down array and vector types into multiple nodes. Perfograph is tested on 6 downstream tasks: device mapping, parallelism discovery, parallel pattern detection, NUMA and prefetchers configuration prediction, thread coarsening factor (TCF) prediction, and algorithm classification. In the device mapping challenge, the new model outperforms ProGraML by 7.4% on the AMD dataset and 10% on the NVIDIA dataset.

Strengths

1. This paper is well organized. 2. This work clearly listed all 3 of its design decisions. 3. This work contains detailed evaluations on multiple targets. 4. Each evaluation contains a carefully described problem definition, dataset, and results.

Weaknesses

1. The improvement of this work over ProGraML seems limited. For example, the 3 main design contributions are not fundamentally new. They are more like minor tweaks on the existing ProGraML system. 2. The evaluation section does not show how each design choice contributes to the overall improvement.

Questions

1. It claims that “Perfograph is built on top of ProGraML” in Section 4, but also claims “A **new** compiler and language agnostic program representation based on LLVM-IR”. Is this work an extension of ProGraML, or is it a new system? This is somewhat unclear. 2. Section 4.1 claims that this work combines multiple writes to the same variable into one node. However, LLVM deliberately separates multiple writes for the ease of certain aliasing analyses, the opportunities of reordering, and optimizing for parallelism. It would be helpful to provide more evidence that this combined representation outperforms separated representation in all (or most) downstream tasks. 3. Section 4.2 mentions that decimal-based encoding is used for numerical constant embedding. This seems counterintuitive since computers natively speak binary. 4. Section 4.3 claims that Perfograph supports composite data types (but only covers arrays and vectors). It would be helpful to further explain how Perfograph deals with programs that use structs (and maybe unions).

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

N/A

Reviewer f7Gk6/10 · confidence 4/52023-07-07

Summary

The research identifies limitations in the current state-of-the-art program representation PROGRAML, in capturing features of numerical values and composite data types. To address these limitations, this work introduces an enhanced GNN-based program representation for LLMV-IR with modifications to the nodes and edges of the program graphs to capture features of numerical values and composite data structures. It also presents a digit embedding approach to enhance numerical awareness within the representation. The evaluation shows that PERFOGRAPH reduced the prediction error rate by 7.4% (AMD dataset) and 10% (NVIDIA dataset) in the Device Mapping challenge compared to PROGRAML. Additionally, it outperforms PROGRAML and traditional rule-based methods in tasks such as determining if a loop is parallelizable, classifying the parallel patterns, and configuring NUMA and prefetchers selections.

Strengths

1. PERFOGRAPH employs effective graph modifications to enhance the learnability of the LLVM-IR graph for GNNs. These methods are well-reasoned and provide valuable domain information for improving program graph abstractions in learning. For instance, PERFOGRAPH unifies the graph nodes for the same local identifier variable in the program graph. It also modifies the LLVM IR graph to add edges from the store instruction to the variables it modified. These modifications all simplify the complex structure of the LLVM-IR graph. 2. PERFOGRAPH also distinguishes between composite data structures and regular data types, which prior approaches did not adequately address. This distinction is crucial because operations involving composite data structures can differ significantly from those involving regular data types. 3. The evaluation of PERFOGRAPH is comprehensive, considering six performance-oriented tasks. The results demonstrate that PERFOGRAPH outperforms PROGRAML in terms of accuracy on these tasks. More importantly, in parallelism discovery tasks, PERFOGRAPH achieves higher accuracy compared to traditional rule-based approaches.

Weaknesses

1. It is unclear to me how the digit embedding ensures that unknown numbers are not encountered during the test phase and what specific information is encoded through the addition of digit and position embeddings. 2. The reasons behind representing multi-dimensional arrays as multiple nodes are not adequately explained. It remains unclear how this representation enhances expressiveness or facilitates reasoning by GNNs. 3. An ablation study that isolates the individual benefits of LLVM-IR graph modification, digit embedding, and support for composite types is absent. Such a study would provide valuable insights into understanding the specific advantages offered by each component. 4. PERFOGRAPH may have higher computational requirements compared to PROGRAML, potentially posing challenges for its practical deployment.

Questions

1. It would be helpful to visualize the digit embedding values of different numerical values to facilitate better understanding. 2. It would be beneficial to compare the cascaded context approach for representing multi-node representation with alternative representations (e.g. storing only the size of one rank) to assess the effectiveness in encoding array information. 3. To provide a more comprehensive analysis, I recommend conducting an ablation study for each major representation enhancement to understand the specific benefits of each component. 4. It would be valuable to measure and compare the inference time of PERFOGRAPH with PROGRAML and other rule-based analysis methods to evaluate the computational efficiency.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

3 good

Presentation

4 excellent

Contribution

3 good

Limitations

N/A

Reviewer f7Gk2023-08-19

Thank the authors for the response. I found the supplementary material answered most of my questions. The ablation study indicates that digital embedding has a lesser impact on accuracy (1%) compared to the composite data type (>5%). The title is a bit misleading in this case as it emphasizes more on numerical awareness. I'm still not fully convinced that digital embedding is effective in capturing the magnitude of numerical values. One observation is that it seems it will cluster numbers with the same digits together (e.g. the point of number 50 is very close to the numbers in [50000-50090] in Fig.2 of the supplementary material. I tend to keep my score unchanged.

Authorsrebuttal2023-08-21

Thank you for sharing your feedback. We acknowledge that a slightly reduced impact of Digit Embedding is observed on the Device Mapping dataset, However as this data set is relatively small, we conducted a further experiment on the DiscoPoP subset of the Parallelism Discovery task by eliminating Digit Embedding. We found our accuracy dropped to **95.26% (3.74% decrease)** with Digit Embedding removed from that experiment. From this observation, we believe it is fair to say that the role of Digit Embedding can be more prominent on more extensive datasets.

Reviewer ubPp2023-08-18

Thank you for your rebuttal. It has further clarified my understanding of the proposed work. Given that I've previously given a 7-accept based on the novelty and originality of the proposed paper, I've chosen to keep my score unchanged.

Reviewer inFa2023-08-21

Many thanks for the authors' detailed response. The new performance data and technical details are very helpful for me to better understand this work. Although I still have some minor concerns about the novelty of this work due to the similarities between PerfoGraph and ProGraML, I would like to change my score to "Weak Accept" because I believe this work contains enough intellectual merits.

Program Chairsdecision2023-09-21

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC