HardCore Generation: Generating Hard UNSAT Problems for Data Augmentation

Efficiently determining the satisfiability of a boolean equation -- known as the SAT problem for brevity -- is crucial in various industrial problems. Recently, the advent of deep learning methods has introduced significant potential for enhancing SAT solving. However, a major barrier to the advancement of this field has been the scarcity of large, realistic datasets. The majority of current public datasets are either randomly generated or extremely limited, containing only a few examples from unrelated problem families. These datasets are inadequate for meaningful training of deep learning methods. In light of this, researchers have started exploring generative techniques to create data that more accurately reflect SAT problems encountered in practical situations. These methods have so far suffered from either the inability to produce challenging SAT problems or time-scalability obstacles. In this paper we address both by identifying and manipulating the key contributors to a problem's ``hardness'', known as cores. Although some previous work has addressed cores, the time costs are unacceptably high due to the expense of traditional heuristic core detection techniques. We introduce a fast core detection procedure that uses a graph neural network. Our empirical results demonstrate that we can efficiently generate problems that remain hard to solve and retain key attributes of the original example problems. We show via experiment that the generated synthetic SAT problems can be used in a data augmentation setting to provide improved prediction of solver runtimes.

Paper

References (21)

Scroll for more · 9 remaining

Similar papers

Peer review

Reviewer 1y8R7/10 · confidence 5/52024-06-20

Summary

This paper presents a procedure for synthesizing hard UNSAT formulas from a given distribution of UNSAT formulas. Concretely, a formula is generated by 1) extracting a core from a seed instance; 2) adding random new clauses; and 3) refining the formula to become harder. In step 3), a GNN is trained to predict UNSAT core of a given formula and the predicted core is relaxed, potentially yielding a larger core. On two problem distributions, LEC Internal and random K-SAT, the proposed method can generate hard instances faster than previous approaches, and the generated instances exhibit similar hardness distribution.

Strengths

- The proposed method strikes a good balance between the hardness of the generated instances and the generation time. It can generate instances much harder than W2SAT, while in a much faster manner than HardSATGEN. - The combination of the two ideas, iteratively relaxing core to generate hard instances and learning to predict cores, is quite clever and elegant. - The fact that the solver performance on the synthetic instances resembles that on the original instances is a good indication that the generated instances are similar in some way to the original instances.

Weaknesses

- It seems that the proposed core refinement technique can be viewed as a post-processing step that can be applied to any initial formulas (e.g., ones generated by W2SAT). Have the authors considered building on top of previous DL-generated formulas? Compared with formulas generated by randomly adding clauses to an original UNSAT core, conceptually, formulas generated by a method like W2SAT, which are trained to embed a formula distribution, seem to be a more justifiable starting point. - The method can still only handle relatively small formulas. - Given that the LEC benchmarks are proprietary, only random K-SAT will be made publicly available. However, one could have considered using dataset such as SATLIB like in previous work (e.g., SATGEN, G2SAT, W2SAT). Minor: - x axis labels are missing in Figure 5. - Line 353: "syntehtic" -> synthetic

Questions

1. We have previously argued that random data differs from real data in important ways that make it unsuitable for machine learning applied to real problems. Could you elaborate?

Rating

7

Confidence

5

Soundness

4

Presentation

4

Contribution

3

Limitations

The authors adequately addressed the limitations. No potential negative societal impact.

Reviewer vCMn4/10 · confidence 5/52024-07-14

Summary

This paper proposes a novel method for generating hard UNSAT problems. The method targets the "core identification" problem and iteratively performs refinement using a GNN-based detection procedure, which preserves the key aspects of the original instances that impact solver runtimes. The experimental results show that the method can generate instances with a similar solver runtime distribution as the original instances in a short time.

Strengths

This paper introduces a novel method to generate hard UNSAT problems in a reasonable time frame, which alleviates the data scarcity problem in the SAT solving domain and improves the performance of deep learning methods in this area. Compared to previous methods, this method can generate SAT problems that are more similar to the original instances.

Weaknesses

1. There is not enough support for the correlation between hardness and core size. For example, in the random 3-SAT problem, the UNSAT core can constitute a significant portion of total instances, even more than 80%, yet modern solvers can easily address these instances. 2. Lack of details about model training. There are no explanations on how to prepare the training dataset. 3. The experimental setting is not convincing. Authors only perform the proposed hard case generation approach on LEC and random k-SAT problems, however, there are no details about how to construct these two datasets. Moreover, it is essential to show results on other datasets to showcase the generalization ability of the proposed method, such as the SAT competition benchmarks. 4. There are some logical problems in the paper’s writing. In the “Core Refinement” part, the paper states that the addition of random new clauses is likely to create a trivial core. However, I find that in the core refinement pipeline, there is no addition of new clause. Instead, new literal is added. The writing here is kind of messy.

Questions

1. Please provide justifications on the correlation between solving hardness and UNSAT core size. 2. Even though considering the minimal UNSAT core, one instance may contain multiple cores. How to handle this one-to-many mapping problem? This problem is significant as it is directly related to the supervision. 3. The random k-SAT and LEC cases have a very strong bias compared to the real instances. The authors should also prove the generalization ability of their proposed approach on more diverse datasets, such as SAT competition benchmarks. 4. In Section 6.2.3, you mention that you use a subset of the training set as the evaluation dataset. This is not the right. 5. When you perform the core refinement step, can the problems generated in the middle be used? How does n, the number of refinement steps, affect the characteristics and performance of the generated problems?

Rating

4

Confidence

5

Soundness

2

Presentation

2

Contribution

2

Limitations

Yes, the authors have stated the limitations of their work.

Reviewer kFMW6/10 · confidence 5/52024-07-14

Summary

This paper introduces HardCore, a novel method for efficiently generating hard Unsatisfiable (UNSAT) Boolean Satisfiability (SAT) problems, addressing the critical challenge of data scarcity. The approach combines a Graph Neural Network (GNN) for rapid core prediction with an iterative core refinement process, enabling the generation of thousands of hard instances in minutes or hours while preserving problem difficulty. HardCore outperforms existing methods in terms of generation speed and hardness preservation, as demonstrated through experiments on both Logic Equivalence Checking (LEC) data and K-SAT Random data. Despite limitations such as applicability only to UNSAT problems and potential scalability issues with extremely large instances, HardCore represents an advancement in SAT problem generation.

Strengths

* The work provides a meaningful contribution to the SAT community by introducing a novel method that generates SAT instances within a reasonable timeframe while preserving the hardness of the original problems. The work has potential applications in various industrial settings, though the scalability limit of the approach might pose a challenge. * The authors conduct extensive experiments, comparing their method against multiple baselines and evaluating various aspects such as hardness preservation, generation speed, and similarity to original distributions. The paper compares against relevant and recent baselines (HardSATGEN, W2SAT, G2MILP), providing a comprehensive view of how HardCore performs relative to the state-of-the-art. * Innovative core refinement process: The iterative core refinement process (Figure 2) is a clever approach to gradually increasing problem hardness while avoiding the creation of trivial cores.

Weaknesses

* The experiments primarily focus on two datasets (LEC Internal and K-SAT Random), with one being proprietary. This somewhat limits the generalizability of the results. * While the paper focuses on runtime distributions and solver rankings, it lacks a deeper analysis of the structural properties of the generated instances (e.g., clause-to-variable ratios, community structure) compared to the original ones. * The core refinement process (Section 5.1) involves adding a single literal to break easy cores. The paper doesn't explore more sophisticated strategies or justify why this simple approach is sufficient. * The paper mentions struggles with "extremely large SAT problems" but doesn't clearly define what constitutes "extremely large" or provide empirical evidence of where the method breaks down.

Questions

How would one extend your technique to more structured classes of formulas?

Rating

6

Confidence

5

Soundness

3

Presentation

3

Contribution

3

Limitations

It would really strengthen the paper if the authors were to study the structural properties (e.g., clause-variable ratio, tree width, hierarchical community structure,...) of the generated instances and identify why they are hard for modern SAT solvers.

Reviewer RF4y7/10 · confidence 4/52024-07-21

Summary

This paper addresses the scarcity of practical data (industrial satisfiability problem instances) for training deep learning methods for SAT solving. The existing data augmentation methods suffer from either the limited scalability or the collapsed hardness of the generated instances. Therefore, the authors introduced a fast augmentation method while preserving the original data's hardness. The primary technical contribution is a fast UNSAT core detection procedure using graph neural networks to monitor the hardness of UNSAT formulae during data augmentation. The empirical evaluation confirmed the new method’s fast generation speed and the preserved hardness of the generated instances. In the application of solver runtime prediction, the augmented data led to a 20-50 percent reduction in mean absolute error.

Strengths

The proposed method achieved the best of both worlds. It has a similar speed to a fast generator while preserving a similar hardness of generated instances to a high-quality but slow generator. The proposed idea is simple and easy to follow, yet effective. The paper is well-written. The authors performed extensive evaluations to demonstrate the preserved hardness of the generated instances in various aspects, such as the preserved average hardness, the hardness distribution over instances, the runtime distribution per solver, and the best-solver distribution over instances. The authors illustrated the effectiveness of their method in a practical application (solver runtime prediction). The augmented data successfully reduced the mean absolute error by 20-50 percent.

Weaknesses

The authors don’t present how well the core predictor generalizes to other datasets. We may need to re-train the core predictor when applied to a new family of instances if it doesn’t generalize well, which would introduce extra overhead not presented in the current evaluation. The technique to eliminate an easy core described in lines 176-191 has been proposed in [1]. The authors should cite the work and move the content to related work. [1] Yang Li, Xinyan Chen, Wenxuan Guo, Xijun Li, Junhua Huang, Hui-Ling Zhen, Mingxuan Yuan, and Junchi Yan. 2023. HardSATGEN: Understanding the Difficulty of Hard SAT Formula Generation and A Strong Structure-Hardness-Aware Baseline. In Proc. of ACM SIGKDD Conf. Knowledge Discovery and Data Mining In line 190, the satisfying solution should be (A=0, B=0, C=1). The axis label is missing in Fig. 5.

Questions

1. How much time did you spend on collecting data and training the core predictor? If the core predictor doesn’t generalize well to a new family of instances, is the re-train time going to be a huge overhead of your method? 2. How many instances are generated from a single seed instance by HardCore and HardSATGEN? If you could use one seed instance to generate multiple instances, why can’t you keep the same number of seed instances for HardCore and HardSATGEN in line 304? 3. Why do you use the extremely small data size (e.g., 10 and 100) in Table 2? It should be considerably insufficient, even after the three-time augmentation, to train a fair runtime predictor.

Rating

7

Confidence

4

Soundness

4

Presentation

4

Contribution

3

Limitations

The limitations have been well addressed. The authors acknowledged that the method can only be applied to UNSAT instance generation and is not suitable for the SAT case. They also mentioned that the results are limited to the current datasets and their method can’t scale to large SAT problems with millions of clauses.

Reviewer vCMn2024-08-11

Thank you for the responses, which clarify most concerns. I'll raise my score accordingly. However, I still have questions regarding Weaknesses 1 and 3. 1. I understand that the proposed approach adds literals to clauses in the UNSAT core rather than creating additional clauses. However, there seems to be a conflict, as mentioned in Figure 2: “As steps (1) and (2) are repeated, the core gradually becomes larger, raising the hardness of the generated instance.” This sentence should be further clarified to avoid misunderstandings. 2. I noticed the global rebuttal results that demonstrate generalization ability. However, the selected "Tseitin" family likely shares a similar distribution with LEC problems, since LEC data is also converted into CNF using the Tseitin Transformation. Could the authors clarify this assumption? I suggest the authors test the proposed approaches on non-circuit-based families and novel cases in SAT benchmarks, such as cryptography and scheduling.

Authorsrebuttal2024-08-11

We express our gratitude to the reviewer for acknowledging our response. 1. *I understand that the proposed approach adds literals to clauses in the UNSAT core rather than creating additional clauses. However, there seems to be a conflict, as mentioned in Figure 2: "As steps (1) and (2) are repeated, the core gradually becomes larger, raising the hardness of the generated instance." This sentence should be further clarified to avoid misunderstandings.*\ \ We agree that the sentence needs to be edited and we will revise it. We will amend the text to clarify that the core we consider at each step is specifically the easiest core (because traditional core detection effectively solves the SAT problem). We also intend to replace references to the "enlarging" of cores with their "hardening" in order to emphasize our crucial assumption as discussed previously. Thus, this phrase would read: "As steps (1) and (2) are repeated, the easiest core of the problem is gradually refined, raising the hardness of the generated instances." 2. *I noticed the global rebuttal results that demonstrate generalization ability. However, the selected \"Tseitin\" family likely shares a similar distribution with LEC problems, since LEC data is also converted into CNF using the Tseitin Transformation. Could the authors clarify this assumption? I suggest the authors test the proposed approaches on non-circuit-based families and novel cases in SAT benchmarks, such as cryptography and scheduling.*\ While it is true that the LEC and Tseitin data share an element of origin, we note that the Tseitin data is made up of problems generated in many different but random ways. For example, the data-points contributed by Elfers in the 2016 Hand-Crafted track \[1\] are generated by providing grid-graphs of varying dimensions to the Tseitin transformation. These grid-graphs are very different from the circuit-graphs seen during LEC. In consequence we see very different runtime-distributions. Comparing Figure 2 in the rebuttal pdf to Figure 6 in Appendix B of the paper and Figure 4 in the text, we note significant differences. Most notably, Solvers 1 and 6 (which we generally have found to be strong for industrial data and weak for random data, are much slower for the Tseitin data). Therefore, we do not believe the Tseitin data shares a similar distribution to the LEC data as the reviewer has suggested. Additionally, while other data such as Cryptography and Scheduling were considered, Tseitin was the family with the most UNSAT problems available in the benchmarks. Since more data enable us to provide more complete distributional experimentation as well as more results on the data augmentation task, we chose the largest dataset: the Tseitin. With that said, we have begun experiments on a dataset of 100 scheduling problems taken from the SAT Competition data using the filter "`family=scheduling and result=unsat`". Of the two families suggested by the reviewer, scheduling had more problems, and so we chose it over cryptography. If time permits for completion before the end of the discussion period we will share results. Otherwise, we will report the fully processed results in the appendix of the revised paper. \[1\] *Proceedings of SAT Competition 2016: Solver and Benchmark Descriptions*, volume B-2016-1 of Department of Computer Science Series of Publications B, University of Helsinki 2016. ISBN 978-951-51-2345-9.

Reviewer 1y8R2024-08-12

I thank the authors for their clarifications and new results. I would like to keep my score.

Program Chairsdecision2024-09-25

Decision

Accept (poster)

© 2026 NYSGPT2525 LLC