Contract design involves a principal who establishes contractual agreements about payments for outcomes that arise from the actions of an agent. In this paper, we initiate the study of deep learning for the automated design of optimal contracts. We introduce a novel representation: the Discontinuous ReLU (DeLU) network, which models the principal's utility as a discontinuous piecewise affine function of the design of a contract where each piece corresponds to the agent taking a particular action. DeLU networks implicitly learn closed-form expressions for the incentive compatibility constraints of the agent and the utility maximization objective of the principal, and support parallel inference on each piece through linear programming or interior-point methods that solve for optimal contracts. We provide empirical results that demonstrate success in approximating the principal's utility function with a small number of training samples and scaling to find approximately optimal contracts on problems with a large number of actions and outcomes.
Paper
Similar papers
Peer review
Summary
This paper studies the problem of contract design, focusing on a scenario where a single principal aims to design the reward structure for different outcomes, followed by a self-interest agent. The principal's utility function, which relies on the agent's best response strategy, is a piecewise affine funciton with points of discontinuity at the boundaries. Based on that, the authors introduce a novel approach by using a discontinuous ReLU (DeLU) network to model the principal's utility function. To infer the optimal contract using a trained DeLU network, the authors propose two inference methods: a Linear Programming (LP)-based approach and a gradient-based approach. The authors provide empirical evidence through various experiments to showcase the optimality, sample complexity, and time efficiency of their DeLU network in comparison to the conventional ReLU network, as well as the effectiveness of their proposed inference techniques.
Strengths
1. The idea of representing the principal's utility function using a neural network is novel and insightful. It paves the way for a new research direction in contract design that harnesses the power of neural networks. 2. The motivation behind the DeLU network is well-founded. The authors thoroughly analyze the piecewise affine linear nature of the principal's utility function, including its discontinuity at boundary points. Subsequently, they construct the DeLU network, which aligns with and preserves these crucial properties. 3. The paper exhibits excellent organization throughout, seamlessly integrating theoretical analysis, methodology descriptions, and the experimental section. The logical flow makes it easy to follow and comprehend the content.
Weaknesses
From my view, one significant weakness of the paper is the lack of motivation for utilizing deep learning in the context of optimal contract design. Why is deep learning a relevant and beneficial approach in this domain? To enhance the paper, it would be essential to address this gap and explain the motivation behind incorporating deep learning techniques for contract design. For instance, researchers have explored deep learning for optimal auction design because the theoretical research on optimal auctions has encountered bottlenecks. Similarly, it is crucial to identify the motivation for leveraging deep learning in contract design.
Questions
1. As I mentioned in 'Weakness', what is the significance of using deep learning in optimal contract design? why do we care about this? 2. Why model the utility function and infer the optimal contract instead of directly modeling the contract and optimizing it based on the utility function?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
The authors have discussed the limitations in Section 6.
Summary
The authors focus on the problem of contract design. In this problem, there is an agent which can take some costly action, each action resulting in a distribution over possible outcomes. A principle gets utility based on outcomes, and incentivizes the agent to take actions which will benefit them by transferring them a payment that depends on the outcome. The agent chooses the action that maximizes their expected received payment minus cost; the principal gets a utility equal to the utility of the outcome minus the required payment. In mechanism design for selling goods (e.g. auctions) there is a recent thread of work on “differentiable economics” for learning good mechanisms. The authors to some extent work within this broad area, although the techniques they represent depart from prior work in interesting ways. In particular, rather than using a differentiable neural network to represent the mechanism or the agent’s utility as a function of agent types, they instead represent the principal’s utility as a function of the mechanism (contract) itself. The authors present a new network architecture (DeLU) well-suited to learning to represent principal utilities (which are always piecewise affine, though unlike ReLU network not necessarily continuous). By training these networks in a supervised manner, the authors approximate the utility of a given contract. They then present various techniques to optimize over network inputs to find a high-utility contract, and show that these approaches do in fact give good-performing contracts.
Strengths
The problem of contract design is relevant and interesting, and the authors tackle it in a novel way. The new network architecture and new techniques for optimizing over its input are interesting technical contributions. The experiments are carefully done and support the empirical claims made.
Weaknesses
A major concern I have is that essentially, the authors take many examples of contracts, use them to supervised-learn an approximate utility function, and then optimize on that utility function. Does this reliably work much better than just taking the best contract observed from the training data, skipping this intermediate step? This general problem also shows up in other mechanism design problems where utility functions are learned via supervised learning -- does learning the utility function and using it significantly improve downstream performance compared to just using the fixed training dataset as a "pointwise" utility function? I don't think this is a trivial issue and if unaddressed it is hard to tell if the paper's contribution serves a purpose or not. This is the main reason for my low score and if it can be addressed and no other problems emerge during the discussion phase, I would significantly raise my score. After author response: this issue is satisfactorily addressed.
Questions
Figure 1, although a valuable and informative plot, is unpleasant looking. Now that you are not up against NeurIPS deadline it might make sense to take the time to do it and make it better looking (e.g. change the strange camera perspective, make axes not cut off, etc.) How would this approach differ from directly doing gradient-based optimization in the space of contracts, similar to RochetNet/MenuNet for auctions? It is possible to embed ReLU networks in a mixed-integer program and globally optimize over their inputs. This approach, along with some preprocessing and pruning, has been shown to work quite well even at relatively large scales (e.g. Tjeng et al., https://arxiv.org/abs/1711.07356), especially since good MIP solvers are so fast these days. Could this technique be used effectively to globally optimize over DeLU networks too?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
4 excellent
Presentation
4 excellent
Contribution
3 good
Limitations
The authors adequately discuss limitations of their work, except for the issue mentioned in "Weaknesses". I think better discussion of the size of problem instances in which their approach can be used would be warranted.
Summary
This paper considers an offline learning problem of optimal contract through neural network. The authors propose a novel neural network architecture, called Discontinuous ReLU (DeLU) network, which models a piecewise affine function with discontinuous boundaries --- a representation that captures the principal (contract designer)'s utility function with respect to the different contracts. With the neural network to model the principal's utility, the paper showcases two methods to determine the optimal contract, linear programming or the gradient-based interior-point method. The gradient based method is shown to be more efficient in the experiments.
Strengths
1. The paper is well written and easy to follow. The authors provide a good introduction to the contract design problem and the related literature. The paper is also well organized, with a clear description of the proposed method and the experiments. The figures in this paper are very well designed for readers to intuitively understand the idea. 2. The paper proposes a novel neural network architecture for training a piecewise affine function with discontinuous boundaries, as well as the gradient based optimization for inference. This architecture is very interesting and useful for many other applications in multi-agent learning. The authors also provide a good explanation of the architecture and the intuition behind it. 3. The paper provides extensive empirical experiments on simulated data with comparison of different network architecture, inference methods.
Weaknesses
1. One major concern on the proposed method of this paper is the fact that it attempts to approximate the optimal contract simply as the argmax to the approximated principal’s utility. This does not seem to be a reasonable choice, because there may be a constant gap between the expected agent response and the actual agent response. In particular, the argmax contract at some boundary of the piecewise affine function in the approximated principal’s utility (as pointed out in Lemma 3) is not robustified from the small inaccuracies at the boundary --- at least I do not see how the proposed method can ensure an accurate boundary can be learnt. And this necessarily leads to a constant drop in the testing against the actual agent in a large class of problem instances. If this understanding is indeed correct, it is actually surprising that the proposed method can still achieve a good performance in the experiments. The authors should provide some explanation on this issue; I suspect this is due to the special structure in the synthesized data. 2. The paper in general lacks theoretical analysis. It would be interesting to see some theoretical analysis on the generalization error/sample complexity of the proposed network architecture. The authors could also provide some theoretical analysis on the convergence of the gradient based optimization method, as well as the convergence of the linear programming method. 3. I also expect the paper to include some discussion on the novelty of neural network architecture. I wonder if similar attempts have been made to design neural architecture for special function structures in the hypothesis class. > Both the first and second concern are resolved given the additional experiments and details provided by the authors in the rebuttal.
Questions
Please answer to my concern in the first point of weakness section.
Rating
8: Strong Accept: Technically strong paper, with novel ideas, excellent impact on at least one area, or high-to-excellent impact on multiple areas, with excellent evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
4 excellent
Presentation
4 excellent
Contribution
4 excellent
Limitations
n/a
Summary
This paper proposes a deep-learning approach for contract design. The rationale is: - the principal's utility function should be learned from the data - the problem of approximating accurately the utility function is non-trivial, and the utility function may be discontinuous --- suggesting the development of better models such as those proposed in the paper In addition to the model approximation problem, the authors improve the available algorithms for training and inference. Finally, the authors experimentally evaluate their paradigm according to several dimensions.
Strengths
The authors clearly posed the problem and the presentation is clear.
Weaknesses
At the current stage, I have some doubts about the significance of the contributions and I need that the authors help me to understand better that issue. My feeling is that the machine-learning contribution is not very strong and that the work is primarily a slight variation of deep-learning tools applied to contract design. That is, from a machine-learning perspective, this paper does not provide substantial advancements. This is not necessarily a critical issue for the acceptance of the paper. Many papers just apply machine learning tools to design important applications. More importantly for me, the advancement in the contract design perspective is not really strong. Yes, the authors are providing better approximations, but it is not clear to me the applicability of the results, e.g., due to the need for thousands of samples for the training and it is unlikely such data are available. I agree with the authors that the main works done so far focus on online/bandit approaches and that the authors are providing a completely different perspective. While online/bandit approaches are directed at a small number of samples, deep learning makes the reverse and, in my opinion, could require a too large number of samples for a good approximation. Personally, I believe that something in the middle would be useful.
Questions
*Technicalities* - are the techniques for training and inference provided by the authors important advancements (i.e., non-trivial, non-direct) of the deep learning techniques? if yes, why and how *Applicability* - is correct my understanding that several thousands of samples are necessary for training? if yes, do you believe that making thousands of queries to the principal is reasonable (using $10^6$ in the plot does not help and I would suggest the authors use semi-log plots to show what happens with a small number of samples).
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
Not applicable.
Summary
This paper introduces an automated optimal contract design method from offline data using deep learning. In this setting, a principle deigns a contract establishing an agreement of payments the principle will undertake for the outcomes arising from the actions of an agent. Given a contract, the agent privately selects an action to maximize its expected utility under an action-outcome transition kernel. The principle, unaware of the agent's actions and the transition kernel, aims to maximize its own expected utility. The authors propose the use of neural network function approximation to learn the principle's utility function from offline samples. Due to piecewise discontinuous nature of the principle's utility function and the continuous nature of existing neural network (eg. ReLU), the authors propose the Discontinuous ReLU (DeLU) network to model the the principle's utility function as a discontinuous piecewise affine function, where each piece represents a particular action taken by the agent. Moreover, the authors present a computationally efficient inference method for contract designing based on interior-point method. Finally, the paper empirically evaluates the validity and the performance of the methods presented here on synthetic data.
Strengths
1) Originality: Noting that I am not quite familiar with this field, this is the first ever use of neural network learning of utility functions for contract design based on the author's claims. The authors not only combine two existing ideas, but also innovate a new neural network architecture as the existing models are not capable of capturing the critical discontinuities in the utility function. Moreover, they propose an efficient implementation of the inference computations by utilizing highly parallelized existing deep learning frameworks. They also propose an alternative concave neural network architecture. 2) Quality The claims made here were supported by sound theoretical and empirical evidence. 3) Clarity Even though I am out of the field, it was very easy to follow the paper. The problem and its motivation are explained very clearly. The theory and the experimental results are stated neatly and discussed properly. It was a pleasure to read this paper. 4) Significance Again, since this is outside my expertise, it is less obvious to me why this problem should be studied. The examples provided by the authors as well as a quick Google search were helpful to clarify this. Aside from the real-world significance, the proposed DeLU network and its concave counterpart are novel and worthy of studying further.
Weaknesses
I don't have much to say here.
Questions
I have some question to the authors out of curiosity. 1) How easy is it to extend your method to contract with mixed payment-penalty structure (ie. negative values for f)? 2) Is there a sequential version of contract design problem with cumulative utility maximization? I am thinking of a case where the principle also is deemed responsible of its actions in the contract design and there is a dynamics interaction between the agent and the principle. 3) Is there a specific reason to pick a mother neural network to models the bias term? It'd be more straightforward to write the bias term as a linear function of activation pattern $ b^{L+1}(\mathbf{f}) = \sum_{l=1}^{L} \mathbf{r_l(\mathbf {f})}^T \mathbf{b}_l $. 4) What are some real world applications where the the offline learning with previously collected data could be feasible and reasonable? Are these type of data readily available in practice? 5) Does it require a significantly new treatment to extend this to multi-agent (single principle) case?
Rating
9: Very Strong Accept: Technically flawless paper with groundbreaking impact on at least one area of AI/ML and excellent impact on multiple areas of AI/ML, with flawless evaluation, resources, and reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
4 excellent
Presentation
4 excellent
Contribution
4 excellent
Limitations
Limitations, societal considerations and future works are all discussed.
Thanks for response
Your new experiments have successfully addressed my major concern. I don't think the full description of the experiments needs to take up space in the main paper, but some mention in the text + full description in the appendix seems important to me. Thanks for your thoughtful responses to the other issues. I will raise my score to "7" as soon as I can figure out where the edit button is on OpenReview.
Thanks for your response! We will incorporate the new experiments into the main paper.
Thank you for your prompt reply! We are pleased that our response addressed your concern. We concur that it is very important to include the new experiments in the main paper. We intend to: - Introduce _`Best Training Sample`_ as a new baseline (around Line 288). - Update Fig. 4 of the main paper to include plots of this baseline. - Provide a (concise) description and analysis of the results (around Line 305). - Describe the setup and results in detail in Appendix C. We hope these revisions can assist readers in better telling the contribution of our work.
Response to the authors
I understand the peculiarity of the problem (learning non-continuous functions) and the technical advancement. I also appreciate the authors' reply about the training data. I agree that similar concerns can be found in other settings in which works on deep learning can be commonly found. However, I keep being skeptical about the actual applicability in real-world applications. I raise my scores accordingly.
Thanks for your response!
Thanks for your feedback! We are delighted that our response can address some of your concerns. We have found the review very helpful, and plan to make the following changes to our paper: - Motivate and discuss the technical novelties from the perspective of deep learning (around Line 71). - Update Figure 6 on Page 9, using semi-log plots to show DeLU performance with a small number of training samples. Also update the accompanying discussion in Sec. 5. - Provide a more extensive discussion about real-world applications of the proposed method in a future work section.
Thanks for the detailed response. The additional experiments on the alignment of decision boundary boosts my confidence in the techniques proposed by this work; to some degree, I find it magical and definitely worth follow-up studies from both theoretical and empirical sides. However, I still have concerns on these new experiment results. Therefore, I am only willing to increase my score if the author could provide some important details in the new experiments: (1) how many samples are used to train the DeLU networks in each instances? (2) Why is this approach reasonable, "If the function is non-linear in some (>20%) directions, we mark the contract as on a boundary."? Can you formally describe your algorithm to check whether a sampled contract is on the boundary? Below are some of my additional comments: - The high variance of "Overlapped boundary points (%)" seems to suggests that there are instances where the decision boundary alignment is bad, and I still suspect there might be adversarial instances where boundary alignment is arbitrarily bad (e.g., sharp drops at every boundary). I also wonder whether these instances where alignment is bad coincides with the instances where "sub-"argmax substantially improves the performance. These findings should help us understand the exact mechanisms undergoing when DeLU approximates the agent's decision function. - I suggest the authors to include (and highlight) these new studies on boundary alignment in the next version of the paper. The current lemmas of Section 3 are well-known in contract design theory and can be significantly enhanced to motivate the study of decision boundary alignment. I think the boundary alignment problem is the key learning challenge in strategic setting, and the paper should deserve higher score if the narrative were to be centered around the "surprising effectiveness of boundary alignment by DeLU networks". - Based on my understanding, the proposed techniques seems to be generalizable to problem setups beyond contract design (e.g., Stackelberg games, security games). It is unclear why the authors choose to focus on the contract design problem (or perhaps it is important that the contract space is unbounded?). I wonder if the authors have any comment on this.
Thanks for your valuable questions and comments! (Part 1)
> (Detail 1) `# Training samples.` All the instances here are trained with 50K samples. > (Detail 2) _`Why is this approach reasonable: If the function is non-linear in some (>20%) directions, we mark the contract as on a boundary.`_ Since the principal's utility function is piecewise linear, the function exhibits linearity in the proximity of an interior point. Conversely, when a point lies on a boundary, there is a jump in utilities within its proximity, rendering it unable to pass a linearity test in some directions. Please note that we use exactly the same approach to check for both true and DeLU boundaries.   > (Detail 3) `The algorithm to check whether a contract is on a true/DeLU boundary` is in lines starting with a square symbol ($\blacksquare$). __Algorithm__ Boundary alignment degree calculation __Input:__ True principal utility function $u$ and its DeLU approximation $\tilde{u}$ * $n_{true}=0$, $n_{ovlp}=0$ `Count # true and overlapped boundary points` * __for__ $k=1,\cdots,K$ __do__: `For K random contracts` * $\mathbf{f}^k\leftarrow$ a uniform random contract * $n_{\text{true-nl}}=0$, $n_{\text{DeLU-nl}}=0$ `Count # directions in which the true and DeLU function is non-linear` * __for__ $n=1,\cdots,N$ __do__: `for N random directions` * $\mathbf{d}^n\leftarrow$ a uniform random sample in $\mathbb{R}^{m}$ * $\mathbf{d}^n\leftarrow\delta\frac{\mathbf{d}^n}{||\mathbf{d}^n||}$ `Normalize` * __if__ $u(\mathbf{f}^k)+u(\mathbf{f}^k+2\mathbf{d}^n)\ne 2u(\mathbf{f}^k+\mathbf{d}^n)$: `If the true utility function is non-linear` * $n_{\text{true-nl}}$+=1 * __if__ $\tilde u(\mathbf{f}^k)+\tilde u(\mathbf{f}^k+2\mathbf{d}^n)\ne 2\tilde u(\mathbf{f}^k+\mathbf{d}^n)$: `If the DeLU utility function is non-linear` * $n_{\text{DeLU-nl}}$+=1 * __if__ $n_{\text{true-nl}}/N>\tau$: `If there are many non-linear directions` * $n_{true}$+=1 * __if__ $n_{\text{DeLU-nl}}/N>\tau$: `Check whether it is also on the DeLU boundary` * $n_{ovlp}$+=1 * return $n_{ovlp}/n_{true}$   > (Comment 1) _`Whether the instances where alignment is bad coincides with the instances where "sub-"argmax substantially improves the performance.`_ Thanks for asking about this. In fact, we do generally observe that the performance improvement from the "sub-"argmax method is related to the boundary alignment degree (Table 5). __Table 5__. Performance improvement provided by "sub-"argmax, in decreasing order of boundary alignment, on instances with __(a)__ # actions=5, $\beta_p=0.9$, $\alpha_p=0.5$. |# outcomes|256|8|128|32|4|16|64| |-|-|-|-|-|-|-|-| |Boundary alignment degree (%)|93.89|85.99|84.49|84.16|73.80|46.42|33.45| |Performance improvement (%)|-0.80|+2.08|+0.46|+1.31|+6.27|+2.40|+12.36| __(b)__ # actions=5, $\beta_p=0.3$, $\alpha_p=0.9$. |# outcomes|4|128|32|8|64|256|16| |-|-|-|-|-|-|-|-| |Boundary alignment degree (%)|98.14|97.88|93.97|92.90|86.90|84.74|66.59| |Performance improvement (%)|+0.63|+0.03|+0.17|+0.77|+22.56|+3.63|+3.10|   > (Comment 2) _`I suggest the authors to include (and highlight) these new studies on boundary alignment in the next version of the paper.`_ We have found this exchange very useful, and agree that studying boundary alignment is informative. We plan to make the following changes: - Motivate and discuss the boundary alignment question after the lemmas in Sec. 3. - Analyze the influence of MSE loss on boundary alignment in Sec. 4.2. - Introduce the "sub-"argmax approach in Sec. 4.2.2. - Add a new experimental subsection studying boundary alignment. Incorporate a detailed description of the boundary check algorithm and experiments on the relationship between alignment degrees, DeLU performance, and improvement provided by "sub"-argmax.
Thanks for your valuable questions and comments! (Part 2)
  > (Comment 3) _`Why the authors choose to focus on the contract design problem.`_ First of all, contract design is a basic problem in economic theory (including the 2016 Nobel Prize in Economics), and is gaining increased attention in the digital economy. For example, it doesn't seem too unlikely that we will soon have contracts for LLM-style actor [1] or generative models as simulators of human decision making and behavior (and in the future, likely firm behavior) [2]. That said, we also agree with the reviewer that DeLU can be explored in other problems. We're interested to bring attention to economics problems involving discontinuities, as studied here through this network model of discontinuous functions and on the specific problem of contract design. We hope to motivate additional applications, for example to Stackelberg equilibria in other contexts (and we've also studied applications to problems of Bayesian persuasion), and we'll expand on this in the future work discussion. Additional methodological work can also consider network models of piecewise discontinuous but _non-linear_ functions. [1] Horton, J.J., 2023. Large language models as simulated economic agents: What can we learn from homo silicus? (No. w31122). National Bureau of Economic Research [2] Park, J.S., O'Brien, J.C., Cai, C.J., Morris, M.R., Liang, P. and Bernstein, M.S., 2023. Generative agents: Interactive simulacra of human behavior. arXiv:2304.03442
Thanks for the prompt response! They resolve many of my concerns and I am very excited about these discoveries. I have increased my score accordingly.
Thanks for your feedback!
We extend our appreciation to the reviewer for delivering high-quality and insightful reviews and feedback, which have not only illuminated the weaknesses within our paper but also spurred the discovery of new insights, greatly enriching the depth of our work.
Thank you very much for addressing my questions as well as other other reviewers' concerns. Based on all the other reviews and authors' responses, I'm in favor of maintaining my score.
Thanks a lot for your response!
We would like to express our gratitude for the reviewer's valuable comments and inputs. We find the questions really helpful in improving the quality of our work. Specifically, we plan to make the following changes to our paper: - Discuss why we use another network to generate the bias of the last layer, instead of a linear transformation (around Line 204). - Provide an extensive discussion about the real-world applications of the proposed method, and whether previously collected data is feasible in these applications (in a future work section). - Discuss possible extensions of our work to multi-agent scenarios, sequential contract design, and mixed payment-penalty settings in the Limitation section.
Decision
Accept (poster)