Differentially private (stochastic) gradient descent is the workhorse of DP private machine learning in both the convex and non-convex settings. Without privacy constraints, second-order methods, like Newton's method, converge faster than first-order methods like gradient descent. In this work, we investigate the prospect of using the second-order information from the loss function to accelerate DP convex optimization. We first develop a private variant of the regularized cubic Newton method of Nesterov and Polyak, and show that for the class of strongly convex loss functions, our algorithm has quadratic convergence and achieves the optimal excess loss. We then design a practical second-order DP algorithm for the unconstrained logistic regression problem. We theoretically and empirically study the performance of our algorithm. Empirical results show our algorithm consistently achieves the best excess loss compared to other baselines and is 10-40x faster than DP-GD/DP-SGD.
Paper
Similar papers
Peer review
Summary
This paper studies DP convex optimization by using second-order methods. The authors present a private version of the cubic regularized Newton method and prove it faster than the first-order methods under strongly convex case. They also provide efficient second-order method for solving the DP logistic regression problems.
Strengths
This is the very first paper to study the second-order methods for DP convex optimization. The author present algorithms for both general strongly convex functions and logistic regressions which are very novel. The Algorithm 3 for DP logistic regression consider the special structure of the object function which is very interesting.
Weaknesses
The algorithm for solving the general strongly convex functions seems straight forward. It is a combination of cubic-regularized Newton and DPGD. The requirement of solving subproblem by DPGD at each iteration is not satisfying and may cause the total computation cost even worse than the first-order method. Can the author present the total number of the iterations (including the inner loop) and compare it with the first-order DP method? Although the author present efficient method for solving the DP logistic regression, Algorithm 3 seems quite different from the meta algorithm for the general strongly convex optimization problems. Can the authors show the connection between the Algorithm 3 and Algorithm1,2?
Questions
See weakness part.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
2 fair
Contribution
3 good
Limitations
N/A
Summary
This work investigates the use of second-order methods in differentially private convex optimization for machine learning. The authors propose a private variant of the regularized cubic Newton method and demonstrate its quadratic convergence and optimal excess loss for strongly convex loss functions. They also design a practical second-order differentially private algorithm for unconstrained logistic regression, which outperforms other baselines in terms of excess loss and is significantly faster than DP-GD/DP-SGD, achieving a speedup of 10-40 times.
Strengths
This paper is good as it has been a really hard to use second-order information while preserving DP property. The contribution is novel and the proof seems to be correct.
Weaknesses
1. Maybe extend to use laplace noise? 2. The presentation of the paper is a little bit awkward. For example, the formal DP theorems and proof of Alg. 3 only presents in Appendix. It is really hard to find the corresponding reference. This paper might need to be rewritten. 3. Experiments: 1. Compare the true running time. It takes a lot of time to compute the CLIP and ADD operations as it needs to compute SVD. So instead of seeing how many steps it needs, please present the true computational time to show the true running time improvement compared with DP-SGD and DP-GD. 2. Compare memory usage. SVD needs tons of memory. 3. DP-SGD brings randomness and sometimes accelerates the training procedure. Please add a comparison with DP-SGD. 4. Compare with different learning rates (I know there does not exist a learning rate in Alg. 3 but a learning rate exists in DP-(S)GD). It would be interesting to see the impact of different learning rates (10, 0.1, or 0.01).
Questions
See Weakness.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
2 fair
Presentation
2 fair
Contribution
3 good
Limitations
See Weakness.
Summary
The paper introduces the use of second-order methods in differentially private optimization. In particular, two algorithms are proposed. One is based on cubic regularized Newton and works for strongly-convex functions. The other one is designed specifically for the logistic regression problem. Numerical experiments on the logistic regression are provided.
Strengths
The topic of using second-order methods in differentially private (DP) optimization is under-explored. In this sense, the paper can be a good starting point and introduce the idea to the DP community. The paper is well-written and easy to follow, with theoretical results that do not exist before and are supported by numerical experiments.
Weaknesses
1. The theoretical justification of the benefit of using second-order methods is not strong enough. For the considered smooth strongly-convex setting, first-order methods can already achieve the same rate in linear time $O(n)$ [arXiv:1703.09947, arXiv:1802.05251, arXiv:2005.04763, arXiv:2102.05855, arXiv:2206.00363]. As a comparison, since the subproblem of the cubic regularized Newton cannot be efficiently solved due to nonsmoothness, the total gradient complexity of the proposed method is $NT\sim n^2$. The main improvement is mostly on the oracle complexity but with more assumptions on the second-order information. Even for the oracle complexity, the proposed algorithm achieves $\sqrt{L_2}/\mu^{3/4}+\log\log(n)$, which is hard to compare with the one achieved by first-order methods (e.g., $(\sqrt{L_1}/\sqrt{\mu})\log(n)$ for private versions of the Nesterov's accelerated gradient descent [arXiv:2102.04704, arXiv:2206.00363]). I am also wondering if an improved lower-bound is possible with second-order information. 2. The convergence analysis of the logistic regression case seems to be not complete. I don't find (also in the appendix) any specific rate, complexity, and comparison with first-order methods. The proposed method requires the local strong convexity assumption at the optimum. Does it hold for the logistic regression loss? 3. The proposed method requires computing the inverse of the second-order information, which could be hard for large-scale experiments. Also, all the numerical experiments in this paper use small models and datasets, where computing this inverse does not cost too much. Given that DP-GD already completes the task within 1 minute, such a 10-40$\times$ improvement is not considered to be so surprising. 4. What does $T^*$ mean in Table 1, or what is the stopping criterion to define $T^*$? Why do Figure 3 & 4 only show the first 10 steps or 0.6s? I guess the noise added for the proposed method is computed according to the theory. Are there any numerical privacy accounting methods used in the experiments to justify the effectiveness of the added noises? In case the privacy analysis is wrong and less noise is added than what is required, this might not be a fair comparison. 5. Minors: It might be good to also summarize related works for non-DP second-order optimization algorithms. What does privacy budget for direction mean in $\lambda_{0,t}$ in line 249.
Questions
See weakness.
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
See weakness.
Summary
This paper considers the problem of differentially-private convex optimization and discusses how second-order information can be utilized to accelerate the optimization process while also achieving the optimal excess error, with DP involved. The main contributions of this paper are the two proposed algorithms, and the corresponding analyses; 1) A second-order DP algorithm that is based on the cubic-regularized Newton's method - for a specific class of convex functions, 2) A second-order DP algorithm for the logistic regression problem.
Strengths
The paper considers an important problem which studies how the performance of a differentially-private convex optimization process can be enhanced with the use of second-order information. The paper is well organized and rigorous proofs have been provided for the claims made. The first algorithm presented in this paper (DP variant of the cubic-regularized Newton's Method) achieves the optimal excess loss and quadratic convergence. The proposed second-order DP algorithm for logistic regression achieves equal or better excess losses and lower computational times compared to the existing DP algorithms, based on the experimental results.
Weaknesses
The authors could clearly state their contributions and remark on the significance of the contributions, as in some cases (for example algorithms 1 and 2), it seems like the authors have simply incorporated DP into existing non-private results. The main paper lacks justification on the privacy guarantees. The authors could comment on how the stated DP guarantees are achieved by the selected noise parameters.
Questions
1) For a given set of inputs in algorithm 3, how can one determine whether the SOI modification is "add" or "clip"? The reader could benefit from a clear description of this part (lines 3-6) in algorithm 3. 2) Do the authors assume any characteristics (i.e., any specific distribution or any statistical characteristics) on the dataset $S_n$ in either of the algorithms?
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
The authors have clearly stated the limited types of functions which the proposed algorithms can be applied on, and have also commented on possible extensions to other classes of functions.
Summary
The paper introduces an algorithm that utilizes second-order information to enhance the speed of private convex optimization, concurrently ensuring optimization excess error is minimal. The outcomes of the experiments indicate that employing second-order information can expedite Differential Privacy (DP) optimization. In addition, it achieves an excess loss that either equals or surpasses that of first-order methods like DP Gradient Descent (DP-GD).
Strengths
The strength of this paper are 1) The application of second-order methods to convex optimization is challenging research area. While some progress has been made, it's still uncertain whether second-order methods can be as practical as first-order methods. This paper unveils a novel second-order approach for Differential Privacy (DP) optimization, which demonstrates optimal efficiency for strongly convex functions. 2) The authors offer an analysis of both the local convergence assurance associated with Hess-clip and Hess-add, as well as the global convergence guarantee for QU-clip and QU-add. 3) The numerical findings indicate that the proposed method outperforms DP Gradient Descent (DP-GD) substantially, demonstrating a speed that is 10 to 40 times quicker for the datasets tested.
Weaknesses
The weakness the paper include 1) The potential impact of this work remains uncertain to me. While it presents strong results in the field of Differential Privacy (DP) private machine learning, it raises the question: can these results or concepts be applied more broadly? 2) The mini-batch version offers interesting numerical results, however, the loss it produces is notably higher compared to the full-batch version.
Questions
On page 1, "One of the major drawbacks of DP-(S)GD is slow convergence. We argue that the main reason for this is the difficulty of choosing the hyperparameters (/eta; T)." Please explain why it is that. How about gradients? For minibatch version, how to make sure the second-order information is still relevant/meaningful from one batch to another batch?
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
Yes, I think so.
Summary
This paper focuses on the problem of private convex optimisation using second order information where the main motivation is the acceleration of private convex optimisation problem as DP-SGD shows slow convergence. This is done in two parts. Firstly, they introduce the privatised version of the cubic method of Nesterov and Polyak when the loss function is strongly convex. In this step, the privatisation is done by solving the optimisation problem given by the global cubic upper-bound privately (DP-Solver). They analyse the algorithm and provide convergence guarantees. Since the cubic method is computationally expensive, for the rest of the paper the authors focus on a method inspired by Newton's method with its specific application to logistic regression. Specifically, they provide a global upper bound for the logistic loss and for which the optimisation the upper bound has a closed form. They then privatise this method by privatising the steps of the optimisation algorithm. They do this in two steps. First by adding the proper amount of noise to the information from gradient and then adding noise at the update step. Finally, they empirically compare the performance of their proposed method for logistic regression to DP-(S)GD and objective perturbation.
Strengths
1. Convex optimisation appears in many settings in Machine Learning and given the increasing attention to privacy, this is a timely problem to study. 2. The combination of ideas to use a method similar to newton's method while making sure the upper bound is global like in the cubic method is nice. 3. The convergence analysis for the proposed algorithms. 4. The details of the experiments are explained thoroughly and for the most part, they seem fair.
Weaknesses
1. While private convex optimisation is an important problem and the techniques are interesting, I feel the scope of the paper is a bit limited as it only allows us to use the results for logistic regression in a private setting. For any other method that uses convex optimisation, like kernel methods, the user would need to calculate the sensitivity of the queries in algorithm 3 at which point the method being faster than DP-SGD might not be justification enough for this method. 2. One of the main motivations for this paper is the slow convergence of DP-SGD. While the authors provide references to other bottlenecks for DP-SGD like the batch size and hyper-parameter tuning, there are no references for the slow-convergence of DP-SGD and the main thing supporting this claim in the paper is Figure 1.
Questions
1. The upper-bound for the convergence analysis for ($(lw) - l(w^*)$) in both Theorem 1 and Theorem 2 is done w.r.t $w^*$ which is the optimal w not attainable in the case where $\rho>0$ as given by the lower-bound given in [BST14]. Does it not make more sense to provide the upper bound w.r.t the set of parameters $w$ which are achievable under privacy constraints? 2. Throughout the paper you mention that your proposed method is 10-40 times faster but looking at Table 1 this is the case for $\epsilon = 10$, otherwise the range is something like 3-40. Is there a reason for mentioning 10-40? 3. [KSJ18] show that the Newton's method globally converges under some conditions which hold for logistic regression. Given this result, why do we need to build the global upper bound of lemma 5.1? 4. Is the idea for building global upper bounds for newton's something done for the first time in your paper? If not, it might be a good idea to add some references. Minor comments: 1. You can add other examples where convex optimisation appears in ML. Some examples are kernel regression and extreme learning machines. 2. I think adding some explanation and details to Theorem 4.2 might help the reader to digest the theorem a bit better i.e. What is the value of $w$ for which the transition between the convergence rates happen? My intuition is that the privacy constraint makes a set of parameters centred around $w^*$)admissible and while $w$ is outside of this range the convergence rate is slower and once we are within those parameters the convergence is much much faster. 3. For theorem 5.6, it might help to mention that the relationship between the semi-norm and the $\ell^2$ norm for ease of understanding.
Rating
6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness
3 good
Presentation
3 good
Contribution
3 good
Limitations
It is similar to the comment I make in the weaknesses. The authors have not clearly mentioned that while the method is faster than DP-SGD, it requires an expert to get the algorithm up and running for other convex optimisation problems. Additionally, it is known that the computational complexity of Newton's method scales badly with dimension d which I think should be mentioned in the paper.
> Putting it all together, the runtime of Algorithm 1 in practice will be no worse than $\mathcal{O}(loglog(n))$ times the runtime of the first-order method we choose as DPSolver. Does this mean that theoretically, the runtime of the proposed method is indeed worse than the DP-GD or other first order methods?
Reply to Reviewer Xt6K regarding the Runtime of Our Algorithm 1
The standard way to theoretically analyze runtime in the optimization theory literature is to consider *oracle complexity*. **Our Algorithm 1 has lower oracle complexity than is possible with first order methods.** That is the point of the result; we did not try to optimize any other complexity measure. If one is interested in the total runtime, then the experimental results are more informative. The oracle complexity of Algorithm 1 does not depend on the DPSolver subroutine. However, the total runtime is $O(\log\log n)$ times the cost of computing a gradient & Hessian plus running DPSolver. For simplicity, we used a first order method as the DPSolver subroutine; it is possible that there are faster subroutines we could use instead. (The cubic function is a much simpler function than the worst-case scenario that witnesses the lower bounds, so it seems likely we could improve.) But we did not explore this, since it doesn't improve the oracle complexity. Comparing the total runtime of Algorithm 1 to first order methods will depend on many factors. Suppose the dataset size $n$ is large compared to other parameters like the dimension $d$. (I.e. $n \to \infty$ and $d = n^{o(1)}$.) Then the most expensive operation in Algorithm 1 is to compute gradients and Hessians. Note that the runtime of DPSolver is independent of $n$. In this case, our algorithm will outperform first order methods. On the other hand, if the dimension $d$ is relatively large compared the dataset size $n$, then it gets messy: The runtime of the DPSolver subroutine may dominate and, if the DPSolver subroutine is itself a first order method, then it seems likely that it would be faster to directly apply a first order method. (In general, second order methods do not work well when the dimension is large.)
Thank you for the detailed explanations. Regarding W2: I understand that the double noise method introduced in Algorithm 3 which privatizes the gradient and the direction (first and second order information), is different from the typical DP-Newton's method that the authors have described. However, it is not very clear how Algorithm 2 stands out from the typical case. While remark 4.5 explains the fact that Algorithm 2 does not impose any restrictions on the Hessian matrix unlike in the typical case, it would be more clear to the reader if the authors can explain the reason/intuition behind this statement.
Reply to Reviewer QyKz regarding the intuition behind Algorithm 1
We thank the reviewer for their reply. The main idea of our result for strongly convex function revolves around using the cubic upperbound and showing that obtaining an **approximate** solution of the cubic subproblems suffices to obtain an optimal excess loss. Notice that the accuracy of the cubic subproblem solver is influenced by the sensitivity of the Hessian, which characterizes the noise scale. In Algorithm 2, we only need the gradient of the cubic function (See Line 5 in Alg.2.). The gradient of the cubic function depends on the Hessian through $H(\theta_i - \theta_0)$, and the scale of the noise depends on the $L_2$ norm of $\|\|H(\theta_i - \theta_0)\|\|\leq M * D$, where $M$ is the smoothness parameter and $D$ is the diameter of the space. This approach is different from the straightforward approach where two independent noises are added directly to Hessian and the gradient. The noise for privatizing the Hessian matrix scales with its Frobenius norm. Therefore, unless the rank is bounded, the scale of the noise for Hessian scales with dimension which results in a suboptimal utility bound (See [ABL21, Appendix E].). In summary, as opposed to the straightforward approach where gradient and Hessian are privatized independently, we only add noise once since the performance of our algorithm depends on the accuracy of the cubic subproblem solver.
Thanks for the response. All my comments are addressed at this point.
Thank you very much for your reply. We will revise our paper according to the constructive comments in the reviews.
The rebuttal addressed my questions. Thank you!
Thank you very much for your reply. We will revise our paper according to the constructive comments in the reviews.
> much smaller than the non-private setting of $\eta=1/\beta$ when the loss is $\beta$-smooth. I don't think the stepsize is $1/\beta$ for SGD in the non-private setting, which is not enough to control the variance of the stochastic gradient. It should scale with $1/\sqrt{T}$ for smooth convex/nonconvex losses. Do you mean GD?
Reply to Reviewer FNjC
Thanks for catching that; indeed this step size is for full-batch GD. In trying to condense our reply we accidentally compared a private, stochastic step size with a non-private, non-stochastic step size. However, for both the full-batch and stochastic settings, it is true that the addition of Gaussian noise for privacy requires a smaller step size to guarantee that the loss decreases in expectation when compared to non-private GD, as Figure 4 demonstrates.
Thanks for your reply. I still have some concerns that the authors do not clearly address. > For the oracle complexity, the proposed algorithm achieves $(\sqrt{L_2}/\mu^{3/4})(\ell_0 - \ell^*)^{1/4} + \log\log(n)$, which is hard to compare with the $\sqrt{L_1/\mu}\log(n)$ complexity achieved by first-order methods. 1. I agree that the proposed second-order method achieves exponential speed-up in terms of dependence on $n$. However, the constant $L_2$ and $\ell_0-\ell^*$ can be pretty large in practice. Also, it is hard to compare because of different assumptions and constants. > I am also wondering if an improved lower-bound is possible with second-order information. 2. Could the authors also comment on the lower-bound? > Reply to W1: The subproblem we solve is a cubic function, which is indeed smooth over the constraint set. 3. Then the smoothness parameter depends on the diameter, which could be very large and will enter the number of gradient and Hessian-vector evaluations. In comparison, first-order methods do not necessarily have a dependence on the diameter in their gradient complexity (considering output perturbation for smooth strongly-convex functions). 4. My last concern is regarding the extension to other settings. Since the understanding of the convergence guarantees on second-order methods is restricted to smooth convex functions even for non-DP optimization, the extension to more practical nonconvex problems could be hard. It only looks promising to first run a first-order method to a small neighborhood of local minimum that potentially satisfies the smoothness and strong-convexity assumptions, and then use a second-order method. However, the existence of DP noise might prevent outputs of first-order methods to be in a small neighborhood of local minimum, unless other structural assumptions are made. Except for these, my other concerns are successfully addressed by the author's rebuttal.
Reply to Reviewer FNjC
Thanks again for your detailed comments. We respond to the remaining points. 1. Indeed, it is hard to compare Theorem 4.2 directly with first-order methods. As mentioned in Remark 4.4, the analysis suggests a phase transition. When we are far from the optimum, second order information does not help much, but once we are close we achieve an exponential speed up. It is possible that in practice the first phase is more important, but in general the optimization theory literature focuses on the asymptotic convergence rate which is determined by the latter phase. We have tried to address questions about practical performance with our experimental results. 2. We haven't thought about lower bounds for second order methods with DP. There are lower bounds for second-order methods without DP and there are information theoretic lower bounds for DP. Other than taking the max of these lower bounds, we don't know what to do here. 3. Note that assuming the loss function is Lipschitz and strongly convex implies a bound on the diameter. So a diameter bound is often implicit in the analysis of first order methods even if not explicitly stated. 4. Applying second order methods to practical non-convex optimization is a challenge even without privacy, although there is [work on this](https://arxiv.org/abs/2002.09018). The main message of our work is that second order methods can work with DP to accelerate optimization. This goes against the commonly held belief that second order methods are too brittle to work with noise. Of course, second order methods have other limitations unrelated to privacy and we do not escape these. We hope that the reviewer is convinced of our main message and is willing to support our submission.
Thanks for the reponse. For the answer 3, the diameter dependence of first-order methods is only logarithmic, i.e., $(L_1/\mu)\log D$ where $L_1$ is the smoothness parameter and $\mu$ is the strong-convexity parameter. As an example, you can find it in Theorem 2 of [arXiv:1703.09947]. At the same time, since the smoothness parameter of the cubic-regularized function depends on the diameter, the diameter dependence of second-order methods can be polynomial. The authors' reply does not clearly address this point.
Reply to Reviewer FNjC Regarding Diameter
Note that we can run a first order-method for a few iterations to get close to the optimum and then switch to our second-order method. This would get the best of both worlds. We did not include this, since it would complicate the algorithm without offering any real insight. However, we will add a discussion of this in the next revision. **More detailed explanation of the performance of first-order methods:** It is not as simple as saying that first-order methods have a logarithmic dependence on the diameter, because this is only true under strong assumptions that imply that the diameter cannot be large. Specifically, [Zhang et al.](https://arxiv.org/abs/1703.09947)'s Theorem 2 assumes the objective function is $\mu$-strongly convex and $L$-Lipschitz. This implies an upper bound on the diameter $D \le 2L/\mu$. The oracle complexity of their result (part 2) is $$T = \Theta\left(\frac{\mu^2+\beta^2}{\mu\beta} \log\left(\frac{\mu^2n^2\varepsilon^2D^2}{L^2 d \log(1/\delta)}\right)\right).$$ If $D = \Theta(L/\mu)$ (i.e. the diameter is as large as possible), then this cancels to $T = \Theta\left(\frac{\mu^2+\beta^2}{\mu\beta} \log\left(\frac{n^2\varepsilon^2}{ \log(1/\delta)}\right)\right).$ To shave the log factor from $T$, we need $D = O \left( \frac{L \sqrt{d} \log(1/\delta)}{\mu n \varepsilon} \right)$. That is to say, their dependence on the diameter is only interesting when the diameter is very small. But small diameter is also the general regime where our Algorithm performs well. Thus, at least in this case, the dependence on the diameter does not appear to be a significant advantage of first-order methods. That said, the results are not directly comparable, so there will be parameter regimes where the result of Zhang et al. beats ours; in that case, we can always combine the algorithms. We can use a first-order method initially and then, once we are close enough, we can switch to our second-order method. We hope that this addresses the reviewer's remaining concerns.
Thanks for your reply. I agree this could be subtle. I am willing to support the submission, as long as the authors mention these limitations in their work. I have raised my score. Thanks again for the discussion.
The rebuttal has addressed my questions and comments. Thanks.
Thanks a lot for your response! We'll revise our paper based on the constructive feedback from the reviews.
Decision
Accept (poster)