Summary
* This paper proposes an economic framework for trading parameters of prediction models.
* Interaction is modeled as a brokered marketplace - Each agent $u$ trains a model characterized by parameters $\theta_u$ using gradient descent. At each time-step $t$:
* Each agent performs a gradient descent step on the previous parameters $\theta_u^{t-1}$, to obtain $\dot{\theta}_u^t$.
* After the GD step, agents relay their model parameters $\dot{\theta}_u^t$ to a trusted broker, which calculates the optimal linear interpolation between them, yielding the possibly-improved combination $\bar{\theta}_u$ for each agent. The broker indicates for each agent the gain-from-trade $\Delta_u^t$.
* The agents participate in Nash bargaining to decide on the value exchange, set trading prices, and $\theta^t_u$ is decided. Each agent has a valuation function $v_u$ for their own parameters, and the other agent’s parameters. Utility function is assumed to be fully known by the broker.
* Analysis:
* Analysis is restricted to the two-agent setting $u\in\{A,B\}$, and a specific form for gain-from-trade $\Delta_u^t$ is assumed. It is assumed that each seller has complete knowledge of the valuation prior, such that Myerson’s revenue-maximizing pricing mechanism (virtual valuations) can be applied based on the information relayed from the broker ($\alpha$, $\beta$, $\Delta_a^t$ - By Theorem 4.1).
* It is assumed that the broker has complete knowledge of the optimal model parameters in advance ($\theta^*$). Combining with the assumptions about the agents’ utility function, a closed-form expression is obtained for the monetary transfer after Nash bargaining.
* In Section 4.1, two upper bounds on convergence rates are presented - One for a scenario where agent $B$ always trades (Theorem 4.2), and one for a scenario where both agents don’t trade (Proposition 4.3). The “always-trade” bound (Thm 4.2) is lower, and the authors claim that this explains why trading is sometimes beneficial.
* Two sets of experiments are presented:
* The first set of experiments is an evaluation of the data-sharing setting, without taking parameters into account (Section 5.1.1 - complete networks, Section 5.1.2 - subsets of networks), on real-world vision datasets (MNIST, CIFAR10, TinyImageNet).
* Finally, the value of trading is evaluated using a synthetic model, and positive results are reported for both cooperative and competitive scenarios.
Strengths
* Problem is well-motivated.
* Leverages recent deep insights about the structure of contemporary learning problems.
* The breadth of work is substantial, and it contains a number of new definitions, theoretical analyses, and empirical evaluations.
* Code is provided.
Weaknesses
* Some key assumptions are not well supported:
* Valuation functions $v(\theta)$ are assumed to be fully known by the broker. For contemporary neural nets, the dimensionality of $\theta$ could be in the order of billions (or even trillions, such as in GPT4), and therefore the parameter valuation would be a function $v:\mathbb{R}^{10^9}\to\mathbb{R}$ - Which may be prohibitively expensive to represent, store and compute.
* Broker is assumed to know the true model parameters $\theta^*$ in advance, which in practice may render the whole learning process unnecessary. In L214, it is claimed that knowing $\theta^*$ “is not necessary in practice”, but I did not find further support of this claim.
* Very specific functional forms are assumed for gain-from-trade but are not sufficiently justified.
* A common prior on agent valuation is assumed (L208), however it is not discussed whether having such prior is realistic and how it would be implemented.
* Mathematical soundness concern regarding one of the claims: In section 4.1, authors claim that trading is beneficial by comparing two performance upper bounds (Theorem 4.2, Proposition 4.3). However, the difference between two upper bounds does not indicate the relation between the true quantities unless they are tight (in other words, proving that $a<100$ and $b<50$ does not indicate that $a>b$ unless there are matching lower bounds for $a$ and $b$). Therefore, if this observation is true, the analysis in Section 4.1 does not address the posed question (whether participation leads to better convergence).
* First set of experiments (Sec 5.1.1, 5.12) only seem to demonstrate the ability to interpolate model parameters, but do contain simulation of parameter trading. Economic behavior is only demonstrated on the synthetic dataset. As the benefit of parameter interpolation has already been demonstrated in the literature to some extent, I did not understand the contribution of the results in sections 5.1.1, 5.1.2 to the understanding of parameter-trading markets.
* Paper only analyzes the case of two agents $\{A,B\}$. Unclear how results extend to multiple agents.
* Trading parameters during the gradient descent process would require agents to train models simultaneously, or suffer significant delays due to trading. This may make such trading impractical.
Questions
* How would the model extend to more than 2 agents? Which results can be used as-is, and which need to be generalized? Would the system behave qualitatively differently in any way when the number of agents is increased?
* What are the computational requirements from the broker?
* Is it possible to quantify the economic implications of the broker not being able to interpolate successfully?
Rating
5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
I feel that limitations could be discussed at more depth. The key assumptions are added gradually throughout the paper, and it is hard to keep track. I feel that the paper can greatly benefit from a thorough discussion of limitations.