Author Response (Part 3/4)
We also note that there are key differences between DICE and gradient-based CFL.
**Motivation**.
- Clustered Federated Learning addresses the challenge of data heterogeneity in federated learning by grouping clients with similar data distributions, without creating personalized models for each client. Gradient-based similarity is commonly used to **cluster clients**, based on the intuition that clients with similar data-generating distributions would share similar gradients.
- In contrast, DICE is motivated by a fundamentally different challenge: **quantifying the contributions of participants** in a decentralized learning system from a data influence perspective. Specifically, DICE provides a mechanism to measure how data at one node influences learning outcomes across the network, enabling the identification of pivotal contributors or potentially malicious actors in decentralized settings.
**Theoretical Formulation**. While gradient-based CFL focuses on **peer-level gradient similarity** in one-hop, i.e., computing similarity metrics between pairs of nodes, DICE extends this concept to evaluate **multi-hop influence propagation** in the network. DICE can systematically quantify how influence from one node diffuses across multiple intermediate nodes in the graph, incorporating factors such as network topology and optimization curvature. Mathematically, DICE generalizes gradient similarity into a **non-trivial extension for decentralized networks** by introducing the notion of $r$-hop influence, which accounts for:
- The topological structure of the communication network.
- The curvature information (Hessian matrices) of intermediate nodes.
- The cascading interaction of gradients over arbitrary neighbor hops.
Specifically, the $r$-hop DICE-E influence $I_{DICE-E}^{(r)}\(z_j^t, z^{\prime}\)$ is given by:
$$
I_{DICE-E}^{(r)}\(z_j^t, z^{\prime}\) =
-\sum_{\rho=0}^{r} \sum_{ \(k_1, \dots, k_{\rho}\) \in P_j^{\(\rho\)} }
\eta^{t} q_{k_\rho} \prod_{s=1}^{\rho} W_{k_s, k_{s-1}}^{t+s-1}
\nabla L\(\theta_{k_{\rho}}^{t+\rho}; z^{\prime}\)^\top
\prod_{s=2}^{\rho}
\(I - \eta^{t+s-1} H\(\theta_{k_s}^{t+s-1}; z_{k_s}^{t+s-1}\)\)
\nabla L\(\theta_{j}^{t}; z_j^t\),
$$
where $k_0 = j$, $P_j^{(\rho)}$ denotes the set of all sequences $k_1, \dots, k_{\rho}$ such that $k_s \in N_{out}^{(1)}(k_{s-1})$ for $s = 1, \dots, \rho$, and $H(\theta_{k_s}^{t+s}; z_{k_s}^{t+s})$ is the Hessian matrix of $L$ with respect to $\theta$, evaluated at $\theta_{k_s}^{t+s}$ and data $z_{k_s}^{t+s}$. For further details, please refer to Proposition 3.
This formulation highlights a key distinction: **DICE evaluates influence across multiple hops and characterizes the interplay between data, curvature, and communication topology**—factors beyond the scope of CFL frameworks. While CFL’s gradient similarity metrics, supported by strong theoretical foundations [3, 4], effectively cluster clients, they are inherently confined to the local, peer-to-peer level similarity, making it insufficient for modeling long-range or cascading influences. DICE extends these concepts by systematically quantifying "influence cascades" through decentralized networks, providing novel insights into how data, topology, and the optimization landscape interact to shape learning outcomes.
We have uploaded the revised version of our paper, which now includes the discussion on multi-hop influence on **page 8**. For your convenience, we provide the official link to the updated version of the paper below:
**https://openreview.net/pdf?id=2TIYkqieKw**