New Application on Breast Cancer Diagnostic Data
We appreciate your feedback and apologize for our previous insufficient response. Due to space constraints, we couldn't include additional figures and results initially. To further illustrate sisPCA's versatility as a plug-in replacement of PCA for disentangling quantities, we provide below a new application, which will be included in the appendix of the final paper, and as a tutorial upon package release.
### **Problem and Dataset**
We used the Kaggle Breast Cancer Wisconsin Data Set (569 samples, 30 summary features, uciml/breast-cancer-wisconsin-data, CC BY-NC-SA 4.0 license). The 30 real-valued features are computed from imaging data of breast mass, which include the mean, standard error and the extremum of quantities like cell nuclear radius, texture, perimeter etc.
Here, our goals are to:
1. Learn a representation for predicting sample disease status (Malignant or Benign, not used during training).
2. Understand how original features contribute to data variability, the learned representation, and diagnosis potential.
### **Experiments and Results**
We focus our comparison on three linear models: PCA, sPCA, and sisPCA, using zero-centered and variance-standardized data as inputs. The diagnosis label used to measure subspace quality remains invisible to all models. Below we summarize the quantitative results, and will include subspace visualization in the final paper.
#### **PCA**
We projected all features (dim = 30) onto one PCA subspace (dim = 6, determined by the elbow rule), explaining 61.6% of total variance. Malignant and benign samples appear well separated in PC1 and PC2. 'symmetry_mean' (loading = -0.223) and 'radius_mean' (loading = -0.219) are the top 2 features that negatively contribute to PC1. That is, the higher the two feature scores, the lower PC1 score and the greater the possibility of the sample being malignant.
#### **sPCA**
From the PC1 loadings, we sought to construct two subspaces to separately reflect nuclei size (‘radius_mean’ ) and shape ('symmetry_mean'). We set $Y_{radius}$ ('radius_mean', 'radius_sd') of $569 \times 2$ as the target variable for the radius subspace, and $Y_{symmetry}$ ('symmetry_mean', 'symmetry_sd') of $569 \times 2$ as the target for the symmetry subspace. The remaining 26 features were projected onto the two subspaces using sPCA (dim = 3, effective dim = 2). *Both subspaces better explained diagnosis status than PCA (**Table S3**) but remained highly entangled.* Specifically, the PC2 loadings of the two spaces have a Pearson correlation of 0.716, and 'perimeter_worst', which is highly correlated to 'radius_mean' (corr = 0.965), also strongly contributes to the PC2 of the symmetry subspace (loading = 0.238).
#### **sisPCA**
We next applied sisPCA ($\lambda$ = 10) to further disentangle the two subspaces, increasing their separation (Grassmann distance increased from 1.593 to 2.058, PC2 loadings correlation decreased to 0.257). Here ‘perimeter’ features no longer contribute to the symmetry subspace. *As a result, the radius subspace remained predictive of diagnosis, while the symmetry subspace became less relevant (**Table S3**).* We confirmed the finding by directly measuring the predictive potential of the target variables $Y_{radius}$ and $Y_{symmetry}$ (Silhouette score = 0.457 and 0.092, respectively).
**Table S3: Predictability of diagnosis status, measured by Silhouette score**
| | Radius subspace (dim = 3) | Symmetry subspace (dim = 3) | Overall subspace (dim = 6) |
|:-|:-|:-|:-|
| PCA (one subspace) | 0.294 (PC 1-3) | 0.013 (PC 4-6) | 0.160 (PC 1-6) |
| sPCA | 0.534 | 0.374 | 0.464 |
| sisPCA | 0.553 | 0.027 | 0.511 |
### **Interpretation**
1. Our results suggest that nuclear size (radius subspace) is more informative for breast cancer diagnosis than nuclear shape (symmetry subspace), aligning with clinical observations [1].
2. Unsupervised PCA and sPCA captured both radius and symmetry aspects as they contribute most to data variability. *Without disentanglement, the two aspects remain intertwined, potentially leading to incorrect conclusions about symmetry's predictive power for diagnosis.*
3. *sisPCA successfully separated the two representations, revealing their distinct relationships to diagnosis.* The results are interpretable: the radius subspace is constructed using features like 'area' and 'perimeter', while the symmetry subspace uses features like 'compactness' and 'smoothness'.
This example demonstrates sisPCA's ability to disentangle different aspects of data variation and uncover underlying relationships. **Importantly, sisPCA improves upon the diagnosis potential of PCA's representation, even through the diagnosis labels were never used during training.**
[1] Kashyap, Anamika, et al. "Role of nuclear morphometry in breast cancer and its correlation with cytomorphological grading of breast cancer: A study of 64 cases." Journal of cytology 35.1 (2018): 41-45.