Summary
The paper proposes 2D-OOB, an out-of-bag estimation framework for jointly detecting both samples as well as their cells that are outliers. In other words, 2D-OOB allows attribution of the value of a data sample to its individual features. Evaluation shows that the framework is also able to localize backdoor triggers in data poisoning attacks.
In comparison to the 2D-Shapley model, which also performs joint valuation of a sample and its features, the 2D-OOB is a more efficient and model-agnostic formulation and builds on the subset bagging model, where each weak learner is trained on a randomly selected subset of features. In the experiments, decision trees as used as the weak learners and trained using randomly selected features. The framework measures (using Eqn. (4)), the average accuracy of the bag of predictors for a selected sample and its features.
Strengths
* The paper is well written.
* Given the background in moving fro DataShapley to the Data-OOB formulation, the transition from 2D-Shapley to 2D-OOB was intuitive and easy to follow.
* The proposed method can identify data poisoning attack triggers (square patches placed in the original images to elicit misclassification) which could be very useful.
Weaknesses
* The paper is low on novelty. Given, OOB-Data formulation and the 2D-Shapley for joint valuation, the 2D-OOB formulation seems like a natural extension -- interesting, but not high on novelty.
* In Fig. 4, panel B, for BadNets poisoning attack, the detection performance of 2D-OOB and 2D-KNN (proxy for 2D-Shapley) are very similar. It is difficult to assess how 2D-OOB will perform for more stronger poisoning attacks. A more detailed study on this aspect as well as analysis showing that 2D-OOB is more likely to identify such triggers, even when attack takes such a detection mechanism into account, will be a good indicator of the strength of this framework.
* Table 1, shows the average run time is much improved from 2D-KNN, however, most of the datasets used (Appendix A, Table 3) have small input dimensions (only three have >= 100 but have small sample size) and hence it is difficult to assess how the technique would scale.
Questions
* One of the arguments mentioned in the description of Fig. 2 is that 2D-OOB detects a majority of outlier cells by examining a small fraction of the total cells. It would be great if the authors could further explain this statement and which part of the formulation in (4) motivates this argument.
* In Fig. 3 (showing 2D-OOBs ability to identify and prioritize the outliers to fix), the test accuracy for the Jannis dataset is relatively low compared to the others and the performance is also similar to 2D-KNN. Any insights as to why this is happening?
* In Section 4.4, were the experiments also performed by varying B using a 2-layer MLP? What is the overhead of training an out-of-bag estimator versus a single model (maybe an autoencoder) and using that for outlier detection.
* Are there any other stronger baselines against which comparison can be made?