Human-Guided Complexity-Controlled Abstractions

Summary

This paper proposes a method for learning discrete representations whose complexity can be smoothly annealed. Experiments demonstrate that, at the appropriate level of complexity, these representations can be useful for downstream classification tasks involving abstract categories.

Strengths

- The work proposes an interesting method for learning a discrete 'codebook' with a controllable level of complexity. - Experiments demonstrate a clear non-monotonic relationship between complexity and usefulness in a downstream task involving abstract categories. - The method outperforms baselines in this setting.

Weaknesses

I have previously reviewed this work, and it seems that the paper has not been revised in a manner that sufficiently addresses the concerns raised by myself and other reviewers. Specifically, a major concern with this work as currently formulated is that the proposed human-in-the-loop approach does not seem very realistic. In the abstract, the authors note that 'humans learn discrete representations ... at a variety of abstraction levels ... and use the appropriate abstraction based on tasks'. This is a compelling motivation, but unlike humans, the proposed approach does not involve any method for autonomously selecting the appropriate level of abstraction for a given task. It is unclear how this human-in-the-loop approach will be scaled in a realistic manner, and the authors do not devote enough attention toward addressing this limitation. It would help if the authors could add more discussion of this issue, and also describe more concretely the scenarios in which they envision this approach being useful. Given pre-trained models over a range of complexity levels, and some downstream task on which a user wants to fine-tune, would it not be much more straightforward to simply use a validation set from the downstream task to evaluate the full set of pre-trained models? This also suggests the possibility of automating the process of selecting the right abstraction level in a meta-learning setup.

Questions

I would appreciate if the authors could devote more attention to the limitations identified in the previous section, and also provide more concrete detail about the intended use case for the proposed approach.

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.

Soundness

3 good

Presentation

3 good

Contribution

3 good

Limitations

There are no discernible negative societal impacts related to this work.

Summary

This paper proposes a framework for human-in-the-loop training of machine learning models where humans select among pretrained models with different complexity levels based on prototypes. The authors demonstrate that finetuning performance is significantly impacted by representation complexity in the experiments considered. Moreover, a user study demonstrates that humans are relatively successful at helping choose the correct complexity level for finetuning.

Strengths

- The idea of using humans to help select abstractions for a given problem is an interesting idea for human in the loop ML. - The proposed VQ-VIB approach seems like a reasonable implementation for this use case. - There seems to be solid empirical verification of the very intuitive connection between representation complexity and fine-tuning performance.

Weaknesses

- The authors only show transfer performance within single benchmarks rather than across tasks or in realistic pretraining and finetuning settings such as ImageNet or LLMs. - The authors do not compare with other methods for human in the loop training in which the amount of human effort can be compared. For example, feature engineering is another way that humans can potentially impact downstream performance -- a technique widely used in practice. - The paper also does not discuss the effort involved in providing this human guidance.

Questions

1. Can you quantify the amount of effort needed by human annotators to select the best abstraction level? 2. Can you quantify the amount of sub-optimality in the human study when the correct abstraction was not selected? 3. How does the effort and efficacy involved in prototype based abstraction selection compare with that of feature engineering? 4. There is something off about the bird abstraction example (line 20). What precisely is the task the abstraction is being used for in this case? Is the expert birdwatcher teaching the child or other experts? What does complexity and simplicity mean in this case?

Rating

6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

4 excellent

Presentation

3 good

Contribution

2 fair

Limitations

The authors do not really address the limitations of the work. I believe the main limitations are the lack of alternative methods for human-in-the-loop training considered i.e. feature engineering and the use of somewhat synthetic benchmarks for evaluation that are disconnected from real-world use cases. It is natural to wonder if humans can be as helpful at prototype based abstraction selection for transfer tasks that are more distant from each other and not drawn from the same benchmark.

Summary

The authors observe that the downstream tasks for pretrained models can rely on representations of varying level of complexity: as a running example, a birdwatcher relies on significantly more complex representations of images to classify bird species, relative to a child who may want to identify the color of a bird. They thus propose pretraining a variety of models with representations of varying complexity, having a human choose the appropriate pretrained model for the task, and then finetuning that model for the downstream task. Relative to finetuning the model with the most complex representations, this should lead to increased data efficiency. The authors suggest a modification of VQ-VIB (vector-quantized variational information bottleneck) for the pretraining in this setting, called VQ-VIB_C (C standing for categorical) – my understanding is that the main difference is that in the authors’ method, the representation is divided into n chunks and each of the n chunks is snapped to a quantized vector (the loss remains the same). I have not spent much time delving into the details as the authors say this is not their main contribution: in particular I may be wrong about the differences from VQ-VIB. The authors implement this idea for a variety of image-based datasets: FashionMNIST, CIFAR-100, and iNaturalist 2019 (iNat). They pretrain an encoder using a reconstruction loss as well as a classification loss for the labels defined by the dataset, and increase the regularization on the representations to generate a variety of checkpoints that produce representations of different complexity. They then evaluate these encoders in a downstream classification task that is coarser than the original classification task (e.g. living vs. non-living for CIFAR-100). Their experiments show that (1) VQ-VIB_C outperforms two baselines (VQ-VIB, $\beta$-VAE), and (2) when the classification task is simple (e.g. 2-3 classes) and there are very few data points (e.g. 1-5 examples per class), it is better to use a pretrained encoder with lower representational complexity (though in all other cases it is typically fine to use the encoder with maximal representational complexity). They also perform a human experiment in which they show that, given visualizations of the learned quantized vectors, humans can correctly identify which encoder will perform best during finetuning.

Strengths

1. To my knowledge, the idea of using simpler representations for downstream tasks in order to improve data efficiency is novel, and I think it is an interesting hypothesis to explore. 2. Although the authors don’t consider it their main contribution, their experiments suggest that the VQ-VIB_C performs notably better than other alternatives on their task. (However, I did not investigate in detail, e.g. it is possible that the baselines were not tuned well while the VQ-VIB_C was.) 3. More generally, the experiments are quite detailed and look at the effects of various hyperparameters in the method. 4. The paper has an experiment with real humans to justify its claim that humans could choose an appropriate model to use for a downstream task. While I think the experiment is pretty different from realistic conditions, this is still better than the vast majority of papers on ideas like these, which often don’t bother testing their claims about real humans at all.

Weaknesses

I have a few concerns: 1. Significance: It seems to me that the author’s experiments suggest that the main idea only has a benefit in very limited settings, which are unlikely to arise in practice. 2. Baselines: For k > 1, the appropriate baseline would be to use the most complex representation, with strong regularization determined using a validation set at finetuning time. 3. External validity: The experiments involve downstream tasks that are pure coarsenings of the pretraining task, whereas in typical settings the downstream tasks will likely not be these “pure coarsenings”, which would reduce performance. 4. Relevance of the human experiment: The author’s experiment seems to “give away” the answer (though there is a case to be made that this reflects a strength of the method, rather than a weakness of the experiment). Overall I feel conflicted about this paper. On the one hand, it has a nice idea with careful technical work done to flesh it out, and a large set of experiments to understand how the idea works in practice. On the other hand, I think the experiments suggest that the idea is not very practically useful (whereas the paper suggests they validate the idea), despite the experiments having some aspects that bias them towards showing the idea to be good. Overall I’m recommending a borderline reject, but I can see the case for acceptance as well. **Significance** Looking at the experiment results (including the ones in the appendix), it appears that if n > 3, or k > 5, or you use any method other than VQ-VIB_C, it is usually best to use the most complex representation (i.e. the one that achieves highest reconstruction loss). Thus the benefits of the idea only occur when n <= 3 and k <= 5, which implies a tiny dataset of 15 examples or fewer. As we might expect from such a small dataset, the resulting finetuned classifiers do not perform particularly well, getting around 70-90% accuracy. Thus it seems like the idea in this paper is only helpful when (1) there is a very simple classification task, (2) there is very limited finetuning data available, (3) you use VQ-VIB_C rather than a different method, and (4) the user is happy with performance numbers of 70-90% accuracy. This seems extremely restrictive, and I find it hard to think of a realistic setting that satisfies all of these constraints (especially #4). Overall, I would characterize these experiments as providing a negative result. (I would actually be more inclined to accept a version of this paper that was upfront about this, and framed the paper as a negative result that has taught us that lower representation complexity only buys you a little bit of data efficiency, that is overwhelmed very quickly by a large enough dataset.) **Baselines** In the paper’s experiments, the finetuning is done the same way for representations with varying levels of complexity (I believe, at least I didn’t see anything to contrary in Appendix 7). However, a natural baseline would be to use the regular (maximum-complexity) representations, but use stronger regularization to learn a simpler classifier. One might reasonably argue that it is unclear how to choose this hyperparameter – but at least when k > 1, it should be possible to take 1 example (or more) per class to form a validation set that is used to tune the hyperparameter. I think it is plausible that this significantly improves performance for high-complexity representations. If it does work it would be a significant improvement, as there would no longer be any need for human input. **External validity of experiments** All of the experiments in the paper have the downstream task labels $Y_t$ being a strict coarsening of the pretraining task labels $Y_p$ (or in other words, $Y_t$ is a deterministic function of $Y_p$). However, this may not be the case in realistic settings – for example, in the running example, there may be birds that are of the same species but have different colors, and so a representation that just identifies the species would be insufficient for predicting the color. Should we expect the method to work even in these cases? I’m not sure; the fact that the pretraining includes a reconstruction loss suggests that it would still work (though likely not as well as in the paper’s experiments). Ideally however the authors would conduct such an experiment to test it empirically. **Relevance of the human experiment** Looking at the pdf for the user study (Appendix 11), the instructions seem to “give away” the answer. In particular, the instructions contain: > Generally, the more images the second robot is using, the less general it will be, and the worse it will do at categorizing the 3 new high-level categories. and > If the visualization shows that (1) the robot is not using many images, and (2) they roughly represent the three high-level categories [...] then the robot should perform well and > For example, here is one robot’s visualization which is perhaps too general, as there is not even three categories being used. From which it is easy to infer “choose the option with fewest categories, subject to having at least three categories” – which then leads to the desired answer. Arguably this represents a strength of the method – the approach for selecting the appropriate model is so simple that it can easily be communicated even to non-experts. However, in this case, it is so simple that there isn’t really any need for human input – we could equally well find the appropriate model using a simple heuristic of choosing the option that has a few more “most important images” than the number of classes in the classification task. (Perhaps this is another baseline which should be compared against.) **Minor issues** Line 91: > Lastly, we assume that the pre-training labels are a sufficient statistic for the task-specific labels: $I(X; Y_p) = I(X; Y_t)$. Intuitively, this states that the pre-training objective must include relevant information for the finetuning task. I don’t think that’s the right criterion. For example, for reconstruction pretraining with red-yellow classification as the downstream task, we have $Y_p = X$ and $Y_t$ is whether the bird is red or yellow. Then $I(X; Y_p) = I(X; X) = H(X) \geq 1$ (since entropy of natural bird images is large), but $I(X; Y_t) \leq H(Y_t) \leq 1$ (since $Y_t$ is a binary variable). I think what you mean to say is that $I(X; Y_t \mid Y_p) = 0$, that is, conditioned on knowing $Y_p$, there is no more information about $Y_t$ that can be learned from $X$.

Questions

1. Could you give examples of realistic settings in which using low-complexity representations would be useful – bearing in mind the four requirements I mention in the “Significance” section in Weaknesses above? (Or alternatively, if I’m wrong about the four requirements, how / why am I wrong?) 2. Is there reason to expect poor performance from finetuning the most complex representation with stronger regularization? (Possibly with a validation set, as described above) 3. How well should we expect the method to work when $Y_t$ is not a deterministic function of $Y_p$? Minor suggestion: Please increase the size of the text in Figure 4(c).

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.

Soundness

3 good

Presentation

3 good

Contribution

2 fair

Limitations

The limitations brought up in the weaknesses section are not mentioned or addressed. In particular, I think the issues raised in “Significance” and “External validity of experiments” should obviously be mentioned (or otherwise addressed); the others are more debatable.

Summary

This paper explores an interesting premise -- how does the level of abstraction captured by a discrete (visual) representation dictate downstream task performance, where downstream tasks can be at arbitrary levels of abstraction. Specifically, the running example from the work that I really like is that of bird-watching; when communicating amongst experts, the extremely specific species names (e.g., "white osprey") is very useful, and captures the right level of information. However, when communicating with small children who are looking at things in the wild, coarser descriptions (e.g., "the white bird" or "the red bird") are much more useful. To study this premise, this paper makes two contributions: first, it introduces a new method for learning discrete representations at various levels of abstraction, the Vector-Quantized Variational Informational Bottleneck - Categorical (VQ-VIB_c). Second, and more importantly, it presents a thorough evaluation, including a human-in-the-loop user study (N = 17), showing conclusively for downstream tasks with "known" abstraction levels, the "best" representation to use is the one that roughly matches that level of abstraction.

Strengths

This paper starts with a strong motivating question, introduces a new method to explore the hypotheses formed from this question, and critically performs a comprehensive evaluation and user study that confirms the proposed hypothesis. I think the user study linking "prototypes" from different models (capturing different levels of abstraction) to the downstream finetuning task's level of abstraction is the most convincing, and most meaningful result in this work. I really like this type of study, and it really helps support the paper's key contributions.

Weaknesses

Unfortunately, I have several concerns with this paper, stemming from the very confusing presentation of the paper (it took me a very long time to understand the experimental protocol, and I found that several details about the pretraining/finetuning procedure were missing or placed in scattered places in the Appendix), to the actual useability of such an approach for more practical applications. First, on the presentation/soundness side -- I ask this question more explicitly below, but what is the exact link between the process of "degrading" a high-accuracy model (pretrained to high accuracy on the "full" classification task), and emerging levels of abstraction? In other words, how exactly does the process of changing the loss hyperparameters lead to models that capture different levels of granularity? From my read of the paper/appendix, this link is never made explicit, and the process is further left underspecified -- if hyperparameters are changing in a staged way (or you're running for a limited number of additional gradient steps given the "high accuracy" model), couldn't a possible confound be the initial batches seen during this "degradation process"? Another confound could just be the per-example/per-class difficulty, which is known to be uneven across classes in the datasets (e.g., iNaturalist has severe label imbalance out of the box, papers from the Distributionally Robust Optimization literature report huge discrepancies in average vs. worst class accuracy for other datasets)? Is this controlled for in any way? Beyond these questions, I find it unclear how such an approach would be useful for practical applications? Is the idea that given some supervised dataset, you should train all models to "best accuracy" then selectively degrade the resulting representations depending on your (family of) downstream tasks? Isn't this expensive/redundant? --- EDIT (Post-Rebuttal): I think many of my soundness concerns have been addressed during rebuttal, and in discussion with the other reviewers. Raising my score to a 5.

Questions

I am very confused at the two-phase approach for varying representational capacity during pretraining. The paper seems to indicate that at first, models are pretrained until they obtain perfect accuracy at the (full) classification problem (e.g., the 10-class classification in FashionMNIST, the 100-class classification in CIFAR10). Then, after that happens, by tweaking the loss hyperparameters (e.g., downweighting "utility" and upweighting "representation conciseness" / "commitment loss"), you compress the representations until they are not meaningful, and look at various saved models that are produced in the course of that process for downstream finetuning (hence the plots that track accuracy on downstream vs. MSE). Why does this approach make sense at all for evaluating the "conciseness/abstraction" of a representation? How are you supporting the claim that the "trajectory of models" between high accuracy and "indistinguishable" representations is capturing different levels of abstraction in the representation space? This feels like a major major connection that needs to be made explicit in the paper.

Rating

5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.

Confidence

4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.

Soundness

2 fair

Presentation

1 poor

Contribution

2 fair

Limitations

The main body of the paper does not explicitly discuss limitations; the appendix discusses some of the limitation of the VQ-VIB_c approach in passing, but a proper treatment of the pros/cons of this approach is not present.

Response to rebuttal

Thanks to the authors for these clarifications and additional experiments. The regularization experiments are interesting, but that is not what I had in mind when I mentioned fine-tuning with a validation set. What I meant was that, given some downstream task, and given models with a range of complexity, a natural solution would be to generate a validation set for evaluating those models based on accuracy for the downstream task. That seems like a much more straightforward solution than having a human look at the prototypes generated by the model, but perhaps I am missing something. I also still feel that the paper needs to more clearly acknowledge the limitations of the human-in-the-loop paradigm, and the need to develop a method for autonomously selecting the right encoder. The paper does not really discuss these issues, and the authors have not indicated that they will revise the paper to include such a discussion.

Thanks for the response

I have read through the authors' rebuttal and the other reviews, and am maintaining my score of 4. ## Baselines Thanks for running the updated baseline! I'm more convinced now that the method shows benefits in the very-low-data regime ($n \leq 3, k \leq 5$). ## Significance I am still not convinced that the method shows benefits for $k > 5$. Looking at $k = 50$ in Figure 10 in Appendix 8, the curves of best fit show a peak at non-maximal complexity, but if you ignore the curves of best fit and just look at the actual data, it is far from clear that there is an actual difference between maximal complexity vs. the point at which the curve of best fit peaks. If there is a difference, it is very small in magnitude. If we instead look at the version with updated baselines (lightest green points in Figure 24a in the author rebuttal pdf), there is even less of a visible difference. To me, these graphs suggest that there is a plateau of maximal performance that includes the maximal complexity representation. However, even if we grant that Figure 10 and Figure 24a show an improvement in the $k = 50$ case for 3-way FashionMNIST, that is a cherrypicked example. If we instead look at Figures 24g / 13 (iNat 3-way), Figure 12 (CIFAR100 20-way), Figure 14 (iNat 34-way), Figure 15 (iNat 1010-way), we see that the best choice at $k = 50$ is the maximal complexity representation. Only Figure 11 / 24d (CIFAR100 2-way) show a peak at non-maximal complexity, and those are even more arguable than Figure 10 and 24a. (Incidentally, I don't trust the curves of best fit. My guess is that the authors are fitting cubic polynomials to the data? When one fits a cubic to a set of data that increases and then saturates / asymptotes creating a plateau, the cubic will tend to put its maximum value in the middle of that plateau -- thus creating an illusion that the maximum value is in the middle of the plateau, rather than at the end of it.) ## Constraints and realistic settings > Using our approach, we could train a robot to identify clothing for sorting (as in our FashionMNIST example) using only 10 examples per class and achieve roughly 93% accuracy. Yes, I would agree that this would be a plausible application. But currently, my sense is that at $k = 10$: (a) the method provides a very small boost (if any) relative to using the maximum complexity representation, and (b) this only happens in some of the settings tested (FashionMNIST), while in others (iNat) there is no boost. So overall this does not seem like a significant improvement, especially given the additional cost of human intervention. To put it a different way, the best case seems to be: "By default, we can train a robot to identify clothing for sorting using only 10 examples per class and achieve roughly 92% accuracy. With our method, we can get costly human input to select an appropriate representation, and then we can train it to 93% accuracy". And it is not clear to me whether even this is achieved, given the experiments in settings like iNat. > Reviewers 99tN and TMXo suggested the hypothesis that the optimal encoder is the one that uses approximately the same number of prototypes as finetuning labels. Such an approach does not succeed. I don't mean to imply that this holds in general. I am saying that your human experiment involved instructions that strongly suggested that the humans follow the rule "choose the encoder that has the minimum number of prototypes, but at least as many as there are finetuning classes", and that in the specific case studied in your human experiment this was the right answer. So I cannot tell from this experiment whether humans would choose encoders appropriately in more complicated settings, like the handbags with / without straps example you give.

Paper reframing per suggestions!

# Paper Reframing We thank reviewers for their excellent comments and active engagement! In light of the discussion and the recommendations suggested by Reviewers 99tN and TMXo, we propose the following reframing of our paper: 1. We explore the hypothesis that pretraining with lower representation complexity is beneficial to downstream finetuning data efficiency. We propose a new architecture, VQ-VIB$_\mathcal{C}$, for generating discrete prototypes with varied representation complexities. 2. We find that, with enough finetuning data ($k$>5 from each class), there are little to no benefits to lower complexity models. Thus, fully autonomous methods for selecting the optimal encoder perform well. However, in low-data regimes (such as the provided robot example or situations where task data can only be provided by human users), there is a benefit to lower complexity representations. Fully autonomous methods *fail* to select the optimal encoder in these cases. 3. To address low-data regimes, we propose a human-in-the-loop framework for selecting the optimal encoder. We demonstrate in a user study that this method is successful at selecting the optimally performing encoder. Overall, these results suggest that in low-data regimes, incorporating human partners in the finetuning process can outperform fully autonomous methods (although this benefit shrinks when we have access to a large enough dataset). We would like to emphasize that this proposed reframing would require *no additional experimental data*, but is simply intended to better emphasize the findings of our detailed technical evaluation while more clearly acknowledging the limitations of the human-in-the-loop method.

Response to Rebuttal

Thank you for your detailed responses to my questions. This definitely helped clarify with respect to some of my concerns. In particular, it was quite interesting to know about the average time taken by the human participants. This was lower than what I would have expected. As a result, I have decided to raise my score after the rebuttal.

Post-Rebuttal Comment - Raising Score to a 5

Thanks to the authors for their clarifications! After reading through these and the global comments, as well as the comments from the other reviewers, I feel like many of my original questions around the paper have been addressed. I still believe that the link between the current process of degrading a representation via an entropy penalty and making broader claims about emerging levels of abstraction to be theoretically tenuous, but the user study and responses from the authors raise many good points, downweighting this initial comment from my review. I think many of my issues around the soundness of the paper have been addressed. I'm still not sure about the experiments controlling for confounds that I mention, and the general viability of the new framing of the paper (the human-in-the-loop approach doesn't seems super motivated, for the reasons that Reviewer TMXo stated) either, but I am raising my score to a borderline accept (5).