Summary
**Paper Summary**
This paper tackles the data-driven estimation of geographical habitat area for a given species; that is, given a certain geographic location (i.e., a "point on the map"), the goal is to predict how likely a certain species is present in this location. Prior efforts train models in an "offline" setting, assuming a finite training data set that indicates (for a fixed set of different species) the presence (or absence) of the species. Datasets that signal the presence (and possibly absence) of a large number of species are readily available.
This paper points out that dataset expansion for species not covered by existing datasets would be cumbersome- we would need to collect presence indication signals for new (or not yet recorded) species worldwide by possibly uniformly sampling presence indication evidence, which can then, in turn, be used to update/re-train existing models.
This paper investigates an interesting angle of this general problem: can we leverage *existing* (already collected) datasets to obtain a "weak" classifier for a new species, which can then guide the data collection efforts to regions, such that with as few samples we can decrease the overall presence entropy *as much as possible*?
This is akin to active learning, where the idea is to label as-informative-as-possible recorded data points. Different to that, in this case, the model suggests geographic regions in which we should sample the data for the new species (i.e., obtain presence/absence signals), such that the range estimates for new species will be as accurate as possible with as few data points (i.e., manual collections) as possible.
The overall idea behind the proposed work is to leverage the similarity between the (new) query species and existing species: this is based on the intuition that "similar species" (in terms of similarity in the feature space) will likely habitat similar territories. Such an approach can then be applied iteratively by incorporating newly-obtained presence/absence signals, suggesting increasingly more informative locations for the data collection.
The idea behind the approach is simple and intuitive: assuming an ensemble of (trained) linear models for a set of species, a model for a new species can be estimated as a weighted combination of existing linear models (where the model weight is proportional to the similarity in the feature space between the new (query) species and species database). This model then provides a "heatmap" over geographic locations, and guides the data collection -- new data points can be collected in the areas where the model is most uncertain about the new species' presence.
**Core Contributions/Insights**
The core contribution of this work is the insight that techniques similar to active learning can be used to streamline the data collection process. This is based on the key insight that existing datasets contain a sufficiently large number of species samples, such that a linear combination of already trained linear classifiers provide a reasonably good predictor for the presence of new species.
**Key Results**
The paper convincingly demonstrates that the proposed hypothesis ensembling outperforms logistic regression, and adding an online-trained model to the ensemble further improves the performance. I cannot comment on whether any relevant baselines are missing, as I am not actively working in species range estimation or active learning, and I am, therefore not familiar with related work. I hope my colleagues can comment on this.
Experimental validation also discusses the impact of the number of existing species (models) for model averaging on the final performance and establishes that the proposed approach is useful when a large number of specie presence models is readily available.
**Potential for Impact**
Specie habitat estimation has the potential to assist scientists in monitoring how the habitats of species are changing over time, which is especially important in the era in which global warming presents an existential threat to several species. Therefore, data-driven methods that can provide reasonable estimates based on (I assume difficult to obtain) evidence of species have important practical implications. As monitoring (and consequentially, data collection) is a continual process, developing models that can guide data collection efforts using active learning techniques makes sense.
**Rating Justification**
I think this paper presents an interesting, well-motivated, and well-executed work with direct practical relevance to monitoring species habitats. At this point, I see no obvious objections to why this paper should not be accepted.
This is a low-confidence review written by a reviewer entirely unfamiliar with bird-species range estimation and only weakly familiar with the field of active learning.
**Post Rebuttal Comments**
I want to thank the authors for their thorough responses. All my questions were addressed, and I see other reviewers are also positive about this paper. I am upgrading my rating to accept.
Strengths
* Application relevance: this work has the potential to guide future data collection efforts for species range estimation. Monitoring species range is important for understanding how the earth's biosphere evolves under rapidly changing environmental conditions.
* Paper presentation: this paper is very well written and summarizes well both the application domain (species range estimation), as well as existing methods that are commonly being used, together with simple and intuitive taxonomy of prior efforts. Core contributions are presented clearly, well-motivated, and experimentally validated. As someone reading a paper on species range estimation for the first time, I really appreciate Sec. 3, which summarizes how this problem is typically tackled in the literature.
* Motivation: the paper makes a convincing case for investigating which areas should be used to collect data in the next training cycles using active learning techniques.
* Results: I find the paper's claims convincingly validated.
Weaknesses
In addition to the weaknesses discussed in the limitation section, it would be fair to point out that the proposed approach is most likely *only* applicable to new species that are sufficiently similar to species for which data is readily available. I assume this approach would fail in the case of potentially newly-discovered species that would hypothetically be quite unlike anything that was observed before. As I am not familiar with the biological aspect of this task, I am not sure if this is a practical concern.
Questions
Did the authors perform any kind of analysis on whether the intuition that most related species are a good indicator for the presence of new species holds? I.e., for a query species, are highly-weighted classifiers indeed representing most related species, and their presence is a good indicator? Did the authors observe any failure cases? For example, the presence of domestic cat, which is related to wild cats, is probably not a good presence indicator for, e.g., lion.
Rating
7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
Confidence
3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Limitations
The paper thoroughly discusses the limitations in a designated section. I find this section insightful.