For neural models to garner widespread public trust and ensure fairness, we\nmust have human-intelligible explanations for their predictions. Recently, an\nincreasing number of works focus on explaining the predictions of neural models\nin terms of the relevance of the input features. In this work, we show that\nfeature-based explanations pose problems even for explaining trivial models. We\nshow that, in certain cases, there exist at least two ground-truth\nfeature-based explanations, and that, sometimes, neither of them is enough to\nprovide a complete view of the decision-making process of the model. Moreover,\nwe show that two popular classes of explainers, Shapley explainers and minimal\nsufficient subsets explainers, target fundamentally different types of\nground-truth explanations, despite the apparently implicit assumption that\nexplainers should look for one specific feature-based explanation. These\nfindings bring an additional dimension to consider in both developing and\nchoosing explainers.\n