Summary
This paper explores the possibility of boosting the recommendation of a song in an automatic playlist continuation system by using a collective strategy of adding the song into the training playlists of the APC at a specific position. The paper shows that adopting a strategy that targets low frequency context makes it possible to very significantly boost the exposition of the song in the output of the APC.
Weaknesses
- Limited scope:
- The idea is tested in the limited context of automatic playlist continuation and is likely not adaptable to broader applications.
- Only one APC model was tested, so the results may be specific to this model. It would have been a good idea to test the impact of the collective action on other models (such as the baseline proposed in the reference paper for the APC model). Also, it’s unclear whether the effect would be robust to hyperparameter changes in the APC model.
- the method is tested in a static context (not in the real world), so the actual impact of the method on usage (for instance, whether a user exposed through the APC to a song boosted by the collective strategy would listen to it or not) is not tested (this would require access to the production of a streaming music service, though)
- Very limited insights on the design of the efficient strategy (DirLof) and why it works. The presentation of the strategy is actually quite unclear, while it’s central to the paper.
- There may be other simpler baselines that would be worth testing. The paper shows that inserting the song at the end of the playlists is less performance than inserting the song randomly, which suggests that inserting it earlier in the playlist sequence may help. So a baseline that would always insert the song at the beginning of the sequence could have decent results (the first place is likely to be avoided as it would mean that the song would never appear as the target in the training of the APC, but low positions that appear regularly as targets in the training could be considered).
Questions
- Could the authors rephrase / clarify the definition of the DirLof strategy and rationale behind it? something which is very surprising is that “the collective selects the anchor songs s0 with the smallest overall song frequency”. But it’s likely that smallest song frequency is for song appearing only once among playlists and these song are then mostly noise in playlists and it’s hard to get why those anchors would help improving recommandation of the inserted song. Also, statistics on songs with low frequency are likely much more difficult to obtain (the statistics will be much less reliable than for popular songs).
- It’s very unclear what the distributions over playlists are (P, P0, P*). It seems it’s almost used as a synonym of dataset, and the expectations are actually just empirical expectations. I’d like the author to clarify this point and to explain why it’s necessary to introduce such notations.
- In table 2, it’s unclear how statistics are estimated when there is no training data to compute them from (while there is still a significant amplification).
Limitations
- As mentioned in the weaknesses, there are other simple baselines that would be worth testing. That would help support the claim that the specially designed DirLof strategy is actually efficient.
- Also the authors should comment on the possible transferability of their results to other APC models, and to modifications of hyperparameters of the model.
On the ethical side, the method is a bit too much presented as a way for artists to attack a recommender system, somewhat rerank their songs and make money out of it, which has some important ethical implications in terms of fairness among artists: As the payment system of most streaming services is based on subscription (so a fixed amount of money), artificially boosting an artist comes at the detriment of other artists.
I would then rather avoid this message of "opportunity for artists" in the paper (and even in the title) and turn it instead as a warning to music streaming platforms that recommender systems may lack robustness to attacks and that they should be aware of it and take action. I think presenting the paper in this second way would solve this ethical issue, but the current message is, to me, ethically borderline.
Minor comments
- The authors claim that “most large platforms have shifted from relying on collaborative-based models for APC to building deep learning based recommenders …” but 1) the references are only Spotify based 2) the references are research papers which doesn’t mean that the shift actually happened and that platforms no longer use collaborative-based models part in their complex architecture.
- In equation (1):
- The authors likely want to specify that the recommended song shouldn’t be part of the playlist i.e S’ **∩** p = **∅** in the argmax
- The argmax actually corresponds to the top K elements in terms of similarity, which is quite straightforward. But the sum in the equation makes it not very clear at first read. It could be worth it to state it explicitly before the equation.
- There are several notations that use the letter h (song embedding, playlist embedding, playlist mapping). I think using different letters may help make things clearer.
- In Figure 3, it seems like s* is subtituted to s_i (there is no s_i in (b)) while just before equation 4, it’s said that only insertions are considered.
- “*Existing data poisoning techniques, for example, operate in different settings, not complying with Constraint 1 and typically assuming white-box access to the model or involving test-time profile manipulations.*” ⇒ this needs references.
- *“Thus, collective action can make a tremendous difference for these artists: suppose an artist’s song is streamed 10,000 times, yielding a revenue for the artist of $40 for royalties of $0.004 per stream [32]; an amplification of 25 would increase this revenue to $1, 000.”* I understand this is purely illustrative, but the figures won’t reflect much the truth. First 0.004$ per stream is an average that depends on the platform, but also on the kind of registration of the user and on the country of the user with quite important variations. Second, several platforms (Spotify, Deezer…) are shifting to a payment model were all streams don’t have the same value, especially recommended streams are “discounted”, and some “real” artist may get a boost. Once again, I get it’s for illustrative purpose, but it brings confusion on how music payment system works, and then should be avoided in my opinion.
- *“Moreover, when considering s* as a relevant recommendation, collective action even enhances the system’s performance”* Isn’t this completely obvious? if presentation of s* is boosted (which happens, given the amplification), other recommendations are barely affected (as shown on figure 6, solid line), and you consider s* a valid recommendation, then for sure recommendation metrics will increase, won’t they?