Can artificial agents learn to assist others in achieving their goals without\nknowing what those goals are? Generic reinforcement learning agents could be\ntrained to behave altruistically towards others by rewarding them for\naltruistic behaviour, i.e., rewarding them for benefiting other agents in a\ngiven situation. Such an approach assumes that other agents' goals are known so\nthat the altruistic agent can cooperate in achieving those goals. However,\nexplicit knowledge of other agents' goals is often difficult to acquire. In the\ncase of human agents, their goals and preferences may be difficult to express\nfully; they might be ambiguous or even contradictory. Thus, it is beneficial to\ndevelop agents that do not depend on external supervision and learn altruistic\nbehaviour in a task-agnostic manner. We propose to act altruistically towards\nother agents by giving them more choice and allowing them to achieve their\ngoals better. Some concrete examples include opening a door for others or\nsafeguarding them to pursue their objectives without interference. We formalize\nthis concept and propose an altruistic agent that learns to increase the\nchoices another agent has by preferring to maximize the number of states that\nthe other agent can reach in its future. We evaluate our approach in three\ndifferent multi-agent environments where another agent's success depends on\naltruistic behaviour. Finally, we show that our unsupervised agents can perform\ncomparably to agents explicitly trained to work cooperatively, in some cases\neven outperforming them.\n