A fundamental aspect of behaviour is the ability to encode salient features\nof experience in memory and use these memories, in combination with current\nsensory information, to predict the best action for each situation such that\nlong-term objectives are maximized. The world is highly dynamic, and\nbehavioural agents must generalize across a variety of environments and\nobjectives over time. This scenario can be modeled as a partially-observable\nmulti-task reinforcement learning problem. We use genetic programming to evolve\nhighly-generalized agents capable of operating in six unique environments from\nthe control literature, including OpenAI's entire Classic Control suite. This\nrequires the agent to support discrete and continuous actions simultaneously.\nNo task-identification sensor inputs are provided, thus agents must identify\ntasks from the dynamics of state variables alone and define control policies\nfor each task. We show that emergent hierarchical structure in the evolving\nprograms leads to multi-task agents that succeed by performing a temporal\ndecomposition and encoding of the problem environments in memory. The resulting\nagents are competitive with task-specific agents in all six environments.\nFurthermore, the hierarchical structure of programs allows for dynamic run-time\ncomplexity, which results in relatively efficient operation.\n