Many cooperative multi-agent problems require agents to learn individual\ntasks while contributing to the collective success of the group. This is a\nchallenging task for current state-of-the-art multi-agent reinforcement\nalgorithms that are designed to either maximize the global reward of the team\nor the individual local rewards. The problem is exacerbated when either of the\nrewards is sparse leading to unstable learning. To address this problem, we\npresent Decomposed Multi-Agent Deep Deterministic Policy Gradient (DE-MADDPG):\na novel cooperative multi-agent reinforcement learning framework that\nsimultaneously learns to maximize the global and local rewards. We evaluate our\nsolution on the challenging defensive escort team problem and show that our\nsolution achieves a significantly better and more stable performance than the\ndirect adaptation of the MADDPG algorithm.\n