Distributed and Democratized Learning: Philosophy and Research Challenges

Due to the availability of huge amounts of data and processing abilities,\ncurrent artificial intelligence (AI) systems are effective in solving complex\ntasks. However, despite the success of AI in different areas, the problem of\ndesigning AI systems that can truly mimic human cognitive capabilities such as\nartificial general intelligence, remains largely open. Consequently, many\nemerging cross-device AI applications will require a transition from\ntraditional centralized learning systems towards large-scale distributed AI\nsystems that can collaboratively perform multiple complex learning tasks. In\nthis paper, we propose a novel design philosophy called democratized learning\n(Dem-AI) whose goal is to build large-scale distributed learning systems that\nrely on the self-organization of distributed learning agents that are\nwell-connected, but limited in learning capabilities. Correspondingly, inspired\nby the societal groups of humans, the specialized groups of learning agents in\nthe proposed Dem-AI system are self-organized in a hierarchical structure to\ncollectively perform learning tasks more efficiently. As such, the Dem-AI\nlearning system can evolve and regulate itself based on the underlying duality\nof two processes which we call specialized and generalized processes. In this\nregard, we present a reference design as a guideline to realize future Dem-AI\nsystems, inspired by various interdisciplinary fields. Accordingly, we\nintroduce four underlying mechanisms in the design such as plasticity-stability\ntransition mechanism, self-organizing hierarchical structuring, specialized\nlearning, and generalization. Finally, we establish possible extensions and new\nchallenges for the existing learning approaches to provide better scalable,\nflexible, and more powerful learning systems with the new setting of Dem-AI.\n