Walk for Learning: Decentralized Learning Algorithms via Random Walk
Ghadir Ayache
Abstract:
Decentralized Machine Learning is a learning framework that allows collaboration on the distributed data over the nodes in a network without involving a central entity for coordination or aggregation. In our work, we study decentralized algorithms via Random Walk. A token holding the model walks in the network in a Random Walk and gets updated upon visiting each node. We propose a new adaptive framework for designing a Random Walk learning process that can adapt to heterogeneous data distributions. We derive the theoretical guarantee for our algorithm and show a near-optimal performance. Last, we compare the performance empirically to existing random walk baselines.