Authors:

Amritanshu Ranjan

Date Created:

Not specified

Course Title:

Professor:

Not specified

About Paper:

Distributed machine learning is a technique of training machine learning models across multiple devices in a network, allowing for large datasets to be processed in less time. The network's devices, sometimes known as "end devices," are tasked with data collection and making predictions. In most approaches to distributed learning, fault-proof end devices and communication are requirements. However, this is not fair to assume in many use-cases due to network connectivity issues, environmental factors, security breaches, etc. Although almost all deep learning models today are trained using the backpropagation algorithm, in distributed learning settings using modern hardware, backpropagation produces communication overhead, scalability limitations, and privacy concerns due to the heavy exchange or synchronization of model parameters. With faulty devices, this can lead to slower training time and poorer accuracy. To maximize the efficiency of end devices in fault- tolerant decentralized learning, we propose utilizing the Forward-Forward (FF) algorithm as an alternative to backpropagation. Not only can FF efficiently train devices with power limitations, but it also cultivates decentralization (and thereby privacy) because its local layer-wise learning can be simulated in devices that individually learn towards a global objective. However, FF has not been explored in the context of fault- tolerance and distributed data. In this study, we adapt FF and investigate its capability to handle device faults in a decentralized learning setting. We demonstrate an implementation of FF in fault-tolerant decentralized learning and compare the accuracy rates of FF versus backpropagation to determine which is better under our constraints.

Source:

Purdue University / 2023

Topics:

No topics listed

Co-authors:

Amritanshu Ranjan