Authors:

Connie Kang

Date Created:

Not specified

Course Title:

Professor:

Not specified

About Paper:

Distributed machine learning has grown in popularity due to data privacy, edge computing and large model training. Increasing the number of components in a distributed system means increasing the probability that some of these components will be subject to failure during the execution of a distributed algorithm. This demonstrates the reliability of a system, which also refer to a system's fault tolerance. Therefore, in distributed machine learning, we sought to improve the fault tolerance of the model against random devices failures and communication attacks. To achieve this, we utilize a blockwise training paradigm consisting of training devices independently with Barlow Twins, a recent self-supervised learning rule at each device. Experiments were conducted on MNIST to provide conclusions and recommendations based on our observation of the blockwise training model performance under different failures. We provide some preliminary analysis and experimental results to showcase the problem and naive baseline methods on a toy problem. This study indicates that the new model increases the fault tolerance level with the use of a self-supervised learning algorithm.

Source:

Purdue University / 2023

Topics:

No topics listed

Co-authors:

Connie Kang