Homogeneous Learning: Self-Attention Decentralized Deep Learning

Federated learning (FL) has been facilitating privacy-preserving deep learning in many walks of life such as medical image classification, network intrusion detection, and so forth. Whereas it necessitates a central parameter server for model aggregation, which brings about delayed model communication and vulnerability to adversarial attacks. A fully decentralized architecture like Swarm Learning allows peer-to-peer communication among distributed nodes, without the central server. One of the most challenging issues in decentralized deep learning is that data owned by each node are usually non-independent and identically distributed (non-IID), causing time-consuming convergence of model training. To this end, we propose a decentralized learning model called Homogeneous Learning (HL) for tackling non-IID data with a self-attention mechanism. In HL, training performs on each round’s selected node, and the trained model of a node is sent to the next selected node at the end of each round. Notably, for the selection, the self-attention mechanism leverages reinforcement learning to observe a node’s inner state and its surrounding environment’s state, and find out which node should be selected to optimize the training. We evaluate our method with various scenarios for two different image classification tasks. The result suggests that HL can achieve a better performance compared with standalone learning and greatly reduce both the total training rounds by 50.8% and the communication cost by 74.6% for decentralized learning with non-IID data.

Homogeneous Learning: Self-Attention Decentralized Deep Learning

Federated learning (FL) has been facilitating privacy-preserving deep learning in many walks of life such as medical image classification, network intrusion detection, and so forth. Whereas it necessitates a central parameter server for model aggregation, which brings about delayed model communication and vulnerability to adversarial attacks. A fully decentralized architecture like Swarm Learning allows peer-to-peer communication among distributed nodes, without the central server. One of the most challenging issues in decentralized deep learning is that data owned by each node are usually non-independent and identically distributed (non-IID), causing time-consuming convergence of model training. To this end, we propose a decentralized learning model called Homogeneous Learning (HL) for tackling non-IID data with a self-attention mechanism. In HL, training performs on each round’s selected node, and the trained model of a node is sent to the next selected node at the end of each round. Notably, for the selection, the self-attention mechanism leverages reinforcement learning to observe a node’s inner state and its surrounding environment’s state, and find out which node should be selected to optimize the training. We evaluate our method with various scenarios for two different image classification tasks. The result suggests that HL can achieve a better performance compared with standalone learning and greatly reduce both the total training rounds by 50.8% and the communication cost by 74.6% for decentralized learning with non-IID data.

Bifurcation control of a minimal model of marine plankton interaction with multiple delays

Plankton blooms and its control is an intriguing problem in ecology. To investigate the oscillatory nature of blooms, a two-dimensional model for plankton species is considered where one species is toxic phytoplankton and other is zooplankton. The delays required for the maturation time of zooplankton, the time for phytoplankton digestion and the time for phytoplankton cells to mature and release toxic substances are incorporated and the delayed model is analyzed for stability and bifurcation phenomena. It proves that periodic plankton blooms can occur when the delay (the sum of the above three delays) changes and crosses some threshold values. The phenomena described by this mechanism can be controlled through the toxin release rates of phytoplankton. Then, a delay feedback controller with the coefficient depending on delay is introduced to system. It concludes that the onset of the bifurcation can be delayed as negative feedback gain (the decay rate) decreases (increases). Some numerical examples are given to verify the effectiveness of the delay feedback control method and the existence of crossing curve. These results show that the oscillatory nature of blooms can be controlled by human behaviors.

Modeling the effect of delay strategy on transmission dynamics of HIV/AIDS disease

In this manuscript, we investigate a nonlinear delayed model to study the dynamics of human-immunodeficiency-virus in the population. For analysis, we find the equilibria of a susceptible–infectious–immune system with a delay term. The well-established tools such as the Routh–Hurwitz criterion, Volterra–Lyapunov function, and Lasalle invariance principle are presented to investigate the stability of the model. The reproduction number and sensitivity of parameters are investigated. If the delay tactics are decreased, then the disease is endemic. On the other hand, if the delay tactics are increased then the disease is controlled in the population. The effect of the delay tactics with subpopulations is investigated. More precisely, all parameters are dependent on delay terms. In the end, to give the strength to a theoretical analysis of the model, a computer simulation is presented.