scholarly journals Super-convergence: very fast training of neural networks using large learning rates

2019 ◽  
Author(s):  
Leslie N. Smith ◽  
Nicholay Topin
Keyword(s):  
Neural Networks ◽  
Learning Rates ◽  
Fast Training ◽  
Super Convergence

Weights Direct Determination of Feedforward Neural Networks without Iterative BP-Training

2010 ◽  
pp. 197-225 ◽  
Author(s):  
Yunong Zhang ◽  
Ning Tan
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Artificial Neural Networks ◽  
Direct Determination ◽  
Bp Neural Networks ◽  
Learning Rates ◽  
The Neural Network ◽  
Gradient Based ◽  
Artificial Neural ◽  
Output Behavior

Artificial neural networks (ANN), especially with error back-propagation (BP) training algorithms, have been widely investigated and applied in various science and engineering fields. However, the BP algorithms are essentially gradient-based iterative methods, which adjust the neural-network weights to bring the network input/output behavior into a desired mapping by taking a gradient-based descent direction. This kind of iterative neural-network (NN) methods has shown some inherent weaknesses, such as, 1) the possibility of being trapped into local minima, 2) the difficulty in choosing appropriate learning rates, and 3) the inability to design the optimal or smallest NN-structure. To resolve such weaknesses of BP neural networks, we have asked ourselves a special question: Could neural-network weights be determined directly without iterative BP-training? The answer appears to be YES, which is demonstrated in this chapter with three positive but different examples. In other words, a new type of artificial neural networks with linearly-independent or orthogonal activation functions, is being presented, analyzed, simulated and verified by us, of which the neural-network weights and structure could be decided directly and more deterministically as well (in comparison with usual conventional BP neural networks).

scholarly journals Artificial Neural Networks in Drug Design 11: Influence of Learning Rate and Momentum Factor on the Predictive Ability

2000 ◽  
Vol 68 (1) ◽  
pp. 57-64 ◽  
Author(s):  
D. Kaiser ◽  
C. Tmej ◽  
P. Chiba ◽  
K.-J. Schaper ◽  
G. Ecker
Keyword(s):  
Neural Networks ◽  
Artificial Neural Networks ◽  
Predictive Power ◽  
Predictive Ability ◽  
Learning Rate ◽  
Data Set ◽  
Lead Compounds ◽  
Learning Rates ◽  
Artificial Neural ◽  
Nanomolar Range

A data set of 48 propafenone-type modulators of multidrug resistance was used to investigate the influence of learning rate and momentum factor on the predictive power of artificial neural networks of different architecture. Generally, small learning rates and medium sized momentum factors are preferred. Some of the networks showed higher cross validated Q2 values than the corresponding linear model (0.87 vs. 0.83). Screening of a 158 compound virtual library identified several new lead compounds with activities in the nanomolar range.

AN ULTRASONIC MOTOR SPEED CONTROL SCHEME BASED ON NEURAL NETWORKS AND ITERATIVE CONTROLLER

2004 ◽  
Vol 01 (03) ◽  
pp. 457-470
Author(s):  
X. H. SHI ◽  
Y. C. LIANG ◽  
X. XU
Keyword(s):  
Neural Networks ◽  
Adaptive Learning ◽  
Speed Control ◽  
Ultrasonic Motor ◽  
Random Disturbance ◽  
Motor Speed ◽  
Neural Network Controller ◽  
Learning Rates ◽  
Network Controller ◽  
Control Scheme

An ultrasonic motor speed control scheme is presented in this paper based on neural networks and iterative controller. Suitable ranges of the adaptive learning rates of neural network controller are presented through the theoretical analysis on the proposed model, which could guarantee its stability. The convergence of iterative controller is also discussed. Numerical results show that the control scheme is effective for various kinds of reference speeds of ultrasonic motors. Comparisons with the existing method show that the precision of control could be increased using the proposed method. Simulations also show that the proposed scheme is fairly robust against random disturbance to the control variables.

Sign in / Sign up

Export Citation Format

Share Document