THEORETICAL RESULTS FOR A CLASS OF NEURAL NETWORKS

1995 ◽  
Vol 06 (04) ◽  
pp. 463-472
Author(s):  
STEVE G. ROMANIUK
Keyword(s):  
Neural Networks ◽  
Theoretical Analysis ◽  
General Class ◽  
Simple Algorithm ◽  
Benchmark Problems ◽  
Network Architectures ◽  
Large Set ◽  
Minimal Network ◽  
Theoretical Results

The ability to derive minimal network architectures for neural networks has been at the center of attention for several years now. To this date numerous algorithms have been proposed to automatically construct networks. Unfortunately, these algorithms lack a fundamental theoretical analysis of their capabilities and only empirical evaluations on a few selected benchmark problems exist. Some theoretical results have been provided for small classes of well-known benchmark problems such as parity- and encoder-functions, but these are of little value due to their restrictiveness. In this work we describe a general class of 2-layer networks with 2 hidden units capable of representing a large set of problems. The cardinality of this class grows exponentially with regard to the inputs N. Furthermore, we outline a simple algorithm that allows us to determine, if any function (problem) is a member of this class. The class considered in this paper includes the benchmark problems parity and symmetry. Finally, we expand this class to include an even larger set of functions and point out several interesting properties it exhibits.

Measures of Network Structure

2010 ◽  
pp. 41-49
Author(s):  
Ani Calinescu ◽  
Janet Efstathiou
Keyword(s):  
Neural Networks ◽  
World Wide ◽  
The Internet ◽  
Network Architectures ◽  
Production Networks ◽  
Related Research ◽  
The World ◽  
Research Questions ◽  
The Brain ◽  
Theoretical Results

Networked systems, natural or designed, have always been part of life. Their sophistication degree and complexity have increased through either natural evolution or technological progress. However, recent theoretical results have shown that a previously unexpected number of different classes of networks share similar network architectures and universal laws. Examples of such networks include metabolic pathways and ecosystems, the Internet and the World Wide Web, and organizational, social, and neural networks. Complex systems-related research questions investigated by researchers nowadays include: how consciousness arises out of the interactions of the neurons in the brain and between the brain and the environment (Amaral & Ottino, 2004; Barabási, 2005; Barabási & Oltvai, 2004; Neuman, 2003b) and how this understanding could be used for designing networked organizations or production networks whose behavior satisfies a given specification.

scholarly journals Verification of Continuous Time Recurrent Neural Networks (Benchmark Proposal)

10.29007/6czp ◽  
2018 ◽  
Author(s):  
Patrick Musau ◽  
Taylor T. Johnson
Keyword(s):  
Neural Networks ◽  
Continuous Time ◽  
Initial Conditions ◽  
Research Area ◽  
Benchmark Problems ◽  
Network Architectures ◽  
Fixed Point Attractor ◽  
Point Attractor ◽  
Superior Approach ◽  
Verification Techniques

This manuscript presents a description and implementation of two benchmark problems for continuous-time recurrent neural network (RNN) verification. The first problem deals with the approximation of a vector field for a fixed point attractor located at the origin, whereas the second problem deals with the system identification of a forced damped pendulum. While the verification of neural networks is complicated and often impenetrable to the majority of verification techniques, continuous-time RNNs represent a class of networks that may be accessible to reachability methods for nonlinear ordinary differential equations (ODEs) derived originally in biology and neuroscience. Thus, an understanding of the behavior of a RNN may be gained by simulating the nonlinear equations from a diverse set of initial conditions and inputs, or considering reachability analysis from a set of initial conditions. The verification of continuous-time RNNs is a research area that has received little attention and if the research community can achieve meaningful results in this domain, then this class of neural networks may prove to be a superior approach in solving complex problems compared to other network architectures.

Measures of Network Structure

2010 ◽  
pp. 926-932
Author(s):  
Ani Calinescu ◽  
Janet Efstathiou
Keyword(s):  
Neural Networks ◽  
World Wide ◽  
The Internet ◽  
Network Architectures ◽  
Production Networks ◽  
Related Research ◽  
The World ◽  
Research Questions ◽  
The Brain ◽  
Theoretical Results

Networked systems, natural or designed, have always been part of life. Their sophistication degree and complexity have increased through either natural evolution or technological progress. However, recent theoretical results have shown that a previously unexpected number of different classes of networks share similar network architectures and universal laws. Examples of such networks include metabolic pathways and ecosystems, the Internet and the World Wide Web, and organizational, social, and neural networks. Complex systems-related research questions investigated by researchers nowadays include: how consciousness arises out of the interactions of the neurons in the brain and between the brain and the environment (Amaral & Ottino, 2004; Barabási, 2005; Barabási & Oltvai, 2004; Neuman, 2003b) and how this understanding could be used for designing networked organizations or production networks whose behavior satisfies a given specification.

Color Space Transformation using Neural Networks

2019 ◽  
Vol 2019 (1) ◽  
pp. 153-158
Author(s):  
Lindsay MacDonald
Keyword(s):  
Neural Network ◽  
Neural Networks ◽  
Color Space ◽  
Reflectance Spectra ◽  
Network Architectures ◽  
Color Spaces ◽  
Natural Materials ◽  
Space Transformation ◽  
Color Space Transformation

We investigated how well a multilayer neural network could implement the mapping between two trichromatic color spaces, specifically from camera R,G,B to tristimulus X,Y,Z. For training the network, a set of 800,000 synthetic reflectance spectra was generated. For testing the network, a set of 8,714 real reflectance spectra was collated from instrumental measurements on textiles, paints and natural materials. Various network architectures were tested, with both linear and sigmoidal activations. Results show that over 85% of all test samples had color errors of less than 1.0 ΔE2000 units, much more accurate than could be achieved by regression.

Recurrent Neural Networks for Edge Intelligence

2021 ◽  
Vol 54 (4) ◽  
pp. 1-38
Author(s):  
Varsha S. Lalapura ◽  
J. Amudha ◽  
Hariramn Selvamuruga Satheesh
Keyword(s):  
Neural Networks ◽  
Recurrent Neural Networks ◽  
Model Performance ◽  
Network Architectures ◽  
Training Procedure ◽  
Resource Constrained ◽  
Compression Technique ◽  
Creative Art ◽  
Comprehensive Survey ◽  
Intelligent Models

Recurrent Neural Networks are ubiquitous and pervasive in many artificial intelligence applications such as speech recognition, predictive healthcare, creative art, and so on. Although they provide accurate superior solutions, they pose a massive challenge “training havoc.” Current expansion of IoT demands intelligent models to be deployed at the edge. This is precisely to handle increasing model sizes and complex network architectures. Design efforts to meet these for greater performance have had inverse effects on portability on edge devices with real-time constraints of memory, latency, and energy. This article provides a detailed insight into various compression techniques widely disseminated in the deep learning regime. They have become key in mapping powerful RNNs onto resource-constrained devices. While compression of RNNs is the main focus of the survey, it also highlights challenges encountered while training. The training procedure directly influences model performance and compression alongside. Recent advancements to overcome the training challenges with their strengths and drawbacks are discussed. In short, the survey covers the three-step process, namely, architecture selection, efficient training process, and suitable compression technique applicable to a resource-constrained environment. It is thus one of the comprehensive survey guides a developer can adapt for a time-series problem context and an RNN solution for the edge.

Global dissipativity and exponential synchronization of mixed time-varying delays neural networks with discontinuous activations

2020 ◽  
Vol 21 (7-8) ◽  
pp. 693-704 ◽  
Author(s):  
Kaifang Fei ◽  
Minghui Jiang ◽  
Meng Yan ◽  
Weizhen Liu
Keyword(s):  
Neural Networks ◽  
State Feedback ◽  
Exponential Synchronization ◽  
Matrix Measure ◽  
Analysis Techniques ◽  
Discontinuous Activations ◽  
Inclusion Theory ◽  
Generalized Halanay Inequality ◽  
New Criteria ◽  
Theoretical Results

AbstractIn this paper, the matters of dissipativity and synchronization for non-autonomous Hopfield neural networks with discontinuous activations are investigated. Firstly, under the framework of extending Filippov differential inclusion theory, several effective new criteria are derived. The global dissipativity of Filippov solution to neural networks is proved by using generalized Halanay inequality and matrix measure method. Secondly, the global exponential synchronization of the addressed network drive system and the response system is realized by utilizing inequality and some analysis techniques and designing the discontinuous state feedback controller. Finally, several numerical examples are given to verify the validity of the theoretical results.

scholarly journals Exploiting heterogeneity in operational neural networks by synaptic plasticity

2021 ◽  
Author(s):  
Serkan Kiranyaz ◽  
Junaid Malik ◽  
Habib Ben Abdallah ◽  
Turker Ince ◽  
Alexandros Iosifidis ◽  
...  
Keyword(s):  
Neural Networks ◽  
Synaptic Plasticity ◽  
Network Model ◽  
Neuron Model ◽  
Linear Operators ◽  
Training Data ◽  
Learning Performance ◽  
Minimal Network ◽  
Hidden Layer ◽  
Hidden Neurons

AbstractThe recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) that are homogenous only with a linear neuron model. As a heterogenous network model, ONNs are based on a generalized neuron model that can encapsulate any set of non-linear operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. However, the default search method to find optimal operators in ONNs, the so-called Greedy Iterative Search (GIS) method, usually takes several training sessions to find a single operator set per layer. This is not only computationally demanding, also the network heterogeneity is limited since the same set of operators will then be used for all neurons in each layer. To address this deficiency and exploit a superior level of heterogeneity, in this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the “Synaptic Plasticity” paradigm that poses the essential learning theory in biological neurons. During training, each operator set in the library can be evaluated by their synaptic plasticity level, ranked from the worst to the best, and an “elite” ONN can then be configured using the top-ranked operator sets found at each hidden layer. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs and as a result, the performance gap over the CNNs further widens.

scholarly journals Optimising the AR Engraved Structure on Light-Guide Facets for a Wide Range of Wavelengths

Optics ◽  
2020 ◽  
Vol 2 (1) ◽  
pp. 25-42
Author(s):  
Ioseph Gurwich ◽  
Yakov Greenberg ◽  
Kobi Harush ◽  
Yarden Tzabari
Keyword(s):  
Theoretical Analysis ◽  
Light Guide ◽  
Spectral Band ◽  
Angular Range ◽  
The Past ◽  
Input And Output ◽  
Wide Range ◽  
The Given ◽  
Theoretical Results ◽  
Shape And Size

The present study is aimed at designing anti-reflective (AR) engraving on the input–output surfaces of a rectangular light-guide. We estimate AR efficiency, by the transmittance level in the angular range, determined by the light-guide. Using nano-engraving, we achieve a uniform high transmission over a wide range of wavelengths. In the past, we used smoothed conical pins or indentations on the faces of light-guide crystal as the engraved structure. Here, we widen the class of pins under consideration, following the physical model developed in the previous paper. We analyze the smoothed pyramidal pins with different base shapes. The possible effect of randomization of the pins parameters is also examined. The results obtained demonstrate optimized engraved structure with parameters depending on the required spectral range and facet format. The predicted level of transmittance is close to 99%, and its flatness (estimated by the standard deviation) in the required wavelengths range is 0.2%. The theoretical analysis and numerical calculations indicate that the obtained results demonstrate the best transmission (reflection) we can expect for a facet with the given shape and size for the required spectral band. The approach is equally useful for any other form and of the facet. We also discuss a simple way of comparing experimental and theoretical results for a light-guide with the designed input and output features. In this study, as well as in our previous work, we restrict ourselves to rectangular facets. We also consider the limitations on maximal transmission produced by the size and shape of the light-guide facets. The theoretical analysis is performed for an infinite structure and serves as an upper bound on the transmittance for smaller-size apertures.

On exponential stability of delayed neural networks with a general class of activation functions

2002 ◽  
Vol 298 (2-3) ◽  
pp. 122-132 ◽  
Author(s):  
Changyin Sun ◽  
Kanjian Zhang ◽  
Shumin Fei ◽  
Chun-Bo Feng
Keyword(s):  
Neural Networks ◽  
Exponential Stability ◽  
General Class ◽  
Activation Functions ◽  
Delayed Neural Networks
Sign in / Sign up

Export Citation Format

Share Document