Evaluating the Learning Procedure of CNNs through a Sequence of Prognostic Tests Utilising Information Theoretical Measures

Deep learning has proven to be an important element of modern data processing technology, which has found its application in many areas such as multimodal sensor data processing and understanding, data generation and anomaly detection. While the use of deep learning is booming in many real-world tasks, the internal processes of how it draws results is still uncertain. Understanding the data processing pathways within a deep neural network is important for transparency and better resource utilisation. In this paper, a method utilising information theoretic measures is used to reveal the typical learning patterns of convolutional neural networks, which are commonly used for image processing tasks. For this purpose, training samples, true labels and estimated labels are considered to be random variables. The mutual information and conditional entropy between these variables are then studied using information theoretical measures. This paper shows that more convolutional layers in the network improve its learning and unnecessarily higher numbers of convolutional layers do not improve the learning any further. The number of convolutional layers that need to be added to a neural network to gain the desired learning level can be determined with the help of theoretic information quantities including entropy, inequality and mutual information among the inputs to the network. The kernel size of convolutional layers only affects the learning speed of the network. This study also shows that where the dropout layer is applied to has no significant effects on the learning of networks with a lower dropout rate, and it is better placed immediately after the last convolutional layer with higher dropout rates.

A Review on BGK Models for Gas Mixtures of Mono and Polyatomic Molecules

We consider the Bathnagar–Gross–Krook (BGK) model, an approximation of the Boltzmann equation, describing the time evolution of a single momoatomic rarefied gas and satisfying the same two main properties (conservation properties and entropy inequality). However, in practical applications, one often has to deal with two additional physical issues. First, a gas often does not consist of only one species, but it consists of a mixture of different species. Second, the particles can store energy not only in translational degrees of freedom but also in internal degrees of freedom such as rotations or vibrations (polyatomic molecules). Therefore, here, we will present recent BGK models for gas mixtures for mono- and polyatomic particles and the existing mathematical theory for these models.

Packing topological entropy for amenable group actions

Packing topological entropy is a dynamical analogy of the packing dimension, which can be viewed as a counterpart of Bowen topological entropy. In the present paper we give a systematic study of the packing topological entropy for a continuous G-action dynamical system $(X,G)$ , where X is a compact metric space and G is a countable infinite discrete amenable group. We first prove a variational principle for amenable packing topological entropy: for any Borel subset Z of X, the packing topological entropy of Z equals the supremum of upper local entropy over all Borel probability measures for which the subset Z has full measure. Then we obtain an entropy inequality concerning amenable packing entropy. Finally, we show that the packing topological entropy of the set of generic points for any invariant Borel probability measure $\mu $ coincides with the metric entropy if either $\mu $ is ergodic or the system satisfies a kind of specification property.

A continuum mechanical framework for modeling tumor growth and treatment in two- and three-phase systems

The growth and treatment of tumors is an important problem to society that involves the manifestation of cellular phenomena at length scales on the order of centimeters. Continuum mechanical approaches are being increasingly used to model tumors at the largest length scales of concern. The issue of how to best connect such descriptions to smaller-scale descriptions remains open. We formulate a framework to derive macroscale models of tumor behavior using the thermodynamically constrained averaging theory (TCAT), which provides a firm connection with the microscale and constraints on permissible forms of closure relations. We build on developments in the porous medium mechanics literature to formulate fundamental entropy inequality expressions for a general class of three-phase, compositional models at the macroscale. We use the general framework derived to formulate two classes of models, a two-phase model and a three-phase model. The general TCAT framework derived forms the basis for a wide range of potential models of varying sophistication, which can be derived, approximated, and applied to understand not only tumor growth but also the effectiveness of various treatment modalities.

Existence analysis of a degenerate diffusion system for heat-conducting gases

The existence of global weak solutions to a parabolic energy-transport system in a bounded domain with no-flux boundary conditions is proved. The model can be derived in the diffusion limit from a kinetic equation with a linear collision operator involving a non-isothermal Maxwellian. The evolution of the local temperature is governed by a heat equation with a source term that depends on the energy of the distribution function. The limiting model consists of cross-diffusion equations with an entropy structure. The main difficulty is the nonstandard degeneracy, i.e., ellipticity is lost when the gas density or temperature vanishes. The existence proof is based on a priori estimates coming from the entropy inequality and the $$H^{-1}$$ H - 1 method and on techniques from mathematical fluid dynamics (renormalized formulation, div-curl lemma).

A Thermodynamic-Consistent Model for the Thermo-Chemo-Mechanical Couplings in Amorphous Shape-Memory Polymers

Chemically-responsive amorphous shape-memory polymers (SMPs) can transit from the temporary shape to the permanent shape in responsive to solvents. This effect has been reported in various polymer-solvent systems. However, limited attention has been paid to the constitutive modeling of this behavior. In this work, we develop a fully thermo-chemo-mechanical coupled thermodynamic framework for the chemically-responsive amorphous SMPs. The framework shows that the entropy, the chemical potential and the stress can be directly obtained if the Helmholtz free energy density is defined. Based on the entropy inequality, the evolution equation for the viscous strain, the temperature and the number of solvent molecules are also derived. We also provide an explicit form of Helmholtz free energy density as an example. In addition, based on the free volume concept, the dependence of viscosity and diffusivity on the temperature and solvent concentration is defined. The theoretical framework can potentially advance the fundamental understanding of chemically-responsive shape-memory effect. Meanwhile, it can also be used to describe other important physical processes such as the diffusion of solvents in glassy polymers.

Entropy inequality and energy dissipation of inertial Qian–Sheng model for nematic liquid crystals

For the inertial Qian–Sheng model of nematic liquid crystals in the [Formula: see text]-tensor framework, we illustrate the roles played by the entropy inequality and energy dissipation in the well-posedness of smooth solutions when we employ energy method. We first derive the coefficients requirements from the entropy inequality, and point out the entropy inequality is insufficient to guarantee energy dissipation. We then introduce a novel Condition (H) which ensures the energy dissipation. We prove that when both the entropy inequality and Condition (H) are obeyed, the local in time smooth solutions exist for large initial data. Otherwise, we can only obtain small data local solutions. Furthermore, to extend the solutions globally in time and obtain the decay of solutions, we require at least one of the two conditions: entropy inequality, or [Formula: see text], which significantly enlarge the range of the coefficients in previous works.

Spontaneous Negative Entropy Increments in Granular Flows

The entropy inequality, commonly taken as an axiom of continuum mechanics, is found to be spontaneously violated in macroscopic granular media undergoing collisional dynamics. The result falls within the fluctuation theorem of nonequilibrium thermodynamics, which is known to replace the Second Law for finite systems. This phenomenon amounts to the system stochastically displaying negative increments of entropy. The focus is on granular media in Couette flows, consisting of monosized circular disks (with 10 to 104 disks of diameters 0.01 m to 1 m) with frictional-Hookean contacts simulated by molecular dynamics accounting for micropolar effects. Overall, it is determined that the probability of negative entropy increments diminishes with the Eulerian velocity gradient increasing, while it tends to increase in a sigmoidal fashion with the Young modulus of disks increasing. This behavior is examined for a very wide range of known materials: from the softest polymers to the stiffest (i.e., carbyne). The disks’ Poisson ratio is found to have a weak effect on the probability of occurrence of negative entropy increments.