Neural network models

Next: Hybrid models Up: Continuous models Previous: Ordinary Differential Equations

Neural network models

Artificial neural networks are mathematical models of information processing originally inspired by networks of neurons in the brain [58]. A neural network typically consists of a collection of nodes, some of which may be designated as input or output nodes, connected by weighted links (see Figure 10). Each node contains a transfer function that transforms a set of weighted input signals into an output signal. These networks can be trained to match particular patterns of activation via a number of learning processes.

**Figure 10:** A standard feed-forward neural network. Nodes are divided into input nodes, output nodes and hidden nodes, which significantly increase the computational abilities of the network. This basic structure may be modified by changing the number of nodes in a layer, the number of layers and the arrangement of the links. In particular adding feedback links from the output layer back to the input layer allows the network to process temporal information, such as grammatical structure and patterns of gene expression.
$\begin{figure} \begin{center} \includegraphics[scale=0.4]{figures/nn} \end{center} \end{figure}$

Mathematically, it is possible to create a mapping between a neural network and a system of ODEs. Conceptually, a relatively straightforward analogy may be drawn between an information processing system in which the constituent elements are neurons and the links are synaptic interactions and a system in which the elements are genes and the links are regulatory interactions. Consequently, a number of researchers have used network architectures and concepts taken directly from neural networks and connectionist models [87,136,137].

The regulatory input to gene i is described as a the sum of the weighted inputs modified by the gene's activation threshold $\theta$ :

$\begin{eqnarray} g_i & = & \sum_jw_{ij}y_j + \theta_i \end{eqnarray}$

where w_ij is the strength of the regulatory interaction between genes i and j. A gene's level of activation is determined on the basis of this regulatory input and degradation:

$\begin{eqnarray} \frac{dx_i}{dt} & = & {\alpha_i}f_i(g_i)-{\gamma_i}x_i \end{eqnarray}$

where $\alpha$ and $\gamma$ are activation and degradation rates and f_i is a sigmoid transfer function as described above.

This type of formalism has been used in several different types of models. Mjolsness et. al. developed a phenomonological model of segmentation in the Drosophila blastoderm that used a neural network model to describe the internal dynamics of a cell as well as a generative grammar that described higher-level developmental processes such as cell division and differentiation [87]. This model has also been applied to other aspects of pattern formation and neurogenesis in Drosophila [100,101,79]. In these models, network parameters were trained such that the dynamics matched observed experimental behaviour.

Vohradský used a similar approach to model the lysis/lysogeney decision in phage $\lambda$ [136,137]. Here, the network structure is determined a priori from known interactions and the interaction weights are learned from experimental data. Several variations on the basic network are investigated, including connected networks and multi-compartment models, in which protein and RNA products are represented by separate network layers [137] (see Figure 11).

**Figure 11:** A two-compartment model of gene expression. Regulatory proteins A, B, C,..., n control the level of expression of gene i within the nucleus. The resulting mRNA, along with additional factors $\alpha$ , $\beta$ , $\gamma$ ,..., v controls the production of the corresponding protein i. Note also the presence of mRNA and protein degradation and an autoregulatory feedback loop. (From [137]).
$\begin{figure} \begin{center} \includegraphics[scale=0.7]{figures/multi-nn} \end{center} \end{figure}$

Neural network models have also been widely used in network inference (see the appropriate sections of [134] for a comprehensive review). In this domain, D'haeseleer has performed a comparison between the performance of network models using both linear and non-linear updating functions and obtained several analytical results [36].

Next: Hybrid models Up: Continuous models Previous: Ordinary Differential Equations

Nic Geard 2004-05-06