A very simple problem can be represented by a single layer of neurons.However, single-layer networks cannot solve certain problems. Multiple feed-forward layers give a network greater freedom. For example, any reasonable function can be represented with a two-layer network: a sigmoid layer feeding a linear output layer.Networks with biases can represent relationships between inputs and outputsmore easily than networks without biases. (For example, a neuron without a bias will always have a net input to the transfer function of zero when all of its inputs are zero. However, a neuron with a bias can learn to have any net transfer function input under the same conditions by learning an appropriate value for the bias.)
5 Benefits of Using Temperature Compensation
Many system peripherals contain information on how the performance of the peripheral changes as the temperature changes. For example, the Freescale MPX10 series of uncompensated 10 kPa pressure sensors contain information regarding how this sensor behaves across its operating temperature range. Many system peripherals contain information on how the performance of the 。
This information is used to create temperature compensation described in AN840
Temperature Compensation Methods that can be found at www.freescale.com. Sensors with internal compensation are also available,but at a higher cost. The benefit of lower overall system cost can be achieved by using a method where the
internal temperature sensor of the S08 microcontroller is used as the reference to perform temperature compensation.
Another benefit of using temperature compensation is extending the operating range of the end application. For example an application that requires accurate serial communications requires an accurate reference clock. If the reference clock loses accuracy at a hot temperature, the end application can not accomplish the serial communications. Using temperature compensation to modify the serial communications at a hot temperature allows the end application to operate across a wider range.
Both of the benefits listed above add value to the end application by using only the available on chip features.
6 Temperatue Compensation Methods
The theory behind performing temperature compensation is straight forward. It involves
22
taking a temperature reading and changing the peripheral parameters based on a knowledge base of how the peripheral performs across temperature.
Peripheral parameters are external or internal factors that affect the operation of the peripheral. In the case of an internal reference clock, such as the ICS internal reference clock on many S08 microcontrollers, the peripheral parameter that affects the output frequency is the TRIM register. Many system peripherals contain a trim register that allows compensation of the output of the peripheral.
Another example of a peripheral parameter are equation variables. For example, if the peripheral is a sensor that outputs an analog voltage and the output contains an offset that is temperature dependant, then the peripheral parameter that must be changed at different temperatures are the equation variables that account for the offset voltage.
To understand the peripheral parameters and how they must be changed, a knowledge base must be referenced. An example of a knowledge base is a specification. For example, the specification for the Freescale MPX10 series contains the parameters that define how the output of the pressure sensor changes across temperature ranges.
A knowledge base can also be generated by data collection. For example, the output of an internal reference signal is measured across the operating temperature range. This data can then be used to modify the system peripheral parameter to compensate for the changes across the temperature range. The following example looks closely at a performing oscillator trim compensation, using a temperature calibration reading to explain temperature compensation.
7 Network Architectures
Two or more of the neurons shown earlier can be combined in a layer, and a particular network could contain one or more such layers. First consider a single layer of neurons. (1) A Layer of Neurons
A one-layer network with R input elements and S neurons follows.
23
In this network, each element of the input vector p is connected to each neuron input through the weight matrix W. The ith neuron has a summer that gathers its weighted inputs and bias to form its own scalar output n(i). The various n(i) taken together form an S-element net input vector n. Finally, the neuron layer outputs form a column vector a. We show the expression for a at the bottom of the figure.
Note that it is common for the number of inputs to a layer to be different from the number of neurons (i.e., R | S). A layer is not constrained to have the number of its inputs equal to the number of its neurons.
You can create a single (composite) layer of neurons having different transfer functions simply by putting two of the networks shown earlier in parallel. Both networks would have the same inputs, and each network would create some of the outputs.
The input vector elements enter the network through the weight matrix W.
Note that the row indices on the elements of matrix W indicate the destination neuron of the weight, and the column indices indicate which source is the input for that weight. Thus, the indices inw12 say that the strength of the signal from the second input element to the first (and only) neuron is w12.
The S neuron R input one-layer network also can be drawn in abbreviated notation. Here p is an R length input vector, W is an SxR matrix, and a and b are S length vectors. As
24
defined previously, the neuron layer includes the weight matrix, the multiplication operations, the bias vector b, the summer, and the transfer function boxes. (2)Inputs and Layers
We are about to discuss networks having multiple layers so we will need to extend our notation to talk about such networks. Specifically, we need to make a distinction between weight matrices that are connected to inputs and weight matrices that are connected between layers. We also need to identify the source and destination for the weight matrices. We will call weight matrices connected to inputs, input weights; and we will call weight matrices coming from layer outputs, layer weights. Further, we will use superscripts to identify the source (second index) and the destination (first index) for the various weights and other elements of the network. To illustrate, we have taken the one-layer multiple input network shown earlier and redrawn it in abbreviated form below.
As you can see, we have labeled the weight matrix connected to the input vector p as an Input Weight matrix (IW1,1) having a source 1 (second index) and a destination 1 (first index). Also, elements of layer one, such as its bias, net input, and output have a superscript 1 to say that they are associated with the first layer.
In the next section, we will use Layer Weight (LW) matrices as well as Input Weight (IW) matrices.
You might recall from the notation section of the Preface that conversion of the layer weight matrix from math to code for a particular network called net is:
25
Thus, we could write the code to obtain the net input to the transfer function as: n{1} = net.IW{1,1}*p + net.b{1}; (8)Backpropagation Overview
Backpropagation was created by generalizing the Widrow-Hoff learning rule to multiple-layer networks and nonlinear differentiable transfer functions.Input vectors and the corresponding target vectors are used to train a network until it can approximate a function,associate input vectors with specific output vectors,or classify input vectors in an appropriate way as defined by you.Networks with biases, a sigmoid layer, and a linear output layer are capable of approximating any function with a finite number of discontinuities.
Standard backpropagation is a gradient descent algorithm, as is the Widrow-Hoff learning rule, in which the network weights are moved along the negative of the gradient of the performance function. The term backpropagation refers to the manner in which the gradient is computed for nonlinear multilayer networks. There are a number of variations on the basic algorithm that are based on other standard optimization techniques, such as conjugate gradient and Newton methods. The Neural Network Toolbox implements a
number of these variations. This chapter explains how to use each of these routines and discusses the advantages and disadvantages of each.
Properly trained backpropagation networks tend to give reasonable answers when presented with inputs that they have never seen. Typically, a new input leads to an output similar to the correct output for input vectors used in training that are similar to the new input being presented. This generalization property makes it possible to train a network on a representative set of input/target pairs and get good results without training the network on all possible input/output pairs. There are two features of the Neural Network Toolbox that are designed to improve network generalization - regularization and early stopping. These features and their use are discussed later in this chapter.
This chapter also discusses preprocessing and postprocessing techniques, which can improve the efficiency of network training.
Before beginning this chapter you may want to read a basic reference on
26