附录：
附录1： 外文文献原文
Mark Beale,Neural Network Toolbox: 1 Neuron Model
Simple Neuron
A neuron with a single scalar input and no bias appears on the left below.
The scalar input p is transmitted through a connection that multiplies its strength by the scalar weight w, to form the product wp, again a scalar. Here the weighted input wp is the only argument of the transfer function f, which produces the scalar output a. The neuron on the right has a scalar bias, b. You may view the bias as simply being added to the product wp as shown by the summing junction or as shifting the function f to the left by an amount b. The bias is much like a weight, except that it has a constant input of 1.The transfer function net input n, again a scalar, is the sum of the weighted input wp and the bias b. This sum is the argument of the transfer function f.
(Chapter 7 discusses a different way to form the net input n.) Here f is a transfer function, typically a step function or a sigmoid function, which takes the argument n and produces the output a. Examples of various transfer functions are given in the next section. Note that w and b are both adjustable scalar parameters of the neuron. The central idea of neural networks is that such parameters can be adjusted so that the network exhibits some desired or interesting behavior. Thus, we can train the network to do a particular job by adjusting the weight or bias parameters, or perhaps the network itself will adjust these parameters to achieve some desired end.
17
All of the neurons in this toolbox have provision for a bias, and a bias is used in many of our examples and will be assumed in most of this toolbox. However, you may omit a bias in a neuron if you want.
As previously noted, the bias b is an adjustable (scalar) parameter of the neuron. It is not an input. However, the constant 1 that drives the bias is an input and must be treated as such when considering the linear dependence of input vectors in Chapter 4, “Linear Filters.”
2 Transfer Functions
Many transfer functions are included in this toolbox. A complete list of them can be
found in “Transfer Function Graphs” in Chapter 14. Three of the most commonly used functions are shown below.
The hard-limit transfer function shown above limits the output of the neuron to either 0, if the net input argument n is less than 0; or 1, if n is greater than or equal to 0. We will use this function in Chapter 3 “Perceptrons” to create neurons that make classification decisions.The toolbox has a function, hardlim, to realize the mathematical hard-limit ransfer function shown above. Try the code shown below.
n = -5:0.1:5;
plot(n,hardlim(n),'c+:');
It produces a plot of the function hardlim over the range -5 to +5.
All of the mathematical transfer functions in the toolbox can be realized with a function having the same name.
The linear transfer function is shown below.
18
Neurons of this type are used as linear approximators in “Linear Filters” in Chapter 4.
The sigmoid transfer function shown below takes the input, which may have any value between plus and minus infinity, and squashes the output into the range 0 to 1.
This transfer function is commonly used in backpropagation networks, in part because it is differentiable.The symbol in the square to the right of each transfer function graph shownabove represents the associated transfer function. These icons will replace the general f in the boxes of network diagrams to show the particular transfer function being used.
3 Neuron with Vector Input
A neuron with a single R-element input vector is shown below. Here the individual element inputs
p1, p2,... pR are multiplied by weights
w1, 1, w1, 2, ... w1, R
and the weighted values are fed to the summing junction. Their sum is simply Wp, the dot product of the (single row) matrix W and the vector p.
19
The neuron has a bias b, which is summed with the weighted inputs to form the net input n. This sum, n, is the argument of the transfer function f. n = w1, 1p1 + w1, 2p2 + ... + w1, RpR + b
This expression can, of course, be written in MATLAB code as: n = W*p + b
However, the user will seldom be writing code at this low level, for such code is already built into functions to define and simulate entire networks.
The figure of a single neuron shown above contains a lot of detail. When we consider networks with many neurons and perhaps layers of many neurons, there is so much detail that the main thoughts tend to be lost. Thus, the authors have devised an abbreviated notation for an individual neuron. This notation, which will be used later in circuits of multiple neurons.
Here the input vector p is represented by the solid dark vertical bar at the left.The dimensions of p are shown below the symbol p in the figure as Rx1. (Note that we will use a capital letter, such as R in the previous sentence, when referring to the size of a vector.) Thus, p is a vector of R input elements. These inputs post multiply the single row, R column matrix W. As before, a constant 1 enters the neuron as an input and is multiplied by a scalar bias b. The net input to the transfer function f is n, the sum of the bias b and the
20
product Wp. This sum is passed to the transfer function f to get the neuron’s output a, which in this case is a scalar. Note that if we had more than one neuron, the network output would be a vector.
A layer of a network is defined in the figure shown above. A layer includes the
combination of the weights, the multiplication and summing operation (here realized as a vector product Wp), the bias b, and the transfer function f. The array of inputs, vector p, is not included in or called a layer. Each time this abbreviated network notation is used, the size of the matrices will be shown just below their matrix variable names. We hope that this notation will allow you to understand the architectures and follow the matrix mathematics associated with them.
As discussed previously, when a specific transfer function is to be used in a figure, the symbol for that transfer function will replace the f shown above. 4 Summary
The inputs to a neuron include its bias and the sum of its weighted inputs(using the inner product). The output of a neuron depends on the neuron’s inputs and on its transfer function. There are many useful transfer functions.A single neuron cannot do very much. However, several neurons can becombined into a layer or multiple layers that have great power. Hopefully thistoolbox makes it easy to create and understand such large networks.
The architecture of a network consists of a description of how many layers a network has, the number of neurons in each layer, each layer’s transfer function, and how the layers connect to each other. The best architecture to use depends on the type of problem to be represented by the network.
A network effects a computation by mapping input values to output values. The particular mapping problem to be performed fixes the number of inputs, as well as the number of outputs for the network.
Aside from the number of neurons in a network’s output layer, the number of neurons in each layer is up to the designer. Except for purely linear networks, the more neurons in a hidden layer, the more powerful the network. If a linear mapping needs to be represented linear neurons should be used.However, linear networks cannot perform any nonlinear computation. Use of a nonlinear transfer function makes a network capable of storing nonlinear relationships between input and output.
21