ۥ-!@ -t(%&j&jj&j&j&j&j&x&<&&&& & &&&2 (((8(8(8(8(8(8(8(:(:(:(:(:(:(X(4(eX(j&X(X(Frustrated Inculcated Synapse Training A Neural Network Training Algorithm David W. Croft Objectives To build a "black box" that will learn to duplicate any desired function. This will be done using the "Frustrated Synapse" neural network training algorithm. The "black box" system will be called a "Neural Optimization Box" or "NOB". Theory A neuron is a device with inputs and an output whose function is determined by a simple "activation rule". An artificial neural network (ANN) is a network of neurons whose interconnections of inputs and outputs are weighted. An artificial neural network (ANN) has the structure to duplicate any function out of the infinite set of possible functions by choosing the appropriate set of interconnecting weights. Choosing the right set of weights to make your ANN duplicate your desired function is very hard because there are an infinite number of possible sets of weights. A neural network training algorithm is method of choosing weights for an ANN. If the algorithm is good, it will cause the weights to converge to the correct or nearly-correct values in a reasonable amount of time. The "neuron state" (Y) describes the state of an individual neuron: firing (+1), inactive (0), or exhausted (-1). The "network state" (Z) describes the present state of all the neuron states in the network. In an ANN of N neurons, there are 3**N possible network states. The "output state" (O) describes the pattern of the network as it changes from one network state to another after is has achieved steady-state. A "Neural Optimization Box", or NOB, is a specialized ANN that uses the "Frustrated Synapse" neural network training algorithm to enable it to duplicate any desired function. Learning Rule A neuron learns by assuming that if it is presented with the same input in quick succession, it failed to respond appropriately to the input the first time. It will then modify its weights in such a way that it will respond differently when the input is presented again -- not firing if it fired earlier or firing if it failed to fire. This learning rule can be implemented by assuming that a neuron has four states: ready, primed, firing, and tired. A neuron begins in the ready state awaiting inputs. If it receives excitatory inputs that are too weak to cause it to fire or are overwhelmed by additional inhibitory inputs, it transitions to the primed state. If it receives a strong net excitatory input, it will immediately transition from the ready state to the firing state. If the neuron is primed and the conditions that caused it to become primed continue, it will remain primed. If it receives a strong net excitatory input, it will transition to the firing state. Otherwise, it will become simply ready again. If the neuron is firing, it will transition to the tired state. If the neuron is tired, it will transition to the ready state. Learning occurs when an excitatory input arrives and the neuron is either tired or primed. If the neuron is tired and one of the synaptic weights attempts to excite it, the weight is weakened as it is assumed that the weight is attempting to fire the neuron repetitiously after unsuccessfully achieving the desired output the first time. If the neuron is primed and one of the synaptic weights attempts to excite it, the weight is strengthened as it is assumed that the weight is attempting to fire neuron after being unsuccessful in doing so the first time. Training for Known States If the desired states of the individual neurons in response to a given input are known, training is a simple three-step process: predict the results of the application of the activation rule for the given inputs, compare the predicted activated responses to the desired responses and change the current neurons states if necessary, then apply the learning rule. If the activation rule dictated that the neuron should transition to a state that was not the desired response state, the learning rule will modify the weights correctly. Activation Response State Weight Prediction Desired Change Learning --------------------------------------------------------------------- Firing Ready Tired Weakens Ready Firing Primed Strengthens Training for Unknown States Training is more difficult if only the desired output is known and the desired states for the neurons between the input and the output are not. As it is unknown what intermediate states will create the desired output, the individual neurons must be trained based upon a comparison of the output results to the desired outputs which indicates only whether the error of the whole network was either small or large. The training algorithm must perform without directly modifying any of the intermediate neuron states with only the network output error as a guide. The training algorithm depends upon the timing of the input presentation. If a neuron sees an input presented repeatedly, it will assume that it responded incorrectly to the input the first time and modify its weights, settling only when the input presentation stops for a sufficient length of time. Where the input is presented indirectly to the neuron, such as in multi-layer networks with hidden layers, the timing becomes complicated as it is difficult to train indirectly connected neurons without untraining directly connected neurons. One strategy is to make the input presentation frequency proportional to the error in the final output. As the error diminishes, the input is presented less frequently, allowing the weights to settle. If the error increases, the input frequency is increased, causing the weights to continue to change until the new network function produces an output closer to the desired output. The boundaries on the input presentation frequency vary between zero and faster than the neuron can react. If the error is still unacceptable after reaching one of these limits, it may be necessary to reverse or gradually oscillate around the desired minimum error frequency to allow the network to settle. Energy Minimization By applying this learning rule, the network will naturally minimize the energy required to perform its function. If a neuron fires unnecessarily to neurons that have already been fired, the weights between them will be decreased. If a neuron fires unnecessarily to neurons that have not fired, they will learn to respond by firing and perhaps changing the inputs that are driving the first neuron to fire unnecessarily. The system will seek stable oscillations as chaotic patterns are likely to cause repetitious inputs which trigger the learning rule. As the learning rule is applied, the system weights change until the repetitious inputs cease, most probably when the network is stable. Pain and Pleasure This training algorithm can be said to put a neural network in "pain" when it is repetitiously presenting the inputs in order to train the network. The neural network will modify its weights until its function is sufficiently different to minimize the output error which is proportional to the input frequency, or the level of pain. The training algorithm's definition of pleasure can be read in the response to the following. David: "Annie, why do you keep punching your head with your fist?" Annie: "Because it feels so good when I stop!" THINGS TO ADD Reinforcement Learning -- doesn't give answer, just yes or no Inculcate, the meaning of the word Universal Computation -- can do any binary function? DATE08/24/93 TIME12:27 PM hwC^YvZoMNRS[\]^bcklrt} VX02  !#fhwy D F  AC^`z|2WYvxQSHJZ\oq!#jKM$$$[Mnprt$$B tB_#)}Mt2 CG Times Symbol "UniversTimes New Roman5Courier New&GreekMathSymbols &Aardvark&Arial &Bangkok&CasperOpenFace &DawnCastle&FrankfurtGothicHeavy&HomewardBound &Jupiter &Monospaced&MusicalSymbols &Mystical &Paradise &Penguin Southern&SwitzerlandInserat)   "hB3Յ&KمEe(Frustrated Synapse NN Training AlgorithmNeural Networks1Lt David W. Croft Your Name