[ver] 4 [sty] ~default.sty [files] [charset] 82 ANSI (Windows, IBM CP 1252) [revisions] 0 [prn] HP DeskJet 550C Printer [lang] 1 [desc] 751441545 6 751105225 154 4 0 0 0 0 0 [fopts] 0 1 0 0 [lnopts] 2 Body Text 1 [docopts] 5 2 [tag] Body Text 2 [fnt] Courier 200 0 32768 [algn] 1 1 0 0 0 [spc] 33 273 1 0 0 1 100 [brk] 4 [line] 8 0 1 0 1 1 1 10 10 1 [spec] 0 0 0 1 1 0 0 0 0 [nfmt] 280 1 2 . , $ Body Text 0 0 [l1] 0 [pg] 4 44 0 84 0 0 0 0 65535 2 Standard 65535 0 0 0 0 0 0 0 0 0 65535 0 0 65535 0 0 0 0 86 0 21 0 0 0 0 65535 2 Standard 65535 0 0 0 0 0 0 0 0 0 65535 0 0 65535 0 0 0 0 114 100 45 32 0 0 0 65535 2 Standard 65535 0 0 0 0 0 0 0 0 0 65535 0 0 65535 0 0 0 0 146 0 0 1025 0 0 0 65535 65535 Standard 65535 0 0 0 0 0 0 0 0 0 65535 0 0 65535 0 0 0 0 [edoc] <+B><:s><:#202,9360><+!>Frustrated Inculcated Synapse Training<-!> <+B><:s><:#202,9360> <+B><:s><:#202,9360>A Neural Network Training Algorithm <+B><:s><:#202,9360> <+B><:s><:#202,9360> <+B><:s><:#202,9360>David W. Croft <+B><:s><:#202,9360> <+B><:s><:#202,9360> <+B><:#202,9360>1993 Oct. 24 <:s><:#202,9360> <:#202,9360><:f200,QCourier,255,0,0>Red: new material<:f> <:#202,9360><:f200,QCourier,0,0,255>Blue: discarded material<:f> <:s><:#202,9360> <:s><:#202,9360><+!>Objectives<-!> <:#202,9360><:f200,QCourier,0,0,0> <+@><:#1010,9360><:f200,QCourier,0,0,0>Our primary objective is to build a "black box" that will learn to duplicate any desired function. It must configure itself based solely on the error between its produced and desired output. This will be done using a neural network t raining algorithm that I call Frustrated Inculcated Synapse Training (FIST). <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><+!><:f200,QCourier,0,0,0>Theory<-!> <:#202,9360><:f200,QCourier,0,0,0> <:#404,9360><:f200,QCourier,0,0,0>A neuron is a device with inputs and an output whose function is determined by a simple "activation rule". <:#202,9360><:f200,QCourier,0,0,0> <:#404,9360><:f200,QCourier,0,0,0>An artificial neural network (ANN) is a network of neurons whose interconnections of inputs and outputs are weighted. <:#202,9360><:f200,QCourier,0,0,0> <:#606,9360><:f200,QCourier,0,0,0>An artificial neural network (ANN) has the structure to duplicate any function out of the infinite set of possible functions by choosing the appropriate set of interconnecting weights. <:#202,9360><:f200,QCourier,0,0,0> <:#606,9360><:f200,QCourier,0,0,0>Choosing the right set of weights to make your ANN duplicate your desired function is very hard because there are an infinite number of possible sets of weights. <:#202,9360><:f200,QCourier,0,0,0> <:#606,9360><:f200,QCourier,0,0,0>A neural network training algorithm is method of choosing weights for an ANN. If the algorithm is good, it will cause the weights to converge to the correct or nearly-correct values in a reasonable amount of time. <:#202,9360><:f200,QCourier,0,0,0> <:#404,9360><:f200,QCourier,0,0,0>The "neuron state" describes the state of an individual neuron: ready, firing, and tired. <:#202,9360><:f200,QCourier,0,0,0> <:#606,9360><:f200,QCourier,0,0,0>The "network state" describes the present state of all the neuron states in the network. In an ANN of N 3-state neurons, there are 3**N possible network states. <:#202,9360><:f200,QCourier,0,0,0> <:#404,9360><:f200,QCourier,0,0,0>The "output state" describes the pattern of the network as it changes from one network state to another as it stabilizes. <:#202,9360><:f200,QCourier,0,0,0> <:#404,9360><:f200,QCourier,0,0,0>The "synaptic learning mode" describes a synapses propensity to learn at a given time: primed, normal, or frustrated. <:#202,9360><:f200,QCourier,0,0,0> <:#606,9360><:f200,QCourier,0,0,0>A "Neural Optimization Box" (NOB) is a specialized ANN that uses the Frustrated Inculcated Synapse Training (FIST) neural network training algorithm to enable it to duplicate any desired function. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>"Supervised learning" trains the neurons given the desired neuron states. <:#202,9360><:f200,QCourier,0,0,0> <:#404,9360><:f200,QCourier,0,0,0>"Reinforcement learning" trains the neurons given only information as to whether the output was correct or not. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>Updating can be either aynchronous, synchronous, or continuous. <:#202,9360><:f200,QCourier,0,0,0>A "tetanus" is a short burst of high frequency excitatory inputs. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><+!><:f200,QCourier,0,0,0>Learning Rule<-!> <:#202,9360><:f200,QCourier,0,0,0> <:#1010,9360><:f200,QCourier,0,0,0>Using FIST, a neuron learns by assuming that if it is presented with the same input in quick succession, it failed to respond appropriately to the input the first time. It will then modify its weights in such a way that it will respon d differently when the input is presented again -- not firing if it fired earlier or firing if it failed to fire. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><+!><:f200,QCourier,0,0,0>Neuron States<-!> <:#202,9360><:f200,QCourier,0,0,0> <:#404,9360><:f200,QCourier,0,0,0>This learning rule can be implemented by assuming that a neuron has three states: ready, firing, and tired. <:#202,9360><:f200,QCourier,0,0,0> <:#1010,9360><:f200,QCourier,0,0,0>A neuron begins in the ready state awaiting inputs. If it receives excitatory inputs that are too weak to cause it to fire or are overwhelmed by additional inhibitory inputs, it remains in the ready state. If it receives a strong net excitatory input, it will immediately transition from the ready state to the firing state. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>If the neuron is firing, it will transition to the tired state. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>If the neuron is tired, it will transition to the ready state. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>Current Neuron State Inputs Next Neuron State <:#202,9360><:f200,QCourier,0,0,0>-------------------- ------ ----------------- <:f200,QCourier,0,0,0>Ready ( 0) <<= 0 Ready ( 0) <:f200,QCourier,0,0,0>Ready ( 0) <;> 0 Firing (+1) <:f200,QCourier,0,0,0>Firing (+1) Any Tired (-1) <:f200,QCourier,0,0,0>Tired (-1) Any Ready ( 0) <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>Neuron_State := Neuron_State + Time_Delta * f ( Inputs, Neuron_State ) <:#202,9360><:f200,QCourier,0,0,0>(working on continuous differentiable equation) <:#202,9360><:f200,QCourier,0,0,0> <:#1616,9360><:f200,QCourier,0,0,0>Learning occurs when an excitatory input arrives and the neuron is either tired or primed. If the neuron is tired and one of the synaptic weights attempts to excite it, the weight is weakened as it is assumed that the weight is attemp ting to fire the neuron repetitiously after unsuccessfully achieving the desired output the first time. If the neuron is primed and one of the synaptic weights attempts to excite it, the weight is strengthened as it is assumed that the weight is attempting to fire neuron after being unsuccessful in doing so the first time. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><+!><:f200,QCourier,0,0,0>Training for Known States<-!> <:#202,9360><:f200,QCourier,0,0,0> <:#1010,9360><:f200,QCourier,0,0,0>If the desired states of the individual neurons in response to a given input are known, training is a simple three-step process: predict the results of the application of the activation rule for the given inputs, compare the predicted activated responses to the desired responses and change the current neurons states if necessary, then apply the learning rule. <:#202,9360><:f200,QCourier,0,0,0> <:#606,9360><:f200,QCourier,0,0,0>If the activation rule dictated that the neuron should transition to a state that was not the desired response state, the learning rule will modify the weights correctly. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>Only those synapses which are triggered are modified. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>Post-Synaptic Neuron State Learning Mode <:f200,QCourier,0,0,0>-------------------------- ---------------- <:f200,QCourier,0,0,0>Firing (+1) Primed (+1) <:f200,QCourier,0,0,0>Ready ( 0) Normal (+0) <:f200,QCourier,0,0,0>Tired (-1) Frustrated (-1) <:#202,9360><:f200,QCourier,0,0,0> <:f200,QCourier,0,0,0>Weight_New := Weight_Old + Time_Delta * ( Neuron_State ) <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><+!><:f200,QCourier,0,0,0>Training for Unknown States<-!> <:#202,9360><:f200,QCourier,0,0,0> <:#1616,9360><:f200,QCourier,0,0,0>Training is more difficult if only the desired output is known and the desired states for the neurons between the input and the output are not. As it is unknown what intermediate states will create the desired output, the individual n eurons must be trained based upon a comparison of the output results to the desired outputs which indicates only whether the error of the whole network was either small or large. The training algorithm must perform without directly modifying any of the int ermediate neuron states with only the network output error as a guide. <:#202,9360><:f200,QCourier,0,0,0> <:#1414,9360><:f200,QCourier,0,0,0>The training algorithm depends upon the timing of the input presentation. If a neuron sees an input presented repeatedly, it will assume that it responded incorrectly to the input the first time and modify its weights, settling only w hen the input presentation stops for a sufficient length of time. Where the input is presented indirectly to the neuron, such as in multi-layer networks with hidden layers, the timing becomes complicated as it is difficult to train indirectly connected neu rons without untraining directly connected neurons. <:#1010,9360><:f200,QCourier,0,0,0>One strategy is to make the input presentation frequency proportional to the error in the final output. As the error diminishes, the input is presented less frequently, allowing the weights to settle. If the error increases, the inpu t frequency is increased, causing the weights to continue to change until the new network function produces an output closer to the desired output. <:#1010,9360><:f200,QCourier,0,0,0>The boundaries on the input presentation frequency vary between zero and faster than the neuron can react. If the error is still unacceptable after reaching one of these limits, it may be necessary to reverse or gradually oscillate ar ound the desired minimum error frequency to allow the network to settle. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><+!><:f200,QCourier,0,0,0>Energy Minimization<-!> <:#202,9360><:f200,QCourier,0,0,0> <:#1212,9360><:f200,QCourier,0,0,0>By applying this learning rule, the network will naturally minimize the energy required to perform its function. If a neuron fires unnecessarily to neurons that have already been fired, the weights between them will be decreased. If a neuron fires unnecessarily to neurons that have not fired, they will learn to respond by firing and perhaps changing the inputs that are driving the first neuron to fire unnecessarily. <:#202,9360><:f200,QCourier,0,0,0> <:#808,9360><:f200,QCourier,0,0,0>The system will seek stable oscillations as chaotic patterns are likely to cause repetitious inputs which trigger the learning rule. As the learning rule is applied, the system weights change until the repetitious inputs cease, most p robably when the network is stable. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><+!><:f200,QCourier,0,0,0>Pain and Pleasure<-!> <:#202,9360><:f200,QCourier,0,0,0> <:#1010,9360><:f200,QCourier,0,0,0>This training algorithm can be said to put a neural network in "pain" when it is repetitiously presenting the inputs in order to train the network. The neural network will modify its weights until its function is sufficiently differen t to minimize the output error which is proportional to the input frequency, or the level of pain. <:#202,9360><:f200,QCourier,0,0,0> <:#404,9360><:f200,QCourier,0,0,0>The training algorithm's definition of pleasure can be read in the response to the following. <:#202,9360><:f200,QCourier,0,0,0>David: "Ann, why do you keep punching your head with your fist?" <:#202,9360><:f200,QCourier,0,0,0>Ann: "Because it feels so good when I stop!" <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><+!><:f200,QCourier,0,0,0>Things to Add or Change <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>What happens when neurons fire faster than desired (plays pattern too fast)? <:#404,9360><:f200,QCourier,0,0,0>Increase precision to slow down? Increase neurons to create delay loops which build over time before firing desired output? <:#404,9360><:f200,QCourier,0,0,0>Show how you can trade off the precision of the inputs by increasing the number of neurons. <:#202,9360><:f200,QCourier,0,0,0> <:#808,9360><:f200,QCourier,0,0,0>Hypothesis: synapses learn best when there is a high correlation between their firing frequencies; may occur at periodic multiples; learn best when phase differences are low. Synapses become weaker (depressed) when continuously fired in phase with the receiving neuron's tired state. <:#202,9360><:f200,QCourier,0,0,0>How is this related to Klopf's work? <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>Optimizer: settles into a small loop of network states; becomes stable <:#202,9360><:f200,QCourier,0,0,0>Sequencer: plays a large loop of network states (music box, movie player) <:#202,9360><:f200,QCourier,0,0,0>Are both content-addressable memories? <:#202,9360><:f200,QCourier,0,0,0>Normal stability rules may not apply to pulse neurons. <:#202,9360><:f200,QCourier,0,0,0>Create a sequencer that plays a sinusoidal. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>Random systems are just chaotic systems with unknown variables. <:#606,9360><:f200,QCourier,0,0,0>If the neural network doesn't have access to all of the variables as inputs it will infer those inputs, create a model of those inputs, or treat them as random variables. <:#202,9360><:f200,QCourier,0,0,0> <:#1212,9360><:f200,QCourier,0,0,0>Supervised training: play neurons with variables clamped at their historical values then let them go for future predictions. Like dreams where the mind infers its model of the dream world from its experiences in the real world and th en computes probabilistic events. Stock market lab: clamp the neurons to their historical values and then release the neurons at the present to see what it will predict for the future. <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>Convert to continuous model. <:#202,9360><:f200,QCourier,0,0,0> <:#404,9360><:f200,QCourier,0,0,0>Test with software that will do generic pattern matching to evaluate the theory and measure the performance <:#202,9360><:f200,QCourier,0,0,0> <:#404,9360><:f200,QCourier,0,0,0>Find mathematical limitations: stability constraints, number of neurons per memory, training time, Universal Computation (any binary function) <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>Compare and contrast to the approaches of Hopfield and Klopf <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>Show biological plausibility <:#202,9360><:f200,QCourier,0,0,0> <:#606,9360><:f200,QCourier,0,0,0>Music is a pattern of sounds as opposed to noise which is patternless or random. Note the human mind's affinity toward learning patterns and relate to stability. <:#202,9360><:f200,QCourier,0,0,0> <:#606,9360><:f200,QCourier,0,0,0>Music Box: collections of neurons represent different frequencies, weights dictate timing and sequence; use for audile observations of instability, error, and training strategies <:#202,9360><:f200,QCourier,0,0,0> <:#202,9360><:f200,QCourier,0,0,0>Inculcate: define the meaning of the word<:f> <:#202,9360> > [Embedded] 00016755