JanB1 wrote: ↑Mon Sep 11, 2017 11:39 pm
Gotta be honest, the exact way neural networks work (and I don't mean how they learn and then use the learned stuff, but how they ACTUALLY work) and how evolving algorithms work is a total mystery to me. Would be glad if someone could enlighten me. Cuz it bugs me, that I don't know this.
ANNs [artificial neural networks] come in about a hundred kinds, but the main kinds about which I know are feedforward, recurrent, and convolutional.
Feedforward networks are one of, if not the, simplest kinds of ANN, and serve as a fine tool to grasp the basics of the rest.
A feedforward network consists of vertices (or neurons) and edges (or synapses). The neurons are arranged into layers. The first of these layers is the input layer, which neurons' values are driven by stimuli. The last is the output layer, which neurons' values are taken as outputs to the problem being solved. All other layers are known as hidden layers, called such because a black-box interpretation of an ANN does not account for them. Neurons in hidden layers and the output layers are driven by the neurons in the previous layer through the synapses. The function which relates the neurons of the previous layer and they synaptic weights to the neuron in question is based on the sigmoid of the sum of the values of the previous layer, each weighted by the relevant synapse's weight. The sigmoid function is an approximately s-shaped function which serves to normalize the sum between either 0 and 1 or -1 and 1, depending on the application. This curve has the interesting property of having a higher slope nearer the origin, which increases sensitivity near middling values and decreases it at extremes. The network learns by changing the synaptic weights. The network as a whole forms a "combinatorial" system; that is, the output is mapped to the input without respect to time or prior values. Because of this, feedforward networks have no capacity for memory except in the genome itself. A feedforward network could be compared to the proportional component of a PID controller, entirely lacking both memory and understanding of change.
Recurrent neural networks are a simple evolution of feedforward networks. In an RNN, one of the input synapses for each neuron comes from itself in the previous simulation frame. This introduces a temporal component, which could develop into memory or change-detection. RNNs are no longer combinatorial, and able to simulate the integral and derivative components of a PID controller as well as the proportional component.
Convolutional neural networks iteratively examine a set of data, usually arranged like a multidimensional array. This makes them particularly well-suited to, for example, image processing, and thus they have been instrumental in much of the recent advantages in computer vision and similar fields. They are arranged largely like the simple feedforward or recurrent networks, save for a steadily decreasing layer size as the network goes deeper to "distill" the data into usable outputs. The inputs of the network are moved across the array (or image), and gain information from a rolling patch of data.
I'm a little unclear on CNNs myself, so forgive me if that section is… lacking.