Artificial intelligence and neural networks are incredibly exciting and powerful machine learning methods that are used to solve many real-world problems. The simplest example of a neural network is the study of punctuation and grammar to automatically create a completely new text with all the spelling rules.
Neural network history
Scientists in the field of computerization have long been trying to simulate the human brain. In 1943, Warren S. McCallough and Walter Pitts developed the first conceptual model of an artificial neural network. In the article “The logical calculus of ideas related to nervous activity,” they described an example of a neural network, the concept of a neuron - the only cell living in a common network that receives input, processes it and generates output signals.
Their work, like many other scientists, was not intended to accurately describe the work of the biological brain. An artificial neural network was developed as a computational model operating on the principle of the functioning of the brain to solve a wide range of problems.
Obviously, there are exercises that are easy to solve for a computer, but difficult for a person, for example, extracting the square root of a ten-digit number. This example will calculate a neural network in less than a millisecond, and it will take a person minutes. On the other hand, there are those that are incredibly simple for a person to solve, but not within the power of a computer, for example, to choose the background of an image.
Scientists spent a lot of time researching and implementing complex solutions. The most common example of a neural network in computing is pattern recognition. The range of applications ranges from optical character recognition and photo recognition , printed or handwritten scans into digital text, to face recognition.
Biological Computing Machines
The human brain is the most complex and most powerful of the known computers. His inner work is modeled around the concept of neurons and their networks, known as biological neural networks. The brain contains about 100 billion neurons that are connected by these networks.
At a high level, they interact with each other through an interface consisting of axon terminals connected to dendrites through the gap - synapse. In simple terms, one transmits a message to another through this interface if the sum of the weighted input signals from one or more neurons exceeds a threshold to trigger transmission. This is called activation when the threshold is exceeded and a message is sent to the next neuron.
The summation process can be mathematically complex. The input signal is a weighted combination of such signals, and weighting each means that this input can have a different effect on subsequent calculations and on the final output of the network.
Elements of the neural model
Deep learning is the term used for complex neural networks consisting of several layers. Layers are made up of nodes. A node is just a place where a calculation takes place that works when it encounters enough stimuli. A node combines input from a set of coefficients or weights that either amplify or attenuate this signal, thereby assigning them significance for the task.
Deep learning networks differ from common neural networks with one hidden layer. An example of training neural networks is the Kohonen network.
In deep learning networks, each layer learns a given set of functions based on the output of the previous level. The further you advance into the neural network, the more difficult are the objects that can be recognized by the nodes, since they combine and recombine objects from the previous level.
Deep learning networks perform automatic extraction of functions without human intervention, unlike most traditional algorithms, and end with an output level: a logical or softmax classifier, which assigns probability to a specific result and is called a forecast.
Black box ANN
Artificial neural networks (ANNs) are statistical models partially modeled on biological neural networks. They are capable of handling non-linear relationships between inputs and outputs in parallel. Such models are characterized by the presence of adaptive weights along the paths between neurons, which can be tuned by the learning algorithm to improve the entire model.
A simple example of a neural network is an architecturally artificial neural network ANN, where:
- Input layer
- Hidden layer - a hidden layer.
- Output layer
It is modeled using layers of artificial neurons or computational units capable of receiving input data and using the activation function along with a threshold value to determine if messages are being transmitted.
In a simple model, the first layer is the input layer, followed by the hidden layer and finally the output layer. Each may contain one or more neurons. Models can become increasingly complex with increasing abstraction and problem solving capabilities, the number of hidden layers, the number of neurons in any given layer, and the number of paths between them.
Model architecture and tuning are the main components of ANN methods in addition to the learning algorithms themselves. They are extremely powerful and are considered black box algorithms, which means that their internal work is very difficult to understand and explain.
Deep Learning Algorithms
Deep learning - this concept sounds loud enough, in fact it is just a term describing certain types of neural networks and related algorithms that consume raw input data through many layers of non-linear transformations in order to calculate the target output.
Maintenance-free feature extraction is also an area in which deep learning exceeds all expectations. An example of training neural networks is the SKIL network.
Traditionally, it is the responsibility of the data scientist or data programmer to carry out the feature extraction process in most other machine learning approaches, along with feature selection and design.
Optimal algorithm parameters
Function learning algorithms authorize the machine to know a specific task using a sophisticated set of learning opportunities. In other words, they learn to learn. This principle has been successfully used in many applications and is considered one of the most advanced methods of artificial intelligence. Appropriate algorithms are often used for controlled, uncontrolled, and partially controlled tasks.
In models based on a neural network, the number of layers is greater than in surface learning algorithms. Small algorithms are less complex and require a deeper knowledge of optimal functions, which include selection and development. In contrast, deep learning algorithms rely more on the optimal model selection and its optimization by tuning. They are better suited for solving problems when a preliminary knowledge of the functions is less desirable or necessary, and the recorded data is not available or is not required for use.
Input data is converted in all their layers using artificial neurons or processor units. An example of neural network code is called CAP.
CAP value
CAP is used to measure the architecture of a deep learning model. Most researchers in this area agree that it has more than two non-linear layers for CAPs, and some believe that CAPs with more than ten layers require too much training.
A detailed discussion of many different model architectures and algorithms of this kind of training is very spatial and controversial. The most studied are:
- Direct neural networks.
- Recurrent neural network.
- Multilayer perceptrons (MLP).
- Convolutional neural networks.
- Recursive neural networks.
- Deep networks of beliefs.
- Convolutional networks of deep beliefs.
- Self-organizing cards.
- Deep Boltzmann cars.
- Folded noise canceling auto encoders.
Top contemporary architectures
Perceptrons are considered first-generation neural networks, computational models of one neuron. They were invented in 1956 by Frank Rosenblatt in Perceptron: A Prospective Model for Storing and Organizing Information in the Brain. A perceptron, also called a direct communications network, transmits information from the front to the back.
RNN recurrent neural networks turn an input sequence into an output sequence that is in a different area, for example, it changes the sequence of sound pressures into a sequence of word identifiers.
John Hopfield introduced Hopfield Net in a 1982 paper, Neural Networks and Physical Systems with Emerging Collective Computing Capabilities. In the Hopfield Network (HN), each neuron is connected to any other. They are trained by setting their values to the desired circuit, after which weights can be calculated.
The Boltzmann machine is a type of stochastic recurrent neural network that can be considered as an analog of Hopfield networks. This was one of the first options that studied internal representations that solved complex combinatorial problems. Input neurons become output at the end of a full update.
Jan Goodfellow's Generative Adversarial Network (GAN) consists of two networks. This is often a combination of Feed Forwards and Convolutional Neural Nets. One generates generative content, and the other should evaluate discriminatory content.
Getting started with SKIL from Python
Deep neural network training using the Python example compares inputs with outputs and finds correlations. It is known as a universal approximator, because it can learn to approximate an unknown function f (x) = y between any input “x” and any output “y”, assuming that they are related by correlation or cause-effect.
In the process of learning, there is the correct “f” or a way to turn “x” into “y”, be it f (x) = 3x + 12 or f (x) = 9x - 0.1.
Classification tasks relate to datasets so that neural networks correlate between labels and data. Known controlled training of the following types:
- face recognition;
- identification of people in images;
- definition of facial expression: angry, joyful;
- identification of objects in the images: stop signs, pedestrians, lane indicators;
- gesture recognition in the video;
- determination of the voice of speakers;
- spam text classification.
Example convolutional neural network
The convolutional neural network is similar to the multilayer network of the perceptron. The main difference is that CNN studies how it is structured and for what purpose it is used. CNN's inspiration was biological processes. Their structure has the appearance of the visual cortex present in the animal. They are used in the field of computer vision and are successful in achieving a modern level of performance in various fields of research.
Before they start coding CNN, they use a library, for example, Keras with a Tensorflow backend, to build a model. First, perform the necessary import. The library helps build a convolutional neural network. Download the mnist dataset via keras. A keras sequential model is imported into which you can add convolution and union layers, dense layers, as they are used to predict labels. The drop-down layer reduces the conversion, and the leveling layer converts the three-dimensional vector into one-dimensional. Finally, import numpy for matrix operations:
- Y = 2 # value 2 represents that the image has the number 2;
- Y = [0,0,1,0,0,0,0,0,0,0,0,0] # 3rd position in the vector is 1;
- # Here the class value is converted to a binary class matrix.
Construction Algorithm:
- Ultraprecise layers and a maximum pool are added to the serial model.
- Add dropdown layers between them. A drop-down randomly disconnects some neurons in the network, which forces the data to find new paths and reduces re-equipment.
- Dense layers are added, which are used to predict the class (0–9).
- Compile a model with categorical cross-entropy loss function, Adadelta optimizer and accuracy metric.
- After training, they evaluate the loss and accuracy of the model according to the test data and print it.
Modeling in Matlab
Let us present a simple example of neural networks of Matlab modeling.
Assuming "a" the model has three inputs "a", "b" and "c" and generates the output "y".
For data generation purposes: y = 5a + bc + 7s.
First, they write a small script to generate data:
- a = Rand (11000);
- b = Rand (1,1000);
- c = Rand (1.1000);
- n = Rand (1.1000) * 0.05;
- y = a * 5 + b * s + 7 * s + n,
where n is the noise specially added to make it look like real data. The noise value is 0.1 and is uniform.
Thus, the input is a set of "a", "b" and "c", and the output is:
Next, use the matlab newff built-in function to generate the model.
Examples of tasks of neural networks
First create a 3 * 2 matrix R. The first column will show a minimum of all three inputs, and the second column will show a maximum of three inputs. In this case, the three inputs are in the range from 0 to 1, therefore:
R = [0 1; 0 1; 0 1].
Now create a size matrix that has the v-size of all layers: S = [51].
Now call the newff function as follows:
net = newff ([0 1; 0 1; 0 1], S, {'tansig', 'purelin'}).
The neural model {'tansig', 'purelin'} shows the function of displaying two layers.
Teach her with the data that was created earlier: net = train (net, I, O).
The network is trained, you can see the performance curve as it learns.
Now they model it again on the same data and compare the output:
O1 = sim (net, I);
plot (1: 1000, O, 1: 1000, O1).
Thus, the input matrix will be:
- net.IW {1}
- -0.3684 0.0308 -0.5402
- 0.4640 0.2340 0.5875
- 1.9569 -1.6887 1.5403
- 1.1138 1.0841 0.2439
- net.LW {2,1}
- -11.1990 9.4589 -1.0006 -0.9138
Artificial intelligence applications
Examples of neural network implementations include online self-service solutions and the creation of reliable workflows. There are deep learning models used for chatbots, and as they continue to evolve, it can be expected that this area will be used more for a wide range of enterprises.
Areas of use:
- Automatic machine translation. It is not new, deep learning helps to improve automatic translation of text using folded networks and allows you to translate images.
- A simple example of the use of neural networks is adding color to black and white images and videos. It can be automatically done using in-depth study models.
- Machines learn the punctuation, grammar and style of a piece of text and can use the model they developed to automatically create completely new text with proper spelling, grammar, and text style.
Artificial neural networks ANN and a more sophisticated deep learning technique are some of the most advanced tools for solving complex problems. Although an application boom is unlikely in the near future, advances in technology and artificial intelligence applications will certainly be exciting.
Despite the fact that deductive reasoning, logical conclusions and decision-making using a computer today are still very far from perfect, significant success has already been achieved in the application of artificial intelligence methods and related algorithms.