Personal tools

Neural Networks Research and Applications

[Artificial Neural Network - SpringerLink]


- Overview

Artificial neural networks (ANNs) are a type of machine learning (ML) process that uses interconnected nodes to teach computers to process data like the human brain.

ANNs are made up of layers of interconnected nodes, each with a different role in data processing. The structure and name of ANNs is inspired by the human brain, mimicking how biological neurons signal to each other. 

In addition to the living world, in the field of ANNs in computer science, a neuron is a collection of inputs, a set of weights, and an activation function. It converts these inputs into a single output. Another layer of neurons selects this output as input, and so on. In essence, we can say that each neuron is a mathematical function that closely models the function of biological neurons.

ANNs are used to solve problems in artificial intelligence (AI). They model the connections of biological neurons as weights between nodes. A positive weight reflects an excitatory connection, while negative values mean inhibitory connections. All inputs are modified by a weight and summed. 

ANNs are made up of three layers:

  • Input layer: Takes data from the network
  • Hidden layer/s: Connections from the nodes in the input layer to the nodes in the hidden layer, and from each hidden layer node to the nodes of the output layer
  • Output layer: Connections from the nodes in the hidden layer to the nodes of the output layer


An example of an ANN is an "OR" gate, which takes two inputs. If one or both the inputs are "On," then the output will be "On". If both the inputs are "Off," then the output will be "Off".

The image above shows a simple feed forward neural network that propagates information forwards.


- Neurons in Neural Networks

The word "neural" was inspired by the word "neuron", and we all know what a neuron is. It helps humans process information and generate output through the brain. This neuron is interconnected with millions of other neurons. The same thing happens in AI. Neural networks also interconnect artificial neurons called "nodes". We also call it an artificial neural network (ANN). 

Neural networks are a type of ML process that teach computers to process data in a way that mimics the human brain. They are a type of deep learning (DL) that uses interconnected nodes, or neurons, in a layered structure.

Neural networks are a key element of DL and AI. What exactly is a neural network trying to do? Like any other model, it tries to make a good prediction. We have a set of inputs and a set of target values ​​- we're trying to get predictions that match those target values ​​as closely as possible. 

Neural networks solve various real-time tasks because of their ability to perform computations quickly and their fast responses. A neural network creates an adaptive system that computers can use to learn from their mistakes and continually improve. 

ANNs therefore attempt to solve complex problems, such as summarizing documents or identifying faces, with greater accuracy.


- The Function of Neural Networks

The design of neural networks is based on the structure of the human brain. Just as we use the brain to recognize patterns and classify different types of information, neural networks can be taught to perform the same tasks on data. 

The layers of a neural network can also be thought of as a kind of "coarse-to-fine filters", increasing the likelihood of detection and output of the correct result. The human brain works in a similar way. Whenever we receive new information, the brain tries to compare it with known objects. Deep neural networks also use the same concept. 

Neural networks allow us to perform many tasks such as clustering, classification or regression. Using neural networks, we can group or rank unlabeled data based on the similarity between samples in the data. Or in the case of classification, we can train the network on a labeled dataset in order to classify the samples in that dataset into different classes. 

ANNs have the unique ability to enable deep learning models to solve tasks that ML models could never solve.


- Artificial Neural Networks (ANNs)

An artificial neural network (ANN), often referred to simply as a neural network, is a computational system inspired by the fuzziness of the biological neural networks that make up animal brains. 

ANNs are based on collections of connected units or nodes called artificial neurons (blue nodes in the diagram above), which loosely model neurons in biological brains. Each connection, like a synapse in a biological brain, can transmit signals to other neurons. 

An artificial neuron (the blue node in the diagram above) receives the signal, processes it, and can send out signals to the neurons connected to it. The "signal" at the connection is a real number, and the output of each neuron is computed by some nonlinear function of the sum of its inputs. These connections are called edges. Neurons and edges typically have weights that are adjusted as learning progresses. The weight increases or decreases the signal strength at the connection. 

A neuron may have a threshold so that it only sends a signal when the aggregated signal exceeds that threshold. Typically, neurons are aggregated into layers. 

Different layers can perform different transformations on their inputs. The signal propagates from the first layer (input layer) to the last layer (output layer), possibly after traversing the layers many times.   

Moraine Lake_Canada_060222A
[Moraine Lake, Canada]

- Neural Networks and AI Research

Neural networks are an important area of ​​AI research and are currently proving valuable for more natural user interfaces through speech recognition and natural language processing, allowing humans to interact with machines in the same way they interact with each other. 

By design, neural networks mimic the biological functions of animal brains to interpret and respond to specific inputs, such as words and intonation. As the underlying technology continues to evolve, AI has the potential to enhance online learning, adaptive learning software, and simulations in ways that more intuitively respond to and engage with students. 

While neural networks (also known as "perceptrons") have been around since the 1940s, they haven't become a staple of AI until the last few decades. This is due to the advent of a technique called "backpropagation," which allows a network to adjust its hidden layers of neurons if the results don't match what the creators wanted -- such as those designed to recognize dogs network, it will misidentify, for example, a cat.

Another important advance has been the arrival of deep learning neural networks, where different layers of a multi-layered network extract different features until it can identify what it is looking for. 

The idea of ​​DL is: use brain simulations with the hope of: 

  • making learning algorithms better and easier to use. 
  • Revolutionary advances in machine ML and AI. 

This is our best chance to move towards true artificial intelligence.


- Why are Neural Networks Important?

Neural networks can help computers make informed decisions with limited human assistance. This is because they can learn and model non-linear and complex relationships between input and output data. For example, they can perform the following tasks.

Make generalizations and inferences. Neural networks can understand unstructured data and make general observations without explicit training. For example, they can recognize that two different input sentences have similar meanings:

  • Can you tell me how to pay?
  • How do I transfer money?


The neural network will know that the two sentences mean the same thing. Or it could broadly identify Baxter Road as a place, but Baxter Smith as a person's name.

Computer vision uses convolutional neural networks (CNNs) to process visual data at the pixel level and deep learning recurrent neural networks (RNNs) to understand the relationship between one pixel and another.

DL algorithms such as convolutional neural networks and recurrent neural networks are used in supervised, unsupervised and reinforcement learning tasks, based on the specific problem and availability of data.


- Some Examples of ANNs

ANNs can be trained using a training set. For example, if you want to teach an ANN to recognize a cat, you can show it thousands of different images of cats so that the network can learn to identify a cat.  

Here are some examples of ANNs: 

  • Feedforward neural networks (FNNs): The most basic type of ANN. In this network, data moves in one direction from the input nodes to the hidden nodes and then to the output layer. FNNs are mainly used for pattern recognition, classification, and regression tasks.
  • Convolutional neural networks (CNNs): A network architecture for deep learning algorithms. CNNs are used for image recognition and tasks that involve processing pixel data.
  • Deep neural networks (DLNs): Deal with training large neural networks with complex input output transformations. One example of DLNs is mapping a photo to the name of the person(s) in photo as they do on social networks.
[More to come ...]
Document Actions