Artificial Neuron

- Overview

An artificial neuron is a mathematical function that models a biological neuron in a neural network. Artificial neurons are the basic units of artificial neural networks (ANNs), which are software programs that simulate how the human brain processes information:

An artificial neuron is a connection point in an ANN. ANNs, like biological neural networks in the human body, have a layered architecture where each network node (connection point) has the ability to process input and forward output to other nodes in the network. 

An artificial neuron takes inputs, applies weights to them, and sums them to produce an output. 

Please refer to the following for more details:

• Wikipedia: Activation Function

- Artificial Neural Networks and Artificial Neurons

Neural networks are sometimes called artificial neural networks (ANNs) or simulated neural networks (SNNs). They are a subset of machine learning (ML), and at the heart of deep learning (DL) models. ANNs are a type of ML process that uses interconnected nodes to teach computers to process data like the human brain.

ANNs are made up of layers of interconnected nodes, each with a different role in data processing. The structure and name of ANNs is inspired by the human brain, mimicking how biological neurons signal to each other. 

In addition to the living world, in the field of ANNs in computer science, a neuron is a collection of inputs, a set of weights, and an activation function. It converts these inputs into a single output. Another layer of neurons selects this output as input, and so on. In essence, we can say that each neuron is a mathematical function that closely models the function of biological neurons.

ANNs are used to solve problems in artificial intelligence (AI). They model the connections of biological neurons as weights between nodes. A positive weight reflects an excitatory connection, while negative values mean inhibitory connections. All inputs are modified by a weight and summed. 

- Deep Learning and Artificial Neurons

In artificial and biological architectures, nodes are called neurons, and connections are characterized by synaptic weights, which represent the importance of the connection. As new data is received and processed, synaptic weights change, and this is how learning occurs. 

Artificial neurons are modeled after the hierarchical arrangement of neurons in a biological sensory system. For example, in the visual system, light input passes through neurons in successive layers of the retina, then to neurons in the thalamus of the brain, and then to neurons in the visual cortex of the brain. 

ANNs work by passing information through multiple layers of interconnected neurons. The neurons receive input, apply an activation function, and use a threshold to determine if messages are passed along. The network learns from mistakes and improves continuously.

As neurons pass signals through more and more layers, the brain gradually extracts more information until it is confident that it can recognize what a person is seeing. In artificial intelligence AI), this fine-tuning process is called deep learning. 

Deep learning models can recognize patterns in data like text, images, and sounds to make predictions and produce insights. For example, some law enforcement agencies use deep learning to detect crimes by matching faces against digital images.

[Neuron and Synapse - GeeksforGeeks]

- Artificial And Biological Neuron

We have heard of the latest advancements in the field of DL due to the usage of different neural networks. At the most basic level, all such neural networks are made up of artificial neurons that try to mimic the working of biological neurons. Understanding how these artificial neurons compare to the structure of biological neurons in our brains and if possibly this could lead to a way to improve neural networks further. 

An Artificial Neural Network (ANN) is an information processing paradigm that is inspired the brain. ANNs, like people, learn by example. An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process. Learning largely involves adjustments to the synaptic connections that exist between the neurons.  

In artificial and biological networks, as neurons process the input they receive, they decide whether or not the output should be passed as input to the next layer. The decision of whether to send information is called bias, and it is determined by an activation function built into the system. 

For example, the artificial neuron may only pass the output signal to the next layer if the sum of its inputs (actually voltages) exceeds a certain threshold. Because activation functions can be linear or non-linear, neurons will typically have a wide range of convergence and divergence. Divergence is the ability of one neuron to communicate with many other neurons in the network while convergence is the ability of one neuron to receive input from many other neurons in the network.  

- Artificial Neuron (Perception)

A neuron is a mathematical function that is used as a model of a biological neuron in a neural network. The mathematical representation of a neuron is called an artificial neuron, or perceptron, and is made up of three components:

• Inputs: Also represented as x1, x2, ..., xn, these are the input values that the neuron receives. These inputs can represent features, measurements, or outputs from other neurons in the network.
• Weights: Also represented as w1, w2, ..., wn, these represent the strength or importance of each input in influencing the neuron's output. Weights can be positive, negative, or zero.
• Activation function: This is the non-linear function that takes the value from the inputs and becomes the neuron's output.

The artificial neuron receives one or more inputs, applies weights to these inputs, and sums them to produce an output. The value is then passed to the activation function to become the neuron's output. 

A neural network is also a mathematical function. The simplest case is a single input node, a weight, and an output node. When multiple layers are added, the neural network becomes a composition of functions as the signal passes from layer to layer. 

An artificial neuron (also referred to as a perceptron) is a mathematical function. It takes one or more inputs that are multiplied by values called “weights” and added together.

- Neural Network Activation Functions

The activation function of a node in an ANN is a function that calculates the output of the node based on its individual inputs and their weights. Neural networks can represent a wide variety of functions with appropriate weights. 

Activation functions can be linear and non-linear, although the most useful ones are non-linear. Non-linear activation functions play a vital role in neural networks and other deep and algorithmic learning models.

Nontrivial problems can be solved using only a few nodes if the activation function is nonlinear.

- Artificial Neuron Example

An artificial neuron is a mathematical function conceived as a model of biological neurons, a neural network. Artificial neurons are elementary units in an artificial neural network. Neural networks are multi-layer networks of neurons (the blue and magenta nodes in the chart above) that we use to classify things, make predictions, etc. 

Above is the diagram of a simple neural network with five inputs, 5 outputs, and two hidden layers of neurons. Starting from the left, we have:

• The input layer of our model in orange.
• Our first hidden layer of neurons in blue.
• Our second hidden layer of neurons in magenta.
• The output layer (a.k.a. the prediction) of our model in green.

The arrows that connect the dots shows how all the neurons are interconnected and how data travels from the input layer all the way through to the output layer. 

Later we will calculate step by step each output value. We will also watch how the neural network learns from its mistake using a process known as backpropagation.