Foundations of DL

- Overview

Deep learning (DL) is a type of machine learning (ML) research that uses artificial neural networks (ANNs) to conduct automated data analysis. It's a subset of machine learning (ML) that's based on representation learning and artificial neural networks (ANNs). 

Deep learning is a collection of layers, with the first layer being the input layer, hidden layers in between, and the final layer generating output. Each layer is made up of a group of units called neurons.

Deep learning models can identify complex patterns in text, pictures, sounds, and other data to create accurate predictions and insights.

DL is used in many fields, including: 

• Computer vision
• Speech recognition
• Natural language processing
• Machine translation
• Bioinformatics
• Drug design
• Medical image analysis
• Climate science
• Material inspection
• Board game programs

DL techniques can capture complex relations between non-related fields. For example, they can capture the relationship between air pressure recordings and English words, or between millions of pixels and a textual description. 

DL is a combination of neural networks, AI, graphical modeling, optimization, pattern recognition, and signal processing.

ML is about computers being able to think and act with less human intervention. DL is about computers learning to think using structures modeled on the human brain. 

- DL Algorithms

Deep learning (DL) is a subset of ML. It's a type of AI that uses neural networks to learn from large amounts of data. 

Neural networks are made up of interconnected nodes, or neurons, that are layered to resemble the human brain. They mimic how neurons in the brain signal each other, which is why they're called "neural". 

DL models can be taught to perform classification tasks and recognize patterns in text, photos, audio, and other data. 

Here are some DL algorithms: 

• Convolutional neural network: Uses filters to learn the features of an image, such as important objects.
• Recurrent neural network: Uses a sequential approach and performs mathematical calculations in a sequence.
• Generative adversarial network (GAN): A class of algorithms that consists of two adversarial networks. One network generates realizations, and the other tries to differentiate real from simulated data.
• Autoencoder: A three-layer neural network that tries to reconstruct the input with minimal error.
• Multilayer perceptron (MLP): A deep learning method that helps with complex computations and increases the prediction accuracy of the training model.
• Decision tree: Uses machine and deep learning to automate complex business processes.
• k-Nearest Neighbor (kNN) classification algorithm: A simple classification algorithm that uses deep learning to classify by measuring the distance between different feature values.
• Logistic regression: Used as a classifier in the final layer of a deep learning. It is fast and simple, so it is used for large datasets.
• Cluster analysis: A clustering algorithm that divides data based on similarities. The grouped data are similar to each other more than the other data in other groups.

- Deep Learning Artificial Neural Networks

Deep learning is essentially a specialized subset of machine learning characterized by the use of three or more layers of neural networks. These neural networks attempt to simulate the behavior of the human brain (albeit far from its capabilities) in order to "learn" from large amounts of data.

Deep learning artificial neural networks (ANNs), also known as deep neural networks (DNNs), are a type of machine learning and artificial intelligence (AI) that mimic the human brain to recognize, classify, and describe objects. 

They are made up of multiple layers of interconnected nodes, or artificial neurons, that are linked together by weights. The weights are positive if one node excites another, and negative if one node suppresses it.

An artificial neural network (ANN) is a computational system inspired by the fuzziness of the biological neural networks that make up animal brains. 

ANNs are based on collections of connected units or nodes called artificial neurons (blue nodes in the diagram above), which loosely model neurons in biological brains. Each connection, like a synapse in a biological brain, can transmit signals to other neurons. 

An artificial neuron receives the signal, processes it, and can send out signals to the neurons connected to it. The "signal" at the connection is a real number, and the output of each neuron is computed by some nonlinear function of the sum of its inputs. 

These connections are called edges. Neurons and edges typically have weights that are adjusted as learning progresses. The weight increases or decreases the signal strength at the connection. 

A neuron may have a threshold so that it only sends a signal when the aggregated signal exceeds that threshold. Typically, neurons are aggregated into layers. 

Different layers can perform different transformations on their inputs. The signal propagates from the first layer (input layer) to the last layer (output layer), possibly after traversing the layers many times. 

ANNs are widely used in a variety of applications, including image recognition, predictive modeling, and natural language processing (NLP). Examples of important commercial applications since 2000 include handwriting recognition for check processing, speech-to-text transcription, oil exploration data analysis, weather forecasting, and facial recognition.  

[More to come ...]