ML Definitions and Concepts

- Overview

Machine Learning (ML) is a subfield of Artificial Intelligence (AI) where computers learn to perform tasks by analyzing large datasets, identifying patterns, and improving their performance over time without explicit programming, essentially mimicking how humans learn from experience; allowing them to make predictions or classifications on new data based on the patterns learned from the training data. 

In simpler terms, ML enables computers to learn from data and make decisions or predictions without being explicitly programmed to do so. At its core, ML is all about creating and implementing algorithms that facilitate these decisions and predictions.

(A) Core Concepts:

1. Learning from Experience: Instead of following fixed, manual rules, ML systems improve their performance over time as they are exposed to more data, mimicking human learning. 

2. Algorithms vs. Models: 

• Algorithms are the sets of instructions or mathematical rules used to process data.
• Models are the output - the specific "knowledge" or program - created after an algorithm has been trained on a dataset.

3. Pattern Recognition: At its core, ML identifies complex relationships within large datasets that would be difficult or impossible for humans to detect manually. 

4. Probabilistic Nature: Unlike traditional deterministic software where "if A, then B," ML is often probabilistic, providing results like "there is an X% chance of B happening".

(B) Main Types of Machine Learning:

• Supervised Learning: The model is trained on labeled data (input with the correct answer provided) to learn how to predict outcomes for new data.
• Unsupervised Learning: The model analyzes unlabeled data to find hidden structures or clusters on its own.
• Reinforcement Learning: The system learns through trial and error, receiving digital "rewards" or "penalties" based on its actions within an environment.
• Deep Learning: A specialized subset of ML that uses artificial neural networks with many layers to solve highly complex tasks like image and speech recognition.

Please refer to the following for more details:  

• Wikipedia: Machine Learning
• Wikipedia: Outline of Machine Learning

- The Evolution of ML

Biological evolution inspires machine learning (ML). Evolution allows life to explore almost limitless diversity and complexity. Scientists hope to recreate such open-endedness in the laboratory or in computer simulations, but even sophisticated computational techniques like ML and AI can't provide the open-ended tinkering associated with evolution.

The earliest computers were designed to perform complex calculations, and their architecture allowed for the storage of not only data but also the instructions as to how to manipulate that data. This evolved to the point where the computer processed data according to a structure model of the real world, expressible in mathematical terms. The computer did not learn but was merely following instructions. 

The next step was to create a set of instructions that would allow the computer to learn from experience, i.e., to extract its own rules from large amounts of data and use those rules for classification and prediction. This was the beginning of ML and has led to the field that is collectively defined as AI. 

A major breakthrough came with the implementation of algorithms that were loosely modeled on brain architecture, with multiple interconnecting units sharing weighted puts among them, organized in computational layers (deep learning). 

Here is a summary of the key milestones in the evolution of MLg:

• Early Calculation Machines: The first computers were designed to perform complex calculations. Their architecture allowed for the storage of both data and specific instructions for manipulating that data.
• Structured Modeling: This evolved into systems that processed data according to structured mathematical models of the real world. During this phase, the computer did not "learn" but simply followed a predefined set of instructions.
• Learning from Experience: A shift occurred when instructions were developed to allow computers to extract their own rules from large datasets. By using these rules for classification and prediction, the field of Machine Learning (ML) was established, eventually leading to the broader field of Artificial Intelligence (AI).
• Deep Learning: A major breakthrough involved algorithms loosely modeled on brain architecture. These systems consist of multiple interconnecting units with weighted inputs organized in computational layers, a technique known as deep learning.
• Biological Inspiration: Scientists continue to look toward biological evolution as an inspiration for ML, hoping to replicate the "open-ended tinkering" and limitless complexity found in nature, which current computational techniques still cannot fully provide.

- Evolution over the Years

ML techniques have been around since 1952. It has changed dramatically over the past decade and went through several transition periods in the mid-90s. Data-driven ML methods emerged in the 1990s. 

From 1995 to 2005, there was a lot of focus on natural language, search, and information retrieval. In those days, ML tools were more straightforward than those used today. 

Popularized in the 80s, neural networks are a subset of ML, which are computer systems modeled after the human brain and nervous system. Neural networks started making a comeback around 2005. It has become one of the trending technologies of the current decade.

Does ML require coding? Yes, ML requires a programming language. First, understanding ML involves algorithms. Mathematics is a required course for learning the concept of algorithms. But when you're implementing ML to solve real-world problems, you do need to code. Python and R are the programming languages of choice in AI and data science.

Here is the breakdown of the evolution of ML and its requirements:

1. Evolution Over the Years:

• 1952: ML techniques have existed since this time, with significant evolution occurring over the past decade.
• 1990s: The field shifted toward data-driven approaches.
• 1995–2005: Focus was heavily placed on natural language, search, and information retrieval. Tools in this era were more straightforward than current ones.
• Mid-90s: The field experienced several transition periods.
• 1980s (Roots): Neural networks, computer systems modeled after the human brain, gained popularity.
• 2005 (Resurgence): Neural networks began making a comeback, subsequently becoming a trending technology in the current decade.

2. Does ML Require Coding? 

Implementing ML to solve real-world problems requires coding and a programming language.

• Key Requirements: Understanding ML involves algorithms, and mathematics is required to learn these concepts.
• Languages of Choice: Python and R are the primary programming languages used in AI and data science.

- Key Concepts in ML

Key concepts in Machine Learning (ML) include: supervised learning, unsupervised learning, reinforcement learning, data preprocessing, feature engineering, algorithms, overfitting, deep learning, computer vision, recommender systems, and model evaluation, with the core idea being that machines can learn from data to make predictions or decisions without explicit programming. 

Breakdown of key concepts in ML: 

• Supervised Learning: Training a model on labeled data where the correct output is known for each input, allowing the model to learn patterns and predict outcomes on new data.
  
• Unsupervised Learning: Discovering patterns in unlabeled data without known outcomes, often used for clustering or anomaly detection.
  
• Deep Learning: A subset of machine learning that uses artificial neural networks with multiple layers to extract complex features from data, often achieving high accuracy in tasks like image recognition and natural language processing.
• Reinforcement Learning: Learning through trial and error, where an agent receives feedback (rewards or penalties) for its actions in an environment, allowing it to adjust its behavior to maximize rewards.
  
• Data Preprocessing: Cleaning, transforming, and preparing data to be suitable for machine learning algorithms.
  
• Feature Engineering: Creating new features from existing data to improve the model's ability to learn patterns.
  
• Algorithms: The mathematical equations and procedures used by a machine learning model to learn from data, like decision trees, linear regression, or neural networks.
  
• Overfitting: When a model learns the training data too well, resulting in poor performance on new data.
• Classification: Predicting a categorical label for new data (e.g., classifying emails as spam or not spam).
• Regression: Predicting a continuous value (e.g., predicting house prices based on features).
• Unsupervised Learning: Where the model is trained on unlabeled data, discovering patterns and relationships within the data without explicit guidance.
• Clustering: Grouping data points into similar clusters based on their features.
• Dimensionality Reduction: Reducing the number of features in a dataset while preserving important information. 
• Computer Vision: Applying machine learning to analyze visual data like images and videos to recognize objects, detect patterns, and understand scenes.
• Recommender Systems: Using machine learning to predict which items a user might like based on their past behavior, commonly seen in e-commerce and streaming platforms.
  
• Model Evaluation: Assessing the performance of a trained machine learning model using metrics like accuracy, precision, recall, and F1-score

[Machine Learning Algorithms]

- When To Use Machine Learning

It’s important to remember that machine learning (ML) can’t solve every type of problem. In some cases, powerful solutions can be developed without using ML techniques. 

For example, ML is not required if you can determine a target value by using simple rules, calculations, or predetermined steps that can be programmed without any data-driven learning. 

Machine learning (ML) is best used for complex, high-scale tasks where writing explicit rules is impossible, such as spam detection or image recognition. 

It excels at identifying patterns in vast data, automating decisions, and adapting to changing environments. 

Avoid ML for simple, rules-based, or high-stakes, precision-demanding problems.

Use ML When:

1. Rules are too complex or cannot be defined: When a task involves too many variables, overlapping factors, or nuanced patterns that are difficult to encode manually (e.g., spam detection).
2. Scale is too large: When the volume of data is too high for manual, human intervention to be efficient.
3. The environment is dynamic: When the underlying patterns change over time, requiring the model to adapt, update, or learn continuously.
4. Data-driven insights are needed: To identify hidden patterns, trends, or anomalies within large datasets.

2. Do Not Use ML When:

• Simple logic works: If the problem can be solved with deterministic, if-then-else, or simple calculations, using ML adds unnecessary complexity.
• High precision is required: If the task requires 100% accuracy (e.g., accounting), ML's probabilistic nature makes it unsuitable.
• Data is unavailable: If you cannot obtain enough relevant data to train a model, it will not perform well.

The following is a classic end-to-end ML pipeline, which contains a real-world process that allows you to understand how all the parts are put together. 

This specific 8-step sequence is a hallmark of high-quality data science projects, often cited in comprehensive guides to move from a messy raw dataset to actionable business insights.

The Complete ML Solution Workflow:

• Data cleaning and formatting: The "unsexy" but vital 80% of the work. You handle missing values, outliers, and ensure data types are consistent for the model.
• Exploratory data analysis (EDA): The "detective" phase. Use visualizations and statistics to find patterns, correlations, and anomalies before touching a model.
• Feature engineering and selection: Creating new variables (like extracting "Hour" from a timestamp) and picking only the most relevant ones to improve model accuracy.
• Compare several ML models: Testing a "baseline" (like Linear Regression) against more complex algorithms (like Random Forest or XGBoost) using a specific metric like RMSE or Accuracy.
• Perform hyperparameter tuning: Fine-tuning the "knobs" of your best-performing model (using tools like Grid Search or Random Search) to squeeze out maximum performance.
• Evaluate the best model on the testing set: The moment of truth. You test your final model on data it has never seen to ensure it generalizes well to the real world.
• Interpret the model results: Moving beyond "black box" predictions. Use techniques like SHAP or LIME to explain why the model made specific decisions.
• Draw conclusions and document work: Translating technical metrics into business value and documenting the process for reproducibility and future maintenance.