CPU vs GPU vs TPU

- Overview

First came CPUs, then GPUs, and now TPUs. As the tech industry grows and finds new ways to use computers, so does the need for faster hardware.

The difference between a CPU, GPU, and TPU is that the CPU handles all the logic, calculations, and input/output of the computer, and it is a general-purpose processor. In contrast, a GPU is an additional processor used to enhance the graphics interface and run high-end tasks. TPUs are powerful custom processors that can run projects made on a specific framework (i.e. TensorFlow).

• CPU: central processing unit. Manage all functions of your computer.
• GPU: Graphics Processing Unit. Enhance your computer's graphics performance.
• TPU: Tensor Processing Unit. Custom build ASICs to accelerate TensorFlow projects.

- AI Chips: NPU and TPU

A Neural Processing Unit (NPU) is a microprocessor designed to accelerate machine learning algorithms, typically by running on predictive models such as Artificial Neural Networks (ANN) or Random Forests (RF). It is also known as a neural processor.

Note that it cannot be used for general-purpose computing such as a central processing unit (CPU). This is mainly because no software support has been developed for such processors for any computing purpose. In fact, developing such a software/compiler can be a challenge, and at the same time, it may provide low performance for non-design tasks.

Neural Processing Units (NPUs) and Tensor Processing Units (TPUs) are specialized hardware accelerators designed to accelerate machine learning and artificial intelligence (AI) workloads. NPUs and TPUs are optimized for mathematical operations commonly used in machine learning, such as matrix multiplication and convolution, and they can be used to accelerate a variety of machine learning tasks, including image classification, object detection, natural language processing, and speech recognition.

- Need for NPU

Over the past few years, we've made incredible progress in applying machine learning and outperforming humans in certain tasks, such as playing games like Go and Chess.At the same time, machine learning applications are taking human life to the next level. 

Some applications include:

• Self-driving car
• Monitoring systems or areas from threats, such as security systems involving real-time facial recognition
• Improving Healthcare Through Accurate Analysis and Treatment
• and Many others

All of this increases the amount of computation involved exponentially, and previous methods using GPUs do not scale well. This paves the way for designing processors that outperform GPUs and address the advances we are making in machine learning. 

The NPU is required for the following purposes:  

• Increase the computational speed of machine learning tasks by several times (nearly 10K times) compared to GPU
• Lower power consumption and improved resource utilization for machine learning tasks compared to GPUs and CPUs

Real life implementations of Neural Processing Units (NPU) are: 

• TPU by Google
• NNP, Myriad, EyeQ by Intel
• NVDLA by Nvidia
• AWS Inferentia by Amazon
• Ali-NPU by Alibaba
• Kunlun by Baidu
• Ascend by Huawei
• Neural Engine by Apple
• Neural Processing Unit (NPU) by Samsung

[More to come ...]