Slimming down AI

The power of AI, on microcontrollers

Since the onset of the COVID-19 pandemic, Artificial Intelligence has assumed a central position in being the primary driving technology of the next decade. This is also why a large body of research is being conducted worldwide in order to find ways to make AI more efficient. Inefficiency has been a major drawback being faced by the entire AI industry, with even small AI-based programs often needing absurd amounts of data and computing power to execute smoothly.

Micro AI

Recent research has shown that powerful AI Vision algorithms can now be squeezed onto simple, low-power computer chips that can run on battery power for months on end. These microcontrollers are relatively simple, low power and low-cost computer chips that can be found inside billions of regular products, including car engines, TV remotes, medical implants and power tools. This is especially significant to the wearable devices and home appliances category, along with industrial sensors and medical gadgets; thereby also allowing these sectors scope for expansion in AI capabilities, such as voice and image recognition. Additionally, by handling the entire process on-site and staying away from on-cloud processing, it also helps to keeps data private and more secure.

Song Han, the MIT professor at the forefront of the aforementioned research, seems rather excited about the project, claiming that it can make the transition from the laboratory to real-world devices sooner rather than later. What researchers are essentially doing in order to make AI more efficient is to trim deep learning algorithms such that these large neural network programs can fire like neurons in the brain. Deep learning has, of course, been crucial to the AI boom, and reducing inefficiencies on that front will be imperative, going forward.

According to Wired: “Deep learning algorithms typically run on specialized computer chips that divide the parallel computations needed to train and run the network more effectively. Training the language model known as GPT-3, which is capable of generating cogent language when given a prompt, required the equivalent of cutting-edge AI chips running at full tilt for 355 years. Such uses have led to booming sales of GPUs, chips well-suited to deep learning, as well as a growing number of AI-specific chips for smartphones and other gadgets.”

The Name’s Efficiency. Algorithmic Efficiency.

Essentially, there are two main approaches to this research. First, the use of an algorithm in order for the exploration of possible neural network architectures which fit the computational constraints of the microcontroller. The second is the development of compact, memory-efficient software libraries running from the network. “The library is designed in concert with the network architecture, to eliminate redundancy and account for the lack of memory on a microcontroller. “What we do is like finding a needle in a haystack,” according to Han.

According to Wired: “The researchers created a computer vision algorithm capable of identifying 1,000 types of objects in images with 70% accuracy. The previous best low-power algorithms achieved only around 54% accuracy. It also required 21% of the memory and reduced latency by 67%, compared with existing methods. The team showed similar performance for a deep learning algorithm that listens for a particular “wake word” in an audio feed. Han says further improvements should be possible by refining the methods used.”

Commercial applications of this technology would facilitate the growth of edge devices and features such as voice recognition without having to push data to the cloud. This can be found in devices such as smart glasses or augmented reality devices than run object detection algorithms constantly.