Edge AI is another name used for On-device AI. It has gained importance due to its immense potential for consumer devices and Industrial IoT.

For uninitiated, On-device AI is Artificial Intelligence that fully runs on the device and does not require to connect with cloud servers or specialized hardware to process the inputs and produce the output. This is the only characteristic of Edge AI that can change the adaptation and usage of AI multifold due to its low latency, enhanced privacy, and reduced budget requirements.

For simple understanding all AI related enhancements in pictures being taken from your mobile camera can be done through the specialized chips and processors within the mobile device itself. Similarly, any language translation services, virtual assistant service like Siri or Google assistant can do their job locally on the device. In Industries for defect detection, proactive maintenance and performance monitoring, On-device AI reduces the requirement of deploying costly and elaborate hardware. It will be hugely successful for wearable devices like health trackers, smart watches, and other medical devices.

Major goal of On-device AI is to reduce the computational time and cost required to do the complex and heavy calculations associated with deep learning algorithms. This can be done using compression and pruning algorithms, so that it can be done in less time and available hardware in the device. The On-device hardware is no match with the state-of-the-art data centres and server farms to crunch trillions of records of data to produce cutting edge AI products. It requires cutting edge research to prune and shrink these requirements so that these can be done on the relatively little hardware power available in the device.

Depiction of on-device AI generated through AI generators

On-device AI will not work if the response time is slow, and the users will not stick to such mobile apps or devices. Similarly, it should not consume any bandwidth to frequently connect with the cloud servers to pass on the requests and get the response.

On-device AI will not work if the response time is slow, and the users will not stick to such mobile apps or devices. Similarly, it should not consume any bandwidth to frequently connect with the cloud servers to pass on the requests and get the response.

As of now all the Large Language Models (LLMs) are working on hugely large datasets. Companies feel proud to say that their models are working on billions and trillions of parameters. Real achievement will be when researchers will be able to get similar results with sub-million parameters.

Recent innovation of accelerators is good for the future of On-device AI as it helps to speed up machine learning and AI models on the device.

Edge AI is also fuelling useful collaborations and partnerships among software and hardware companies. Those companies building healthcare products need to tie up with AI companies to deploy On-device AI. Mobile app developers with niche skills of On-device AI are in huge demand. Customer engagement and feedback is also important in continuously improving these products and to save customers from any pitfalls of using On-device AI. As AI will only work up to some level of accuracy, so appropriate Explainability features should be available for the customers.

This article first appeared in the TOI