As technology companies rush to incorporate generative AI into their software and services, they face significantly higher computing costs. Running generative AI on mobile phones, rather than through the cloud on servers run by Big Tech groups, could make chatbots development much cheaper.
Table of Contents
A hybrid AI to reduce costs
Last week, Google claimed to have managed to run a version of PaLM 2, its latest large language model, on a Samsung Galaxy.
Although it has not publicly shown the reduced model, called Gecko, the move is the latest sign that a form of AI that has only required computing resources in one data centre is rapidly beginning to make its way to many other places.
This shift could make services such as chatbots much cheaper for companies to run and pave the way for more transformative applications using generative AI.
Read also: Chatbot development in 2023: the latest trends
How to exploit smartphones
When the launch of ChatGPT at the end of last year brought generative AI to everyone’s attention, the prospect of bringing it to mobile phones seemed distant.
In addition to training the so-called large language models that underpin these services, the inference work, i.e. running the models to produce results, is also computationally demanding. Mobile phones do not have the memory to hold large models like the one underlying ChatGPT. Nor the processing power to execute them.
Generating an answer to a question on a device could also reduce the latency, or delay, in using an application. When a user’s personal data is used to refine generative answers, keeping all processing on a handheld device could also improve privacy.
More than anything else, generative artificial intelligence could make it easier to perform common tasks on a smartphone. For instance when it comes to producing text.
Smaller language models, such as Meta
The rapid progress of some underlying models has changed the equation. The largest and most advanced, such as Google’s PaLM 2 and OpenAI’s GPT-4, have grabbed the headlines.
But an explosion of smaller models has made some of the same capabilities available in technically less demanding ways.
These have benefited in part from new techniques for tuning language models based on more careful attention to the datasets they are trained on, reducing the amount of information they must contain.
In addition to the much smaller size, the open-source nature of models like this has made it easier for researchers and developers to adapt them to different computing environments.
What will Apple do?
The surprising speed with which generative artificial intelligence is beginning to arrive on smartphones is bound to increase the focus on Apple. Which has so far stayed away from the speculative frenzy around this technology.
Well-known shortcomings of generative AI, such as the tendency of large models to have ‘hallucinations’, i.e. when the chatbot responds with made-up information, have made it unlikely that Apple will include the technology in the iPhone’s operating system for some time.
With Apple’s Worldwide Developers Conference starting on 5 June, preceded by Microsoft’s developer event called Build, the fight for developers’ attention is about to get intense. Generative artificial intelligence is still in its infancy, but the race to get it into the hands – and pockets – of many more users is already getting into full swing.
Read also: And now ChatGPT also comes to our smartphones: Altman’s deal with Apple
