- Anthony Corletti
Recently I built turbo, a GPT3 powered executive assistant.
Seeing that everyone is excited about the budding use-cases of GPT3 applications like midjourney, dall-e, stable diffusion, etc, I wanted to see if AI systems like GPT3 would be ready to take on tasks humans might view as daily digital tasks or chores, i.e. scheduling calendar invites.
After building turbo and testing it out with a few users, I think that Narrow AI systems like GPT3 are falling short of completing tasks that can save time for everyday people.
So where did GPT3 fall short for turbo? Well there were a few interesting use cases.
I gave GPT3 the freedom to infer all relevant guest email addresses. Despite using a structured prompt with specific keywords that matched the prompt exactly, GPT3 was still more likely to replace
firstname.lastname@example.org the literal
GPT3 would not accurately infer the correct year for the meeting. Which is usually something that humans automatically infer. Rarely do we schedule a meeting for 1 year in the future, or 1 year in the past, or 2 years in the future.
GPT3 would also sometimes strangely respond with a prompt-like response. For example; I would structure an example payload to OpenAI's Completion API like so.FROM: Anthony Corletti <email@example.com>TO: Phil Smith <firstname.lastname@example.org>"Thanks Phil! Speak to you on November 29th, 2022 at 12:00 PM eastern time. Cheers!|guest_emails|when|meeting_duration_num_minutes|location|title|description|"
And GPT3's response would literally be|meeting_id|meeting_url|meeting_password|meeting_start_time|meeting_end_time|meeting_timezone|meeting_created_at|meeting_updated_at|meeting_created_by|meeting_updated_by|meeting_created_by_email|meeting_updated_by_email|meeting_created_by_name|meeting_updated_by_name|me|
Which is not a valid response for a payload I could use to send to the Google Calendar API. GPT3 is telling me what columns to use for a database table!
Overall, of about the 47 emails that were sent, only 3 were successful. The rest were either a failure, or a prompt-like response.
Maybe I'm using a bad prompt. Maybe I shouldn't be using GPT3 for something like this. At any rate, it goes to show that there are still applications of Narrow and General AI that aren't suited for these kinds of daily tasks yet, and so, in order to automate these tasks, humans will need to have programmatic access to more advanced General AI systems and hyper specialized networks of Narrow AI.
What do I mean by Narrow and General AI? Let's define a few concepts to get there.
All artificial intelligence is automation, but not all automation is artificial intelligence.
Let's define automation: technology applications where human input is significantly minimized.
Let's define AI (Artificial Intelligence): theory and development of computer systems able to perform tasks that normally require human intelligence, such as visual perception, speech recognition, decision-making, and language translation.
We can further divide AI into two subgroups, Narrow AI, and General AI.
We have lots of Narrow AI now. Narrow AI is good for specialized, particular tasks and it will never be as general as human intelligence. For example, AI that generates pictures given an input prompt, or AI that can generate auditory responses given an input prompt, are examples of Narrow AI.
An advanced general AI would be able to schedule a meeting using only the input of a 30 second phone call, while walking your dog and picking up it's droppings on the sidewalk, while making sure the person on the bike coming down the sidewalk doesn't hit the dog. You get the picture – we're a long way off from this kind of advanced General AI.
But even scheduling a meeting is more general than you think, it requires context about who the sender would want to include based on context of the meeting, timezones, whether or not to include certain kinds of documents in the meeting invite, and whether or not preferences for Zoom, Google Meet, or MS teams are taken into account.
I think this opens the gate for specialized kinds of narrow AI to handle this task. Maybe only a narrow AI for intros on Google Meet would be something that simplifies the problem where a human inputs more preferences to connect the sender and receiver. In this case however, AI isn't necessary, as some code that extracts the senders and receivers from an email object is all that's needed to meet the value proposition of automating the task at hand.
Narrow AI applications are really strong today and are great for generative tasks (G stands for generating in GPT3; Generative Pre-trained Transformer 3). Supplementing human creativity is a fantastic use case for generative, Narrow AI applications.
Inferring what to do for specific tasks with many contextual variables is still very hard for most AI systems to do. This is where BI (or Business Intelligence), consisting of simpler statistical calculations to inform human decision making, yields better return on investment over narrow or general AI systems.
I am hopeful that AI will one day be able to handle more digital chores, automating more of the mundane for humanity and enabling more opportunity for creative and boundary breaking endeavors. We're certainly not there yet.