A study showed that some generative artificial intelligence chatbots, including ChatGPT, outperformed humans in creativity tests. Is the machine really ready to replace the human being?
Creatives have been among the first to fear being replaced by artificial intelligence (AI). Threatening graphic designers, illustrators, advertisers, photographers, artists, and so on are software such as Dall-E, Midjourney, or Stable Diffusion, which can generate AI images by drawing from the Internet’s sea magnum and then generating new ones in the same style.
Many large companies, such as Coca-Cola, have since already opened the doors of their marketing departments to AI. But the question of whether technology can really replace humans and their inventiveness remains.
Table of Contents
The study: is AI more creative than humans?
“Creativity,” the study states, “has traditionally been considered an exclusive ability of humans. But the rapid development of artificial intelligence has led to the creation of generative AI chatbots capable of producing high-quality works of art. Thus raising questions about the differences between human and machine creativity”.
The research presented in Scientific Reports then compared the creativity of 256 humans with that of 3 current chatbots (OpenAI’s ChatGPT and GPT-4 and Copy.Ai, built on GPT-3) by applying the Alternate Uses Task (AUT). Literally, “alternative uses task,” which is taking an object and asking people to think of new and unusual ways to use it.
Participants-human and non-human-were instructed to prefer quality over quantity. Each chatbot was tested 11 times for each of the 4 objects chosen for testing.
The researchers used two methods to evaluate the AI and human responses. The first was an algorithm that evaluated how close the suggested use for the object was to the object’s original purpose.
The second was to ask six human evaluators (who were unaware that some responses were generated by AI systems) to rate each response on a scale of 1 to 5 in terms of creativity and originality.
The results
On average, say the study’s authors, the chatbots outperformed the human participants, scoring higher on the test. “While the human responses also included poor-quality ideas, the chatbots generally produced more creative responses.” However, the best of the human ideas were equal to or higher than those of the chatbots.
According to scholars, this shows that while the research highlights the potential of AI as a tool for enhancing creativity, it also underscores the unique and complex nature of human creativity. Which may be difficult to replicate or overcome completely with technology.
The machine is machine
For Ryan Burnell, a researcher at the Alan Turing Institute, “demonstrating that machines can perform well on tasks designed to measure creativity in humans does not prove that they are capable of approaching original thinking. Because we don’t know exactly what data they have been trained on or how they generate their responses.”
They may in fact have “simply drawn on what they saw in their training data. Which may include just the Alternate Uses Task.” And “in this case,” Burnell concludes, “we are not measuring creativity but the model’s prior knowledge about this type of task.
Also for Anna Ivanova, a postdoctoral researcher at MIT who studies language models and did not work on the project, one has to “keep in mind that although chatbots are very good at completing specific requests, it can take only small tricks such as rephrasing a request to prevent them from achieving the same performance.”
Read also: Boom of generative artificial intelligence companies in 2023: the top 6
