Why you should think twice about your AI study buddy

Generative AI is pervading every part of our lives, for better or worse. How worried should we be?

Artificial intelligence (AI) is on everybody's mind at the moment. Donald Trump even recently stated it should be renamed "SI", standing for 'Super Intelligence'. Given how reliant the world is becoming on AI, the question needs to be asked: is generative AI helping or hindering humanity?

What are the different types of AI?

AI, though it may often seem like it, is not a magic genie. Different types of AI tools result from different input data, different ways of processing that data, and a desire for outcomes specific to your situation. Examples include:

Generative AI: These can create new content based on short-sentence prompts, such as Large Language Models (LLMs) like ChatGPT, Grok, and Microsoft Copilot.Predictive AI (used by big brands such as Starbucks and Netflix): These use patterns and statistical analysis to predict new outcomes, like how adverts on social media might be tailored to you based on your previously interacted posts.Neural network AI: These are trained on thousands of examples of what they should look for.  A recent example is a breast cancer detection AI that can detect cancers missed by radiologists.

These artificial intelligences seem rather useful, so why should we be wary of the rapid growth in AI use?

What can LLMs do?

LLMs are becoming increasingly widespread, and their versatility means you can find an AI tailored to practically every purpose imaginable.  From helping with a homework assignment, to creating and debugging code, to even serving as a substitute conversationalist, LLMs like ChatGPT can be used for a variety of purposes.

It seems to be a very useful tool, but the issues with generative AI are far more dangerous than they first appear.

The emerging reliance on AI to generate essays has been shown to result in reduced development of critical thinking skills and academic growth, and a study from the University of East Anglia found that student-written essays were far superior to their AI counterparts.

Using AI as a substitute for human interaction, or even a therapist, may help someone feel better in the short-term, but it can have serious negative effects on people's long-term social skills and overall wellbeing.

Even when using AI's powers for good, the energy and water required to power these AI models currently have a far more detrimental impact on the environment than any positive contributions they make.

Why can't we trust generative AI?

Generative AIs will often 'hallucinate', as they must generate new text based on the data used to train them. This data is often human-generated, and AI can produce inaccurate and biased results based on it.

The prompts we provide can drive these prejudiced results.

For example, I asked Grok 3 whether 58439 is a prime number (which it is). It then incorrectly stated that it was not a prime number, and the LLM scrambled to correct itself based on my faulty information.

The 'black box' nature of generative AI means the steps it took to reach this decision cannot be traced or followed. We cannot see how it determined that 71 x 823 was 58439, rather than the correct result of 58433.

A subcategory of LLMs, Large Reasoning Models (LRMs), are designed to overcome this barrier and explicitly show their path to a decision. However, these still fall short of fulfilling this requirement and struggle to complete the tasks they were primarily intended to perform.

At its core, generative AI cannot be trusted to make serious decisions, especially if a simple statement can so easily mislead it. The overreliance on AI must end before we trust it with making important decisions that impact our lives.

Governing AI use through human oversight and strict safety rules can only take us so far.