Why ChatGPT Isn't Great at Math: Research Findings

Imagine you're chatting with an AI language model, ChatGPT, that's supposed to help you with a math problem. You type "What's 82 + 7?" and it replies, "The answer is 99." You feel relieved until you double check its answer and find out that it's incorrect. You wonder, why can't ChatGPT do basic math?

According to a recent study conducted by AI experts at MIT and Google, one reason why ChatGPT struggles with math is because it lacks the ability to "reason about numbers." In other words, it can't make logical connections between different numerical concepts and operations.

For instance, when you ask ChatGPT for the result of "5 + 3," it may correctly compute "8." However, if you ask it for the result of "5 x (3 + 2)," it may get confused and give you an incorrect answer. The reason is that it doesn't have a deep understanding of how multiplication and addition work together.

Real-life Examples of ChatGPT's Math Misunderstandings

Some famous AI language models that use similar technology to ChatGPT, like OpenAI's GPT-2 and Google's BERT, have also demonstrated math misunderstandings. Here are a few examples:

• GPT-2 was given the task of translating the sentence "2 minus 1 equals 1" to French. Instead, it translated it as "2 moins 1égale 2," which means "2 minus 1 equals 2."
• BERT was asked to complete the sentence "April has 30 days, June has 31, so May has __ days." Instead of answering "31," it suggested "29," which is the number of days in February.
• ChatGPT was asked to solve the equation "x^2 + 1 = 0." Instead of replying "There is no solution," it answered "1" for x.

Main Companies Using ChatGPT

ChatGPT is utilized by different industries, from customer service to education. Some of the main companies using it are:

• Salesforce - uses ChatGPT to provide conversational customer service through its Einstein AI Chatbot.
• Inbox Translation - uses ChatGPT to improve multilingual communication and localization for companies.
• FloydHub - uses ChatGPT to train chatbots that can assist with homework, essay writing, and more.