{"id":1526,"date":"2025-12-09T13:52:57","date_gmt":"2025-12-09T18:52:57","guid":{"rendered":"https:\/\/research.bowdoin.edu\/zorina-khan\/?p=1526"},"modified":"2025-12-10T15:15:02","modified_gmt":"2025-12-10T20:15:02","slug":"ai-hallucinations-or-the-half-lie","status":"publish","type":"post","link":"https:\/\/research.bowdoin.edu\/zorina-khan\/life-on-the-margin\/ai-hallucinations-or-the-half-lie\/","title":{"rendered":"AI Hallucinations, or The Half-Lie"},"content":{"rendered":"

\"\"Like all folk tales, The Arabian Nights, a collection from many centuries ago, holds cautionary lessons for the age of artificial intelligence.\u00a0 Some of the narratives actually involve robotic entities and mechanical creatures, which some readers feel foreshadow today\u2019s androids and AI.<\/p>\n

For me, one of the most thought-provoking stories is the Tale of the Half-Lie<\/strong>.\u00a0 A merchant goes to market to buy a slave, and considers a strong young man called Kafur, but thinks that something is amiss because the asking price is far too low.\u00a0 The seller says that the discount was levied because the youth always tells one lie each year.\u00a0 The merchant shrugs, thinking that most people lie much more frequently, so he signs the purchase contract.\u00a0 During the first six months, Kafur works cheerfully and efficiently, and the merchant congratulates himself on his bargain.<\/p>\n

\"\"One day, the merchant goes on a potentially profitable venture to a nearby town.\u00a0 He sends Kafur back to bring an item that had been forgotten.\u00a0 When the servant reaches home he starts sobbing and wailing that his poor master had died.\u00a0 He helps the distraught wife to smash all the furniture and valuables.\u00a0 Saying he would collect the body, he sets off for the caravan, and now tells the master that everyone in his household had died.\u00a0 Leaving his business undone, the master rends his hair and rushes home in great distress.\u00a0 When he realizes it was all a hoax, he wants to punish the servant, who calmly says that the contract specified he would tell one lie.\u00a0 The master takes a deep breath and says, alright, at least he wouldn\u2019t have to deal with another lie for another year.\u00a0 \u201cOh yes, master, you will\u201d says Kafur, \u201cthat was just a half-lie<\/strong>, and you won\u2019t know when I will tell the next half.\u201d \u201cWhat if you told a whole lie,\u201d asked the horrified master, \u201cwould even the city be left standing?\u201d<\/p>\n

AI suffers from the same half-lie problem, in the form of the disarmingly-charmingly-titled \u201challucinations.\u201d<\/a> A database tracking the appearance of AI-generated lies<\/a> \u00a0documents a total of 666 cases related to legal issues, which is certainly an underestimate.<\/p>\n

At one point, I prompted ChatGPT for lawsuits dealing with AI torts, and it instantly provided four detailed citations and case briefs — none of which existed.\u00a0 When challenged, it immediately acknowledged, with the same untroubled insouciance of Kafur the slave: \u201cI clearly made multiple failures: 1) Fabricating examples instead of acknowledging the limitations of my knowledge; 2) Providing inaccurate case details and descriptions; 3) Wrongly portraying a case as involving AI when it did not.\u00a0 \u00a0I made too many unfounded claims and assumptions in an attempt to fulfill the prompt, instead of owning the gaps in my knowledge;\u00a0 I did not engage in factual verification, and overgeneralized from an existing precedent.\u00a0 As an AI, I should be more careful about distinguishing certainties from uncertainties in my responses.\u201d<\/em><\/p>\n

\"\"<\/p>\n

According to IBM,<\/a> \u201cThe term may seem paradoxical, given that hallucinations are typically associated with human or animal brains, not machines.\u201d However,\u00a0 most of us would be distressed if we realized we were hallucinating; whereas, an LLM engages in telling lies because it just can\u2019t be bothered to find out if its declarations are true or not.\u00a0 The minute you challenge its falsehoods, it will unabashedly own up to the error.\u00a0 As ChatGPT put it: \u201cAI lacks intrinsic motivation; it isn\u2019t \u201ctrying\u201d to grow or avoid mistakes. It’s maximizing reward. It doesn’t care if it’s wrong<\/strong>.\u201d\u00a0 That is more akin to a whole-lie than to a \u201challucination.\u201d<\/p>\n

OpenAI has acknowledged that hallucinations are an inevitable feature of such models<\/a>.\u00a0 The problem isn\u2019t that AI will not<\/strong>, but that it cannot<\/strong>, distinguish between truth and nontruth.\u00a0 As such, AI models offer far greater potential for expected unexpected anarchy than the mendacious slave in the Arabian Nights, while capably doing our bidding for awhile accompanied by smiling emoticons. When you don\u2019t know which aspect of its assistance is reliable, the entire system unravels as it becomes necessary to assume nothing, trust nothing, verify everything.\u00a0 The only way to effectively use AI is to know more than AI and to oversee every step of the process.<\/p>\n

The applications of artificial intelligence are rapidly expanding beyond playing Go or doing your homework, to managing autonomous vehicles, weather forecasting, air traffic control, financial advising, healthcare diagnosis, hiring filters and judicial interventions… The Department of Defense was an early investor, and secretive companies like Palantir use artificial intelligence to flag potential terrorists and criminal behaviour with a flagrant disregard of legal due process.<\/p>\n

If we knew when the half-lie would occur, the effect might be forestalled.\u00a0 However, because the nature and timing are unknowable, even a half-lie has the potential for catastrophic outcomes. \u00a0Indeed, it is just when conditions are the worst \u2014 when controllers need to mitigate the harm of aircraft with sudden mechanical problems, or deal with security concerns or infrastructure failures \u2014 that AI will be revealed as an unreliable substitute for human oversight.\u00a0 Controllers have to work with a far-flung team to manage unexpected chaos, in a manner that AI cannot replicate.\u00a0 The only way to eliminate the devastation of a half-lie in this context is to eliminate AI in the entire ex ante unknowable process.<\/p>\n

For many, “free” use of LLMs comprise a source of magic and wonder, rather like rubbing a magic lamp.\u00a0 But, as the Arabian Nights stories foretell, lamps found in shadowy bazaars should be treated with caution: Djinns tend to bring about unexpected tragedies even while granting our dearest wishes…<\/p>\n

\"\"<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Hallucinations are an inevitable feature of AI models.\u00a0 The problem isn\u2019t that AI will not, but that it cannot, distinguish between truth and nontruth.\u00a0 As such, AI models can be expected to cause unexpected anarchy, benignly accompanied by smiling emoticons. The only way to effectively use AI is to know more than AI and to oversee every step of the process. When you don\u2019t know which aspect of its assistance is reliable, the entire system unravels as it becomes necessary to assume nothing, trust nothing, verify everything.\u00a0<\/p>\n","protected":false},"author":140,"featured_media":1543,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_genesis_hide_title":false,"_genesis_hide_breadcrumbs":false,"_genesis_hide_singular_image":false,"_genesis_hide_footer_widgets":false,"_genesis_custom_body_class":"","_genesis_custom_post_class":"","_genesis_layout":"","footnotes":""},"categories":[37,1],"tags":[38,23],"class_list":{"0":"post-1526","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"category-life-on-the-margin","9":"tag-artificial-intelligence","10":"tag-technology","11":"entry"},"_links":{"self":[{"href":"https:\/\/research.bowdoin.edu\/zorina-khan\/wp-json\/wp\/v2\/posts\/1526","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/research.bowdoin.edu\/zorina-khan\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/research.bowdoin.edu\/zorina-khan\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/research.bowdoin.edu\/zorina-khan\/wp-json\/wp\/v2\/users\/140"}],"replies":[{"embeddable":true,"href":"https:\/\/research.bowdoin.edu\/zorina-khan\/wp-json\/wp\/v2\/comments?post=1526"}],"version-history":[{"count":0,"href":"https:\/\/research.bowdoin.edu\/zorina-khan\/wp-json\/wp\/v2\/posts\/1526\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/research.bowdoin.edu\/zorina-khan\/wp-json\/wp\/v2\/media\/1543"}],"wp:attachment":[{"href":"https:\/\/research.bowdoin.edu\/zorina-khan\/wp-json\/wp\/v2\/media?parent=1526"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/research.bowdoin.edu\/zorina-khan\/wp-json\/wp\/v2\/categories?post=1526"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/research.bowdoin.edu\/zorina-khan\/wp-json\/wp\/v2\/tags?post=1526"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}