{"id":26568,"date":"2024-04-11T22:34:42","date_gmt":"2024-04-11T22:34:42","guid":{"rendered":"https:\/\/davidgerard.co.uk\/blockchain\/?p=26568"},"modified":"2024-05-17T07:16:58","modified_gmt":"2024-05-17T07:16:58","slug":"pivot-to-ai-hallucinations-worsen-as-the-money-runs-out","status":"publish","type":"post","link":"https:\/\/davidgerard.co.uk\/blockchain\/2024\/04\/11\/pivot-to-ai-hallucinations-worsen-as-the-money-runs-out\/","title":{"rendered":"Pivot to AI: Hallucinations worsen as the money runs out"},"content":{"rendered":"<ul>\n<li aria-level=\"1\">Amy is still busy, so David wrote this one up. Expect <em>proper<\/em> spelling and eschewing the AP style guide.<\/li>\n<li aria-level=\"1\">We need your support for more posts like this. Send us money! Here\u2019s<a href=\"https:\/\/www.patreon.com\/amycastor\"> Amy\u2019s<\/a> Patreon, and here\u2019s <a href=\"https:\/\/www.patreon.com\/davidgerard\/\">David\u2019s<\/a>. Sign up today!<\/li>\n<li aria-level=\"1\">If you like this post \u2014 please <b><i>tell just one other person.<\/i><\/b><\/li>\n<\/ul>\n<blockquote><p>\u201cGPT5 has averaged photos of all website owners and created a platonic ideal of a sysadmin: [<em>insert photo of <a href=\"https:\/\/knowyourmeme.com\/memes\/people\/parked-domain-girl\">Parked Domain Girl<\/a><\/em>]\u201d<\/p>\n<p style=\"text-align: right;\">\u2014 <a href=\"https:\/\/social.tchncs.de\/@jookia\/112254653523184343\">Jookia<\/a><\/p>\n<\/blockquote>\n<p>&nbsp;<\/p>\n<p><a href=\"https:\/\/davidgerard.co.uk\/blockchain\/2024\/04\/11\/pivot-to-ai-hallucinations-worsen-as-the-money-runs-out\/deepmind-dogs-eyeballs\/\" rel=\"attachment wp-att-26588\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-26588\" src=\"https:\/\/davidgerard.co.uk\/blockchain\/wp-content\/uploads\/2024\/04\/deepmind-dogs-eyeballs.jpg\" alt=\"\" width=\"510\" height=\"315\" srcset=\"https:\/\/davidgerard.co.uk\/blockchain\/wp-content\/uploads\/2024\/04\/deepmind-dogs-eyeballs.jpg 680w, https:\/\/davidgerard.co.uk\/blockchain\/wp-content\/uploads\/2024\/04\/deepmind-dogs-eyeballs-300x185.jpg 300w, https:\/\/davidgerard.co.uk\/blockchain\/wp-content\/uploads\/2024\/04\/deepmind-dogs-eyeballs-348x215.jpg 348w\" sizes=\"auto, (max-width: 510px) 100vw, 510px\" \/><\/a><\/p>\n<p style=\"text-align: center;\"><small><i>Deep Dream output, 2015 \u2014 a Biblically accurate doge. Switch the site to dark mode for best results.<br \/>\n<\/i><\/small><\/p>\n<p>&nbsp;<\/p>\n<h3>Lucy in the skAI with diamonds<\/h3>\n<p>A vision came to us in a dream \u2014 and certainly not from any nameable person \u2014 on the current state of the venture capital-fueled AI and machine learning industry. We asked around and several others who work in the field concurred with this assessment.<\/p>\n<p>Generative AI is famous for \u201challucinating\u201d made-up answers with wrong facts. These are crippling to the credibility of AI-driven products.<\/p>\n<p>The bad news is that the hallucinations are not decreasing. In fact, the hallucinations are getting worse.<\/p>\n<p>Large language models work by generating output based on what tokens statistically follow from other tokens. They are extremely capable autocompletes.<\/p>\n<p><i>All<\/i> output from a LLM is a \u201challucination\u201d \u2014 generated from the latent space between the training data. LLMs are machines for generating convincing-sounding nonsense \u2014 \u201cfacts\u201d are not a type of data in LLMs.<\/p>\n<p>But if your input contains mostly facts, then the output has a better chance of not being just nonsense.<\/p>\n<p>Unfortunately, the venture-capital-funded AI industry runs on the promise of <i>replacing<\/i> humans with a very large shell script \u2014 including in areas where details matter. If the AI&#8217;s output is just plausible nonsense, that\u2019s a problem. So the hallucination issue is causing a slight panic among AI company leadership.<\/p>\n<p>More unfortunately, the AI industry has run out of training data that isn&#8217;t tainted with previous AI output. So they\u2019re seriously considering doing the stupidest thing possible: training AIs on the output of other AIs. This is already well known to make the models collapse into gibberish. [<a href=\"https:\/\/www.wsj.com\/tech\/ai\/ai-training-data-synthetic-openai-anthropic-9230f8d8\"><i>WSJ<\/i><\/a><i>, <\/i><a href=\"https:\/\/archive.is\/27tnr\"><i>archive<\/i><\/a>]<\/p>\n<p>AI companies are starting to talk up \u201cemergent capabilities\u201d again \u2014 where an AI suddenly becomes useful for things it wasn\u2019t developed for, like translating languages it wasn\u2019t trained on. You know \u2014 magic.<\/p>\n<p>Every claim ever made of \u201cemergent capabilities\u201d has turned out to be an irreproducible coincidence or data that the model was in fact trained on. Magic doesn\u2019t happen. [<a href=\"https:\/\/hai.stanford.edu\/news\/ais-ostensible-emergent-abilities-are-mirage\"><i>Stanford<\/i><\/a><i>, 2023<\/i>]<\/p>\n<p>The current workaround in AI is to hire fresh master\u2019s graduates or PhDs to try to fix the hallucinations. The companies try to underpay the fresh grads on the promise of future wealth \u2014 or at least a high-status position in the AI doomsday cult. OpenAI is notoriously high on the cultish workplace scale, for example, not understanding why anyone would want to work there if they weren\u2019t <a href=\"https:\/\/davidgerard.co.uk\/blockchain\/2023\/11\/18\/pivot-to-ai-replacing-sam-altman-with-a-very-small-shell-script\/\">true believers.<\/a><\/p>\n<p>If you have a degree with machine learning in it, gouge them for every penny you can while the gouging is good.<\/p>\n<p>Remember when AI had <em>proper<\/em> hallucinations? Eyeballs! Spurious dogs! [<em><a href=\"https:\/\/www.fastcompany.com\/3048941\/why-googles-deep-dream-ai-hallucinates-in-dog-faces\">Fast Company<\/a>, 2015<\/em>]<\/p>\n<h3>When the money runs out<\/h3>\n<p>There is enough money floating around in tech VC to fuel the current AI hype for another couple of years. There are <i>hundreds of billions<\/i> of dollars \u2014 family offices, pension funds, sovereign wealth funds \u2014 that are desperate to find returns.<\/p>\n<p>But in AI in particular, the money and the patience are running out \u2014 because the systems don&#8217;t have a path to profitable functionality.<\/p>\n<p>Stability AI raised $100 million at a $1 billion valuation. By October 2023, they had $4 million cash left \u2014 and couldn\u2019t get more because their investors were no longer interested in setting money on fire. At one stage, Stability ran out of money for their AWS cloud computing bill. [<a href=\"https:\/\/www.forbes.com\/sites\/kenrickcai\/2024\/03\/29\/how-stability-ais-founder-tanked-his-billion-dollar-startup\/\"><i>Forbes<\/i><\/a><i>, <\/i><a href=\"http:\/\/archive.today\/2024.04.08-032052\/https:\/\/www.forbes.com\/sites\/kenrickcai\/2024\/03\/29\/how-stability-ais-founder-tanked-his-billion-dollar-startup\/\"><i>archive<\/i><\/a>]<\/p>\n<p>Ed Zitron gives the present AI venture capital bubble three more quarters (nine months), which would take it through to the end of the year. The gossip concurs with Ed on this likely lasting another three quarters. There should be at least one more wave of massive overhiring. [<a href=\"https:\/\/www.wheresyoured.at\/peakai\/\"><i>Ed Zitron<\/i><\/a>]<\/p>\n<p>Compare AI to bitcoin, which keeps coming back like a bad Ponzi. It\u2019s true, as <a href=\"https:\/\/davidgerard.co.uk\/blockchain\/2021\/04\/11\/desperate-investors-neoliberalism-and-keynes-how-to-increase-returns\/\">Keynes<\/a> says, that the market can stay irrational longer than you can stay solvent. Crypto is a pretty good counterexample to the efficient market hypothesis. But AI doesn\u2019t have the <a href=\"https:\/\/davidgerard.co.uk\/blockchain\/2018\/01\/04\/why-you-cant-cash-out-pt-3-bitcoin-is-not-a-ponzi-scheme-it-just-works-like-one\/\">Ponzi-like structure of crypto<\/a> \u2014 there\u2019s no path to getting rich for free for the common ex-crypto-degen that would sustain it that far beyond all reason.<\/p>\n<p>AI stocks are what\u2019s holding up the S&amp;P 500 this year. This means that when the AI VC bubble pops, tech will crash.<\/p>\n<p>Whenever the NASDAQ catches a cold, bitcoin catches COVID \u2014 so we should expect crypto to go through the floor in turn.<\/p>\n<h3>AI will soon be doing reasoning! Well, \u2018reasoning\u2019<\/h3>\n<p>Financial Times headline, Thursday 11 April: \u201cOpenAI and Meta ready new AI models capable of \u2018reasoning\u2019\u201d. Huge if true! [<a href=\"https:\/\/www.ft.com\/content\/78834fd4-c4d1-4bab-bc40-a64ad9d65e0d\"><i>FT<\/i><\/a><i>, <\/i><a href=\"https:\/\/archive.is\/zLwB0\"><i>archive<\/i><\/a>]<\/p>\n<p>This is an awesome story that the FT somehow ran without an editor reading it from the beginning through to the end. You can watch as the splashy headline claim slowly decays to nothing:<\/p>\n<ul>\n<li aria-level=\"1\">Headline: AI models capable of \u2018reasoning\u2019 are nearly ready. According to the subheading, they\u2019ll come out this year!<\/li>\n<li aria-level=\"1\">First paragraph: well, they\u2019re not <i>ready<\/i> as such, but OpenAI and Facebook are \u201con the brink\u201d of releasing a reasoning engine \u2014 trust us, bro.<\/li>\n<li aria-level=\"1\">Fourth paragraph: the companies haven\u2019t actually figured out yet how to do reasoning. But \u201cWe are hard at work in figuring out how to get these models not just to talk, but actually to reason, to plan\u2009&#8230;\u2009to have memory.\u201d Now, you might think they\u2019ve been claiming to be hard at work on all of these for the past several years.<\/li>\n<li aria-level=\"1\">Fifth paragraph: it\u2019ll totally \u201cshow progress,\u201d guys. It\u2019s \u201cjust starting to scratch the surface on the ability that these models have to reason\u201d \u2014 that is, the models don&#8217;t actually do this.<\/li>\n<li aria-level=\"1\">Sixth paragraph: current systems are still \u201cpretty narrow\u201d \u2014 that is, the models don&#8217;t actually do this.<\/li>\n<li aria-level=\"1\">Thirteenth paragraph, halfway down: Yann LeCun of Facebook admits that reasoning is a \u201cbig missing piece\u201d \u2014 not only do the models not do it, the companies don\u2019t know how to do it.<\/li>\n<li aria-level=\"1\">Fourteenth paragraph: AI will one day give us such hitherto-unknown applications as journey planners.<\/li>\n<li aria-level=\"1\">Seventeenth paragraph: \u201cI think over time\u2009&#8230;\u2009we\u2019ll see the models go toward longer, kind of more complex tasks,\u201d says Brad Lightcap of OpenAI.<\/li>\n<\/ul>\n<p>In the final paragraph, LeCun warns us:<\/p>\n<blockquote><p>\u201cWe will be talking to these AI assistants all the time,\u201d LeCun said. \u201cOur entire digital diet will be mediated by AI systems.\u201d<\/p><\/blockquote>\n<p>It\u2019s hard to see that other than as a threat.<\/p>\n<h3>His lips are moving<\/h3>\n<p>Lie detectors don&#8217;t exist, but there&#8217;s no sucker like a rich sucker. Speech Craft Analytics managed to get a promotional flyer printed in the FT for its purported voice stress analysers.<\/p>\n<p>Voice stress analysis is <a href=\"https:\/\/en.wikipedia.org\/wiki\/Voice_stress_analysis\">complete and utter pseudoscience.<\/a> It doesn\u2019t exist. It doesn\u2019t work. Fabulous results are regularly claimed and never reproduced. Anyone trying to sell you voice stress analysis is a crook and a con man.<\/p>\n<p>But lie detectors are such a desperately desired product that merely being a known fraud won\u2019t stop anyone from buying them.<\/p>\n<p>The FT story markets this to professional investors and securities analysts \u2014 they claim that they can tell when a CEO is lying in an earnings call.<\/p>\n<p>How do they do this? It\u2019s uh, AI! Totally not handwaving, magic or pseudoscience.<\/p>\n<p>The use case they don\u2019t name but obviously imply is, of course, to use this nigh-magical gadget on your own employees \u2014 whether it works or not. The article even admits that this sort of claimed AI use case risks becoming an engine for racism laundering. [<a href=\"https:\/\/www.ft.com\/content\/ee2788dd-aca5-4214-8a08-d88081eac1b9\"><i>FT<\/i><\/a><i>, <\/i><a href=\"https:\/\/archive.ph\/2023.11.12-215729\/https:\/\/www.ft.com\/content\/ee2788dd-aca5-4214-8a08-d88081eac1b9\"><i>archive<\/i><\/a>]<\/p>\n<h3>But consider: everyone involved is in dire need of leashing<\/h3>\n<p>For anyone who doubts that AI is precisely the same rubbish as blockchain, we present to you: the AI UNLEASHED SUMMIT! [<a href=\"https:\/\/www.aiunleashedglobalsummit.com\/summit\"><i>AI Unleashed Summit<\/i><\/a><i>, <\/i><a href=\"http:\/\/archive.today\/2024.04.11-193921\/https:\/\/www.aiunleashedglobalsummit.com\/summit\"><i>archive<\/i><\/a>]<\/p>\n<p>This amazing event ran in September 2023. The \u201cExponential Ai strategies\u201d were promises to teach you how to do prompt engineering.<\/p>\n<p>&#8220;Experience the ultimate skill mastery like being plugged into a Matrix-like machine!&#8221; \u2014 someone who&#8217;s totally watched <em>The Matrix<\/em>.<\/p>\n<p>The promotional trailer video looks like a parody because it appears largely to be stock footage. [<a href=\"https:\/\/www.youtube.com\/watch?v=-DWcHumgGtc\"><i>YouTube<\/i><\/a>]<\/p>\n<h3>Impromptu engineering<\/h3>\n<p>A card game developer told PC Gamer how it paid an \u201cAI artist\u201d $90,000 to generate card art \u2014 because \u201cno one comes close to the quality he delivers.\u201d The \u201cartist,\u201d who totally exists, isn\u2019t on social media. The cards are generic AI ripoffs of well-known collectible card games but with extra fingers and misplaced limbs and tails. It turns out PC Gamer was tricked into running a promotional story on an NFT offering \u2014 in 2024. [<a href=\"https:\/\/www.pcgamer.com\/games\/card-games\/champions-tcg-ai-artist\/\"><i>PC Gamer<\/i><\/a><i>, <\/i><a href=\"http:\/\/archive.today\/2024.04.11-052955\/https:\/\/www.pcgamer.com\/games\/card-games\/champions-tcg-ai-artist\/\"><i>archive<\/i><\/a>]<\/p>\n<p>Where is GPT-5 getting fresh training data? It looks like it\u2019ll be getting a lot of valuable new authentic human interactions from honeypot sites designed to waste the time of spammy web crawlers. [<a href=\"https:\/\/mailman.nanog.org\/pipermail\/nanog\/2024-April\/225407.html\"><i>NANOG<\/i><\/a>]<\/p>\n<p>Baldur Bjarnason: &#8220;Tech punditry keeps harping on the notion that nobody has ever successfully banned \u2018scientific progress\u2019, but LLMs and generative models are not \u2018progress\u2019. They\u2019re products and we ban those all the time.&#8221; [<a href=\"https:\/\/www.baldurbjarnason.com\/2024\/they-ban-products-dont-they\/\"><i>blog post<\/i><\/a>]<\/p>\n<br><br><div align=\"center\"><p><a href=\"https:\/\/www.patreon.com\/bePatron?u=8420236\"><img src=\"https:\/\/davidgerard.co.uk\/blockchain\/wp-content\/uploads\/2021\/10\/become_a_patron_button.svg\" alt=\"Become a Patron!\" title=\"Become a Patron!\" width=217 height=51><\/a><br><p style=\"align:center;\" class=\"patreon-badge\"><i>Your subscriptions keep this site going. <a href=\"https:\/\/www.patreon.com\/bePatron?u=8420236\">Sign up today!<\/a><\/i><\/p><\/div>","protected":false},"excerpt":{"rendered":"<p>Remember when AI had proper hallucinations? Eyeballs! Spurious dogs!<\/p>\n","protected":false},"author":1,"featured_media":26588,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"inline_featured_image":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1],"tags":[3515,3767,3769,3666,2295,444,484,3770,2020,3517,3768,3766,3764,3765],"class_list":["post-26568","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorised","tag-ai","tag-ai-unleashed-summit","tag-baldur-bjarnason","tag-brad-lightcap","tag-ed-zitron","tag-facebook","tag-financial-times","tag-kane-minkus","tag-nft","tag-openai","tag-pc-gamer","tag-speech-craft-analytics","tag-stability-ai","tag-yann-lecun"],"jetpack_featured_media_url":"https:\/\/davidgerard.co.uk\/blockchain\/wp-content\/uploads\/2024\/04\/deepmind-dogs-eyeballs.jpg","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/davidgerard.co.uk\/blockchain\/wp-json\/wp\/v2\/posts\/26568","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/davidgerard.co.uk\/blockchain\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/davidgerard.co.uk\/blockchain\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/davidgerard.co.uk\/blockchain\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/davidgerard.co.uk\/blockchain\/wp-json\/wp\/v2\/comments?post=26568"}],"version-history":[{"count":31,"href":"https:\/\/davidgerard.co.uk\/blockchain\/wp-json\/wp\/v2\/posts\/26568\/revisions"}],"predecessor-version":[{"id":26711,"href":"https:\/\/davidgerard.co.uk\/blockchain\/wp-json\/wp\/v2\/posts\/26568\/revisions\/26711"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/davidgerard.co.uk\/blockchain\/wp-json\/wp\/v2\/media\/26588"}],"wp:attachment":[{"href":"https:\/\/davidgerard.co.uk\/blockchain\/wp-json\/wp\/v2\/media?parent=26568"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/davidgerard.co.uk\/blockchain\/wp-json\/wp\/v2\/categories?post=26568"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/davidgerard.co.uk\/blockchain\/wp-json\/wp\/v2\/tags?post=26568"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}