This is a great illustration of how AIs don’t “know” anything – they generate an answer one word at a time based on a huge corpus of text, predicting which the most likely next word is based on what it thinks is relevant to the answer.
Even though Bing “knows” that Sunak is PM, as you can see from the second question, it can’t use that in an answer about public school members of the cabinet because the corpus of training data trends towards talking about Johnson’s cabinet (for a good reason – his percentage of public schoolers was much higher than that of Truss, so many people wrote about it).
Google’s bard has even less accuracy:
Almost every fact in this response is wrong. Johnson went to Eton, but is no longer PM; Sunak is no longer chancellor and went to Winchester, not Eton; and Truss is no longer in the cabinet and went to a state school.
The counterpoint to this is the idea that AI is only at the start of its journey, and all this will be ironed out “eventually”. My view is the opposite: I don’t think that, as currently constituted, large language model-based AI Is capable of much improvement. Like almost every kind of AI research in the last 30 years, it’s a one-trick pony rather than a generalised system. And the story of AI research since its foundation is littered with one-trick ponies which can’t be grafted onto a more generalised intelligence.
Animal-style intelligence is a set of emergent properties that evolved in parallel, not separately. Our ability to do vision and other sensations, abstract reasoning, and communications – which covers most of what we think of as intelligence – continually interacted with and reinforced each other over millions of years. We didn’t evolve any of those capabilities in isolation.
And that’s why all machine learning efforts that solve one thing at a time will fail to produce truly intelligent systems. You can’t just “solve the vision problem” then graft on a large language model, then crowbar in an abstract game-playing system and have something intelligent. It’s like putting together a jigsaw by ignoring the shapes and just cutting off bits of the pieces till they “fit” – you lose the complete picture.
