Roblog

The buzz this week has been about Microsoft’s launch of its AI interface to the Bing search engine. Simon Willison, among others, has documented the frankly insane responses that people have managed to coax out of it; it’s clear that, from a safety perspective, this AI really isn’t ready for prime time.

But how does it fare on accuracy? Nick Diakopoulos of Northwestern University fact-checked some of Bing’s answers – and the results aren’t pretty:

“But, when it comes to accuracy it’s a different story. I found factual inaccuracies in 7 of the 15 responses (47%). There were also several responses that provided references for a sentence which did not include evidence of the claim in that sentence. Sometimes the claims were accurate, and other times not accurate, but either way there’s a sort of unwarranted credibility conveyed, where the citations to news outlets give a trust signal, but don’t actually support the claim made.”