\n<\/p>\n
\u201cOnce, men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.\u201d \u2014 so said [Frank Herbert] in his magnum opus, Dune<\/em>, or rather in the OC Bible<\/em> that made up part of the book\u2019s rich worldbuilding. A recent study demonstrating \u201ccognitive surrender\u201d in large language model (LLM) users, as reported in Ars Technica<\/em><\/a>, is going to add more fuel to that Butlerian fire.<\/p>\n Cognitive surrender is, in short, exactly what [Herbert] was warning of: giving over your thinking to machines. In the study, people were asked a series of questions, and \u2014 except for the necessary \u201cbrain-only\u201d control group \u2014 given access to a rigged LLM to help them answer. It was rigged in that it would give wrong answers 50% of the time, which while higher than most LLMs, only a difference in degree, not in kind. Hallucination is unavoidable<\/a>; here it was just made controllably frequent for the sake of the study.<\/p>\n The hallucinations in the study were errors that the participants should have been able to see through, if they\u2019d thought about the answers. Eighty percent of the time, they did not. That is to say: presented with an obviously wrong answer from the machine, only in 20% of cases did the participants bother to question it. The remainder were experiencing what the researchers dubbed \u201ccognitive surrender\u201d: they turned their thinking over to the machines. There\u2019s a lot more meat to this than we can summarize here, of course, but the whole paper is available free for your perusal<\/a>.<\/p>\n Giving over thinking to machines is nothing new, of course; it\u2019s probably been a couple decades since the first person drove into a lake on faulty GPS directions, for example. One might even argue that since LLMs are correct much more than 50% of the time, it is statistically wise to listen to them. In that case, however, one might be encouraged to read Dune<\/em>.<\/p>\n Thanks to [Monika] for the tip!<\/p>\n<\/p><\/div>\n