A bad chatbot might luck its way to victory if the judges aren’t familiar with tell-tale signs of chatbot-ness. That’s usually of less importance when your panel includes experts in the field of computer science. In this case, it included an actor from Red Dwarf and a member of the House of Lords, both of whom are incredibly accomplished and by all indications brilliant minds, but not specifically trained in this field.
David Auerbach argues that “Eugene Goostman” did in fact pass the Turing test – but that the test itself has a fatal flaw:
Trashing the Reading results, Hunch CEO Chris Dixon tweeted, “The point of the Turing Test is that you pass it when you’ve built machines that can fully simulate human thinking.” No, that is precisely not how you pass the Turing test. You pass the Turing test by convincing judges that a computer program is human. That’s it. Turing was interested in one black-box metric for how we might gauge “human intelligence,” precisely because it has been so difficult to establish what it is to “simulate human thinking.” Turing’s test is only one measure.