Computer passes turing test for first time

[kch49er]

http://www.gizmodo.com.au/2014/06/what-i...ring-test/

[Rynn]

This is a non-story IMO. This article has a good criticism of why: https://www.techdirt.com/articles/201406...tter.shtml

Okay, almost everything about the story is bogus. Let's dig in:

It's not a "supercomputer," it's a chatbot. It's a script made to mimic human conversation. There is no intelligence, artificial or not involved. It's just a chatbot.

Plenty of other chatbots have similarly claimed to have "passed" the Turing test in the past (often with higher ratings). Here's a story from three years ago about another bot, Cleverbot, "passing" the Turing Test by convincing 59% of judges it was human (much higher than the 33% Eugene Goostman) claims.

It "beat" the Turing test here by "gaming" the rules -- by telling people the computer was a 13-year-old boy from Ukraine in order to mentally explain away odd responses.

The "rules" of the Turing test always seem to change. Hell, Turing's original test was quite different anyway.

As Chris Dixon points out, you don't get to run a single test with judges that you picked and declare you accomplished something. That's just not how it's done. If someone claimed to have created nuclear fusion or cured cancer, you'd wait for some peer review and repeat tests under other circumstances before buying it, right?

The whole concept of the Turing Test itself is kind of a joke. While it's fun to think about, creating a chatbot that can fool humans is not really the same thing as creating artificial intelligence. Many in the AI world look on the Turing Test as a needless distraction.

In general I'm also very skeptical of the utility of the Turing test. Taking it to basics what does it actually demonstrate? In it's generic form of a conversation with the goal of appearing human all the Turing test really tests is the sophistication of a chatbot. It doesn't demonstrate general intelligence and if anything computer science (and automation in general) have shown over the past several decades that acting like a human isn't required for being better than a human at a task. Even if you could generate software and hardware that is generally equal to humans in all tasks that doesn't prove consciousness or human-like intellect. All it demonstrates is human-like capability which I would argue is far more desirable as it avoids a host of ethical concerns.

[kch49er]

I must admit I just saw the headline and didn't delve into it. I didn't know they had picked the judges.


Who cares if it isd a chatbot or supercomputer?

Yes the orginal test in turing paper had the computer and a male attempting to impersonate a human female, so imitating a 13 year old boy (even 100%) isn't passing the orginal, how much of a stickler do you want to be?


I don't see anything wrong with it impersonanting a non english person.

In OA we have turinggrade computers if anything this serves as a good example of how turing grade is not equal to can pass turing test. In OA there are most likley non sopohont ais that can pass the turing test and sophont ais that can not pass the turing test. I'm sure they have more sophistacted methods for determining intellegence.

[Rynn]

Yes the orginal test in turing paper had the computer and a male attempting to impersonate a human female, so imitating a 13 year old boy (even 100%) isn't passing the orginal, how much of a stickler do you want to be?

I don't see anything wrong with it impersonanting a non english person.

It's a very important distinction. Not only did the researchers tell judges that the conversation would be with a 13 year old boy but a 13 year old boy from Ukraine. This means that the judges are far more likely to ignore strange sentences or misunderstandings as they must take into consideration language barriers and age.

This in combination with the self-selected judges means that the claim is entirely invalid. A proper test would have a large pool of randomly selected and diverse judges and software that attempts to mimic an average person with no qualifiers.

In OA we have turinggrade computers if anything this serves as a good example of how turing grade is not equal to can pass turing test. In OA there are most likley non sopohont ais that can pass the turing test and sophont ais that can not pass the turing test. I'm sure they have more sophistacted methods for determining intellegence.

Turing grade as used in OA is synonymous with sentient which is not what the Turing test can actually show. All it can show is the sophistication of a chat bot which has good utility should it ever be realised but is not the same at all as an artificial general intelligence. In OA there would be far better metrics for determining sophonce yes.

[kch49er]

I don't see anything wrong with it pretending to be a non english person.[/quote]

It's a very important distinction. Not only did the researchers tell judges that the conversation would be with a 13 year old boy but a 13 year old boy from Ukraine. This means that the judges are far more likely to ignore strange sentences or misunderstandings as they must take into consideration language barriers and age.

This is a case of youth and second language rather than english per se. If it could hold it's own in Ukranian conversation that would be more significant. Again I'm not sure that it is cheating to even use the second language excuse, if the goal is to mimic a human, then it shouldn't matter what the human is like. Blind, deaf, dumb, boy, girl, foreign whatever. If anything that is a poorly defined test.

This in combination with the self-selected judges means that the claim is entirely invalid. A proper test would have a large pool of randomly selected and diverse judges and software that attempts to mimic an average person with no qualifiers.

Agree self-selected judges is the biggest invalidator. It is always possible a randomly selected judge could be Ukranian. If the judges are self selected then a Ukranian speaking Judge can be avoided.

If it was attempting to mimic an average person then the AI should speak Mandarian. Why should it even speak English? I'm sure there are intellegent Mandarian monoglots.

In OA we have turinggrade computers if anything this serves as a good example of how turing grade is not equal to can pass turing test. In OA there are most likley non sopohont ais that can pass the turing test and sophont ais that can not pass the turing test. I'm sure they have more sophistacted methods for determining intellegence.

Turing grade as used in OA is synonymous with sentient which is not what the Turing test can actually show. All it can show is the sophistication of a chat bot which has good utility should it ever be realised but is not the same at all as an artificial general intelligence. In OA there would be far better metrics for determining sophonce yes.

replace chat bot with computer and I agree. Yes the current turing test does not show true machine intellegence, however it has changed since the original so by Y11K it may have changed yet again to be a far better metric for determining sopohonce.

Or it may be that it is just a handy byword for sophont AI.