Something I figured out that was handy for guessing some human pieces was trying to figure out what the prompt would have been for generating the image. If I couldn’t formulate with words what the image was, then it was more likely a human artist.
On the other hand, looking at light, shadow, and reflections to see if everything made sense really didn’t help.
66% correct! I did better than expected! It’s interesting that most people taking the test do better than 50/50. People aren’t just guessing, they can actually tell the difference, though that might be because there’s a few obvious ones with 6 fingers.
Also interesting are the couple photos that are picked wrong often. One is wrong 84% of the time. It’s a landscape in oil on canvas style most believed is human made. I guess both the style and subject are rigid enough that it’s hard to tell. Ask 10 artists to draw the same landscape with oil on canvas and they’d all end up coming out similar making it easy for AI to replicate.
The abstract ones are most difficult.
It’s not clear to me what the purpose of that test is, other than a gotcha. Are we going to see a snarky reveal of the hit and miss rates in some clickbaity newsicle next week?
A few images were quite clearly in the supersaturated colour scheme that “genAI” favours; others were artworks or artist’s styles I recognised, like Basquiat and Miro. The harder ones to identify were the generic DeviantArt artworks, and a couple of false positives made by “genAI” that were quite obviously trained on fine art masters.
But again, not a Turing test, only a test of the participants’ art history skills and visual literacy 🤷
I think the purpose is to give a rough estimate about how hard it is to tell humans from AI art. This was adapted from a test posted on ACX, whose author generally likes to look at graphs of things over time, so I imagine he’ll likely post a couple more over the next few years and compare. I wouldn’t say that’s a gotcha.
Yeah, it’s not exactly a Turing test, no; the Turing test is meant to be interactive after all. That’s sort of harder to do with this kind of art than it is with text. You clearly have a greatly above average understanding of art history; I think the vast majority of people don’t really know the styles of more than a couple famous artists or know much about art histories. So for most people this is meant to be a test on vibes; can they detect “humanity” vs “genAI-ness” in a piece. I think you could more or less call it a Turing test.
Thanks for the context, I didn’t see that information on the website at all. Generally, I think it’s just good form to let people know how and to what ends their results are used — but of course, we know it’s not necessarily how things work
The site mentions it’s from ACX at the top. The results on this version are probably not going to be collected though, and it’s just for fun. The original ACX posting was a google form; when the site says “X% of people get this wrong” it’s most likely in reference to the population polled at that time.
The site mentions it’s from ACX at the top
…no? There’s a link mentioning Scott Alexander which does lead to ACX, but without any background or context. It’s a fairly big assumption that others know what “ACX” is and how SA is connected to it.
I didn’t connect the dots immediately, but turns out I tried reading Unsong back in the day. I quit when I realised Alexander’s ties to rationalist and effective altruist thought 🤷
Fascinating, but i hate all the red herrings
From a UX perspective that site is horrible - the auto scroll thing it’s doing is possibly the most disruptive way to do multiple questions I’ve come across
Quite interesting, even though the dataset clearly favors generated art.
What do you mean by that? I think there are 25 of both.
Yes, but genereated pictures are deliberetly chosen to contain less elements that “AI” struggles with and human made ones contain quite a bit of bad anatomy ones just to confuse you. It also contains abstract art, which literally strips proper shapes from drawn objects and calls it a stylistic choice.
What I’m getting at is that it’s not a random selection from both categories, they were hand picked. And in my opinion, the selection methodology favors generated art.
This was really interesting! There were some cases where I felt like the low res quality of the image made it harder than it should be but maybe that was the intention. Does anyone know of a similar resource that also trains you to spot AI creations (outside of the obvious things like weird hands and feet)?
I got 80% correct but I really took my time. 4 of the AI images that I thought were human (and 6 vice versa).
It was more fun than I thought, but I imagine when it gets even harder to tell it will be an agonising task to perform.
Go team human!
