Stable diffusion fraud

publish #needswork - add a date opened Backlink: Musings on philosophy. Or Musings on tech? Related to Post-truth media.

It just blows my mind how rapidly we’ve gotten here.

Update: 2024-04-15: seen for the first time in the wild.

My first time

My first observation with AI image fraud happened on Facebook on 2023-03-03 in a hobbyist Michelin chef page I follow:

Only one guy gets it

This Facebook account is super active. He posts 2-3 food pics per day, all expertly-prepared and plated.

The worst part is how long I hesitated before concluding this account was a fraud. All the dishes being posted look plausible - there’s even recipes in the descriptions for all of them!

What an elaborate lie this guy is running.

In the end what tipped me off was the cover photo of this self-proclaimed “chef”:

No way this is real. Four of the same dish using four totally different loaves of bread? Not to mention the dishes are flawless.

So we’ve established this dude is a fraud but where are these recipes coming from? Are they AI too? The fact this guy documented exact nutrition facts for his dishes suggests maybe it’s more likely a copy-paste from a website:

… but wait, nothing shows up for this specific (but also pretty common serving of chimichurri sauce):

So was the recipe AI-generated too?

Probably, yes. I’m not sure where the calorie count is coming from. The similarities in wording are too glaring for this to not be at least partly the work of an LLM. Like the exact wording on “salt and freshly ground pepper,” the qualifiers after exactly (and only) the asparagus, steak, potatoes, and cherry tomatoes.

This is nuts. I think I only saw through this because of how much I’ve been playing with Google’s stable diffusion AI at work. Will internet users develop skepticism fast enough to see through frauds like this before something bad happens? Will any of us know what’s real in a few years?

I wonder still: who would choose to lie about something as trivial as chef status on a Facebook image board? I’m guessing this profile is a social experiment. Surely there’s a sociology PhD behind the scenes of that account collecting data.

I still shudder to imagine what a malicious actor could achieve.

Also: how striking that of all the fields AI dominated first, it was the arts, photography, and music. Back in middle school I always imagined seeing artificial intelligence developed in my lifetime. But I didn’t envision it taking over the arts first.