Stable diffusion fraud

publish Opened 2023-03-03. Backlink: Musings on philosophy. Or Musings on tech? Related to Post-truth media.

It just blows my mind how rapidly we’ve gotten here.

Update: 2024-04-15: seen for the first time in the wild.

My first time

My first observation with AI image fraud happened on Facebook on 2023-03-03 in a hobbyist Michelin chef page I follow:

Only one guy gets it

This Facebook account is super active. He posts 2-3 food pics per day, all expertly-prepared and plated.

The worst part is how long I hesitated before concluding this account was a fraud. All the dishes being posted look plausible - there’s even recipes in the descriptions for all of them!

What an elaborate lie this guy is running.

In the end what tipped me off was the cover photo of this self-proclaimed “chef”:

No way this is real. Four of the same dish using four totally different loaves of bread? Not to mention the dishes are flawless.

So we’ve established this dude is a fraud but where are these recipes coming from? Are they AI too? The fact this guy documented exact nutrition facts for his dishes suggests maybe it’s more likely a copy-paste from a website:

… but wait, nothing shows up for this specific (but also pretty common serving of chimichurri sauce):

So was the recipe AI-generated too?

Probably, yes. I’m not sure where the calorie count is coming from. The similarities in wording are too glaring for this to not be at least partly the work of an LLM. Like the exact wording on “salt and freshly ground pepper,” the qualifiers after exactly (and only) the asparagus, steak, potatoes, and cherry tomatoes.

This is nuts. I think I only saw through this because of how much I’ve been playing with Google’s stable diffusion AI at work. Will internet users develop skepticism fast enough to see through frauds like this before something bad happens? Will any of us know what’s real in a few years?

I wonder still: who would choose to lie about something as trivial as chef status on a Facebook image board? I’m guessing this profile is a social experiment. There’s probably a sociology PhD collecting data from that account.

Also: how strange it is that, of all the fields AI dominated first, it turned out to be art, photography, and music. Back in middle school I always imagined seeing artificial intelligence developed in my lifetime. But I always envisioned it taking factory jobs, not the jobs of artists.

I shudder to imagine what a malicious actor could achieve with this power.

Update (2025-10-30): And now we don’t have to imagine. It’s happening: https://reddit.com/r/BlackPeopleTwitter/comments/1ojydgq/ai_is_already_getting_used_to_lie_about_snap