It’s the neural net equivalent of shouting “enhance!” at a computer in a movie - the resulting photo is MUCH higher resolution than the original.
Could this be a privacy concern? Could someone use an algorithm like this to identify someone who’s been blurred out? Fortunately, no. The neural net can’t recover detail that doesn’t exist - all it can do is invent detail.
This becomes more obvious when you downscale a photo, give it to the neural net, and compare its upscaled version to the original.
As it turns out, there are lots of different faces that can be downscaled into that single low-res image, and the neural net’s goal is just to find one of them. Here it has found a match - why are you not satisfied?
And it’s very sensitive to the exact position of the face, as I found out in this horrifying moment below. I verified that yes, if you downscale the upscaled image on the right, you’ll get something that looks very much like the picture in the center. Stand way back from the screen and blur your eyes (basically, make your own eyes produce a lower-resolution image) and the three images below will look more and more alike. So technically the neural net did an accurate job at its task.
A tighter crop improves the image somewhat. Somewhat.
The neural net reconstructs what it’s been rewarded to see, and since it’s been trained to produce human faces, that’s what it will reconstruct. So if I were to feed it an image of a plush giraffe, for example…
Given a pixellated image of anything, it’ll invent a human face to go with it, like some kind of dystopian computer system that sees a suspect’s image everywhere. (Building an algorithm that upscales low-res images to match faces in a police database would be both a horrifying misuse of this technology and not out of character with how law enforcement currently manipulates photos to generate matches.)
However, speaking of what the neural net’s been rewarded to see - shortly after this particular neural net was released, twitter user chicken3gg posted this reconstruction:
Biased AIs are a well-documented phenomenon. When its task is to copy human behavior, AI will copy everything it sees, not knowing what parts it would be better not to copy. Or it can learn a skewed version of reality from its training data. Or its task might be set up in a way that rewards - or at the least doesn’t penalize - a biased outcome. Or the very existence of the task itself (like predicting “criminality”) might be the product of bias.
In this case, the AI might have been inadvertently rewarded for reconstructing white faces if its training data (Flickr-Faces-HQ) had a large enough skew toward white faces. Or, as the authors of the PULSE paper pointed out (in response to the conversation around bias), the standard benchmark that AI researchers use for comparing their accuracy at upscaling faces is based on the CelebA HQ dataset, which is 90% white. So even if an AI did a terrible job at upscaling other faces, but an excellent job at upscaling white faces, it could still technically qualify as state-of-the-art. This is definitely a problem.
A related problem is the huge lack of diversity in the field of artificial intelligence. Even an academic project with art as its main application should not have gone all the way to publication before someone noticed that it was hugely biased. Several factors are contributing to the lack of diversity in AI, including anti-Black bias. The repercussions of this striking example of bias, and of the conversations it has sparked, are still being strongly felt in a field that’s long overdue for a reckoning.
Bonus material this week: an ongoing experiment that’s making me question not only what madlibs are, but what even are sentences. Enter your email here for a preview.
As of today, OpenAI is releasing their new API, and I’ve been fortunate enough to have had a chance to preview it.
Folks, I’m going to be doing a LOT of posts on this.
Like GPT-2, they used internet text as training data, and it can use that knowledge to look at text you give it and predict what comes next. The new OpenAI API is REALLY good at following all sorts of prompts. Set up the first two lines of a chat and it will stay in character.
Me: Hello, who are you? Bot (I wrote this line for it): Foolish mortal! I am the god of the internet. Me: Prove it. Bot: No need to prove it, you will bow down and worship me and the wonders of the internet. Me: What if I don’t? Bot: Then I shall unleash a bot attack on your computer. Me: Go ahead and try it. Bot: But I told you who I was! You must worship me! Me: Let’s see the bot attack first. Bot: Oh, for the love of… Fine! (insert name of random bot here) attacks you! Me: That… is not a bot attack. Literally nothing happened. Bot: … Seriously? Me: Yeah dude, sorry.
It also is pretty good at style transfer. By giving it a few examples to start out with, I got it to do the following rewriite:
This API is versatile. And one thing it does well is make Twitter bots. For those who are unfamiliar, @dog_rates is a twitter account that posts user-submitted dogs, introduces them, and then gives them a rating from one to ten. All the dogs are rated at least 10/10 because they’re very good dogs.
In the past when people have built bot versions of twitter accounts like @dog_rates, the resulting twitter bots really aren’t that on-topic. The original account usually isn’t consistent enough for the neural net to figure out how to sound like it. But after showing this API just 15 recent quintessential tweets, it was rating dogs convincingly and consistently.
Ok, some of its commentary is a little weird.
(I used artbreeder to manually generate the dog images to go with these tweets)
Because I preceded each of the 15 example tweets with its name and topic, I can even get it to generate tweets with the name and topic of my choosing. Here’s what it did with (Cthulhu, tentacle).
And here’s what it did with (Godzilla, stomp):
What is super impressive about this is how well it copied voice and cadence and then adapted them plausibly to entirely new topics. Why yes, I did do a quick test to see what a twitter bot of myself would look like.
Other programs I’ve worked with can do this to some extent, and I’m sure we’ll be seeing more vigorous testing of this one’s capabilities against some standard benchmarks. And not all of the API’s tweets are usable. For the dog_rates tweets, around 19/20 are reasonable, and the remaining 1/20 are way off-target or even contain drug references. For the janellecfakes tweets, it’s got a less consistent format to follow and maybe 1/3 are entertaining and 1/20 are plausibly something I might tweet. They’re not ready for unsupervised use. But I’m impressed with what I’ve seen so far, and will be building a lot of neat stuff with this.
Bonus material: more AI attempts to introduce and rate dogs. You can enter your email here, and I’ll send them to you.