Can You Find This Human?

September 28, 2021

Today we travel to a future where your face is findable no matter where you go.

Guests:

Scott Myers — former editor of the Perplex City Wiki
Laura E. Hall — game designer and author of Planning Your Escape
Tom-Lucas Säger — technologist & Satoshi finder
Stan Alcorn — senior reporter at Reveal from the Center for Investigative Reporting
Erin — long lost camp friend
Dr. Kate Klonick — assistant professor of law at St. John’s University School of Law

Further Reading:

Here is the Billion to One card:

Here is the photo I used to find Erin:

Episode Sponsors:

BirdNote Daily: BirdNote Daily is a short, two-minute podcast that answers all of your burning bird-related questions — like, how do birds stay perched in a tree when they’re asleep? What’s the difference between a raven and crow? What’s the biggest bird that’s ever lived?
The World As You’ll Know It: a podcast about the forces shaping our future.
BetterHelp: Affordable, private online counseling. Anytime, anywhere. Flash Forward listeners: get 10% off your first month at betterhelp.com/flashforward
Shaker & Spoon: A subscription cocktail service that helps you learn how to make hand-crafted cocktails right at home. Get $20 off your first box at shakerandspoon.com/ffwd.
Tab for a Cause: A browser extension that lets you raise money for charity while doing your thing online. Whenever you open a new tab, you’ll see a beautiful photo and a small ad. Part of that ad money goes toward a charity of your choice! Join team Advice For And From The future by signing up at tabforacause.org/flashforward.
Tavour: Tavour is THE app for fans of beer, craft brews, and trying new and exciting labels. You sign up in the app and can choose the beers you’re interested in (including two new ones DAILY) adding to your own personalized crate. Use code: flashforward for $10 off after your first order of $25 or more.
Purple Carrot: Purple Carrot is THE plant-based subscription meal kit that makes it easy to cook irresistible meals to fuel your body. Each week, choose from an expansive and delicious menu of dinners, lunches, breakfasts, and snacks! Get $30 off your first box by going to www.purplecarrot.com and entering code FLASH at checkout today! Purple Carrot, the easiest way to eat more plants!

Flash Forward is hosted by Rose Eveleth and produced by Julia Llinas Goodman. The intro music is by Asura and the outro music is by Hussalonia. The episode art is by Mattie Lubchansky. Amanda McLoughlin and Multitude Productions handle our ad sales.

If you want to suggest a future we should take on, send us a note on Twitter, Facebook or by email at info@flashforwardpod.com. We love hearing your ideas! And if you think you’ve spotted one of the little references I’ve hidden in the episode, email us there too. If you’re right, I’ll send you something cool.

And if you want to support the show, there are a few ways you can do that too! Head to www.flashforwardpod.com/support for more about how to give. But if that’s not in the cards for you, you can head to iTunes and leave us a nice review or just tell your friends about us. Those things really do help.

That’s all for this future, come back next time and we’ll travel to a new one.

FULL TRANSCRIPT BELOW

transcripts provided by Emily White at The Wordary

▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹ ▹▹

FLASH FORWARD
S7E12 – “Can You Find This Human?”

[Flash Forward intro music – “Whispering Through” by Asura, an electronic, rhythm-heavy piece]

ROSE EVELETH:
Hello and welcome to Flash Forward! I’m Rose and I am your host. Flash Forward is a show about the future. Every episode we take on a specific possible, or not-so-possible, future scenario. And this episode we’re actually not going to start in the future like we normally do. Instead, we’re going to start in April of 2005.

SCOTT MYERS:
One of the challenges for me, whenever I’m explaining what Perplexity is or you know, what was it, how did it work, what do you do, is just trying to figure out where do I even begin talking about it? Because it had so many weird angles, you know.

ROSE:
This is Scott Myers, he’s a software developer. And in April of 2005, a new game that he had been waiting for was released. It was called Perplex City. Or, as he says it, ‘Perplexity’.

SCOTT:
You know, I didn’t think about that, actually. Yeah, I guess… I just… I pronounce it just ‘perplexity’. But it’s true. I guess… I mean, clearly it is Perplex City. It’s this city of, you know, perplexing stuff.

ROSE:
Perplex City was a long-term, alternate reality game.

SCOTT:
We take the idea that the world is your playground and we can do anything in it. And maybe, you know, instead of having our normal humdrum daily jobs where we go about doing boring things like boring people, we kind of get to be these secret agents, or we get to be covert operatives, or we get to be people who are just communicating with people in a different place, who know secret codes and we might be tapped for special operations at any moment.

ROSE:
Alternate reality games, or ARGs have been around for a while. The earliest versions existed before the internet and relied on communication through letters, payphones, and newspaper classifieds. In 2001, one of the first online ARGs, called The Beast, coined the phrase “This is not a game” – an idea that is usually central to these experiences.

Like many ARGs, Perplex City was big and complicated. It had all kinds of layers and ways to play. But the basic storyline was a mystery that players had to solve.

SCOTT:
The hook to Perplex City is that there’s this cube, and it was a really long time before we knew what kind of cube. But it’s a cube, and it turned out it’s… yeah, it’s a physical cube that was stolen. It was stolen from the academy and no one knew what the academy was. And how was it stolen? What’s this cube? And what is it, where is it? Ah, but where is it? That’s the question.

ROSE:
Andwhoever found the cube didn’t just get the satisfaction of winning the game. They would also win £100,000, which at the time, was close to $200,000.

SCOTT:
Oh by the way, it was actual money on the line. So that was exciting; that was a big draw.

ROSE:
There were a bunch of different ways you could play Perplex City. Sometimes there were live events put on by the game makers.

SCOTT:
Where we all get together and we just…you know, we just have some drinks and have a good time. And then someone drops a secret note that has a secret message, that has an encrypted thing. And suddenly the room, you know, goes crazy with, like, “Oh, my God, what does this mean? Oh, heavens.”

ROSE:
The game also posted regular updates online and sent out clues by email.

SCOTT:
So for me to play, what I would do is I would then, you know, go out just trying to digest it all, and make sense of it, and put it into this wiki to make sense of it. Kind of very similar to Wikipedia, right? I was just chronicling the world, just what was going on in this place.

ROSE:
Scott wound up becoming the editor of the Perplex City wiki, chronicling everything he could find about the game. And pretty quickly, players from all over the world started joining.

LAURA HALL:
At the time, I didn’t really think of myself as being a puzzle person. You know, when you’re with an art community, it’s people from tons of different and fascinating backgrounds and disciplines. And that was the part that was really intriguing to me, right? It’s the idea of all of these people coming together for the limited amount of time that the game is running to work toward a common goal.

ROSE:
This is Laura E. Hall, a game designer and the author of a recent book about escape rooms called Planning Your Escape.

LAURA:
Eventually, Perplex City got so big that it spawned its own separate message board forum, the Perplexorum.

ROSE:
So every time there was a scrap of new intel, Scott would diligently update the wiki and players would gather on the forum and discuss this latest clue. And along with the various events and updates online, there was another piece of the game.

LAURA:
Perplex City released puzzle cards in foil packs so you could go to a real-life store and buy them.

ROSE:
These cards also did contain clues that might help you find the cube.

SCOTT:
There was ultraviolet ink on a couple of cards that, you know, only if you have a blacklight, you can find the hidden message, that kind of thing.

LAURA:
There were 256 cards in total and they ranged in difficulty from red, easiest, all the way up to silver, which are the nearly impossible ones.

ROSE:
And when Laura says impossible, in some cases she really means impossible.

LAURA:
One of them is to solve the Riemann Hypothesis, which is like an impossible math problem, right?

SCOTT:
They’re numbered from 1… a very simple red card would be number 1, all the way up to number 256. Satoshi, by the way, is number 256. So, we expected it to be a very hard puzzle. We expected it to be a very challenging thing that people are going to take years to solve.

ROSE:
And the Satoshi card that Scott is talking about, that’s the card that got me interested in Perplex City. The card is actually called Billion to One.

LAURA:
On it, there’s a selfie of a man taken with a very distinctive European background. Down the side in Japanese writing it says, “Find me”. And then at the time, there was a hint line that was running for the game, and the hint for this card was, “My name is Satoshi”. So all you have is this man’s face, the location that he was in, and the name.

ROSE:
Now, Laura wasn’t initially super drawn to this particular card. There was a whole other game to play, a cube to find, and a reward to be won!

LAURA:
I mean, it was always an interesting challenge, I always thought it was cool. But for a while, you know, we were, as a community, just sort of looking at what was left to do. You know, there’s the ongoing game, the interactions with the characters, the story of their lives playing out. And then on the side is these puzzle cards, right? And you know, there was, sort of, a core group of people that were tackling all of the silver ones and trying to move that forward.

ROSE:
The official Perplex City game ended in 2007. A man named Andy Darley found the cube buried in a park in England and won the prize money. The team that created the game tried to do a second season but ended up canceling it before it even started.

But even after the game officially ended, there were still these silver cards that hadn’t been solved.

SCOTT:
For the longest time, there are only about four cards that were unsolved. So there’s the Riemann equation, which is actually unsolvable. There was this card called the 13th Labor, which is a reference to a Herculean feat. Hercules had the Twelve Labors of Hercules and we were proposing a 13th one, which was to basically decrypt a string of numbers. And that was incredibly difficult.

And this other one that basically… for a very long time was unsolved and it just amounted to a particular cipher using a particular deck of cards. It was just a very difficult puzzle that took a long time to solve.

ROSE:
Someone solved the deck of cards puzzle later in 2007. The game makers, sort of, soft retracted the Riemann Hypothesis card. And then someone finally solved the 13th Labor card, three years after the cube had been found. By 2010, the only silver card left to be solved was Billion to One.

SCOTT:
And it was just like, “Okay Satoshi, where are you, man?”

LAURA:
Somebody had started a website that had a premise of, there’s reward money and then everybody in the chain who helps actually locate this person will split that reward.

ROSE:
Now, Laura had some questions about this card. Did this guy consent to this? Did he know all of these people were looking for him? She didn’t want to work on something that was invading someone’s privacy, so she reached out to the organizers and asked.

LAURA:
And they confirmed that he consented to it and knew that people were looking. At the time, also, the way that cards were solved was that they would resolve to a keyword or a code of some kind that you would enter into a website. And so the idea for that card was that he knew the answer to a question that you had to ask him in person. So he carried that password with him the whole time.

ROSE:
And so Laura got involved and started coordinating the search, updating a website called FindSatoshi.com. That was in 2006.

LAURA:
I didn’t doubt that it was possible. I knew it would be a challenge. I did not think it would be a 14-year long challenge.

ROSE:
That’s right, it took 14 years to find this man. And the entire time, Laura never gave up. And the way they found him is fascinating, and a little bit terrifying. So here’s what happened.

The spirit of the card was the concept of six degrees of separation – the idea that all people are, on average, six or fewer social connections away from one another. And it was in some ways a test of early social networks. Could enough people share this image that someone who knew this person would be like “Oh! Hey! I know that guy!” And then they could contact him that way.

LAURA:
Yeah, so in the early days, really, the strategy was just to get it in front of as many eyes as possible, right? So that meant putting all of the info on the website, trying to translate that into different languages. I was on the news randomly talking about it. And yeah, we sent out press releases, really. It was a, sort of, traditional campaign to get the word out about something.

ROSE:
Laura did radio interviews, and YouTube interviews, and posted all over the place. She even visited the place in the background of the image on the card.

LAURA:
In the background of the picture, there’s this sort of storybook-looking village. It’s on a river, the photo’s taken from a bridge, and there’s these, like, very cute little houses. And I can’t remember exactly who found it. It was somebody on one of the forums, but they identified the background as being Kaysersberg in Alsace in France. And I actually… I mean, completely by coincidence, was in that area for a work trip. And my boss drove me over and we found the bridge that the photo was taken from. That’s the kind of stuff my mind still boggles a bit about. The coincidence of that happening. I didn’t travel a lot for that job, and so for this random meeting to take place there at that time is still pretty wild. [laughs]

ROSE:
And that was actually one of the more exciting events in the hunt for a while. Other than that… there was kind of nothing. No leads. Just… silence.

And that was how it was for years. Every so often some unsolved mystery show would come across the story and feature it. And Laura would always say yes, hoping that maybe this time someone would recognize Satoshi. So when she got a request early last year from a YouTube channel called Inside A Mind, Laura said yes.

[clip from YouTube:]

Host:
And you’re still looking for him to this day, correct?

Laura:
Yes. I don’t think that I will ever stop until this is done. I would love for there to be a community of people who are, like, dedicated to respectfully looking for this person.

Host:
If you ever feel like joining in the hunt to find this mysterious person, then you can do just one simple thing: Spread the word.

LAURA:
This was right at the beginning of the pandemic, of course, and lockdown for a lot of people. And so, I think it was sort of a perfect storm of getting it in front of a lot of people all at once and in a time where people perhaps had more time on their hands and more time at their computers.

TOM-LUCAS SÄGER:
I found a really interesting video about this story about this man who’s not been found for so many years. And of course, the story of Laura, who is looking for him all this time and stuff.

ROSE:
This is Tom-Lucas Säger, a technologist from Germany. He saw this video and he was like, “Oh, that’s interesting.” But then he actually forgot about it for a while. I mean, there are so many weird and cool things you come across online, right? You cannot investigate them all. He didn’t remember it until he went home for Christmas and his family started talking about this weird thing that happened to his sister.

TOM:
She received a letter from people she didn’t know, and she was opening it up, and it said that they found a message in a bottle that she threw in the water, like, 14 years ago. And now they… yeah, they found the letter again. So they did not find it… They found it 14 years ago when she threw it in the water, and then they forgot about it, and then they wrote her this message.

ROSE:
And hearing them talk about this, all he could think about was Satoshi.

TOM:
So I went home that evening and I thought, “Okay, I can give it a try.”

ROSE:
Now, a lot has changed since 2006, and so the first thing Tom tried wasn’t Facebook, or LinkedIn, or a press release. Instead, he went to a website called PimEyes, which is a facial recognition system. And he plugged in the picture of Satoshi.

TOM:
So I uploaded the picture to PimEyes and the third result was looking, kind of, okay. So I uploaded it to Reddit and said that I tried this, if anyone else tried it… I just wanted to start a conversation. I just wanted to get involved in this thing, and I was not really sure that it was him. It was just okay. The AI says it could be him, so yeah, okay.

ROSE:
In this picture, there’s a group of people, and on the end is a guy that does look a lot like Satoshi, holding a beer.

TOM:And then I later found out that someone in the company was called Satoshi.

ROSE:
But Tom actually didn’t think it was him.

TOM:
Yeah, okay, there’s someone called Satoshi, but it could not be him because he’s the executive of such a big corporation and he’s standing there, I think… I don’t know, in sweatpants. He was wearing some kind of sweatpants with a beer.

ROSE:
So the next question was, “Was this man, this particular Satoshi, ever in the place in the picture on the card?” What would bring this Japanese executive to Kaysersberg, France? The answer wound up being a half marathon.

TOM:
And we could take the starting number that was clearly visible and put it in. And the name of the person was Satoshi.

ROSE:
At this point, Tom was very confident that he had the right guy. So he emailed Satoshi’s company, and he also contacted Laura. Of course, Laura had already watched this whole thing go down on Reddit.

LAURA:
Somebody contacted me and said, “You should look at this thread.” So I headed over there and I was like, “Oh my God. This is it. They found this guy. They really found him.” So, I had a friend in Japan call the office to let them know what was going on, for one thing, but also to get his work email and make sure it was okay to get his work email. And then they helped translate a message into Japanese. We sent it in Japanese and English to ask, are you this person? And he wrote back and said yes.

ROSE:
And that was it. They had finally found Satoshi.

TOM:
And it was just too crazy because I… exactly then I knew that a lot of people would find this really cool and really interesting. But I was just in bed sitting there. So it was really weird because I was, like, a little bit famous in some weird kind of corner of the internet. But here, yeah, no one really cared.

ROSE:
Scott remembers hearing about the card finally being solved.

SCOTT:
And I didn’t recognize the name of the guy that found him. I was pretty sure he wasn’t, you know, one of the old-timers. He wasn’t anyone I knew. He wasn’t anyone that I recognized. So, immediately there’s this thought of, like, “Who in the heck is this?” You know, the more I read, the more surprised I am, honestly. It’s just more and more surprising the more I learn about it.

ROSE:
Now that they had found Satoshi, they could get the code that he was keeping and finish his puzzle. Or at least, in theory. It turns out, over the last 14 years, Satoshi had… understandably, forgotten the riddle he was supposed to give to whoever found him.

LAURA:
Because to him, you know, somebody just asked, “Hey, do you want to be on this card?” He’s like, “Sure.” And then that’s the last he’s heard about it for 14 years.

ROSE:
But the game organizers of course had it, so here is the clue that Satoshi, in theory, would have given to whoever found him: [riddle spoken in Japanese]

Which translates to this question: Who died after giving birth to flames?

In case you love riddles and you want to try and figure this out, I won’t give the answer just yet. Listen to the very end of the episode if you want to hear the solution to Satoshi’s riddle.

Now, there are two things that kind of blow my mind about this story. The first is that Laura spent 14 years working on this. Not actively all the time; it was always kind of in the background. But still! 14 years!

LAURA:
I really did always believe it was possible somehow. But I think probably around the 20-year mark I would have maybe given up.

ROSE:
And for many of those years, she actually could have found out the answer very easily.

LAURA:
So the other part of this story is that I actually ended up marrying one of the people who worked on that game. We met through this ARG.

ROSE:
That’s right, Laura literally shares a home and a marriage with a person who could have just… told her who Satoshi was. At any point!

LAURA:
Well, I made him promise not to, right? Because to me, it would be… well, it would be such a waste of effort to just be told. It’s like, you know, you want to scratch your own itch, right?

ROSE:
Do you ever hear something and you’re like, “Wow there are really just so many different kinds of people on this planet”? This was one of those moments for me because it is absolutely unfathomable to me that she didn’t just ask for the answer. For all those years!

TOM:
I think if my wife has a secret, she couldn’t hold it for me for like five minutes or stuff like that. And they just… day in and day out and… Yeah, I think this is really, really crazy.

ROSE:
But at the same time, it’s really sweet, right? I mean, what is love if not withholding the answer to a question from your partner at their request for 14 years so they can have the satisfaction of answering it for themselves?

LAURA:
There’s a sense of intrigue and mystery around alternate reality games and the, sort of… it’s a promise that there’s more to life, right? There’s an adventure just around the corner, or there’s this story layer over everything that you’ve seen every day and suddenly it becomes magical in some way. There’s a promise inherent in that. And I think that we both really appreciate and treasure that part of those games.

ROSE:
The second thing that really struck me about this story is that Laura spent 14 years trying to figure this out the “traditional” way. And it took Tom just a couple of minutes to find this guy using a public facial recognition system.

TOM:
The solution to the puzzle took me, like, in reality, maybe five seconds. It was just uploading the picture and that was basically it.

SCOTT:
So it’s interesting, really, that it was ultimately solved through very different means. That’s kind of cool to me, I think.

LAURA:
You know, the original point of the game was to do it through six degrees of separation, but ultimately was supposed to be about the power of the internet. Can you get this out in front of as many people as possible? And so, in a way, we’ve done that. Then it has a further, more contemporary element with the facial recognition part of it. So I’m glad in some ways that it was the power of the internet ultimately that led to it and not something random… you know, it was a deliberate search that came from having the exposure to the large YouTube audience.

ROSE:
But at the same time, I don’t know. There’s something a little… almost disappointing about this kind of solution. Because it wasn’t about connecting people, or sharing, or socializing, or trying to form a great human chain. It was solved in just a couple of minutes using a technology that, as we have talked about extensively on this show, can have a real dark side.

LAURA:
And yeah, it’s… I really do think that it’s a harbinger of privacy issues to come. The fact that it actually was possible to find this person after 14 years using a facial recognition search. The problems, of course, lie in somebody who doesn’t want to be found. This game was consensual.

ROSE:
Laura is a game designer. She now makes these kinds of games and experiences. And I asked her if she would ever create a puzzle like Billion to One today.

LAURA:
I don’t think I would make a puzzle like this today, and it’s mostly because, to achieve the scale that you do need, the saturation to make it work, to get it in front of as many people as possible, you’re moving outside of your core community of people who actually understand the deal. It’s a game. These are all real human beings and so on, right? The spread of it is where the problem lies.

And I think there have been a lot of examples of things going viral which ruin the thing that has gone viral. Fields of sunflowers where an influencer takes a picture and then suddenly they have to shut it down because all the flowers are being trampled, or things like the silver obelisk in the desert, right? And people came and dismantled it because the visitors who were coming to it were not respectful of the terrain and it was starting to degrade all of the stuff that made that thing so beautiful.

And so I think that is really the issue with a lot of viral stuff, you know. And anything… I wouldn’t make something that had the potential for that. I cherish the small community things that I have been able to experience because I can see how badly wrong it goes when things move outside of that community.

ROSE:
In the end, this was a fun game where everybody involved was respectful and knew what was going on. But this kind of thing can easily get… weird and kind of dark. Think about the true crime sleuths on Reddit, for example, who hound victims’ families for more information. Or the people who have died trying to find a supposedly buried treasure. And when you add facial recognition into the mix, you wind up with a kind of classic science fiction trope: this idea that in the future, you could pick out anybody in a crowd, run them through a system, and find out who they are.

And in fact, you might have already had some kind of experience with this sort of thing. If you’ve ever put photos into albums on Facebook, or Google, or Apple Photos, often the software will ask you to tag your friends’ faces so they can categorize who is in which photo. And sometimes the system picks up a face from the background. Someone you don’t know.

TOM:
And I didn’t like that because I liked my photo library to be really clean, and you have those faces, and then you had those, and I didn’t know how to handle them. I wanted to name all faces in the pictures, but I don’t even know who this is. And maybe if I have one picture from this person, it doesn’t make any sense.

ROSE:
But what if, in the future, the system could actually tell you who that person was? What if you could find out just how many photos you’re in the background of; staring at a castle, or on the beach, or at Disneyland? This is a question I’ve actually kind of been obsessed with for a long time, it’s a puzzle I sometimes try and think about and guesstimate. How many photos am I in the background of, unknowingly?

I used to try and guess this by thinking about how many tourist destinations I’d been to and try and guess how many pictures I took, how many people were in the background of my photos, so they probably took about the same number of photos as me… But now that method doesn’t really work because people take photos everywhere. On the street, at restaurants, in coffee shops, in bars. That’s not a bad thing, it’s just reality. Which means that you are probably in a LOT of photos. We all are.

But can we be found in those photos?

This is what Julia and I wanted to find out, because it’s possible that Tom just got really, really lucky in finding Satoshi. So when we come back, we are going to try and find some people in the background of our own photos.

ROSE (on call):
How do I explain? How do I explain that I found you via facial recognition without making her run away screaming? Um…

More on that, when we return.

ADVERTISEMENT: BETTERHELP

Today’s episode is sponsored by BetterHelp.

Look, let’s be totally honest, you and me, the world is kind of terrible right now for a lot of people. And I am a firm believer that, for most people, some form of therapy is a good idea, even if you don’t think you need it.

But finding a therapist that is available, and affordable, and a good fit can be really, really hard. So if you’ve never tried therapy because you’re freaked out by how hard it seems to get on the phone and find someone, or if you’ve tried it but you’ve never really found a good match, BetterHelp could be a good solution for you.

BetterHelp is professional therapy done securely online. When you sign up, they will assess your needs and match you with your own licensed, professional therapist who you can start communicating with in under 48 hours. There’s a broad range of expertise available, and you can use BetterHelp from anywhere in the world. Just log into your account anytime and send a message to your therapist. You’ll get timely and thoughtful responses, plus you can schedule weekly video or phone sessions. And BetterHelp is committed to facilitating great therapeutic matches, so they make it easy and free to change therapists if you need to.

Visit BetterHelp.com/Flash Forward and join the over two million people who have taken charge of their mental health with the help of an experienced professional. In fact, so many people have been using BetterHelp that they are recruiting additional therapists in all 50 states. And BetterHelp is offering a special deal for Flash Forward listeners. Get 10% off your first month at BetterHelp.com/Flash Forward.

ADVERTISEMENT END

ADVERTISEMENT: BIRDNOTE

Today’s episode is supported in part by BirdNote. BirdNote Daily is a short, two-minute podcast that answers all kinds of your burning bird-related questions like: How do birds stay perched in a tree when they are asleep? What is the difference between a raven and a crow? What is the biggest bird that’s ever lived?

Right now, they’re exploring the wonders of bird migration, and let me tell you, some of this stuff is absolutely wild. For example, some songbirds will consume their own organs for energy on long flights. Indigo buntings have proteins in their eyes that let them see magnetic fields. The veery, which is a kind of thrush, predicts hurricanes, sometimes better than computer models can.

Hear all about it on BirdNote Daily. Get it wherever you get your podcasts or at BirdNote.org.

ADVERTISEMENT END

ROSE:
Okay, so how well does this stuff really work? To test out this facial recognition system, PimEyes, Julia and I both started with photos of ourselves.

[commentary from PimEyes search]

Julia: I accept the terms of service… Oh boy. Okay. I’m not going to read the privacy policy at this moment, but I am going to open that in a new window because I feel like we might want to look at that later.

Rose: Okay, it found me in a bunch of places. It found my headshot… Actually, that’s funny. It did find my headshot on a lot of sites because that’s the one I use for professional purposes, which maybe I should have thought of before I used it as this one.

Julia: Okay, so the photo that I uploaded is one that I use for, like, bio photos a lot. So that came up on the NPR website.

Rose: It also found me at a couple of conferences, at On Air Fest. It did find a bunch of pictures of memes that were made of my face when the GamerGate stuff was happening.

Julia: The most disturbing one is… there’s this photo of me at a protest that I honestly didn’t… I didn’t know that this photo of me existed. I’ve never seen this photo of me before.

So, the system found our faces and found our faces in some places that we didn’t even realize that it would. This is one reason why, often, people who organize protests suggest for participants to not take identifying photos of other people’s faces. And in fact, this is a question that photojournalists are grappling with as well. How do you document a protest without potentially putting an attendee at risk of being identified and harassed? Facial recognition has also been used to find and out white supremacists who attend racist events.

But Julia and I wanted to do more than just find ourselves. The real goal here was to try and find people we didn’t know. So we decided to challenge each other with some pictures. We both uploaded some photos of us in various tourist destinations, like the Liberty Bell, Disneyland, Canyonlands National Park.

We also both uploaded a few photos of people who we’ve lost touch with. Julia went on a cruise with their family about ten years ago and met some friends who they have photos of but haven’t been in touch with. I uploaded some pictures of people I met in college, or on a trip to Tokyo, or a picture of my ex-boyfriend hanging out with one of his friends that I actually really liked but lost touch with. And then we both set off to see if we could find the other person’s faces.

[commentary from PimEyes search]

Julia: I’m going to try looking for a couple of the ones from Disneyland to see if any of the people in these photos come up. It’s not… it does recognize them as faces, they’re not super clear photos.

Rose: Let’s see what we find here…

Julia: Yeah, okay. This one… There are only low score results. That’s not promising.

Rose: So it doesn’t give us any faces from the background for that one.

Julia: Okay, so it looks like it only recognizes the faces of the people you were there with. Robert definitely comes up.

Rose: I’m now doing a photo from, it looks like, a national park in the Southwest. It did not see any faces in that photo. Okay.

Julia: But I wonder if I just try, like, cropping it so that you can only see the other faces. Maybe that would be better.

Rose: So it doesn’t like that. It doesn’t like that. Okay, let’s see. Those are children. I don’t know if I want to do it on kids. None of these people are looking at the camera. Hmm.

So… maybe the good news is that we did not manage to locate or identify anybody in the background of our photos.

Julia: I was kind of surprised by, like, how bad it was at finding certain people also. Like, you know, it makes sense that for us, because we both have, like, author photos on websites, and for me, that’s basically all that came up. I was actually surprised by how few photos came up of me other than those two weird ones that I had never seen before. But for some people… And I mean, I think partly it could be because, like, the photo was not as good or it was someone in the background.

But we did actually find other surprising and somewhat nerve-racking things.

Rose: Okay, let’s try this photo. Okay, this person does look… this does look reasonably correct. Oh… This is an adult cam. Yeah, this is an adult cam person, which is fine, just maybe not someone we want to out.

So for at least one of the people we searched for, their online sex work profile came up. Which, to be very clear, is totally fine – sex work is work, and they should do whatever they please. But it was a really clear reminder that these kinds of facial recognition sites can pull up stuff about someone that they maybe don’t want the entire world to know about.

And we did have some success matching names to faces for people who weren’t in the background of these photos. Well, I should say, Julia mostly had success. They did way better than I did at this little exercise. By the way, we’re going to bleep out the people’s names and identifying information that we find, because the point here is not to actually tell you who they were, it’s to show you how hard, or easy, it was to find them.

[commentary from PimEyes search]

Julia: Okay, wait. I had another question, though, which is so you have this photo called [censored]. Do you know who those people are already?

Rose: I know who one of those people is. Did you find them?

Julia: I found one of the people, I think, because she has a blog called [censored].

Rose: That is correct. That is her.

Julia: Amazing. I just wanted to be right, really, but…

Rose: Yes. [laughs] You got it. You got that one for sure. Yeah.

Julia: Amazing. I really thought that I would be able to find something for this person who was, like, your ex’s roommate. But I don’t think I found who they are. But they do have a doppelganger who’s, like, a musician.

For some reason, their photo was on this other contemporary Czech folk artist’s page. Oh, and then… Okay, it was similar artists. Okay, [censored]?

Rose: That’s right! That’s him!

Julia: Oh that’s him? Oh, okay. Yeah, then I found him. I totally found him. Amazing.

Rose: That’s totally him. He plays the banjo, yeah.

Julie: Oh my god. Okay, this was one that I was least sure about. But so… yeah I found, like, his Tumblr, his SoundCloud. You can totally get in contact with him if you want to.

Julia also found something interesting about a person in my Disneyland photos.

Julia: Your friend that you went to Disneyland with, I did find a really funny, like, weird photo of them at, like, a 2015 Brooklyn music festival that’s just, like… he just happens to be in the photo. And I was like, “If you want to freak him out, you could just be like, ‘Hey, what were you doing at the Northside Music Festival in 2015?’” and see how he reacts. [laughs]

Rose: Oh yeah. I do want to freak him out.

STAN ALCORN (mono):
Okay, let’s see what Rose sent me. “Have you ever seen this photograph of yourself?” Huh… That’s weird. I don’t… I don’t recognize this at all.

ROSE:
This is my friend Stan Alcorn. He’s a reporter with the investigative podcast Reveal.

STAN:
I’m in the front row at some sort of concert, I guess. What shirt am I wearing there? Do we recognize any other people? Oh, you think it’s a deepfake? It’s weird; I’m in a concert with, like, an old guy next to me. I’m in the front row, but I’m, like, holding a beer. Yeah, I don’t… I do not recognize this scenario. I don’t think I’ve ever seen this photo.

ROSE:
So, it is very funny to me that Stan immediately thought that I had made a deepfake of him, which suggests that he knows me very well. But this time, in fact, it was not quite so devious.

STAN:
Like, I zoomed in on my fingers because I was like, “Are those really my fingers?” But it’s not quite sharp enough to tell, like, do I have long nails or not? My hand looks a little weird, but I don’t know, maybe it’s just that hands look weird when you zoom in

ROSE:
Here’s what he said when I revealed to him that this was not, in fact, a deep fake. It was instead somehow both less and more creepy than that.

STAN:
Oh, my god! It was from when I saw The Very Best with Andy. Yeah, the Northside Festival in Williamsburg. Whoa… So it just used a picture of my face to find this on the internet?

ROSE:
But the thing that I think surprised both Julia and I the most was what happened when we looked for a picture of me, as a child. When I was in middle school, I went to this summer camp. And at that summer camp, I had a couple of friends who I’d always see there. And for a couple of years, there was this one friend in particular who I would always be really excited to see. And then at some point, she stopped going to camp. This was before kids had smartphones or emails, and our parents hadn’t thought to trade information… so that was that. I never saw her again. But I have this photo of us from camp. We’re probably, like, 12 or 13 in this picture. So I uploaded it first just to see if it would recognize my face when I was younger.

[commentary from PimEyes search]

Rose: Let’s see what happens if I do… that. [gasp] It 100% percent found me. That’s actually really terrifying.

Okay, yep, that’s me, that’s me. That’s what I look like. It, uh, definitely found me, so that’s… cool.

And then I looked for the other face in the picture.

Rose: [gasp][censored]. PPC? What is a PPC expert? [Censored] has specialized in paid search for five-plus years. She was voted number four most influential PPC experts of 2016. I don’t even know what PPC is! I think this is this person. There she is on Facebook.

This is definitely her. Okay, this is sort of stressful because now I’m like, “Do I want to reach out to her or is she going to think this is fucking creepy?” She works at [censored], though. Maybe she won’t think it’s creepy. She’s on Twitter. She tweets. She tweets! [laughs] Oh no.

After going back and forth with myself for a while, I decided to send her a message on LinkedIn.

Rose (typing): Hi, [censored]. This is sort of weird, [laughs] but did you go to summer camp at [censored] as a kid? I think maybe we were friends at camp.

I am… how do I explain? [laughs] How do I explain that I found you via facial recognition without making her run away screaming? Um, and that I want to talk to her on the phone.

Send. [gasp] Oh god. I just sent her a message.

[ringing]

ROSE (on call):
Hi, is this Erin?

ERIN:
It is!

ROSE:
Hello, it’s Rose.

ERIN:
Nice to talk to you after so long.

ROSE:
Yeah. I am glad that you were not scared away by my slightly weird messages.

ERIN:
I get a lot of weird LinkedIn messages, so I’m like, “Oh, this is actually someone I know. I’ll take it.”

ROSE:
Let me… I should show you this picture.

ERIN:
Oh my god. We’re so young!

ROSE:
Isn’t that wild?

ROSE:
What do you remember about this summer camp?

ERIN:
I was thinking about that last night. There’s a couple of things I remember. I am from a really small town in the mountains, and I think I learned about Blink 182 this summer. And so, I remember, like, listening to Blink-182 nonstop, talking about them. It was, like, old for everyone else and new for me. So I remember that. And I also remember there was a big dance. Was there a big dance towards the end? And I remember, like, all year I was thinking about what I would wear to that dance. And I’m sure… I don’t think I had any prospects. [laughs] But it was like… It was like the highlight of my summer, was the dance.

ROSE:
Yeah. Yeah, I mean it was, like, specifically nerd camp. I think we were, like, all not very cool.

ERIN:
No. [laughs] Yeah. And the cicadas! Do you remember one year, there were cicadas and we would save all our money and buy that weird juice with caffeine? Surge? Remember that?

ROSE:
I do. I didn’t remember it until you just said it, so I’m having a memory happening actively. [laughs]

I do remember… Now that, like, I was thinking about this, I was like, I feel like I remember being like, “Oh, I wonder if Erin will be at camp this year,” because there was no way for us to know if the other person was going to be there until you showed up.

ERIN:
And how many years… I thought we went for, like, three or four years. And then in high school… Actually, in high school, I went to boarding school so my parents stopped sending me to camp. They were like, “You’re not home very much anymore. If you’re not going to be home during the year, then you should be home during the summer.” So, camp ended for me.

ROSE:
Gotcha. I’m trying to remember… Because I remember my mom being like, “Oh, maybe we should have, like, exchanged contact information with these people.” And I was like, “Oh yeah. That probably would have been a good idea.” We didn’t do that! So it was like, “Well, now…” Who knew that, like, what, 13 years later, however many years later, I would find you official recognition!

ERIN:
That was not on your mom’s radar back then.

ROSE:
I don’t think it’s on my mom’s radar now, frankly.

ERIN:
Actually, I have to tell my parents about this conversation. I think they will be fascinated.

ROSE:
Yeah, it is very weird to me that the system was like… It wasn’t even like, “Oh, maybe it’s this person.” It was like, “I found this person in, like, seven seconds. Here she is. This is her name. This is where she works now.” I was like, “Oh god, I’m creepy.” How did you feel about that? Hearing that it was facial recognition?

ERIN:
You know, not creeped out, which is maybe concerning. Like, when you said it, I was like, “Oh, this is logical.” My phone looks at my face every day, no matter what expression I’m making, and recognizes me. I use, like, face search in Google photos all the time and it finds me. So it just felt, like, logical. Like, “Oh, okay.”

All right, I’m going to call my parents now and tell them it’s all good.

ROSE:
Amazing, amazing. All right. Well, thank you so much. I’m excited we got to talk and that you weren’t just like, “Uh, delete message from crazy lady.” [laughs]

ERIN:
No. It was such a pleasure. I hope we hang out soon. Thanks for reaching out to me.

ROSE:
Sweet. Thank you. Have a good rest of your day.

ROSE (mono):
So Erin and I did exchange phone numbers and we are going to try and meet up in person some time. I will post the photo that I used to find her on the Flash Forward page and social media as well so you all can see it.

[commentary from PimEyes search]

Rose: Do you feel like this made you feel more or less nervous about some of this technology?

Julia: I think it made me less nervous overall about this technology because, honestly, it didn’t really work that well. It was… like, the weirdest thing was seeing photos of myself that I didn’t know existed, which is really, like, an older problem than the search engine existing. It’s just, kind of like, being in public, sometimes you’re going to be in a random photobomb, or a news crew took a photo of you that you didn’t know about, which is just sort of weird. But it’s like, yeah, that feels like an older problem, really, than the face searching thing.

Rose: Yeah. It is interesting because, like, in some ways, it’s comforting to know that it’s not like magic, right? It’s not like any photo that you’re in the background of, like, any creep like us can just be like, “Hmm, I wonder what that guy’s name is,” you know? But at the same time, I’m disappointed. [laughs]

Julia: Right.

ROSE:
So yeah, this is not magic. We are not yet in the, sort of, sci-fi Minority Report, “enhance” kind of situation where you can be located in ANY picture that anybody has taken that you happen to be in. And I don’t want to overstate how good this technology is right now. There is a tendency, and I think I do it on this show too sometimes, to give a little too much credit to the power of technology in the service of trying to scare people into caring about privacy. But at the same time, we were able to find images of ourselves, and other people, and identify them from even childhood photos. And that’s… creepy enough, probably?

So where does that leave us? And what does this mean for the future? Should we ban this technology? How do you regulate something like this? And what does the future of privacy look like?

Can You Find This Human?

What Is The Future Of Gender?

What If We Could Shrink Technology?

You may also like