WTF?! Technology’s invasive reach is expanding at an alarming rate. Two enterprising students have demonstrated how easily off-the-shelf programs can be used to create smart glasses capable of doxing strangers on the street. This unsettling project serves as a stark reminder of how thin the line is between privacy and surveillance – a line that is blurring even more in the age of smart devices and AI.
Two Harvard students, AnhPhu Nguyen and Caine Ardayfio, have created a controversial project called I-XRAY, which combines facial recognition technology with smart glasses that not only can identify people but also retrieve their personal information – including their home address, phone number, and family members – in real-time.
The project uses Meta’s commercially available Ray-Ban smart glasses and a custom-built system that can quickly look up a person’s face and provide identifying details on the user’s phone.
“To use it, you just put the glasses on, and then as you walk by people, the glasses will detect when somebody’s face is in frame,” Ardayfio told 404 Media. “After a few seconds, their personal information pops up on your phone.”
According to an accompanying Google Doc describing the project, I-XRAY relies on several existing technologies, including PimEyes, which is a facial recognition service used to look up people’s faces, and data brokers that collect personal information. Large language models (LLMs) are also used to infer personal details from scraped web content.
Essentially, they have used mostly off-the-shelf products to build a system that automatically doxes people in public spaces.
After a photo is snapped, the system automatically visits the PimEyes website and uploads the photo. PimEyes provides the URLs where the matching photos can be found and I-XRAY then opens those URLs, which can be anything from yearbook photos to profiles on an employer’s website. The LLM is able to determine the person’s name, job, and other personal details from those URLs.
Once it has the name, I-XRAY then taps a commercially accessible data broker, which provides home addresses, phone numbers and information about family relationships.
The creators showed off I-XRAY’s capabilities in a video posted on X and in a live demonstration for 404 Media. The system was able to identify classmates, strangers on the subway, and even pull up childhood photos of subjects.
Are we ready for a world where our data is exposed at a glance? @CaineArdayfio and I offer an answer to protect yourself here:https://t.co/LhxModhDpk pic.twitter.com/Oo35TxBNtD
– AnhPhu Nguyen (@AnhPhuNguyen1) September 30, 2024
Not surprisingly, the people doxed were shocked. As one subject told Ardayfio: “Dude, holy shit, this is the craziest thing I’ve ever seen. How do you know my mom’s phone number?”
The students claim their motivation was primarily curiosity and interest. However, they quickly realized the potential dangers. “Some dude could just find some girl’s home address on the train and just follow them home,” Nguyen said.
Indeed, one of the live demonstrations showed how easily the technology could be used for nefarious purposes. Ardayfio approached a woman he called Betsy armed with information he gleaned from the face match on his phone. He introduced himself claiming they had met through a specific foundation that had been included in the search results.
The woman’s surname was hidden in the video but 404 Media said it was easily able to identify her based on the information in the demo. The publication said it was unclear whether she consented to be used as a test subject.
The system is not foolproof, though. Sometimes it provided the wrong name, according to some of the responses in the demonstration.
Not surprisingly, the project raises significant privacy concerns.
When asked for comment, a Meta spokesperson emphasized that facial recognition technology could be used with any camera, not just their Ray-Ban smart glasses. They also pointed to their terms of service, which state users are responsible for using the glasses “in a safe, lawful, and respectful manner.”
PimEyes, the facial recognition service used in the project, expressed surprise at how their technology was being applied.
Nguyen and Ardayfio say they are not releasing the code for I-XRAY. They view their project as a way to raise awareness about the current state of technology and its potential misuse. “I think people could definitely take the idea and run with it,” Ardayfio said. “I would hope that awareness that we’ve spread on how to protect your data would outweigh any of the negative impacts this could have.”
