The House That

Spied on Me
The reason I smartened up my house was to find out
whether it would betray me

Kashmir

In December, I converted my one-bedroom apartment in San

Francisco into a “smart home.” I connected as many of my
appliances and belongings as I could to the internet: an Amazon
Echo, my lights, my coffee maker, my baby monitor, my kid’s
toys, my vacuum, my TV, my toothbrush, a photo frame, a sex
toy, and even my bed.

“Our bed?” asked my husband, aghast. “What can it tell us?”

“Our breathing rate, heart rate, how often we toss and turn, and
then it will give us a sleep report each morning,” I explained.

“Sounds creepy,” he said, as he plopped down on that bed, not

bothered enough to relax instead on our non-internet-connected
couch.

I soon discovered that the only thing worse than getting a bad
night’s sleep is to subsequently get a report from my bed telling
me I got a low score and “missed my sleep goal.” Thanks, smart
bed, but I know that already. I feel like shit.

Why? Why would I do this? For convenience? Perhaps. It was

appealing to imagine living like the Beast in the Disney movie,
with animated objects around my home taking care of my every
need and occasionally serenading me. As a result of the
apartment upgrade, I could watch what was happening in the
house when we weren’t there. I could use voice commands to
turn on the lights, coffee maker, and music. I could exchange
voice messages with our toddler (and her caregiver) through a
toy. I got reminders from my toothbrush to brush and tips on
how best to do it. If I got cold in the night, my bed could warm
me up. And I no longer had to push a vacuum around the house,
instead activating a robot to do it for me with a press of a
smartphone button.

Thanks to the Internet of Things, I could live in my very own

tech-mediated Downton Abbey. That’s the appeal of smart homes
for most people, and why they are supposed to be a $27 billion
market by 2021. But that wasn’t my primary motivation. The
reason I smartened up my house was to find out whether it would
betray me.

I installed internet-connected devices to serve me, but by making

the otherwise inanimate objects of my home “smart” and giving
them internet-connected “brains,” I was also giving them the
ability to gather information about my home and the people in it.
The company that sold me my internet-connected vacuum, for
example, recently said that it collects a “rich map of the home”
and plans to one day share it with Apple, Amazon, or Alphabet,
the three companies that hope to dominate the smart home
market. Once I made my home smart, what would it learn and
whom would it tell?

One person I knew it would be leaking to was my colleague,

Surya Mattu, because he built a special router to monitor the
devices monitoring me.

Surya
Yes, I am basically Kashmir’s sentient home. Kashmir wanted to
know what it would be like to live in a smart home and I wanted
to find out what the digital emissions from that home would
reveal about her. Cybersecurity wasn’t my focus. (I wasn’t
interested in hacking her sex toy or any of her other belongings.)
Privacy was. What could I tell about the patterns of her and her
family’s life by passively gathering the data trails from her
belongings? How often were the devices talking? Could I tell
what the people inside were doing on an hourly basis based on
what I saw?

Using a Raspberry Pi computer, I built a router with a Wi-Fi

network called “iotea” (I’m not very good at naming things) to
which Kashmir connected all of her devices, so that I could
capture the smart home’s network activity. In other words, I
could see every time the devices were talking to servers outside
the home.

I had the same view of Kashmir’s house that her Internet Service
Provider (ISP) has. After Congress voted last year to allow ISPs
to spy on and sell their customers’ internet usage data, we were
all warned that the ISPs could now sell our browsing activity, or
records of what we do on our computers and smartphones. But in
fact, they have access to more than that. If you have any smart
devices in your home—a TV that connects to the internet, an
Echo, a Withings scale—your ISP can see and sell information
about that activity too. With my “iotea” router I was seeing the
information about Kashmir and her family that Comcast, her ISP,
could monitor and sell.

There was a lot to see. Since the router was set up at the
beginning of December, there hasn’t been a single hour of
complete silence from it, even when there was no one in the
house.

After a week of living in my newly smartened home, I could tell

why the Beast was always in such a bad mood: The animate
objects in my home were becoming a constant source of
annoyance. I thought this was going to be a story about privacy,
but instead I was finding out how infuriating it is to live in a
janky smart home.

Our 1970s apartment building did not offer enough electrical

outlets for this 2018 smart home, so we had power strips
and outlet expanders everywhere, to the point where I was
worried I was going to spark a fire and burn our smart home
down. (This actually might have been cathartic.)

I had to download 14 different apps to my phone to control

everything, which meant creating an account for each one of
those apps. (Yes, my coffeemaker has a log-in and a very long
terms of service agreement.) After setting them up, I thought I’d
be able to control all the devices by issuing voice commands to
Alexa via the Echo—the smart speaker that we’ve been using for
the last year as a glorified timer and music player— but this did
not go as well as I had hoped.

The fantasy of the smart home is that it will save us time and effort.

It took at least two hours to get all of our Christmas lights

plugged into smart plugs from WeMo and Sonoff, and then to get
those plugs online with their apps, and then to get those apps to
talk to the Alexa app. The first night I said, “Alexa, turn on the
Christmas lights,” they all turned on in sparkly synchronicity and
it was magical. But one day, Alexa stopped recognizing
“Christmas lights” as a group, and I could not figure out how to
fix it, so I had to ask Alexa each night to turn off the lights one-
by-one. (“Turn off kitchen Christmas lights.” “Turn off living
room Christmas lights.” “Turn off bookcase lights.”) This was
way more annoying than turning them off manually. The fantasy
of the smart home is that it will save us time and effort, but the
friction involved in getting various devices from different
companies to work together meant that many things took longer
to do.

I could tell when the lights were being turned on and off. And
even when Kashmir’s family wasn’t using them or weren’t home,
those smart plugs were constantly talking to their home servers.
One of them checked in erratically, around four times per day at
random intervals. The other smart plug, that promised insights
about how much electricity you were using, was far more chatty,
pinging home almost every hour.

Smart coffee was also a world of hell. The Brewgenie Smart

Coffeemaker I ordered first was “smart” in that it had Bluetooth
connectivity, so I could use its custom app on my phone to make
it run a pot of coffee, but it wasn’t compatible with Echo, so I
couldn’t say, “Alexa, make me coffee.” To remedy this, I ordered
the Behmor Connected Customizable Temperature Control
Coffee Maker which promised Alexa compatibility. I came to
regret this while setting up my Eight Sleep Tracker—a sensor
layer that went over my mattress that could track sleep, warm
the bed, and connect to “any wifi-enabled device in your house.”
Its instruction manual, along with dire warnings about possibly
setting my bed on fire, informed me that it could automatically
brew me coffee when it sensed I was waking up ... but only if I
had a WeMo coffeemaker.

I refused to order a third coffee maker for my smart home.

I made do with the Behmor, which makes incredible coffee but
had a prickly relationship with Alexa. Each morning, my husband
and I begged Alexa to put the coffee on. She would only react to
a very specific phrasing of the request and then would only do so
sometimes.

“Ask Behmor (pronunciation: Be-more) to brew me coffee.”

“Behmor,” she would respond. “A passion for coffee. How can I

help you?”

“Brew me coffee.”

“I don’t understand,” she would respond. This was especially

aggravating for two caffeine-addicted people who had not yet
had their coffee. Sometimes we would keep rephrasing the
question until she got it, but more often, one of us would just get
up, walk to the kitchen and press the button on the coffeemaker
rather than doing it the “smart” way.

And that’s when I ran into the next issue: the Withings Home Wi-
Fi Security Camera with Air Quality Sensors that I had set up in
our living room. When the camera detects motion or noise, it
automatically records what it’s seeing. That’s great if you’re
worried about break-ins or how people treat your kid when
you’re not around, but not great for protecting the intimacy of
your home. The day after I set it up, it caught me walking
through the living room naked, resulting in the very first nude
video of me (that I know about), which was promptly sent to the
cloud and saved to the Home Cam app on my phone. This
appears to be a common problem for the smart home set.

The camera was constantly sending huge amounts of data, which

makes sense given it’s sending video. But good news—it was all
encrypted, so someone monitoring your network will not get
access to what the camera sees, including nude videos.

“You set up a camera!??” my husband asked, horrified. Four

clicks deep into the app, I found out it keeps everything it
records by default for two days but that you have to sign up for a
“premium account” if you want to save anything longer.

“It deletes the video after two days,” I told my husband.

“How comforting,” he replied, without looking up from his

phone, his voice doing the eye roll for him.

When I went away to Spain for a few days for work in December,
I was looking forward to being able to check in on my family via
the camera. But eight hours after I left, during my layover in
Toronto, I opened the app on my phone and it reported back that
the camera was offline; the most recent video was from that
morning, showing my daughter in my husband’s arms in the
kitchen.

“The camera’s not working,” I texted my husband.

He replied that he had unplugged it.

“It was staring at me while I made coffee,” he texted back.

I told him I needed it plugged in for the story to monitor the data
flows and he said he would do it. But as the days passed in Spain
it remained offline and I stopped bothering him about it. (The
acceptance of complete surveillance had not been part of our
marriage vows.)

When I got back from Europe, I moved the camera to the

nursery. It would be more useful there, and thankfully our 1-
year-old daughter is too young to care about privacy invasions.
The downside of this was that the move somehow screwed up the
camera’s power cord, so it started randomly losing its connection
to the router; when that happened, the camera would glow
bright orange to inform me that it was offline. This is not a great
design for a camera billed as a baby monitor, as the bright
orange light in the otherwise dark nursery would then wake my
daughter.

The whole episode reinforced something that was already

bothering me: Getting a smart home means that everyone who
lives or comes inside it is part of your personal panopticon,
something which may not be obvious to them because they don’t
expect everyday objects to have spying abilities. One of the
gadgets—the Eight Sleep Tracker—seemed aware of this, and as
a privacy-protective gesture, required the email address of the
person I sleep with to request his permission to show me sleep
reports from his side of the bed. But it’s weird to tell a gadget
who you are having sex with as a way to protect privacy,
especially when that gadget is monitoring the noise levels in your
bedroom.

The Eight Sleep tracker sent its data through a nonstandard port
that I wasn’t monitoring, so I wasn’t able track what was
happening in the bedroom.

One of the decisions we made early in the process was not to

break the encryption on the devices in Kashmir’s house. We
wanted them in their normal, off-the-shelf state. We’ll leave the
encryption-breaking to the growing number of computer-science
academics who are building their own smart homes to try to
figure out the future of corporate surveillance.
When the data streams were unencrypted, which was the case
every time someone watched Hulu on the Vizio smart TV, I could
see exactly what was being sent. When they were encrypted, as
the majority of the data turned out to be, I could see only the
metadata—the volume of data being sent and to where, which is
like seeing the outside of an envelope but not being able to read
the letter inside. But sometimes, metadata is the message. I
know, for example, when the family wakes up, because the
Amazon Echo usually starts playing songs from Spotify between
6 a.m. and 8 a.m., even if I don’t know which songs. I also know
that Kashmir likes to use the Alexa Sounds app—which loops
ambient sounds such as rain, oceans, and fireplaces—between 6
p.m. and 8 p.m., which is when she puts her 1-year-old daughter
to sleep.

Getting a smart home means that everyone who lives or comes inside it is part of your
personal panopticon.

It turns out that how we interact with our computers and

smartphones is very valuable information, both to intelligence
agencies and the advertising industry. What websites do I visit?
How long do I actually spend reading an article? How long do I
spend on Instagram? What do I use maps for? The data packets
that help answer these questions are the basic unit of the data
economy, and many more of them will be sent by people living in
a smart home.

The thing in the smart home that most fascinated me, because of
its value to advertisers, was the television. In Kashmir’s house, it
doesn’t get turned on every day, but when it does, it’s usually
between 8pm and midnight. A typical day of TV viewing looks
like this in the data:
An outlier to this trend was Christmas Day, when, it looks like,
the television was on throughout the day. I couldn’t tell if it was
for watching a show or streaming music, but there was a
consistent stream of activity.

I didn’t like it, but the TV was tuned to basketball for most of the
day on Christmas. Thanks for the reminder, Surya/smart home!

Similarly, it looks like 2017 ended with a low-key TV night:

Okay, yes, we stayed in on New Year’s Eve. Don’t shame me. We

have a baby! We watched Phantom Thread on DVD. There was
no internet involved. Why do you even know that?

Even though the “smart” part of the TV wasn’t being used that
night, it was still sending data about its use.

When the TV is on, it’s usually tuned to Netflix or Hulu. I couldn’t

see what they watched on Netflix because Netflix encrypts
streams. But I discovered that Netflix doesn’t encrypt images, so
I could see the shows being recommended to them, which is
revealing in that it shows what Netflix thinks they should like:

Meanwhile, Hulu sends its traffic unencrypted, so I could spy on

exactly what they watched, like when Netflix spied on A
Christmas Prince addicts. I was able to figure out which nights
they were binge-watching Difficult People on Hulu (which, based
on the traffic, seems to be one of their favorite shows).

I wasn’t the only one watching what they were watching. Their
TV is telling Scorecard Research, a digital behavior tracker, and
Rewardtv.com, a website owned by Nielsen, what they stream on
Hulu.
Here’s what the reward TV request looked like; it included the
name of the episode in plain text:

“Surya wants to know who is binging on Difficult People,” I told

my husband, the responsible party.

“Our TV is watching us?” he asked, surprised, even though he’d

been the one to connect the TV to the router surveilling us.
“Wow, I forgot.”

This must be what it’s like to be in a documentary or in a reality

TV show. The cameras eventually move to the periphery of your
vision and then disappear altogether. If homes become sentient,
and it becomes the norm that activity in them is captured,
measured, and used to profile us, all of the anxiety you currently
feel about being tracked online is going to move into your living
room.

Talking to the human who actually got to see and analyze my

smart home’s activity made me realize just how deeply
uncomfortable it is to have that data pooled somewhere.

After two months of data collection, I was able to pick up a bunch

of insights into the Hill household—what time they wake up,
when they turn their lights on and off, when their child wakes up
and falls asleep—but the weirdest one for me personally was
knowing when Kashmir brushes her teeth. Her Philips Sonicare
Connected toothbrush notifies the app when it’s being used,
sending a distinctive digital fingerprint to the router. While not
necessarily the most sensitive information, it made me imagine
the next iteration of insurance incentives: Use a smart
toothbrush and get dental insurance at a discount!
The larger pattern that emerged about the smart home was that
all of the devices phoned home daily, even if they hadn’t been
used, telling the companies that made them, “Hey. I’m still here.
I’ve still got power. Have any updates for me?”

An exaggerated version of this was seen in the Echo and Echo

Dot, which were in constant communication with Amazon’s
servers, sending a request every couple of minutes
to http://spectrum.s3.amazonaws.com/kindle-wifi/wifistub-
echo.html. Even without the “Alexa” wake word, and even when
the microphone is turned off, the Echo is frequently checking in
with Amazon, confirming it is online and looking for updates.
Amazon did not respond to an inquiry about why the Echo talks
to Amazon’s servers so much more frequently than other
connected devices.

The funniest “conversation” that happened over the two months

was a week in January when Kashmir was out of town. I could tell
the house was empty because the amount of data being sent out
slowed, but her home remained active despite being empty. All of
her devices, from her TV to her WeMo smart plugs, continued to
send out information every day. But the Behmor Connected
coffee machine seemed to miss its inhabitants because it
completely freaked out. The coffee machine, which typically
pings its servers a few times a day, phoned home over 2,000
times on Thursday, January 24th.

A comparison in data:

When we asked Behmor about it, they had to check with Dado
Labs, the third-party that makes their devices smart. “We pretty
much stay at arm’s length. We don’t even have our customers’
email addresses for log-in,” said Joe Behm from Behmor. Dado
Labs reported that a server was down that day, meaning the
coffee machine just kept calling and calling a line that wouldn’t
pick up.

Overall, my takeaway is that the smart home is going to create a

new stream of information about our daily lives that will be used
to further profile and target us. The number of devices alone that
are detected chattering away will be used to determine our
socioeconomic status. Our homes could become like internet
browsers, with unique digital fingerprints, that will be mined for
profit just like our daily Web surfing is. If you have a smart home,
it’s open house on your data.

I was looking forward to the end of the experiment and getting

rid of all the internet-connected devices I’d accumulated, as well
as freeing up the many electrical outlets they’d been hogging.
The Internet of Shit Twitter account is right. Smart homes are
dumb.

But the truth is that my house will remain smart, just like yours
may be. Almost every TV on the market now is connected—
because otherwise how do you Netflix and chill?—and over 25
million smart speakers were sold last year alone, with Apple soon
to release its version, the HomePod, meaning a good percentage
of American homes have or will have an internet-connected
assistant waiting patiently for someone in the house to say their
wake word.

In fact, the most disturbing product in my home was in it before

the experiment even started: the Vizio Smart TV. Beyond finding
out that my TV was letting data brokers know what Hulu shows
we watched, the experiment led me to actually visit Vizio’s
privacy policy, which was full of horrifying language about the
TV’s ability to collect second-by-second information about
everything we watched—from shows to video streams to DVDs to
commercials—and sell it to advertisers, who could then track our
activity on other devices on the same IP address to see whether
we went to a particular website because of what we saw on TV.

This ability was turned on by default for the 11 million smart TVs
Vizio had sold to consumers since 2014. Our TV had likely been
tracking everything we watched until 2017, when the Federal
Trade Commission and the New Jersey Attorney General sued
the company for unfair and deceptive practices. Vizio wound
up settling the suit for $2.2 million and turning off the tracking
unless the TV owners chose to turn it on (in the pursuit of well-
targeted ads?).

We may already be past the point of no return: internet

functionality is a necessary component for the operation of many
devices in our home, and it increasingly gets added on as a
feature even when it’s not strictly necessary. So when you get a
sex toy, it connects to the internet just in case your partner
wants to be able to please you from afar. But once the data is
going over the wires, companies can’t seem to resist peeking at
it, no matter how sensitive it is. The Canadian company We-Vibe
wound up paying millions of dollars to its customers because its
internet-connected sex toy was collecting stats about their
orgasms, for “market research.”

What our experiment told us is that all the connected devices

constantly phone home to their manufacturers. You won’t be
aware these conversations are happening unless you’re
technically savvy and monitoring your router like we did. And
even if you are, because the conversations are usually encrypted,
you won’t be able to see what your belongings are saying. When
you buy a smart device, it doesn’t just belong to you; you share
custody with the company that made it.

That’s not just a privacy concern. It also means that those

companies can change the product you bought after you buy it.
So your smart speaker can suddenly become the hub of a social
network, and your fancy smart scale can have one of its key
features taken away in a firmware update.

There was one connected device I came to love as a result of the

experiment: the iRobot Roomba 890 vacuum. (It doesn’t make a
map of my home, by the way, because it’s not the newest 900
series.) It terrified my daughter, who cried and ran to me every
time the low, round robot rolled in as if the clown from Stephen
King’s It had just entered the room, but I loved it. The Roomba
did what robots do best: easy, boring, monotonous work. But it
didn’t do so independently; like my other “smart” products, it
used its internet connectivity to pop up notifications on my
phone. Being smart meant it could nag me:

“Roomba requires your attention: Your Roomba is stuck.”

“Roomba requires your attention: Your Roomba’s bin is full.”

“Roomba’s cleaning job was canceled.”

I thought the house would take care of me but instead everything

in it now had the power to ask me to do things. Ultimately, I’m
not going to warn you against making everything in your home
smart because of the privacy risks, although there are quite a
few. I’m going to warn you against a smart home because living
in it is annoying as hell.