Tracking Tech: Do phones really listen to what you’re saying?

And what is Big Tech actually doing when they ‘sell your data’ to advertisers?

Thank you for taking the time to read this and I hope you walk away with a new perspective of how we use everyday technology. If you enjoyed this post, please do share with your network.

---

Unsplash

The age old question. Everyone has a freak story of them talking about something, only to be shown an ad for it. It's so common and makes people uncomfortable to the extent there is a website documenting people's testimonies of freaky occurrences. However, this has been denied on multiple occasions, for example an ex-Facebook Ad Targeting Leader talking about the sheer impracticality of recording everything from your phone. 

The short answer to the question: yes and no. No in the sense your phone is not listening to your conversations and processing the information to understand what you are talking about, but yes in the sense your phone may be using your phone microphone to understand the context of your environments, like if you are outside or indoors, etc. 

Audio processing and storing uses large amounts of computing power and storage. If [insert social media or phone brand here] really does listen to everything you say, then these companies would rapidly run out of storage and use insane amounts of processing. This isn't including the data costs that you would incur as a result of transferring everything to remote databases.

But if your conversations aren’t being listened to, then how the hell do advertisers pinpoint your conversations so well? 

---

Programmatic Advertising - a brief overview

To understand that, we need to dive a little into how adverts are bought on the internet today. The point of advertising is to make sure that people who might interested in your product see that your product exists so that they go buy your product. 

This might get a little complicated but I promise to make it as clear and simple as I can. Understanding this will help you grasp what is actually happening on your phone/browse the web and Big Tech gather data on you. Everything will make sense once you understand this model.

The traditional model of advertising

Let’s say Nike has a new pair of shoes they want to advertise to you (I’m using Nike as literally all the examples I found online when researching this was Nike?). How do they reach you? 

Firstly we need to look at the transaction. When you look at the transaction, there are two parties:

• Advertisers - these people have something to sell, in our case, Nike and their shoes. They want to buy ad slots.

• Publishers - these people have real estate or ad slots, either digitally or physically, where ads can be placed. This could be a billboard or a little space on a popular app. 

Previously, Nike would approach the person who owns a large billboard by a popular place, say like a train station, and buy out that place for a short while in the hopes that people will see the ad while they rush to work. But that is old fashioned - you are slapping on a massive advert by a popular place, where say 100,000 people will see it, but only about 5,000 will actually care, of which maybe 500 will actually go buy a pair of shoes. Besides, not everyone even leaves the house that often anymore. 

The advertiser, Nike, has to maintain multiple contracts with multiple publishers and decide who to engage with when they have an ad campaign

Over time with the new technology boom, using the traditional ad buying model also meant that Nike had to maintain relationships with multiple people - the guys who own billboards in an area, the TV Networks, the local newspapers, websites, etc. This became a bit of a faff, and gave the rise to Ad Networks. These guys maintained the relationships with the publishers, and Nike would then have deals with the Ad Networks to get their advert buying activities done. 

An example of an Ad Network is Google Display Network (GDN), or Google. So, in our example, Nike would have a deal with GDN to display an ad in say, London, and GDN would deal with talking to all the publishers in the London area to ensure that all the publishers in London had Nike’s ad. This saved a headache for Nike, publishers got their money and Google made a lil bonus in the process by charging Nike a fee. 

The advertiser, Nike, has one simple contract with an Ad Network, who in turn negotiates contracts with multiple publishers

This wasn’t necessarily bad. This model also allowed smaller publishers to sign up to GDN and get access to Nike’s ad - something they may not have previously be able to!

But then, there were loads of Ad Networks. So advertisers were back to square one. Managing too many relationships. 

So then Ad Exchanges were made. These guys maintained relationships with multiple Ad Networks and publishers. For example the Microsoft Advertising Exchange (Google do have an exchange but I want to separate the entities so you don’t get confused).

The advertiser, Nike, has a contract with an Ad Exchange, who in turn has contracts with Ad Networks, who in turn has contracts with publishers

So now, we have an advertiser, like Nike. They have a shoe ad that they want to show in London locations and people who live in London. They make a deal with the Microsoft Advertising Exchange, who in turn has conversations with GDN and other Ad Networks in the London area. These guys in turn talk to publishers in the London area. Eventually your digital billboard in London has an ad shown on it of the Nike shoe as well as your Facebook feed while you browse if you’re in London. 

As you can see, this model is doomed to get bigger and bigger. So what was proposed? 

The current age of advertising

So now we get to the foundation to how ads are sold today and how you end up with that super specific ad on your Instagram feed. 

Today, the Ad Exchanges have become more like stock markets, which I’m sure is just as alien to you, and if you’ve read the Flash Boys, you’ll realise that not even the bankers who work on the trade floor fully understand what is going on in there. 

Advertisers use this digital software called ‘Demand-Side Platform’ or DSPs and publishers use ‘Supply-Side Platforms’ or SSPs. They use this so that this intermediary can talk to the Ad Exchange - like how you would use Trading 212 or or Hargreaves Lansdown to talk to the stock exchange to buy stocks from a stock exchange. Publishers can work with Ad Networks or directly use SSPs themselves.

The advertiser, Nike, uses a platform called DSP to connect to the Ad Exchange where they buy ad slots. Publishers connect to SSPs to connect to the Ad Exchanges so that they can sell ad slots, similar to the stock exchange

Nike would use a DSP, or their digital platform that talks to the Ad Exchange, to broadcast ‘Hey I have a shoe ad and I’ll pay £5 for every 100 people who click on it’. Then a publisher, say theverge.com (who have ads everywhere) would go on there SSP, which is their digital platform that talks to the Ad Exchange, and be like ‘We have some space here on our website, and that £5 per 100 clicks look pretty tasty to us’, and then BAM, a deal is made, and Nike show their shoe adverts on the Verge and the Verge get paid. 

Types of Ad Sales

There are a few types of ad sales today: 

1. Programmatic Guaranteed - Nike want to specifically advertise on theverge.com. In this case it’s a guaranteed sale and Nike talk to The Verge directly. 

2. Private Market Place - The Verge say, ‘Hey we only care about big boys in Tech, so if you aren’t Apple, Google, Amazon, Microsoft, Facebook or Twitter, get out’. In that case no one can advertise there unless the stated companies send out a request to advertise. 

3. Real Time Bidding (RTB) - this is when companies bid for an ad slot. This is the one we care about most here as this is why you get hyper personalised ads.

Real Time Bidding (RTB)

So you’ve read this far and at no point have I mentioned where the Big Tech data tracking stuff comes in and how you get those scarily relevant ads. Get ready as it’s all popping off now. 

So with RTB advertisers bid for the ad slot in the stock exchange. But why would anyone want to bid in particular for an ad spot? Because of the individual who is visiting the ad slot. 

So say I visit theverge.com. Engines around the internet keep tabs of who I am and log it in a Data Management Platform, or a DMP (I promise this is the last time that I will throw in an acronym). So this DMP has your browsing habits and a mental picture of who you are due to all the cookie crumbs you leave around the internet (as in, they track you through cookies). So, from my internet activity, this DMP knows that I am looking to get into shape and lose them lockdown kgs, and I’m looking up running advice and forums. I am also someone that this DMP has worked out that is willing to spend on some expensive running trainers from my previous shopping browsing history and my predicted income depending on where I live, which they got from my IP address. 

A DMP stores your tracking information, which sells that information to advertisers during the bidding process

So I click on theverge.com. As soon as I click on theverge.com, The Verge send out a sale to the Ad Exchange, saying ‘Hey we have space to sell ads, who wants it’, meanwhile Nike can buy information from the DMP. The DMP tells the Ad Exchange ‘Hey this guy has been looking to get into fitness again, lives in an middle-income bracket area that could afford some £40-£70 shoes and has indicated he’s willing to buy some trainers. We know this from all the tracking we’ve done across the internet and you really might sell some trainers to this guy’. At this point, running shoe companies start bidding, depending on how much that company believes I am their target market and am worth getting a sale from. If Nike and Adidas know that I have a high chance of buying some shoes, they would fight to get their shoe advert in-front of my eyes - therefore start bidding more and more for the ad slot on theverge.com that will be eventually shown to me. Whoever bids the most will get to see their ad shown to me (unless I block it using ad blockers muahahaha). 

Everything I just described happens in an instant, so I won’t even realise that’s happening. The important component here is the DMP. In the example, I only used things they gathered from my web browsing, such as cookies, IP Address (my location), etc. But imagine they had access to my Google Fit data and could see I went for runs? Or my location data and see I have previously gone into sports shops? 

So what happens that make you feel like your phone is listening into your conversation?

There are multiple steps involved in this process. There's some anecdotal explanations out there, but I'll also explain here. 

You walk past a billboard by the train station with a Nike advert on. You mention to your friend about the Nike advert. Nike have exact geographic location of their billboard. 

From there Nike can go to Google being like "Hey, anyone who is looking to buy sports shoes, is sporty, can afford it, and goes past this train station billboard in a 100m radius, I wanna show them an ad for the Nike shoes that was advertised".

You on the train go on Instagram. Instagram shouts that have an ad space, and the DMPs out there say ‘Hey this person is into running, has money and just walked past the Nike advert’, and then Nike buys this data and is like ‘BINGO, THIS IS OUR CONSUMER KEEP BIDDING FOR THIS INSTAGRAM AD SLOT TILL WE GET IT’. 

You come across the Nike advert. In your head you are like ‘Wait, I just spoke about this! How is it on my Instagram feed!?’.

Another example? 

I’ll quote the reddit thread of an individual from the point of view of a Nike marketing exec:

Adelphic sells ConnectedTV data. What does this mean for you? Say Nike is my client, and you’re watching an ad supported show through one of my the various apps on your smartTV. A Nike commercial comes on and you ignore that s*** by scrolling through your phone. Your smarTV has a little microphone inside of it. Adelphic partners with the TV brands and tells the microphone to listen to key words in the Nike commercial playing. Since your TV is connected to WiFi, and your phone is connected to WiFi, Adelphic now knows your household IP address. So as you ignore the Nike commercial on your TV, I can target your WiFi devices (phone, tablet, computer) with a Nike ad, because I bought that information from Adelphic.

There are SO MANY data points that can be collected. A gyroscope can tell when you’re lying down - perfect time for some pillow and mattress ad. They can see what you spend most of your time on, etc. You can see a small list of things that are tracked here. 

A list of Google tracking categories

Is there any point in blocking trackers?

So what right? At the end of the day, you will be tracked and what does a cheeky advert for a bag from ASOS do? 

The principle idea of targeted ad isn’t necessarily bad - and I’m not here to hate on the ads industry. They need to get their product known somehow! For me it’s the lack of care of the management of all the macro-surveillance these companies do.

This level of tracking is also nothing radically new - companies have been hyper tracking you and targeting you with specific ads for a while now. Ever wonder why stores really want you to sign up to loyalty cards? There’s a story from 2012 about how Target worked out a teen girl was pregnant before she told her father from her purchases at Target.

It falls down to your risk tolerance and how much you care about handing over your data. Security is the balance of convenience and privacy. How much do you trust Big Tech with your data to genuinely use it the way they say they will? 

This data has been abused 

I wrote in my previous article about AI Ethics that a big problem with data collection is that once it’s in the hands of corporations and/or researchers is that the user who has had their data collected does not have control on how it is used, should the data collectors decide to use the data for something else. 

For example, location data can be collected by a weather app to get your local weather. But once they have a list of all the locations you have visited over a month, you have no control over if they decide to use the data for something else. 

Google claim that they do not disclose your religion and political leanings, among other things. Assuming Google is telling the truth, we know this is not the case with other data collecting services. 

I’ve mentioned this before, but Cambridge Analytica is one of many examples of the data collected from you being aggressively abused, and Facebook didn’t do due diligence on how such sensitive data was being used. Cambridge Analytica used the pages you liked on Facebook, your friends, your interactions (like likes and comments) among other things to determine your political leanings and how likely you are to being swayed by news and ‘fake news’. It is claimed they had a sway in the outcome in the US Elections to vote Trump into office and the Brexit vote. 

Any interaction on any platform is useful data points to someone. Ever wonder why Facebook expanded beyond the like reactions and Twitter is looking to do the same? Advertisers can get a much better understanding of your mood by seeing whether you ‘laughed’ at something or ‘cried’ at something. You might think this is all trivial, but it all adds up, and advertisers pay good money for this information! Otherwise Big Tech wouldn’t be Big Tech.

For the Apple users out there who think they’re not being tracked? You definitely are, they just claim that they don’t sell the data to advertisers…yet. They do however, plan to work out the state of your mental health and whether you have autism. Imagine. They have enough data points on how you use your phone to accurately medically diagnose medical conditions. 

I am not trying to scaremonger here - these are all things that have happened to your data. Your data can be used to give you a better experience without it being misused and abused. You have the right to know what is being done with your data. 

How can I block these trackers if I want to?

So first off - there is no way to truly block everything, unless you literally go off the grid, don’t use the internet, phones, and literally do not go out in the public. 

What you can do is make it a little harder and hand over less data, with the following steps:

1. Decline tracking cookies

   1. When a website asks for to track you, try decline them. They may ignore you, but at least you did your bit.

2. Use a VPN

   1. A VPN will give you a different IP address when you use your services on your phone or your PC, giving the trackers inaccurate information of your location activities.

3. Turn off your background location on your phone

   1. In theory your location data won’t be collected. Although Google has outright ignored this before.

4. Block trackers through your browser

   1. Firefox is great for your personal privacy. If you want a chromium browser (something similar to Chrome), Brave browser is also good for blocking.

5. Reset your Advertising ID

   1. Your phone has a unique advertising ID that leaves crumbs around the internet and your profile is built and linked back to this ID. This can be reset and therefore breaking the chain and disassociating yourself from the profile advertisers have created on you. To do this:

      1. On Google: Google Settings → Ads → Reset Advertising ID

      2. On Apple: Settings → Privacy → Advertising → Reset Advertising ID

Will these make you untraceable? No. But it will make it a little more harder for advertisers to track you. It will also mess up some of the data collected on you, making the data inaccurate and therefore the profile on you less valuable. You can also go to this website and see how you fair for tracking on the browser.

This is more of a part one of a multi-part series. I will revisit this, where next time I’ll discuss how ‘ghost profiles’ are created. 

Until then, remember: just because this stuff is normalised and Big Tech tells you that it’s ‘inevitable’, doesn’t mean it’s ok. 

---

If you have a better idea than I do, if I’ve missed out anything or you think I am talking absolute rubbish, feel free to reach out either by commenting on the post, or by emailing me on tanvirtalks@substack.com

If you enjoyed this post, subscribe to Tanvir Talks, where I publish a newsletter once a month breaking down the big questions asked in tech into digestible chunks for you to consume, the average consumer. I also have a podcast where I do the same thing!