I think the responses to this can be broken down into a 2x2 matrix: level of concern vs. understanding of technology.
1) Don't understand ML; not concerned - "I have nothing to hide."
2) Don't understand ML; concerned - "I bought this device and now people are spying on me!"
3) Understand ML; not concerned - "Of course, Google needs to label its training data."
4) Understand ML; concerned - "How can we train models/collect data in an ethical way?"
To me, category 3 is the most dangerous. Tech workers have a responsibility not just understand the technologies that they work with, but also educate themselves on the societal implications of those technologies. And as others have pointed out, this extends beyond home speakers to any voice-enabled device in general.
In conversations about this with engineers the response I've gotten is essentially: "Just trust that we [Google/Amazon/etc.] handle the data correctly." This is worrying.
I'm in the 5th category. 5) Understand ML; concerned - won't allow any of these things in my house period because they will always use them for things behind the scenes that they won't state. I don't care how well trained they are, or "ethical." Ethical... according to who, and at what time period in the future? Ethics change. The data they have on you won't. Look at all of the politicians and other people getting in trouble for things they said 15 years ago, which were generally more acceptable at the time but we've "progressed" since then. Who will be making decisions about you in the future based on last years data? Just don't give it to them.
Before I go further: do you own a smartphone? If not, you can ignore the rest.
So, assuming you have a smartphone. That is a device which has:
- Permanent network connection. Doesn't matter what you do on your Wifi, it has the cellular data network. Which is controlled by an independent processor with its own firmware
- Excellent noise-cancellation microphones. They may not be able to pinpoint the sound source like an Echo would, but they are still pretty sensitive
- Accurate location updated via GPS. If you block GPS, it can still use the cell phone towers as an approximation, plus any SSID beacons nearby
- Very powerful processor, capable of listening for 'interesting' data before sending anything
- Similar functionality to an Echo / Google Home / Apple whatever. They'll be listening for "hey siri" or "ok google"
And so on. Plus, anyone you interact with will have such a device in their pockets.
Given that. why would an Echo be of any particular concern? At least you can monitor their network activity easily. Not so with a smartphone, which is a much higher threat. And yet most people sleep with them next to their beds.
> Similar functionality to an Echo / Google Home / Apple whatever. They'll be listening for "hey siri" or "ok google"
This seems to be a pretty big assumption, without which your whole argument falls apart. In my case, at least, I have no such functionality enabled. You could argue that the manufacturer of my phone could push a software update that enables it without my knowledge (in fact, they likely couldn't, because I don't have automatic updates turned on), but even if this were true we're now arguing about a very different thing - the theoretical ability of MegaCorp to spy on me as opposed to actual known spying.
EU governments and telcos spy on you by law. 10 years ago there was a law across entire EU that forced telecom companies to store the position and call numbers of users for the past 6 months. Germany got even fined because it did not implement the law.
And of course, the police have been caught abusing this data. Data acquired this way is likely to get to you in another type of cases: divorce proceedings often use this sort of data as well, and it's been used in trade disputes too.
Given that all that already happens with cellphone data, what's the point of doing anything about it at this point ?
> In my case, at least, I have no such functionality enabled
How do you know? Is your device _incapable_ of performing that task, or did you just click a checkbox somewhere? Because if you just politely _asked_ the OS to disable it, it doesn't really matter, does it? We are back to the matter of trusting the vendor.
> (in fact, they likely couldn't, because I don't have automatic updates turned on)
So you say. The software may or may not agree with your statement.
> the theoretical ability of MegaCorp to spy on me as opposed to actual known spying.
There's no "actual known spying". Spying implies intent, none has been demonstrated so far.
Where your argument falls apart (yeap, I can use that line too) is this: you are interacting with other people also carrying powerful listening devices of their own. They may or may not have disabled a "ok google" functionality, and they may or may not have disabled updates. Again, where do you draw the line? Do you ask everyone to leave their devices behind before interacting with them?
Yes, but there's still a difference from what we are worried might be doing with our smartphones, and what we KNOW they are doing with 'voice assistants'.
I turn off voice assistant on my smartphone and don't have any google home/alexa devices either. I have a TV with no voice activation. I won't use voice activation devices.
Is it possible my phone is listening to my anyway and sending recordings to someone? It's possible. If I were smart I would never say anything that would be 'dangerous' if someone heard it with my phone in the room, although Im sure I'm not that careful, it's probably not possible.
If I found out for sure my phone was listening to me surreptitiously, rather than just knowing that it's not impossible -- I'd do something about it.
One affect of arguing "it's all the same, your phone COULD be spying on your voice conversations anyway" is, perhaps ironically, to make people care LESS about it. If it's doing it anyway (or just if I can't prove it's not?), what do I care about Google Voice doing it too? But one disincentive for companies doing it is precisely that people will be upset if they find out. (And they're gonna find out eventually).
You don't need to go live in the woods to care about it. It's good to care about it. You don't need to either "draw the line" at living in the woods without electricity, or draw no lines. We can draw lines. They can move. If your argument is that since you can't be sure that nobody's spying on you, you shouldn't care about the people you DO know are spying on you and shouldn't get upset about it -- that's not how any of this works.
Insisting on all or nothing gets you nothing (cause it turns out 'all' isn't really an option), and who wants nothing?
I'm just inserting a comment here in the hopes that it will come before a more deeply nested soon-to-be-written XKCD comic. Although I'm fairly confident XKCD has covered this in the past already.
On my smartphone I can at least turn off the apps that would automatically listen for voice commands; I can use the phone just fine without them. And if I thought there was enough of a threat that Google or someone else was listening in using my phone's microphone without my knowledge, even with the apps turned off, I would switch to a rooted phone that would give me more control.
For the Echo/Google Home/etc. devices, I don't have any of the above options, so I simply don't buy them and don't use them.
Even a rooted phone will not help, since the monopolistic baseband processor will do it. The big ARM CPU is just for the users, the baseband is for control of the public masses and mass surveillance.
The surveillance problem is not black and white. A rooted phone may not help you if a state actor is after you, but it will help against companies that make money by building an invasive profile of who you are and what you do. Again how much a rooted phone helps in that depends on how many ties you want to cut with the digital-advertising ecosystem.
Here's a paper describing the privacy invasive and potentially harmful properties of pre-installed apps [1]. I specifically saved this paper in my files so I can have a comprehensive answer when people ask me why I go through the trouble of rooting my devices.
P.S. People who know more than me are saying that the baseband processor can be restricted with hardware practices such as IOMMU [2]. I don't really know how effective that can be.
The baseband processor isn't running Siri or Google Voice on my phone. Yes, if the state has a back door into the baseband processor there's not much I can do about it, but that's not the threat model that I'm avoiding by not using devices like the Echo or Google Home.
> that's not the threat model that I'm avoiding by not using devices like the Echo or Google Home.
100% this. IF a state intelligence agency is after my data they'll get it somehow (most likely just by asking). My concern is the rampant dragnet aggregation of personal data that I didn't expressly provide them by companies like Google and Facebook. I mean, Google having access to your emails that you store in Gmail? Kind of understandable. Google buying MasterCard transaction records to cross-reference against your web browsing data as tracked by any site using Google Analytics? No.
Which defeats the whole purpose of having it, namely, to respond to voice commands.
> You can turn voice off just the same as you can on a phone.
I don't have to turn off voice altogether on my phone, I just have to turn off the apps that I don't trust. On an Echo, either I turn it off and might as well not have it, or I turn it on and it's recording everything. There's no middle ground where I get some basic functionality without being spied on.
> On an Echo, either I turn it off and might as well not have it, or I turn it on and it's recording everything. There's no middle ground where I get some basic functionality without being spied on.
I'd guess that most people could effectively find a middle ground by treating the Echo as if it were someone in a cordial but not really friends relationship with a family member, such as a classmate of one of you kids over to discuss a school project, or a member of your spouse's church over to discuss planning the annual church picnic.
If they are doing things that they would not mind such a person overhearing, have the Echo on. If they are doing things that they would not do until such a person leaves, have the Echo off.
It would be interesting to give these home voice assistants faces, including eyes that look at and track people it is listening to when they speak, to help people remember that it is still listening to them. That might make it easier to remember to turn it off before discussing sensitive things in its presence.
The analogy you make is helpful, but it immediately brings attention to one thing: you generally don' invite "someone in a cordial but not really friends relationship with a family member" to stay with you all the time, day and night. People you do let stay with you are people whom you either trust, or expect to trust as the relationship develops. There's no relationship development with an Echo - it's forever an agent of a faceless corporation that's indifferent to you.
Turns out the NSA has an implant for smart tvs that among its features is keeping the tv on and recording you while turning the indicator light that indicates its on status off.
You don't own the device despite paying for it. It updates itself when it feels like it and does whatever its actual owner says including compromising itself if told to.
I use Lineage OS (https://lineageos.org) with the cameras taped. It's not perfectly secure, but it's the best tradeoff I can make at the moment. Here's a few better options I want to switch to in the future:
+ Does not use Android or iOS. The Librem 5 comes with the mobile version of our FSF-endorsed operating system PureOS by default, and is expected to be able to run most GNU+Linux distributions.
+ CPU separate from baseband, isolating the blackbox that the modem may represent and allowing us to seek hardware certification of the main board by the Free Software Foundation.
+ Hardware Kill Switches for camera, microphone, WiFi/Bluetooth, and baseband.
+ End-to-end encrypted decentralized communications via Matrix over the Internet.
+ We also intend the Librem 5 to integrate with the Librem Key security token in the future.
It's not quite what you're asking for, but Apple does still sell the iPod Touch, and they recently revved it. One could pair that with a mobile hotspot with a power button...
I won't have one either. Well, I have my Android, which I guess is essentially the same.
I was visiting my neighbor the other day. We were wondering something about special counsel mueller, and she decided to ask Alexa. This was new, and I was surprised she had this surveillance device, but I didn't say anything.
There was some back and forth, then I said to her "I think you're ordering motor oil "
"Alexa, stop! Alexa, stop!" As if a dog was chewing on the furniture.
And now Amazon knows that I was discussing politics, and my opinion. That kind of analyzed data is going to be lucrative in some future military or law enforcement or political contract.
Do they know it was you, specifically? I would say they know your neighbor was talking with someone else, and possibly that politics were being discussed (not really something they seem to have put a lot of effort into recognizing, since it doesn't lead to shopping...) If you're interacting with an Echo or gHome you can always ask it "who am I?" GOOG is sneaky enough to recognize you on any other device, Alexa, not so much (and in this case you shouldn't be on their radar to being with).
They can eventually know who I am by voice recognition, especially they can get corroboration by the fact that we live in the same building.
I imagine we probably said something that would suggest I live nearby (... "Checking database" ...) or that I'm a regular visitor (... "Checking database." ...).
They could be sending an inaudible tone, to be captured by an app on my phone. I don't have an Amazon app, but some other app might capture the tone and sell that fact to Amazon, so that they don't leave money on the table.
Imagine 20 years from now you decide to run for political office. Do you want someone "ethically" going through all your old recorded conversations looking for statements that will be politically incorrect 20 years from now?
Not only you, but your kids. People have lost sponsorships and contracts because of things one of their parents said before the person in question was even born. Look at last year's incident where Eli Lilly pulled their NASCAR sponsorship of Conor Daly over something his father had said a decade before Conor existed. https://www.indystar.com/story/sports/motor/2018/08/24/eli-l...
Or intentionally don't watch what you say and stop giving this kind of system any power. If you run for office in 20 years planning to hide what you think now, or apologize for it, then you're in for a losing battle. Say what you think, admit when you're wrong, learn from your mistakes.
>won't allow any of these things in my house period because they will always use them for things behind the scenes that they won't state.
As OP stated, "this extends beyond home speakers to any voice-enabled device in general".
Unless you're living like the Unibomber, it's not a matter of "just don't give it to them". The moment you step outside your house and socialize with almost anyone anywhere, it's liable to being taken from you.
"oh but i deleted all my data through the helpful portal they provided. they said i had full control?"
Data is Googles game. Once they have it, they have it. Nothing else matters. You own nothing. Your image, your voice, your ideas, your conversation, your habits, your medical records. Google wants it all. Thats is Googles world.
Google aint giving that up. Everything they do is about getting more and more of it. Everything to the contrary is just PR spin.
Additionally, in order to be 'ethical' by any means, they still have to collect the data in the first place. Regardless of actual intentions around ethical/defined practices, that always leaves the possibility for someone else to access that data and do unintended things with it.
I have been casually tracking who actually owns these types of devices and have never met anyone under 25 who would even consider it whereas I go to the homes of people in their late 30's and 40's and see them all the time.
If you visit your friends or relatives, you cannot fully protect yourself from this data collection by simply choosing not to own these devices yourself.
What's strange about this conversation is that your refusal to trade privacy for convenience is passed off as some kind of no-loss decision, when in reality there are all kinds of downsides to living your life the way you claim you do.
The conversation can't even be had until the facts are presented honestly, and when you present your position this way, it's not honest.
This classification is very useful to discuss this issue.
The difference between 3 and 4, noble as it is, can be caused by feasability concerns that push people into 3, not just ignorance of the privacy impact. Human labelling of training data sets is a big thing in supervised learning. Methods that dispense with this would be valuable for purely economic reasons beyond privacy - the cost of human labelling of data samples. Yet we don't have them!
Techniques like federated learning or differential privacy can train models on opaque (encrypted or unavailable) data. This is nice, but they assume too much: that the data is already validated and analyzed. In real life modelling problems, one starts with an exploratory data analysis, the first step being looking at data samples. Opaque encrypted datasets also stop ML engineers from doing error analysis (look at your errors to better target model/dataset improvements) which is an even bigger issue, IMO, as error analysis is crucial when iterating on a model.
Even for an already productivized model, one has to do maintenance work like checking for concept drift, which I can't see how to do on an opaque dataset.
It's not wrong for humans to label training data. It's wrong to let humans listen to voice recordings that users believed would be between them and a computer. The solutions are obvious: sell the things with a big sticker that says, "don't say anything private in earshot," revert to old fashioned research methods where you pay people to participate in your studies and get their permission, or ask people for permission to send in mis-heard commands like how Ubuntu asks me if I want to send them my core dumps.
> ask people for permission to send in mis-heard commands
Note that you also want the "correctly" heard commands, because some of them will have been incorrect. It's frustrating when an assistant gives the "I don't know how to do that", but it's even more frustrating to get "OK, doing (the wrong thing)".
Also, another alternative: provide an actual bug reporting channel. "Hey Google, report that as a bug" "Would you like to attach a transcript of the recent interaction? Here's what the transcript looks like." "Yes."
To be fair the system already has something like that. If you complain to the Home it'll ask if you want to provide feedback and give you a few seconds to verbally explain what went wrong.
I'm not sure if humans will then review that feedback of if it goes through a speech to text algorithm first but the mechanism for feedback is there.
Yeah, i think I've experienced that. I was driving with Maps directions, and while i was driving Google decided to show me new things Maps can do.
I tried to voice my way back to directions, unsuccessfully.
I said "Fuck you Google."
"I see that you're upset," followed by some instructions on how to give feedback. While I was driving. It sounded almost exactly like "I'm sorry Dave, I can't help you."
> like how Ubuntu asks me if I want to send them my core dumps
While I like how Ubuntu does it, I actually like better how Fedora does it. Not only do they ask to submit core dumps but gives you the ability to annotate and inspect what gets sent as well as gives you a bug report ID which you can use to follow up on.
Agreed, I'd like to support Ubuntu development, I often run it on bleeding edge hardware I'd like to submit crash reports for, but the inability to sanitise the data causes me not to unless it's a "fresh" device.
Just give participants the choice to opt in for a chance to get early access to new products. Make it invite only to feel exclusive. They will have millions of willing test subjects.
good point, there's precedent from hospitals wrt IRB and other infrastructure involved w/ data gathering. Hospitals/research institutions self-regulate in this regard, doesn't appear tech does
Handling the data in an ethical way doesn't need to be handling the data in an completely anonymous fashion. That would be one solution, but you can also create a tust-based system for how the data being labeled is handled, similar to HPAA. In addition, there are simple operational methods that could help ensure the data is processed as close to anonymously as possible. For example with voice data, you could filter the voices, work with the data in segments, and ensure that metadata for the samples is only accesible by trusted individuals certified under the above framework.
In trust-based systems like HIPPA or Clearances, there is a fundamental aspect of requiring 2 conditions to access data: privilege, and the necessity to know. Taking data and mining for valuable insights isn't a "need to know" it's a "need to discover something unknown". This is where the security breaks down. In a conventional HIPPA system, only your doctor needs to access your info. You don't have to worry about some other doctors accessing your information in bulk to try and conduct a study on cancer rates. They don't NEED to know your info, they just WANT to know. When you WANT to know how to accurately fingerprint people by their voice, then obfuscating it is counterproductive.
>You don't have to worry about some other doctors accessing your information in bulk to try and conduct a study on cancer rates.
This not only happens, it's my job (though I'm not a doctor). Of course, it's tightly controlled on my end. I work for the government, but health systems have their own analysts. As part of my job, I have access to sensitive and identifying information.
This isn't to be contrairian. There are existing systems using very personal data in bulk for analysis. The wheel doesn't need reinvented.
My car has a little blurb that explains that they collect data to use for training and gives me the choice to participate or not. Opting out doesn’t affect any functionality. Why can’t Google do the same thing?
The shareholder value theory is rubbish, because it has no predictive or descriptive powers for why one decision was made over another.
I can just as easily say that the best way to maximize shareholder value is to minimize public scandal, scrutiny, and potential for legislature.
Nearly every single decision, including contradictory ones, made by every single company, everywhere, can be retroactively justified to have been done in the name of shareholder value.
> I can just as easily say that the best way to maximize shareholder value is to minimize public scandal, scrutiny, and potential for legislature.
Scandals can get free marketing, for example, Nike and Colin Kaepernick. Attention is always better than no attention at all for a business. Every single decision is made to increase profit but there might be many things that need to be accomplished first so its hard to see the big picture. For example, a developer might want to improve a feature because they want more people to use their product. A manager gets approval to pay that developer because the investment is deemed a profitable one. What does the person who gave them that money care about the number of users. It's not their invention and they don't even use the service? They give the money because they know that More users = more market share = more ads to sell = a return greater than the initial investment. Until a business can run with people working for free, the person paying for things always dictates what is bought and thus the direction the company is headed.
Let's say that direction is contrary to the direction of another prominent member of the business wants it to go. Whether you want to believe it or not, the same calculus goes on in every person's mind: Is this the potential payoff of Option A greater than the potential loss of Option B given the risk?
This is a wonderfully condescending response but it answers nothing. The question was, why can’t google do it differently? This doesn’t answer the question. We can plainly see this from the fact that other companies, operating under the same conditions you describe, make different choices.
This is the business equivalent of saying “because physics.” It’s not wrong, it’s just not useful.
Sorry, I didn't mean to be condescending. To answer your question, the reason Google can't do things differently is that they have already established themselves as first and formost and advertisement company and the way to do that best is to know their audience very intimately. Other businesses like Apple have established themselves as a hardware company first so they aren't dependent on user data as much so they took advantage of that and established themselves as the "Secure" phone. Google is too large and it makes too much money from its core business which is ad drive. As long as search and ads are their cash cow they cannot change in the way you hope.
That's what is so great about capitalism. If one company starts to take advantage of its users for profit, it opens up a niche for another company to take a different approach.
Then explain why they changed to Alphabet. Shareholders were sick of things like project loon, siphoning cash from google search. You are extremely naive if you think there are many concerns of higher importance than profit. Everything else is about maintaining and growing profit even if that means doing an ad campaign convincing people you are fighting the good fight..for profit.
Because now they have to report it to their shareholders where the money is going so that if the board doesn't like it they can replace the CEO. Before since it was all google, the money went where they said it went, there was no oversight. They had this massive R&D budget that was opaque to the investors. Money that could have been paid to shareholders as a dividend or return was instead spent on projects they had no idea about.
>To me, category 3 is the most dangerous. Tech workers have a responsibility not just understand the technologies that they work with, but also educate themselves on the societal implications of those technologies.
Do you think its possible to be educated on the societal implications of these technologies and still not be concerned? Seems like you've written your own viewpoint into the only "logical" one here.
That's a fair point. I do think it is possible to be educated on the societal implications of these technologies and still not be concerned. I would disagree with that opinion, but it is certainly valid.
Maybe these philosophical debates are going on behind closed doors. If so, this should be communicated to the public/end consumers. Much like in legal proceedings, the process itself is just as important, if not more so, than the outcome.
That being said, based on conversations that I've had with people working on these very products, the interest level and incentive structure for engaging with the tech side of things far exceeds that of engaging with the broader societal implications. Creating the tech earns your salary, questioning its morality may get you fired. So many choose simply to not engage in the philosophical discussion, which to me is a big problem in the industry.
>To me, category 3 is the most dangerous. Tech workers have a responsibility not just understand the technologies that they work with, but also educate themselves on the societal implications of those technologies. And as others have pointed out, this extends beyond home speakers to any voice-enabled device in general.
Yes I'm frequently amazed how many coworkers I have that are still completely plugged into Google, Facebook, Amazon services/spyware, fill their homes with internet enabled "smart devices", have alexa/google assistance etc, and yet they act like I'm paranoid when I try to discuss security concerns or just flat out don't care.
As much as I hate to say it, I think there needs to be a massive breach or abuse of power from one of these organizations/services that has severe real world consequences for those that utilize/support them. Until then nothing will change.
It sounds like you're hoping for this as evidence that you were right to be concerned; have you considered that you might be wrong? What if your coworkers are right, and the risk is actually extremely low? How would you determine that?
> It sounds like you're hoping for this as evidence that you were right to be concerned; have you considered that you might be wrong?
An alternate interpretation is that if a massive breach of trust is intentional then it would be better if it happened sooner rather than later.
> What if your coworkers are right...? How would you determine that?
That is a pretty classic appeal to popularity; what most co-workers believe is not evidence of anything in this case. If they are right, they are right. If they are wrong, they might be a self-selected group of the people who don't see a risk for what it is.
At any rate 'the risk' is a bit vague, but moral panics and witch hunts are things that happen. When the tech companies get involved in one, which will happen, it could be very nasty. There is clearly some sort of new risk here making it easier to quickly and accurately identify minorities. Even ignoring rogue employees finding creative ways to use data to enrich themselves.
>It sounds like you're hoping for this as evidence that you were right to be concerned
Not at all, if the constant security breaches and lack of response from consumers, regulators, companies, etc. isn't enough evidence for you that there is a serious problem then you fall into the group I'm describing
What constant security breaches? There are two major examples of security breaches I can think of that happened recently at amagoofaceoft: mis-stored passwords (fb/insta, and google for a short period earlier this year), and variants of the Cambridge analytics attacks, which steal public information but at scale. While those certainly aren't good, I wouldn't classify either as a security breach. The first was a loss of defense in depth, and the second, like I said, just got public info, but lots of it.
Are you saying that breaches at other companies mean we shouldn't trust the big ones to be secure? Like because Equifax has terrible security practice, google by definition must also have bad security? Or...what?
Breaches was probably the wrong term to use in my example, as it brings attention to the wrong issue. The point I am trying to make is not that we shouldn't trust large companies to be secure as you say (although based on my experience with enterprise infosec I wouldn't be surprised if a majority of companies handling/storing personal data don't have appropriate security controls enforced).
The point I am making is that many large organizations such as Google and Facebook are performing worldwide, largely unchecked mass surveillance with data collection and analytical capabilities far beyond what is available to the majority of the world, and people simply don't care despite how knowledgeable they are about technology. There's also little to now way to escape it as Google and Facebook technology is so ingrained in the existing internet. While Google may not have poor security practices and may never experience a breach where data is stolen (although again I highly doubt that), as far as I'm concerned Google itself, as well as Facebook, are malicious actors in my own life and personal opsec as a huge portion of their business model is based on collecting and monetizing user data by any means possible with little to no concern for the negative impact on users such as mental health problems.
Frankly, I don't like companies that make money spying on people, particularly those that abuse psychological techniques that make it more difficult for people to make informed decisions or choices about the technology they're using.
Beyond that, these technologies are sold/rented or otherwise provided to governments, law enforcement and intelligence agencies, dictators, authoritarian regimes, and others that can and are being for personal gain.
So no, I don't believe we should trust Google, but not because other companies have experienced data breaches. That is just one of the many reasons I believe people should value their data and personal privacy far more than most do
I find it funny that I often get defensive questions from Google and Facebook engineers about their technologies/organizations when I post initial pro-privacy comments on HN, but after being called out and explaining in more detail I never get a response. I guess there's no point for them to argue it further as they're aware of the negative impact, but have made a conscious decision to choose money over morals
You said they have constant breaches, then immediately recanted when asked for details. Your argument which got no reply was an idealogical argument which appears to be constructed to shut down debate (they shouldn't be trusted because you don't like them) as opposed to lead to a meaningful discussion. You even threw out the casual line about them not having poor security and breaches, invalidating your argument in the post they replied to.
In other words, they seemingly care about whether the technical argument has merits. Once it's clear that there's no technical substance and it moves on to your personal crusade against modern companies, people lose interest.
Disclaimer: I don't work anywhere near the companies in question
>You said they have constant breaches, then immediately recanted when asked for details
No, I didn't, go back and read again
>Your argument which got no reply was an idealogical argument which appears to be constructed to shut down debate
It wasn't meant to shut down debate. If he wants to argue the ethics of spying on people and using psychological tactics for financial gain I'd be more than happy to discuss
>You even threw out the casual line about them not having poor security and breaches, invalidating your argument in the post they replied to
Again, no I didn't. I never said google had poor security or breaches, and I clearly stated that was just a generic example I used which brings attention to the wrong things, as demonstrated by you focusing on "breaches" rather than the point I was really trying to make and elucidated in my reply.
>In other words, they seemingly care about whether the technical argument has merits. Once it's clear that there's no technical substance and it moves on to your personal crusade against modern companies, people lose interest.
That's the entire point, and why I regretted saying "breaches". You are focusing 100% on the wrong thing. The problem that I have is not a technical argument about whether or not breaches could occur
As the original person, this almost exactly. I can totally understand why someone might hold those opinions. I don't share them, and argument won't be productive. Litigating values doesn't get anywhere.
> Are you saying that breaches at other companies mean we shouldn't trust the big ones to be secure? Like because Equifax has terrible security practice, google by definition must also have bad security? Or...what?
Are you asserting that Google detects 100% of significant breaches, and promptly notifies the public of all of them?
My experience tells me that neither assertion is likely to be true.
"... I think there needs to be a massive breach or abuse of power from one of these organizations/services that has severe real world consequences for those that utilize/support them..."
One of my greater fears is the knowledge that, should this happen, there's a nonzero chance that no one would care.
Yes, the Equifax breach is one of the reasons I included "severe" as a qualifier. It has been demonstrated that even fairly serious breaches will be ignored by the general public. It needs to be something that makes people genuinely fear for the safety of their finances, possessions, and/or health
> there needs to be a massive breach or abuse of power from one of these organizations/services that has severe real world consequences for those that utilize/support them. Until then nothing will change.
I think even more darkly: that the consequences of something severe enough to cause that change would be so much as to effectively destroy modern civilization. Consider that much of the modern economy is driven by tech companies not only ignoring privacy but often actively violating it.
> Tech workers have a responsibility not just understand the technologies that they work with, but also educate themselves on the societal implications of those technologies.
I think this goes well beyond tech workers. I think it's time for society to legally recognize the balance between the value of ML systems and the privacy concerns of customers of ML.
Doctors and lawyers obviously should understand the value of privacy, but we, as a society, have also created legal rights and duties for them. Conversations with lawyers and doctors are legally privileged; at the same time, there are specific consequences for medical companies or lawyers who do not protect that information.
Companies like Google, Apple, Amazon, etc. certainly have the resources, intelligence, and sophistication to comply with a similar regulatory regime. IMO it should be possible to construct a law that allows companies to collect, store, and tag customer data for purposes of training ML systems, but sets serious duties, with consequences, on them to do it right.
Right now, what is to keep employees at these companies from abusing these systems to stalk, to surveil, to harass, or even just to feed their own curiosity? These data systems are core trade secrets for these companies, which means they are opaque to any kind of oversight from outside the company.
The free market can't create the necessary balance because customers need information to make decisions--information that they don't have. The result will be an increasingly chaotic "hero/shithead rollercoaster" as customers make snap judgments based on scanty or wrong information about what these companies are actually doing.
This is a classic case for regulation, which prevents a "race to the bottom" of sketchy practices for short term gain, while also protecting the ability of people and companies to use this technology to create value.
Doing this right will help data-leveraging companies in the long run, just like attorney-client privilege and HIPAA have helped lawyers and doctors build trust (and therefore value) in their customer relationships.
One thing that I think gets lost in engineers (and humans) is scale.
Googazon doing {thing} might be "meh" for 10 people. But the implications look very different when it's doing {thing} for 10%+ of a country's population.
At 10 people, I may find out Ted likes to eat Italian. At 10%, I may find out an Italian chain has a sudden health issue and short their stock.
Which is in essence their original playbook: do things that only work at a scale that only we can play at.
The person in the story said they thought they heard domestic violence in some of the recordings.
I know some people who are into “consensual nonconsent“, a form of BDSM which I do not understand, but as these acquaintances tell me they like being on the receiving end I have reason to trust them.
Any system or person which incorrectly identifies one of these groups as the other, in either direction, has life-altering negative consequences. Note that in the UK, some forms of consensual BDSM have been prosecuted as serious assault, and the person on the receiving end has been prosecuted for conspiracy to commit assault because they consented to it.
Any system which prevents DV information reaching the police is bad. Any system which reports BDSM to the police as DV is bad. I don’t even know what the relative frequencies of the two acts are, so I cannot even make a utilitarian ethical judgement.
> Note that in the UK, some forms of consensual BDSM have been prosecuted as serious assault, and the person on the receiving end has been prosecuted for conspiracy to commit assault because they consented to it.
Can someone versed in UK law clarify how such a stupid system emerged out of their laws? I'd guess that's an unintended consequence of how laws were worded, but it is mind boggling.
I'm not versed in UK law, but live in the UK and am aware of a few cases. Bottom line - British law does not recognize the possibility of consenting to actual bodily harm.
I'm not aware of any recent case where the person on the receiving end was prosecuted for consenting, perhaps the OC can cast some light on that.
My acquaintances mostly reference Operation Spanner which is about 30 years old now. I don’t know if that counts as recent or not, however while the Law Commission recommended in 1994(!) that this law be altered, it appears their recommendation was never adopted.
Anyone remember the 3d printed gun stuff from a few years back? I think this isn't very different from it. You can take these raw pieces and explain how they are simple and good and draw these simple ethical conclusions from them, but then you add it up and the bigger picture doesn't feel quite the same way. 3D printers are good, sharing 3D printing plans is good, it's good to help your neighbor, no regulations and we're experiencing tremendous growth in the 3D space, people are inventing new stuff, starting new businesses, etc.. all good stuff. but letting any jackass off the street print a working gun when we have how many mass shootings a year? People don't feel the same way. All the pieces are totally okay until you've got a more questionable global intention, and how can you regulate intention?
Google using the data to train models is just a tool, it's a baby step, they aren't doing that to sell the models or in and of itself, they're doing it so that they can generate data that they might consider theirs and not yours from your voice data and then feed that in to other systems which generate tremendous profits for them in ways you don't even know. They have intended uses already. Is it a remotely fair question to talk about ethical training in this context without some idea as to the intended use and distribution of the meta data?
5) Understand ML; concerned - "Why do other people in the ML industry think it's OK to use and store peoples data in without informed consent (which are only those in group 3, and group 1+2 don't have informed consent)"
I don't see the distinction between "if we wish to do a thing, we must do it ethically when doing so" (4) and "we must act ethically if we do a thing" (5).
If training data can't be collected ethically, do you think group 4 still would, or something? That just seems like trying to out yourself on an ethical high horse without a real distinction in action.
The distinction (from how I'm reading the GP's comment) is that Group 4 presupposes that data collection is necessary, and seeks to minimize unethical means of collecting that data, while Group 5 presupposes that ethics are paramount, and seeks to establish whether or not data collection can actually be ethical at all.
That is: Group 4 would be more willing to compromise ethics if absolutely necessary to get the data said group needs, while Group 5 would be more willing to compromise data collection if there's no ethical way to collect that data.
I agree. The way way the categories are worded essentially excludes the possibility that maybe we shouldn’t be training these models at all. We have banned potentially insightful experiments in both medicine and psychology because they are unethical. I see no reason ML should get a pass.
The Mycroft project has a better approach to this:
"Mycroft uses opt-in privacy. This means we will only record what you say to Mycroft with your explicit permission. Don’t want us to record your voice? No problem! If you’d like us to help Mycroft become more accurate, you can opt in to have your voice anonymously recorded."
Also there's a huge difference between short anonymized voice clips and taking a transcript of your entire house's audio, as a complete dataset, with your name and address on it.
I'm perhaps in a subcategory of (3) that falls under "Understand ML; concerned".
Knowing what I know about how people I have worked with have come close to or have actually mishandled data despite the best of intentions, I do not trust any of these teams without an explicit accountability mechanism that is observable by an outside entity. I'm not looking to punish slip-ups, because mistakes happen, but I am looking for external enforcement to keep people honest.
It's not that I think the engineers using this data are mustache twirling villains, it's that I think mishandling is inevitable due to inattention (yes, even you make mistakes!), and we have to design our data pipelines against that.
Exactly. Having worked in teams which handle personal data of consumers I know how easy it would be to misuse the privilege.
The legal and marketing teams that come up with the jargon and slogans about privacy are so far removed from the day to day operations that they have no clue about the reality. I don't they would care even if they did.
There’s a different dimension that may or may not understand ML, but are cognizant that any data created will be viewed at least by the company that creates it.
I fall into that category as I have time, nor do I trust any evaluation methods, to determine if a company is using my data ethically. If I create data and store it something that’s not mine, then I only do that in situations where I’m comfortable with the owner doing anything they want with it.
I understand ML and know that Google has to at least use it for training. I’ve also worked on IT long enough that even in super tight controlled environments data are misused by administrators.
> In conversations about this with engineers the response I've gotten is essentially: "Just trust that we [Google/Amazon/etc.] handle the data correctly."
No one is afraid of power when it's in their own hands. A common failure mode is that people assume a given power that's in their hands today will always be.
4 because not being explicit about the practice is misleading at best, outsourcing the difficult task of keeping the analysis private show how unimportant it's considered, and because big techs have a tendency to decrease privacy over time. Using clients who paid for the product as a dataset generator is also wrong.
But 3 at the same time because well, it's important to evaluate the performance of the product in the field not just in the lab. There were so many cases of catastrophic failures for ML models (ex. classifying black people as gorilla) that having a tight feedback loop is important.
It has to be done right, but evaluating a product that was primarily developed for (or at least by) English speakers and transfered to other domains seem like the right thing to do.
All in all, I don't and wouldn't use one of those assistants because 4 outweigh 3, but it's not binary.
>Tech workers have a responsibility not just (to) understand the technologies that they work with
Ok, I agree completely with you, 100%. However, based on my limited worldview, tech workers barely understand the tech they work with at all [0]. Asking for the ethical implications to be mulled over is unlikely to happen considering the near-weekly HN threads on "interviewing sucks, heres how to fix it, lol". We can't even figure out how to hire someone let alone how to impedance-match with them on deep issues like ethical implications of ML/AI.
Get real, obvious, informed consent by asking if you would like your voice prompts to be improved on / heard by real live humans as an opt in. I bet 1/500 of the population would opt in to it.
And the first one to do it should be apple itself.
Assuming categories 1 and 3 are sufficiently large (and I assume that is the case), this is easily resolved by allowing users to choose whether to donate their data for training or not.
If the training already only happens on a 1/500 sample, skewing the sample towards "people who don't care about their privacy" will probably not significantly impact the quality of the data.
I'm surprised this wasn't already the case, but hopefully the article will help the people responsible make better decisions in the trade-off between minimizing onboarding friction and respecting user's privacy in the future.
Asserting your point of view as "educated" and "correct" while labeling people who don't share it as dangerous. Doesn't sound like a great way to start a discussion.
I'm between 3 and 4: I just want proof that they remove PII from the audio files. If it's a bunch of audio files with unique IDs and metadata like time of day, count me as a member of group 3.
Even if I trust them to do what they say they're doing with the data I may not trust every party who comes to possess that data. And I may not trust their possession/use of it in all future contexts - as their privacy policy slowly drifts into the unknown year after year.
If they're collecting it in a way that can be requested by governments (for instance) or could be leaked by hackers that's another layer of valid "concern" not related to my understanding of the ML aspect of this.
The meta-issue in the United States is that once your data is accessible to a third party, you have no sovereignty over it, and abuse by private actors is "agreed to" by click-wrap and access by government actors is a simple subpoena.
The law needs to catch up. Sharing should require specific informed consent and legislation needs to establish a scope where data stored as a "tenant" on a third party server is given 4th amendment protection.
I agree, but I think this issue is incredibly mishandled by reporting. The title in the linked article being a great example.
There is absolutely no proof of number 2 in your list, but that is by far the widest-held belief.
It's infuriating, because we can't have a useful societal dialog about the issue if the largest chunk of concerned people are, essentially, conspiracy theorist.
TBH its not the Googles I'm worried about. They're under a fair amount of scrutiny, but ultimately, I think its mostly chicken little screaming regarding the 'how' of how they'll use this data. The ones I'd be more concerned about are the ones not on your radar. Google knows security, anonymity, they have share-holders, governments know to regulate them, etc.
What you really have to worry about is the companies getting data on you with out your permission that you have ZERO knowledge they are doing so, and as well, almost no ability to know they are taking data on you.
The reality is, most people are almost completely ignorant of how much real world data on them is out in the wild with no google or facebook or any tech giant involved. I can't get into too much detail on it, but the data is there. You just have to be savvy enough to find it, process it, predict, and refine.
The reason companies use lots of live data is because people get upset when facial recognition doesn't work on people with different skin colors, voice recognition doesn't work for people with accents, etc. Doing #3 is literally the result of #4.
Keep in mind that there's no rule that says there must be a correct answer. If the only way to do something is approach A or approach B, and people don't like either approach, then other options include figuring out a new approach C or not doing the thing.
This is in no way 'the reason'. THE reason is that using live data without having to bother with consent is 'fast, easy and cheap'.
Having to get consent is seen as a nuisance, and num's the word as we don't want to wake sleeping dogs. And we don't want to pay for generating training data, so yeah, we basically just steal the data from our uninformed users without their consent. I hope the GDPR will pursue this as a landmark case.
I don't think this is just a ML issue. Any device with a mic and connected to the internet could at any time be updates with new code that send audio back outside of what is needed for ML.
I think that issue is much bigger than just training ML based off real world data.
Laws might give you some recourse if you can prove this is happening or not. But really we need to start thinking about ways we can ensure the entire stack is under the users control.
I have said this in the past. Free software and open sourc movements have worked decades to open up software and hardware for greater transparency and user control. Yet within a few years we have wiped that out with all the closed hardware and software we happily hold in out hands and place in our homes.
We need more open source hardware and software projects offering alternatives to these systems that have become a daily part of our lifes.
I fall into (3) but not for the reason you mentioned. "Privacy" is largely an industrial age phenomenon and practically speaking we can never go back. I think we need new constructs to talk productively about issues like this and the research in this area hasn't even really started to take shape yet. We are only beginning to understand the implications of social graphs, information security, data as a product - let alone critically analyzing their intersection and having informed discussions thereabout.
Privacy is not a Industrial Age stuff... I could be private in Rome, just as I'm now, just a random dude working... A small cog, no one gets to know me, no on cares to gossip about me... Be it now, be it on Renascence in Venice, be it on Rome just before the senators kill Caesar, be it on Luoyang as the tensions rose between the Warring States...
At any point in time, I could have a private conversation, write and hide a document, like some stuff that was socially unacceptable and keep it a secret, meet someone in secret, and, in general, lead a private life... Stop comparing urban life as if it is something modern... Of course it is much more prevalent now, as the ratio vs rural residents shifted, and sure, it is much harder to hide something on a small village of some dozens of people, or at a farm with a huge family, but in any major city of any era, you could be as hidden and private as today, if not more...
Privacy is not new and examples go back as far you you look. Curtains, doors, walls, envelopes, seals for envelopes. Some types and bits of clothing.
Privacy is built into a awful lot of what people have done when you look for it.
Good breakdown. But I don't understand why any company thinks it is ethical to ever listen/record/send home data from -inside- a home. There are a plethora of public, or semi public places they could use to obtain real conversation(think of a subway, restaurants, DMV, etc). Set up a booth and let people talk to it. Using real speech from within someone's private quarters is disturbing IMO.
I dunno if I agree with you assessment, if you buy a device that purposefully made to record your voice and send it to a company, I wouldn't find it unreasonable that the company could listen to what I sent to them. It's not like they are being sneaky about when it is recording or where the recordings are going.
What we need is an AI to label the training data for ML to cut out the need for human workers to listen to people scream "Alexa" during sex. Or new laws to stop people from naming their kids Alexa or variations of it such as Alexander.
I am sure we can whip up some such AI easily with some quantum computing. Preferably in a blockchain so we can verify correct operation and scale better.
I understand ML and I'm concerned, but my position isn't "how can we train models/collect data in an ethical way?" Its more like "Do we really need to train models like this?"
I continually come back to the same thought: The Amish (to take a random example) consume probably thousands of times less than I do. They do without the conveniences of technology which I "enjoy". They have neither computers nor an endless stream of distracting media and social interactions.
Yet they are probably, on average, as happy as or happier than other Americans.
These technologies don't really exist to serve people - they exist to create profit, and they do that by meeting superficial needs.
There is great potential for machine learning to help the human race. But the vast majority of energy dedicated to the question by the information megaopolies isn't invested in that direction except under the most facile sorts of hypercapitalist justifications.
We should, in short, regulate data collection and analysis into oblivion, with exceptions made only for democratically determined use cases which serve the public good.
I'd like to offer a counterpoint to romanticizing the Amish way of life.
Love it or hate it, our current world would not have developed the way it has if everyone was following the Amish way of life. They enjoy a massive number of benefits that were born out of alternative-to-them life styles. For example we may have "an endless stream of distracting media" but we also have a hugely increased life span and reduced mortality rate due to advances in modern science.
To me, this is a bit like the "self-made" billionaire ignoring all the societal infrastructure afforded to them..
It doesn't matter if a company takes your data, does a poor job of anonymizing it, and then decides to label it as "training" for their "AI" or if they just stick it all in a flat .txt files and process it in Fortran.
It's the exact same thing. You should be mad about the data being saved and used. Splitting hairs over implementation details to try and find a loophole is just a waste of time.
You could also compare this to hospitals sending voice files to India to be transcribed. This is not automated at all. It's not clear that hospitals are any better at getting informed consent for this than Google.
1) A person going into a hospital, having their voice recorded, and then having the recording sent to another hospital where it might help treat and/or save their lives
vs
2) A company exploiting the lack of regulation and public knowledge/education on the dangers of mining personal data, to mine personal data and make a profit with no regard for the safety of the individual
If explicit consent had to be obtained, with the requirement that the person consenting be fully informed on the details of what they're giving up, in which of the scenarios above do you think people would be more likely to refuse consent?
Identity theft, regular theft, harassment, stalking, sexual assault, discrimination, reputational harm, etc.
Example scenario:
I tell a friend that I voted for Trump, my Google home hears it, a Google employee eavesdrops, leaks on twitter that I voted for Trump along with my home address, the likely times I'll be in my home, and even the pin to disable my alarm, etc. Then a group of left-wing extremists uses that information to harass/rob/murder me.
Alternate scenario:
Google employee uses their access to find an attractive woman with a Google home, steal nudes, spy on conversations, etc. That escalates into stalking, and eventually sexual assault and/or murder.
Both of those scenarios are possible today, and we're just supposed to "trust" Google is being responsible because they say so.
Whether these threats are realistic depends on how good Google's internal controls are. It's likely that there are Internet companies where internal controls are very weak (random Internet of things companies) and others where they are stronger. Stalking cases have happened, so you can say it's "possible," but to assess risk we need to do better than making a binary distinction between possible versus impossible.
In the case of the contractor described in this article, it sounds like they are pretty well isolated, so I don't see these scenarios happening: On the one hand, the audio snippets are more personal, being recorded in the home. On the other hand, having any idea who they're listening to will be rare, the snippets are short, and they are unlikely to hear the same person twice. I don't see them getting enough data to do damage.
You might compare with a store employee or waitress hearing a bit of conversation, or someone eavesdropping on your conversation or screen on a bus or plane. While people should be on guard, often they're not, and an eavesdropper can find out a lot more of any one person's data.
Other Google employees might have different access (for example tech support), but they'd be foolish to basically give employees remote root on Google Home devices, and I don't think Google security is that foolish.
I don't get your point here. You start off by questioning if the threats are realistic, then questioning if they're even possible, then you end by saying it's not that bad because waitresses can overhear your conversations too.
1) Those threats are 100% possible and realistic. If you think they're not just because the guy in this article is a contractor, then you're being incredibly naive and shortsighted.
2) Google employees have complete access to this data, and to think that they don't means you've decided to trust their word. Maybe you like Google, and that's fine, but it's not smart to trust them on this whether you're a fan or not. If their internal security policies for this type of data are terrible, they're never going to admit it and will definitely lie about it.
3) What people say in a restaurant and what they say in the privacy of their own homes are completely different. Can't believe I have to explain that.
> but they'd be foolish to basically give employees remote root on Google Home devices, and I don't think Google security is that foolish.
Why would you need remote root access when Google Home already uploads conversations to Google servers by default? That's the only part that matters.
Why do you think "Google employees have full access to this data?"
It seems strange that they would have permission, unless there were some reason it was necessary for the job.
This is sort of like assuming telephone company employees can listen to whatever conversations they want. Wiretaps exist, but it's not like just anyone gets to use them.
> Why do you think "Google employees have full access to this data?"
Because they do. It's literally there on their servers. You're assuming that they have some really good policies to prevent employees from accessing that data. Maybe they do, I don't know. But it doesn't matter because those are just internal policies. If some employee just says "fuck it" and ignores those policies, then if they're caught they'll just be silently fired and we'll never hear about it. There's no external audit; this is all unregulated territory.
Since this is HN, I'll give you a scenario that might hit closer to home: let's say you want to apply to work at Google. You send in your perfect application/resume, but you never hear back because your recruiter peaked into your Google Home files and noticed that you once told your friend that the Dodgers suck. Since your recruiter is a Dodgers fan, they decided to just throw your resume in the trash.
The one thing about these stories that keep coming out about the home assistants... they kind of create the impression that this is an issue specific to home speakers, and you can avoid it, by simply not buying them.
That's misleading.
Any voice command you use to operate any internet connected tech gadget, from phones to smart TV's, is potentially stored and flagged for human review.
You really have to avoid using voice commands at all, on all of your devices. Even that is probably insufficient. You probably have to go even further and actively disable voice command features on all of your devices, assuming they actually support such a setting. Otherwise here's still the possibility of an accidental recording taking a journey through the clouds, to a stranger's ears.
And not to be anywhere near anyone else's listening devices.
Isn't there a law in some US states that there needs to be consent before recording someone? How does this fit in and who would be held responsible, the owner of the listening device or the company behind it?
Yeah, this was a big issue about a decade ago when police officers could sue someone for recording said officers doing something illegal in Massachusetts.
So they redacted that law then had to fast track an updated version of it a few years later when someone got arrested for taking upskirt photos and then it wasn't illegal. My cousin was an aide to a state lawmaker and had to explain to his boss what it meant at the time.
> And not to be anywhere near anyone else's listening devices.
No assistant records all the time, not Google, Amazon, or Apple. They listen for the "Wake Word" onboard using more primitive (and lower power consumption) Speech Recognition and only utilize The Cloud after the "Wake Word" (or phase) is spoken. You can confirm this using Wireshark.
You can view and listen to your recorded speech on the Google Account Dashboard.
No, they don't, however, accidental activation is still highly likely.
I've had other people's devices activate during conversations where I couldn't figure out which part of the sentence activated them, it just happens.
"Ok, Go get..."
"Ok, Good ..."
"All except..."
"Sir, I..."
Then you have television shows and adverts that intentionally use language to activate these assistants.
Usually the assistants are going to say something when it accidentally detects a wake word, but if you're in another room or don't hear it for some reason, it can easily capture a conversation without you knowing.
I use the phrase “are you serious?!” a lot when I am frustrated. Almost always it wakes up Siri on at least one of my iDevices. Which makes me say “are you serious?!” again and it just spirals from there.
"Isn't there a law in some US states that there needs to be consent before recording someone?"
I doubt it - there would have been a test case involving one of the millions of people who have mobile phones capable of recording video/audio. If you're in a public place you should have no expectation of (that sort of) privacy.
I'm kind of surprised that Massachusetts's very strong laws about wiretapping permit storage of training data from Amazon Echo/Google Assistant/Apple Siri/Microsoft Cortana/Xbox, given that by their nature they naturally sometimes record incidental conversations of people who didn't intentionally trigger them.
It's a fairly mainstream view that MA wiretapping law requires written consent from every participant in a conversation for the recording of their conversation in a non-public place and does not permit implicit agreement (there isn't any kind of common-sense carveout for "you should have known you were being recorded in the background by the Echo at your neighbor's house"). See MGL chapter 272 section 99 (https://malegislature.gov/laws/generallaws/partiv/titlei/cha... , sample writeup https://www.masslive.com/news/2014/06/massachusetts_wiretap_... )
Now, MA has a bunch of laws on the books that no one actually enforces. There is a law against jaywalking which provides a $1 fine and tickets are never issued. Or for a bigger example, there is a law that requires you get a temporary permit from the Alcoholic Beverages Control Commission before importing any quantity of alcohol into the state, including for example buying beer at a NH liquor store or flying home from Europe with a bottle of wine. Last time I looked I think that law provides for a $2500 fine or 6 month jail time if violated, although it was hard to tell. ABCC will absolutely insist that this is a real requirement if you ask, and will even provide a copy of all such permits they have issued for the year under freedom-of-information rules -- I once asked and was given a copy of 46 permits issued in 2015, many to a single person who reviews wine and stubbornly files a permit for every shipment he orders from out of state apparently to protest the requirement, causing so much administrative overhead that the ABCC tried to issue him a special blanket approval to get him to go away, which he refused to accept.
To the extent that wiretapping laws are similarly not really enforced against the technology companies who make, retain, and distribute the recordings, this seems like an unknowably large regulatory risk a lot of companies are taking. Sure, the state loves its big local employers (IIRC Alexa development is in Cambridge?), and wouldn't want to lose their tax revenue, but what if the political winds change?
I think when you buy a Google Home you basically accept some terms and conditions in the app to set it up. Buried in there is probably your consent to analytics etc.
You already consented by agreeing to the terms of service. If someone else is talking to their smart device while you're talking, it's ostensibly their responsibility. There's no reasonable laws that prevent you from being overheard in the back of a recorded phone conversation
Most states are either one party or two party consent states. One party = you can unilaterally record anything (not sure this includes things you're not actively involved with, e.g. spying). Two party = you must have consent of everyone in the recording.
By a plain reading of two party consent statutes, people are in violation if their home speaker records a guest without obtaining consent.
I'm sure Google and Amazon's lawyers would try to weasel out of compliance via claimed anonymization, but that's definitely not the spirit of the law.
You're also going to bump up into specific wording on whether a given statute covers only telephone conversations or oral conversations, as most of these are phone wiretap laws that may or may not have been worked ambiguously.
Additionally, there are federal statutes that likely also bear.
Know that even in all party consent states, if you continue talking after being made aware that the conversation is being recorded implies consent. This is why devices like google home are legal, they make a loud warning sound before they begin recording. For example in CA the law states that:
> (a) A person who, intentionally and without the consent of all parties to a confidential communication, uses an electronic amplifying or recording device to eavesdrop upon or record the confidential communication, whether the communication is carried on among the parties in the presence of one another or by means of a telegraph, telephone, or other device, except a radio, shall be punished...
(b) For the purposes of this section, “person” means an individual, business association, partnership, corporation, limited liability company, or other legal entity, and an individual acting or purporting to act for or on behalf of any government or subdivision thereof, whether federal, state, or local, but excludes an individual known by all parties to a confidential communication to be overhearing or recording the communication.
> they make a loud warning sound before they begin recording
This contradicts my personal experience last week with a google-controlled music player. Music was the only response to a voice command to play music, and silence was the only response to a voice command to turn it off.
My understanding of recording law is that in One Party states you need to be part of the conversation to record it. Speculation: This would mean that the device / owner would be in violation if the owner was not in the room?
Sorry for the extreme analogy, but a gun owner is responsible if their nephew accesses their gun when they're not home.
It is not farfetched to imagine that, to be in compliance with the law, you would need to unplug your listening devices to avoid them accidentally going off.
The owner of a listening device might be ignorant of audio being stored and audited by human listeners. It's farfetched to think a gun owner might be ignorant of a gun's dangers because they didn't read the terms of service in detail.
So I buy one of these things; install it; put it online, but i'm not responsible for it? I don't understand. Would you be happy with that defence from a hotel if your wife stayed in a hotel room and was killed by carbon monoxide from a faulty heater?
Not in the case of the device being someone else's. I don't have any Google or Amazon smart listeners, so I never accepted them. Yet if my voice is in any of these recordings, well...
As long as you are warned that you are being recorded then the law considers you to have given consent if you decide to continue to talk. This is why all the home devices make a loud sound that you cannot disable before they begin recording.
Except that I don't know what that sound means. I know what the one my device makes is, but I've never heard the others. Unless it's a human voice saying, "This conversation is now being recorded" I can't be expected to know what a random beep from a device means. It could just mean the person got a notification or something. (And even if it is a recognizable sentence, it assumes I understand the human language the device is set to.)
The rules regarding audio recording are different from video. This is why many security cameras do not actually record audio.
I think the laws are different primarily due to the different pace of audio vs. video recording technology. Audio recording of phone calls etc. has been feasible for a long time so laws were written about that. Ubiquitous video recording has really only become a thing in the past 2 decades or so.
Absolutely. I feel like the pace of adoption has been mostly driven by per-bit storage costs falling (and high efficiency codecs).
Above all else, people will do useful things with computers once the price to do so matches the utility. And we're far on the other side of that with cameras.
I can't wait to see what the next decade+ does to all the Facebook-esque camera startups. It's going to be hard to monetize your customer's video feeds once regulation clamps down.
I don’t believe there are laws prohibiting video surveillance in public by businesses. Some states have laws prohibiting filming in locations where one expects privacy. Other states allow filming in private spaces as long as the business notifies employees and customers they are being filmed.
Yea most states also have a pretty clear "if the device is obvious" law. It's also why businesses put up "smile you are on camera" signs, especially if their camera isn't immediately recognizable.
There are some rules concerning security cameras as well. I recently found out that private CCTV (at least in the UK) can't record public areas (e.g only your porch). Someone got sued over this recently.
There have been some stories in HN about opting out of face recognition as well. Maybe the laws for video are different as the other reply says, but there are privacy concerns in there as well.
edit: here's a list of GDPR fines (not comprehensive as I only see 2 in the UK). If you filter by CCTV you'll find a couple of examples from Austria:
http://enforcementtracker.com/
Face coverings are not legal in public in all countries, but even where they are, why should someone concerned about creeping surveillance go to great lengths to modify their own behaviour because someone else is unwittingly breaking the law because a product they bought was made by someone who couldn’t be bothered to do it right?
So this outrage is in the Netherlands. Security cameras may not be pointed by businesses at public space (which we have a lot of unlike the US). The local government itself may place cameras though but private parties, what kind of nightmare situation is that?
And in public or private space when there are cameras there needs to be signs everywhere to warn and inform you.
So in the Netherlands at least.. Google recording a conversation with someone who doesn't know Google is recording is definitely illegal.
The question is: will they prosecute? Then it becomes a geopolitical question because we are a small country with a disproportionate number of Google datacenters.
So to summarize:
- This is definitely illegal in the Netherlands
- There is no consent of others participating and you really do need that
- Fine print is not consent: consent of terms and conditions requires a majority (determined by polling or common sense of a judge) of users to be aware and knowledgeable what they consented to.
- there won't be prosecution by the Dutch public prosecutor.
- there will be a lobby for the EU to buttrape Google but it may use different reasons or context
I find this reasonable; you wouldn't sue the manufacturer of a recording device if someone made a secret recording with it. I do think this means that the smartphone owner is liable, and should be fined or jailed.
But in this particular case, it's the company doing the recordings, not the owner of the device, so it's a slightly different situation, in my opinion.
That said... The owner could be liable if for example it were necessary to explicitly inform of the existence of such devices the possibility of being recorded.
Not to mention all the crap you type into your search bar, Gmail, images I upload, basically anything with any sort of machine learning backed enhancement. Any of that stuff could be sampled and reviewed by humans. Admittedly anonymised, but it's still potentially personal stuff just as the voice print data is.
Yes that’s true. But: with another device like a phone it is optional to turn a voice assistant on. Not so with a device with the sole purpose of being a voice assistant.
Whenever I'm in the home of a friend who's enthralled with their smart speaker gadget, I like to make red-flag-raising requests to their gadget. Asking for good sniper vantage points in Washington DC, a safe rohypnol dosage for 10-year-olds, the soonest flight to a non-extradition country, those sorts of things.
When I give a voice command, I know that my voice could be recorded. What I'm actually concerned about is devices which are always listening, even when I'm not giving a voice command, and recordings from when I'm not interacting with the device potentially falling in the wrong hands.
So Google’s response is (paraphrased as fairly as I can while removing the sugar-coating):
’Yes, we hire people to listen in to and transcribe some conversations from the private homes of our customers (so as improve our speech recognition engines); but the recordings aren’t linked to personally identifiable information.’
Even assuming they have only the purest intentions here, I still don’t understand how they can possibly guarantee that these recorded conversations are not linked to personally identifiable information!
For example, what’s to stop me from saying “Hey Google, I am <full legal name / ID> and my most embarrassing and private secret is <...>”?
One might argue that they could detect this in the recognized text and omit those samples, but presumably the whole purpose of hiring people to create transcripts is because the existing speech-to-text engine isn’t perfect, and they need more training data.
“I rue the day I married you, Steven Robert Parker, you HIV-infected cheating scumbag! I wish I had never lied to the FBI about those classified documents you stole!”
It seems even worse than this - I'd argue your voice is personally identifiable information! The vast majority of these clips open with "Hey Google".
Meanwhile, Android allows you to personalize voice commands based on its ability to recognize that a specific person is the one saying "OK Google". Voice authentication has already reached high accuracy with a few seconds of unconstrained text, or a few words of fixed text. Voice identification on open sets takes more data, but sub-minute clips are still reasonably effective.
At the very least, Google itself could make a credible attempt to identify whether the speaker in any voice clip heard by Google Home is a regular user, and plausibly de-anonymize users of OK Google. More alarmingly, we're told that about 1 in 500 Google Home clips is heard by a human, and this employee apparently shared "thousands" of clips with a news organization. It seems plausible that anyone with access to any large voiceprint database could attempt to obtain clips from a random contractor and de-anonymize the most interesting or salacious content.
You paraphrased it in a different way and that might be why you're confused.
Google says "the excerpts are not linked to personally identifiable information." To me that means the metadata is stripped, not that they strip anything out of the audio.
Thank you, good catch. I’ve edited my paraphrase to make it more accurate in this way.
That said, it still sounds like Google is trying to convince us that the data they capture (not just the metadata) is never linkable to personally identifiable information, which if true would genuinely ease many privacy concerns here.
As far as I know, just because data is not explicitly annotated with PII doesn’t erase the legal (and ethical) responsibilities associated with handling data that contains PII.
So even if they worded their response so it’s truthfulness is legally/technically defendable, it’s still a bit of a ‘red herring’ at least (I don’t think anyone is accusing Google of explicitly associating these audio recordings with user IDs).
> For example, what’s to stop me from saying “Hey Google, I am <full legal name / ID>
Even more fun, if you call a bank, you often have to key-in your account number (which can be easily decoded if your phone sounds back the tones, which most do), then tell you name, your address and sometimes your other PII like Social Security number or part of it. Record that call and that's a complete identity theft package, nicely wrapped, just replay it to the bank (which name you've also have recorded, if the user called on speaker, which they did because who wants to keep the phone pressed to your head all the time while you're waiting and listening to the muzak) and you get full access to the user's bank account.
I'm not sure Google devices can make calls, but if they could, the only part that would be sent to Google (which is what these people would have to analyze) is "hey Google, call bank"
From what I understand, you don't have to call bank using Google Home device, enough that you'd call bank while Google Home device is within the earshot while something else says "OK google" while you're talking.
I would count a recording of my voice as "personally identifiable information" right off the bat. Voice printing is a thing, and anyone will also tell you that they recognize the voices of people they interact with regularly. If someone played an audio clip of someone I know talking to Google Assistant to me, I would recognize who it was based on their voice.
This sent me down the rabbithole of learning how identifiable voiceprints are. As you might guess, the answer is "very", although to my surprise our voices change enough that recordings lose a great deal of fidelity over a few years.
Authentication on fixed phrases is reasonably accurate within a very few words, so at minimum it should be possible to associate "Hey Google" clips with regular users of Google Assistant voice control (i.e. "OK Google"). Identifying whether someone is present in a large dataset on open phrases is much harder, but a ~30s clip could do the job fairly consistently for anyone with access to a significant amount of voice data. And if this employee (who isn't directly working for Google) shared 'thousands' of clips with a news org, the cautious bet is that some other employee might share them with anyone willing to pay for the records.
> ‘personal data’ means any information relating to an identified or identifiable natural person (‘data subject’); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;
Anything that falls in that 'personal data' segment above that belongs to me has to be obtained with my prior, explicit consent. And these bits of data with my name or other details in them must be included if I send Google a "right to be forgotten" or "show me all the data you've got about me" request. That's GDPR in a nutshell.
This might be a grey area for now, as both GDPR and listening devices are both quite new. But Google, Amazon & co aren't super popular with EU regulators and governments, so they might side with users' rights on this one.
Without the metadata PII, there’s no evidence adversarial utterances like that are true.
It’s hard not to feel like this outrage is trumped-up anti-Google FUD. So many more worthy fronts to assail Google et al. on!
After all, they let you upload photos and video that are, per various policies and with some non-zero frequency, reviewed by humans — and users are begging them to do it more often.
"The man, who wants to remain anonymous, works for an international company hired by Google. "
So not a Google employee at all, a probably low paid contractor who is in possession of thousands of audio files. Your privacy matters, except when the bottom line is involved.
What is doubly concerning here is that the contractor was in a position to demonstrate how the system worked to the reporters. That would seem to indicate they have access to that data in a non-secured environment.
I'm not familiar with EU law around these things, but I would imagine there is some kind of whistleblower mechanism available, and a right for authorities to audit/inspect such activities?
I would expect that a telecoms employee who was doing similar work on quality etc to be quite securely vetted.
If I was doing now what I did back in the day For BT / Dialcom (I had root on the UK's main ADMD) I would probably have to pass DV vetting (TS in USA terms)
What about all the telecom APIs like Twilio? They have raw access to millions of phone calls every day. I doubt they have ‘secure rooms’ for debugging.
The person is probably a temp/vendor from a consulting company (think accenture or cognizant), who should've signed the same NDA agreements as anyone working on that stuff.
But whose machine monitoring, security, and use habits almost definitely do not match the requirements Googmazon would require for its own employees. These vendors, time and time again, end up being the weak spots in companies' and governments' handling of sensitive information.
and how are they going to enforce the NDA? contractors are being paid peanuts in bad working conditions - firing them might land them a better job elsewhere, and they don't have any assets to sue for any kind of monetary damages.
That can still bankrupt the person on the receiving end of the lawsuit. Just because damages don’t necessarily fully compensate for the loss, doesn’t mean it’s not a massive deterrent to the behaviour.
Does it matter how much they're payed? They're probably payed the right amount relative to the work they are doing.
Also how is having access to small samples of audio a privacy issue? Are they also receiving enough information to attach an identity to the audio clips? How long are the clips? Are they randomly assigned to humans? Do those humans get to listen to multiple clips from the same Home device and can they tell that's the case?
Home, Siri, Alexa, M, they all do. I have friends that work on this field transcribing the audio, and measuring its accuracy. Sometimes it's multiple layers of contractors: An employee hands the task to a contractor, another contractor verifies the speech to text, and they're all managed by a contractor.
“... In some cases, teams use the audio of users’ voice requests as training data—all anonymized, Apple says.
> We leave out identifiers to avoid tying utterances to specific users, so we can do a lot of machine learning and a lot of things in the cloud without having to know that it came from [the user],” Joswiak said. In other words, Siri can learn things about users as a whole without tapping into individuals’ personal data.
> Apple holds on to six months’ worth of the user voice recordings to teach the voice recognition engine to better understand the user
> After that six months, Apple saves another copy of the recordings, sans user ID, for use in improving Siri, and these recordings can be kept for up to two years.
> The training happens on Apple’s servers, but the models only start practicing what they’ve learned when they’ve been deployed to your device.
> Once on the device, the models begin to run computations on things you type or tap into your device, or on things that are seen in the device’s camera, heard through the microphone, or sensed by the device’s sensors. Over time, this creates a massive pile of personal data on the device, as much as 200MB worth.
First hand source, it was the first job several of my Brazilian friends (or their spouses) got when they relocated to the Bay Area. They use companies like Moravia or Welocalize. Take a look at some of the job posts from my link above.
Apple still stores the audio, but they said they can't allow you to download/GDPR request your recordings (like Amazon and Google allow you to do) since they're not associated with your Apple ID whatsoever. I wouldn't be surprised if they also human-review some audio.
I'm not familiar with HomePod, but if I ever get an Alexa/Siri kind of assistant, it will be one that analyses my voice locally rather than sending it to the cloud.
Ah, who am I kidding? I just bought a new Android phone, which probably does exactly the same thing. Time to install LineageOS on it, I guess.
I grew up as a kid in a country ruled by Securitate [1], one of the few institutions that rivaled the East-German Stasi when it came to spying on its own citizens, and as such I'm very, very perplexed of why would anyone bring in a listening device in his/her own house out of his/her own volition. And those people even pay for the privilege of having their home-lives actively monitored and listened to almost all the time, it's crazy.
I would imagine that for the people that didn't grow up in such a country ruled by Securitate do not have the experience to make them fear being listened in. Not saying that they are wrong (they may turn out to be right), just that we are all products of our experiences.
Do you have a smartphone? Why would you bring that listening device (the smartphone) into your own house out of your own volition? Please explain, because I am very perplexed.
This is such a specious argument and yet repeated ad nauseam. Please stop. For one, it doesn't make the listening and harvesting of data ok, just because it may already be happening. Also, it's just condescending. You don't think the parent or other people of reasonable intelligence and valid concern haven't thought about that? Then, it also just misses the point anyway -- no, I'm not OK with my cell phone harvesting anything and everything it can get (cadence of my walk, say). Yes, I like having access to maps on the go. These concerns aren't mutually exclusive, and most of us choose not to live like Stallman. We still live in a rich and complex ecosystem of law and precedence, highly dependent on where we are on a given day, what nationality we are, etc. None of that invalidates the quite reasonable expectation to privacy (not to mention that solutions exist such as opt-in, but cavalier people working in tech choose instead the race to the bottom of privacy and chasing cost-per-click).
Right, I'm not saying it makes it ok, I'm saying OP is a hypocrite or doesn't understand what technology is capable of. But of course if you believe any device capable of listening is listening, then you have to forfeit your smartphone along with your home automation device.
It's relatively straightforward on a smartphone to monitor the battery and bandwidth consumption that active listening and device<>server communications are occurring.
Of course - then, the next apologist will always say: "the devices only transmit when hearing the 'Wake Word'"...
Yeah, I don't think it takes an elite programmer to store/buffer audio, until the device hears the 'Wake Word', and then sends a "slightly" larger payload... (or batches several payloads)
The next level of enthusiast evangelist will then tell us that it is impossible for these devices to buffer/store that much audio data, because they have limited memory...
Yet, the same devices when used to play audio streams from the internet obviously have some sort of storage capacity to handle network glitches.
Spoken audio (which is probably recorded at standard telephone/cellphone sampling rates - after all, why train on audio that is significantly different than the audio coming from mobile phone devices) does not take up a dramatic amount of storage space.
I think that may have been true in the past but I would not at all be surprised if modern smartphones can record 24/7 with negligible effect on battery life, and upload the audio while being charged.
I do have a smartphone, and I also do know that by default my phone is not actively listening on me (if it matters I do not use Siri-like services and I don't intend to). This is not the case with all these Alexa-like devices, but correct me if I am wrong.
I don't have a smartphone (never have) and I don't take my dumb nokia with me unless the battery is out.
You're right. But it doesn't mean that these home voice assistants are okay. It just means there's a much, much worse problem with smart phones people aren't willing to acknowledge. And those of us that do and act accordingly are treated as weird paranoids.
Not only that, but people still buy computers even though the Nazis used IBM computers to help them perpetuate the Holocaust. Surely you shouldn't use computers if the NAZIS used computers!
I think Frank Herbert hit it a bit more on the nose:
“Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.”
Frank captures the motivation, the fact that it is desirable is the scary part.
Akin to the genetic engineering scene from Gattaca.
Iain Bank's Culture series deals with self-conscious machines with the same rights and freedoms as humans. Each machine has a value denoting its mental equivalence to a human, and some starship Minds are close to gods (similar to comparing a human to amoeba).
'Never forget I am not this silver body, Mahrai. I am not an animal brain, I am not even some attempt to produce an AI through software running on a computer. I am a Culture Mind. We are close to gods, and on the far side. ‘We are quicker; we live faster and more completely than you do, with so many more senses, such a greater store of memories and at such a fine level of detail. We die more slowly, and we die more completely, too. Never forget I have had the chance to compare and contrast the ways of dying.’ [1]
Robot series is good, Foundation series by him is brilliant.
Also check out The Culture novels by Iain M Banks (he was the closest recent author to be a great the equal of Wells, Huxley or Orwell (imo) just a phenomenal writer).
The Polity series by Neal Asher are well imagined and he world builds brilliantly.
What you want to read by Asimov (much more relevant to the present topic), is this story "The Dead Past",
(do not click on the below link as it would spoil it):
However the final words by Araman (which I won't cite, again to not spoil the effect) are memorable and very, very suited for these times.
The April 1956 "Astounding Science Fiction" where it was originally published is available via Internet Archive, read the story first (it is just 40 pages):
Sorry, it should have been "framing your relationships". Today having a good profile in social media does have an impact on your social status. It's not difficult to imagine a future where things you say at home have an impact on how your profile gets presented to other people.
I'm not sure why it's unreasonable for customers to have assumed this is like every other product: they developed it with employees and voluteers before shipping.
After all, the voice recognition in my Prius doesn't send recordings of my "call mom and dad mobile" back to Toyota and it certainly doesn't send random snippets of conversations my wife and I have to low paid contractors around the world.
> After all, the voice recognition in my Prius doesn't send recordings of my "call mom and dad mobile" back to Toyota and it certainly doesn't send random snippets of conversations my wife and I have to low paid contractors around the world.
I mean, I would assume that humans are used to improve the speech recognition capabilities of voice assistants. Accents alone are a big challenge to voice recognition, not to mention regional dialects, and the more complex languages.
People listening to what is said to a voice assistant after the prompt doesn't really alarm me. What would be alarming is if the voice assistant was secretly recording normal conversations.
What is usually shown, time and time again, is that the voice assistants aren't secretly recording. They are triggered because they heard something that sounded like a hotword.
Maybe you were talking about what dog you want you buy. Someone asks if a Greyhound would be alright and you said "No way, Poodle!". Well, the device picked it up as "Ok Google" and begins listening for a command. It makes a bleep, and lights up and you start talking about dogs or whatever.
You maybe didn't notice, so it picks up 15 seconds of a conversation, has no idea what to do, and probably just does an image search for dog pictures or even just the "I don't know how to help with that".
Not to say you shouldn't be annoyed that it picks up unrelated stuff... I just always think "secretly recording" is such a misnomer.
Your phone has more sensors on it than these devices and most people have their phones on them 24/7. And most phones have the same voice-controlled features too. Why people are concerned about narrow scoped voice-devices more than their phone is a curious thing. Is it because it talks back to you that makes it more apparent?
Most people assume that the system is entirely automated, and that their commands will be processed by mere computers, not listened to by actual people who might even be their own neighbours.
I own 4 Home Minis, 1 Home and 2 Home hubs I honestly don't care so long as my data is used to improve the functionality and stability of my investment. It is quite another thing if they are selling my conversations to third-party vendors.
1. If the company has recordings, governments can hoover them up. And once one govt agency has them, sometimes other agencies get access.
2. Sounds like you're okay with Google using your private communication to enhance its products. So they can target you with ads based on your conversations with your spouse, children, or colleagues.
3. I just hope the algorithms understand context:
Your spouse: son, put down the gun and come to dinner. [jokingly] Oh no, don't shoot! Argh you killed me, now eat your peas.
You: we're going to kill it this quarter. They'll never know what hit them.
TV in background: "kept their body chained in a basement. That and more at 11"
Only if the government subpoenas your data individually.
Also, the recorded data in this case is stripped of PII, meaning it's not linked to you. If you're still worried because you told Google your social security number, you can review what is stored, opt-out, and even mass delete the recordings at myactivity.google.com.
> The man, who wants to remain anonymous, works for an international company hired by Google. His job is to listen to the audio clips and to write out what he hears, so that Google can improve the speech assistant.
So,
> It is quite another thing if they are selling my conversations to third-party vendors.
The concern remains the same though: Making user data available to a non-Google entity.
Do you care if Google makes money off of that trade? Really, they get value one way or another, because the analysis is incorporated back into the product.
I mean. Of course they are. Do you expect to be able to do any meaningful level of training on data that hasn't been properly labeled? At some point, a human has to go in and correct the software when the software gets it wrong. If you want services that do what Google Home does, you have to have this.
Even with that, I'm sure that the engineers are flagging voice requests that happen more then once, or where some one has to manually change or correct what the software thought was the request.
This is only creepy if you don't understand how the software works.
I wonder how people felt about handing all their personal photos over to a stranger just so they can get developed and printed, back in the 1990s. By today's standards, that would be incredibly creepy. But everyone did it because... having photos is pretty awesome.
It would be nice if companies like Google made it very clear that specific forms of user-data can/will be human reviewed for development/operational purposes. Personally, I just assume that anything I say after activating a digital assistant will be anonymized but listened to by a human one day. And I still think that Digital Home Assistants are pretty awesome despite that.
> I wonder how people felt about handing all their personal photos over to a stranger just so they can get developed and printed, back in the 1990s. By today's standards, that would be incredibly creepy. But everyone did it because... having photos is pretty awesome.
This had its own set of norms associated with it: people would not take in photos that could be considered "indecent". Possibly by the extremely conservative standards of your local chemist. Then Polaroid invented a camera where you didn't have to submit your photos to the judging eye of someone else ...
Anyone who's worked in a photo lab before will tell you that "norm" gets ignored all the time. Some people just do not think about the fact that an actual human is going to see their BDSM photos with the secretary.
I worked in a photography store in Italy in the summer 30 years ago or so. That's not always what was happening.
Neither I nor anyone else working there ever looked at anyone's private pictures, but the guy delivering them from the lab would sometimes joke about or allude to customers' photos, always from some other store. Even as a kid, I found that troubling. It also made me wonder what he told other stores about our clients' pictures.
How about Google PAY MONEY to generate training data or gather it from informed people? (“Make $100 by recording your voice for Google’s machine learning algorithms.”) not “Arms full with an infant? This $50 device will solve all your problems” and then shipping those recordings to third party contractors in unsecured facilities.
Figure out how to run your business unit ethically or shut down the business unit. They don’t have the right to turn their transfer learning problem into an abusive privacy policy.
Yes, if they just paid people to record their voice they would not get training data for real use cases.
I cannot find the blog post now, but quite a few years ago I recall some Google employees noticed a large number of queries for "cha cha cha cha cha..." from Android users in New York. All of the queries were done using voice search, so they listened to a few of the recordings. It turns out that their speech-to-text models were interpreting the sound of the NYC metro pulling into a station as speech.
Obviously they didn't have enough training data of people trying to talk next to a train.
We test our medicines on a small group that is representative of the entire worlds population. We build soil models based on sampling a small region. We don't test your entire blood to do a medical test. I don't know what you mean by "real data", but representative sampling is how work gets done in every single domain in the world. Google can do this.
Representative sampling is how we formerly did this kind of work. It wasn't particularly good or effective, but we didn't have the methods or compute to go beyond that. No longer.
You're free to have your own opinion, but anything specific beyond "it doesn't work"? I work in pharma, and we use representative sampling every single day in every single thing we do, and it works.
Representative sampling, does 'work' in the sense that it may or may not 'prove' whatever it is you had a question about. But the issue is that you effectively building in your assumptions about what is 'representative' into your sample. Its (imo) the central issue in the reproducibility crisis: our assumptions about the world and how that impacts the questions we ask about it.
It was previously intractable to do a census rather than a sample, and maybe for your purposes a sample is good enough or a census remains intractable. In my field , this is how things were done for decades (and still largely is), and even though (imo) it did a piss-poor job, it was good enough for some purposes. As piss-poor job is still better than knowing nothing. Maybe this is good enough for your purposes.
There's a third way however, which is to move beyond sampling and to perform a census. This is the difference I'm speaking of. We're at the point where we don't have to sample because we can measure. Effectively, this is what modern data science is. We've always had the ability to sample and interpolate. It doesn't work very well (imo: https://en.wikipedia.org/wiki/Replication_crisis) and usually is reflecting back to us something about our assumptions in how we sampled. But thats just it. We don't have to rely on a sample if we can take a census.
>But the issue is that you effectively building in your assumptions about what is 'representative' into your sample.
Even if I agree with your premise, Google is not going to build a custom voice model for every individual anyway. There will be simplifications made. There will be assumptions made, and they will end up with a representative model anyway. So you're actually just bolstering my point. It makes a ton of sense to record people in a known, controlled environment and tweak variables one by one- such as the size of the room, the location of the microphone, introducing varying amounts of background chatter, etc etc. This is how normal science happens all the time, and it has worked for us so far. And we haven't even addressed the ethics of spying on people in such a blatant manner. That is a whole another conversation.
Modelling aggregate human behavior/psychology is not a proper science. The same is true of macro economics and other such non-exact fields. Problems in those fields do not apply across other fields.
This is a very different kind of problem than the ones you listed. One drop of blood is going to be very similar to any other in and individual. That's not true when it comes to language data (or many other types of data for that matter). The data you would record in a prepared setting (i.e. reading from some predefined set of phrases) is typically not even close to representing the full distribution of phrases/dialogues that human's use.
Furthermore, Google/Amazon/FB do use representative sampling of real user data, it's not feasible to transcribe every interaction with Google Home/Alexa/Siri. This akin to what you're suggesting but it no way addresses the privacy concerns. The only real way to do that is to use authorized data or scripted interactions, which, as described above, are not actually representative samples. It is complicated and nuanced problem.
>One drop of blood is going to be very similar to any other in and individual. That's not true when it comes to language data (or many other types of data for that matter).
Why? Please do explain. If you claim that our biology doesn't change at all in one domain, but varies significantly in another, it would be easy to show this scientifically, or more specifically, how this variance is applicable in this context of voice recognition.
Just to take a simple example of blood glucose. Using continuous glucose monitors attached at various sub-cutaneous sites over the body, it is trivial to show how the local glucose is not identical at all sites.
I'm sure they did. And probably still do. But no amount of paid for training data is going to cover all situations that occur in real life. Accents, mannerism, speech imperments - you can't cover all possible premutations.
> I'm sure that the engineers are flagging voice requests that happen more then once
As a user, I expect to see feedback declaring that it's having trouble and specifically requesting permission to send my unparseable requests to some review queue. I expect to see feedback indicating what could resolve the issue: was it something I said? was there too much noise in the background? was there a software fault and a fix will be deployed at such-an-such date?
The fact that I do not get any feedback whatsoever when things go wrong leads me to have no faith that the problem will be resolved in any satisfactory manner. When combined with a complete lack of consumer/business interaction options in general (w.r.t. Google), it leads to some very dissatisfied consumers.
You think engineers are flagging voice requests that happen more than once? How would engineers have access to that data in the first place if it's supposed to be anonymous and private?
>The fact that I do not get any feedback whatsoever when things go wrong leads me to have no faith that the problem will be resolved in any satisfactory manner. When combined with a complete lack of consumer/business interaction options in general (w.r.t. Google), it leads to some very dissatisfied consumers.
I guess you can ask for your money back? Lets stay grounded in reality here, Google is paying money on your behalf to improve your experience. They could, and from a financial perspective maybe should try getting you to annotate your own data. But all that annotated data gets pooled and used to improve the models. I don't think their goal here is to give you faith they are resolving your issue. The algorithm is going to get some things wrong. Its a matter of improving the overall accuracy and precision in the algorithm.
>You think engineers are flagging voice requests that happen more than once? How would engineers have access to that data in the first place if it's supposed to be anonymous and private?
100% they have to be. Its too costly and time intensive to slog through all the data. Probably* they have a flag that goes off on a voice request when something is either not intelligibly interpreted so many times in a row. Or if a use has to go manually do something after repeatedly making a request. This flags the interaction for manual review. Then it processes through some algorithm to strip it of identifying data. Then it gets put in front of a warm body for review.
Trust me that the company wants to avoid putting things in front of warm bodies as much as possible. Its expensive.
> Lets stay grounded in reality here, Google is paying money on your behalf to improve your experience.
Lets stay grounded in reality here, Google is not doing this for me or on my behalf. Google just invests a bit to take a bunch of very valuable user data and then monetizes it for far more than it took to obtain it. This relies on the fact that most users are unaware of how valuable their data is.
As a business model there's nothing illegal about this. But in any other sense it's no different from tricking an uneducated individual into selling their kidney for $2000 just so you can resell it for $20.000 and pretend you're doing them a favor by giving them money.
And as long as the vast majority of customers are in the dark regarding the value of their data and what they're actually trading when using such a system then yes, this is Google (and not only) abusing ignorance to line their pockets.
Can you imagine how valuable voice data could be if it could be mined to show what products, politics, opinions are being discussed in the real world?
Can you imagine how valuable your voice data would be to a marketing campaign which you didn't even know you had participated in?
Can you imagine how much valuable information is contained in the vocal enunciation of A/B testing? Every little "hmmm" or "how do I go backwards?" or "what does this button do?" that people don't even realize they're saying.
Can you imagine how much that violates someone's privacy?
> other then by selling devices that use voice recognition
That's one way but what's the actual question? My point is they are not doing this as a favor to the user (as GP seemed to suggest), they are doing it for a profit by getting the user's data far too cheaply. And for this they rely on the user staying unaware of the value of their data, how it will be used, and how much is collected in the first place.
The voice recognition tech itself (and any ancillary parts) can be sold/licensed to so they can build similar systems. The actual data obtained by the voice recognition can be used exactly like any other data Google collects. They literally have access to what you say around their microphone. You can't not see any way they could make money from this.
20 years ago monetizing the free search was just as much of a mystery for many, including seasoned investors.
It is still slightly creepy because of which humans are doing it. Is there any reason that these systems can't ask the users themselves about the accuracy of the speech to text conversion? Not everyone would do it but a percentage would.
Unsupervised learning is totally a thing! Word embedding models are achieving STOA results on massive unlabeled corpuses. That's what's powering most of the new results in NLP.
Clustering and Dimensionality Reduction folks just sitting here thrown out to dry too...
I thought only samples of audio from speech that is addressed to the Home device (starts with "OK Google...") is sent to humans. That by definition makes it not a conversation in the privacy of your home, it's a conversation with the assistant just like having a phone conversation with someone else in the privacy of your home implies that someone else has access to the conversation.
You will note that the mute button is not a physical switch that cuts the signal from the microphone, but a soft button. It will probably do what it claims most of the time, but if Google for whatever reason wanted to secretly unmute it remotely, I have no doubt they could.
Wireshark? Sure, but what am I even looking for? They could be holding on to recordings in storage to send them later when the device is unmuted again. They could be embedding audio in encrypted form into other innocuous-looking packets -- I doubt a device like this is quiet on the network even when muted.
Until all the software is open source and auditable and the switch verifiably breaks the physical signal path to the mic, a device like this can never be trusted.
It's precisely knowing how software works, and the limitations of the current technology, what makes it so creepy. With this system there's no way of avoiding humans hearing a clip they shouldn't have heard.
- Ensure that the persons doing this review do not have physical access to the systems outside of a secured environment (one in which outside audio and video recording devices are not allowed, and for whose presence is monitored, with physical access controls, etc.) Basically, not remote workers or a typical office environment. Most finserv call centers do this, so it's not particularly crazy to think they can do the same.
- Mask the voices such that they are intelligible but not identifiable. Maintain a limited set of "high access" taggers who can hear the raw clips if there is an issue with the masking.
- Limit the length of the clips (sounds like they already do this).
- Have pre-filters for anything personally identifiable in the audio. The metadata for the audio might already be de-identified, but what if the audio clip consists of the person reading out a phone number, credit card number, username, etc.? They should have their "high access" team building detectors for that and flag those portions of the audio or whole clips and route them to a limited access team.
- Make it more clear to customers that their audio, including accidental captures, can and will be sent to their servers. Make this very explicit, rather than burying it in TOS and using terms like "audio data". "The device may accidentally interpret your bondage safe word as a trigger and send your private conversations to our real-live human tagging team for review."
- Provide a physical switch that can temporarily disable the audio recording capability.
- Pay money, like cigarette companies do, to help fund a public education campaign that informs the general public about these listening bugs and mass surveillance issues so that people are aware of industry practices and how it affects them.
Edit:
I like what others are saying about explicit opt-in, as well as paid end-users. For quality/safety control, I don't know that they can exclusively use paid end-users. They probably need to sample some amount of their real live data. For that, explicit opt-in makes sense.
How about 2 Google Homes: The $100 Privacy version and the $30 Opt-in Testing version. That way some people get their privacy and I get Chicken Little to subsidize my cheaper IoT products. =)
The problem with this is that "Chicken Little" is probably not an adventurous person, but rather a poor person. This makes privacy another privilege that can be bought. This devolves into another way to exploit poorer people.
>From a computer science perspective, what should Google do to train its models in a privacy conscious way?
Install these devices in the homes of google employees, executives and offices and allow the public to listen in. What’s good for the goose is good for the gander and all.
Maybe when google has trained the systems enough to not need to train them by collecting and listening to customers conversations, then they enter them into the stream of commerce.
I'm not sure if Google has a diverse enough employee base for this to work. I'd imagine most Google employees are tech workers so the training data collected would not accurately represent the general population.
Not necessarily. I've had Google Voice (And maybe iphone?) ask me if they can use my voice-mail to improve transcription data, and I always say yes. But it's an explicit ask for each time (If I remember correctly), which is nice.
Use test subjects who sign up explicitly and specifically for this purpose, rather than burying it in a massive TOS written in legalese that can change at any time and is applied indiscriminately to all customers
Not true. Apple has demonstrated how to do it and has published a paper on the topic.
A. Bhowmick et al. Protection Against Reconstruction and Its Application in Private Federated Learning, 2018.
Don’t just dismissively say “oh that’s different and doesn’t apply” until you read it and understand the range of problems Apple is tackling with this and similar approaches. A lot can be done when the organization makes privacy a priority.
Let’s not forget I am replying to a person who has chosen to use a throwaway account for this topic. Why did they do that? Hmmm.
Hah! Good one. I hope not. I think the Siri people at Apple have just gotten lazy. Maybe they are too impressed with themselves and focusing on BS things like sports scores. I hope they get more ambitious soon.
^This. I'm sure they're optimizing to only transcribe data they are pretty confident they got wrong (multiple requests; use has to go in and correct), but if you want software to do what the Google Home software does, you have to do this.
The biggest issue IMHO is how the average consumer has been deceived into the belief that current AI is pure AI, when in reality a lot of humans are looking at your pictures, listening to your recordings, crawling through your inbox and analyzing your browsing/purchasing/streaming history, right now: https://imgs.xkcd.com/comics/trained_a_neural_net.png
I think a lot of people here are under the assumption that voice commands, on any device, have the potential to be human reviewed. I am not sure whether or not the general public has that same assumption.
That being said, my biggest concern is the fact that many of these device don't have a hardware microphone kill switch. I feel better when I know I can control when a device is listening in. I've read reports that some Alexa devices have them, but I don't own any so I am unable to verify that.
I want all of my devices with microphones to hardware based kill switch for the mic; that's my phone, laptop, tablet, everything.
> I think a lot of people here are under the assumption that voice commands, on any device, have the potential to be human reviewed. I am not sure whether or not the general public has that same assumption.
That's because we, as an industry, have fooled them into thinking that AI is real, and that people who don't know it's real are idiots. We don't think students deserve a proper tech education, so unless they are professionally techies, they have to learn from marketing materials (designed to convince them to buy things.)
Assuming $0.3/audio clip and base wage of $10/hr, that equates to 33.3 audio clips/hr = 266.4 audio clips/day that are being monitored by any one 'language expert'.
However, Google does not specify how long a 'conversation' is. How many sentences make up a conversation? When is the cutoff point?
Google also says '1 in 500' conversations are monitored. That means for any one 'language expert', there are approx. 133,200 conversations/day that have a chance of being monitored.
So basically, you have a 0.2% chance that your conversation is being picked up by any particular 'language expert' per day.
The number of people in this thread who believe that this is ok because, 1) it's obviously the only way Google could train their voice system and thus 2) people clearly knew what they were getting into, is horrifying.
It's no coincidence that companies like Amazon market their Echos as "stocking stuffers" for the holiday season. I've wondered how Google Home and these "smart home" devices were always able to be priced as low as they are. Goes to show that paying for the product doesn't exempt you from still being part of the product.
People have never heard of ML. They think that there's a computer that listens to what they say, then does it. If they are interested in technical stories on tv news, they may think that what they say is sent to a computer at Amazon/Apple/Google/Wherever and the command is sent back; if they are not, they think that their phone is the computer.
I mean, we've told them that we're on the verge of AI (and to fear it), and we can tell a two year old to do something and they do it without having to hire somebody to help them understand us.
A bit tangential, but I tried sharing this link with a few friends on Facebook Messenger, and noticed it's blocked because it "violates Community Standards" [1]. Even shortened bit.ly links are blocked.
Anyone know why that would be the case? I'm trying to not assume malice (eg. maybe it got misflagged?) but it certainly feels like censoring and is yet another push for me to drop Messenger too.
Some companies tell me on every phone call that I am being recorded. Some don't (but maybe are recording anyway?)
Should Voice Assistants be required to annoy that they are recording every time they received an activation? Until the user explicitly approves long term? Should that approval require daily or monthly renewal?
Should the device detect new voices and give them the same warning? Should these recommendations become law?
This is common knowledge for us tech oriented people but with time (and stories such as these) it is (hopefully) becoming more and more general knowledge.
Personally I'm ok with it: If I use a cloud based voice enabled device, I accept that my voice might be heard by someone working there. Similarly, if one sends nude pics as a "private" message in messenger, they shouldn't be surprised if someone at FB sees them. Know the risk, evaluate the benefit, and make an informed decision.
One caveat is the explicitness of "put online": The above holds for things put up deliberately, not for things uploaded accidentally or covertly. In these cases the data is illegally obtained and therefor, imo, not important for this discussion. That's because I (naively) don't think big companies would risk a large scale purposefully illegal enterprise (and if they do, there should be major ramifications) and 3LA can spy on anyone anyway if they want, and that's (currently) legal/above the law.
One thing that Google Home does noticeably poorly with is disabled voices and non-native speakers. Google Home would be really handy for a family member with down syndrome who can't read nor write but can easily learn voice commands, but Google Home doesn't currently understand him at all. My partner with a Malaysian Chinese accent also really struggles to use Google Home.
That said, I notice the quality of voice recognition is rapidly improving and I'm thankful for the work the Google engineering team are doing; voice recognition and language translation has the potential to improve the lives of so many people.
The current headline seems to imply more than what's actually happening. What's happening is when google hears "ok google" the recording is saved and may be manually reviewed for ML training purposes.
The problem is hearing "ok google" is not reliable and it sometimes captures normal conversations.
I do think it's an ethics issue that people may not be told clearly when buying these products that these recordings may leave their homes and can be heard by people (though only up to a minute and without any clear PII unless someone says something identifiable).
I don't get why anyone is surprised by this or why the outrage is restricted to only Google Home. Google employees and contractors are listening, viewing, and reading stuff from time to time in order to provide labels to their deep learning systems. Moreover, you've consented to this when you created your account or in some cases just used their services without an account.
You can't really have useful machine learning without lots of annotated data quite yet, and Google prides itself on useful machine learning.
I've been toying with the idea that companies that use user data passively (while the user is not interacting with the system in a way to expect that data usage) should have to notify the user about the usage. For instance, if data you produced was randomly selected to be listened to by someone, you should get a notification somewhere about it.
I think something along these lines would help increase user awareness.
Of course they are! There are user researchers, people looking for spam and abuse, people tagging data, among other things who are absolutely looking at your: private messages, search queries, profiles viewed, Alexa conversations, Tinder messages, bookmarks, call history, purchase history, geolocation history... and on and on. I guess there are a lot of news stories left to write on this topic.
The main problem here is that Google hired someone like this person who leaked days they were trusted to protect. It's a concern for users (Insider Risk at Google), and something we need laws to protect. Surely if Google Corporation is a poor steward of this data, the specific malicious actor here who violated both his employment policy and his users' trust is especially bad.
I unplugged my family’s Alexa, since they weren’t using it but left it on. They haven’t used it at all for months and haven’t plugged it back in. It sits on their shelf. I wonder if this is common, where families overestimate the usefulness of voice assistants and then rarely use them after a couple months.
Well of course you will have some qa / quality monitoring just as you will with telecoms and I seem to recall that the Dutch go in for monitoring of phones in a big way.
Also does that article remind me of the Monty python sketch "news for wombats"
Now it's making me wonder if the light on my Google Home Mini is on if that means it's passively listening to me. I always do a hard reboot since the indicator light suddenly stays active for a long period of time.
"(...) honey, what's our card number? 123 (...)"
"(...) honey, what was the password? 123 (...)"
"(...) honey, i got us 10 first bitcoins, we need to printout mnemonic 123 (...)"
There are two big things right now on my list of things I will never use: closed remotely operated home assistants with cameras or microphones that are not physically switched, and Facebook Libra.
My biggest frustration is going to a friends house which is blanketed with Echo/Google coverage and realizing that all my words could be used against me at a later time.
I mean, it's roughly the same level of exposure you run into literally every minute you're out in public anyway - all those phones around you have microphones, and more and more, they're always listening in exactly the same fashion. This is how always-on "Hey Siri" or "Okay Google" works.
Which isn't to say it isn't frustrating, just that your frustration at your friend is misplaced - they're not exposing you to anything which you aren't already exposed. If your fear is that these tech companies are going to misuse casually-overheard conversations from these devices, they're just as likely to use it from all the phones you're near.
If your fear is that your friend is going to use this tech, however, maybe you need better friends. There's nothing stopping an unscrupulous friend from putting plain-old audio recorders through their house anyway.
Discussions held in private can be very different from the ones held in public.
I think it's very human to want privacy. Having confidence that someone isn't listening/watching to you can have a tremendous effect on how you act. It's just human and I'm not sure if you can change that by telling people that it's not feasible in today's world. I know I acted differently in private when I was just a kid - it's not something I learned, but was natural to me.
The tech would actually be fine if it was provably contained to the local network. As soon as it leaves the local network, I can't trust it - doesn't matter if Amazon/Google says they don't use it for X purpose.
> they're always listening in exactly the same fashion. This is how always-on "Hey Siri" or "Okay Google" works
Afaik it is not, actually. The devices are taught to recognize the wakeup words (like "ok google") completely offline, and only the stuff recorded afterwards gets uploaded to the cloud of contractors.
The phones listen in the same fashion as the in-home devices, is what I mean - they both work on the wakeup words. So if you're mistrustful of the in-home devices, you should be petrified of your morning commute.
Scary though isn't it ? With increasing moves in these companies to govern human thought, I can imagine a time when it'll be extremely difficult to have any independent thought of your own. It's already this way in the academia/media, where anything outside the norm is considered 'blasphemy'.
Your only recourse is to stop going to that friends house. Also, remember you and others still have a internet connected microphone in your pocket at all times. A zero day means you could be owned at any time.
Telling the truth about your religion or sexual orientation can get you beaten or killed in many parts of the world.
That's not likely to happen in the present day US, but there are plenty of less dramatic but definitely undesirable things that can happen when your home audio gets passed around to a bunch of random contractors.
I don't see the problem to short anonymous fragments to help resolve recognition problems. Full conversations, even anonymous, seems problematic though.
google employees liste to the recordings made with the smart google home speakers and via the google assistant app on smartphones. worldwide ex.belgium and netherlands people listen to those recordings to make the search engineer smarter. VRT NWS was able to listen to more 1,000 tracks thats why the employees must be listeing to the home conversations.
google employees listen to the recording made with the smart google home speakers and via the google assistant app on smartphones as well as worldwide in belgium and netherlands , people listen to those recordings to make the search engine smarter. VRT NWS was able to listen t more than 1000 tracks these are pieces usally spoken conciously.
I've long been curious how GDPR should be applied to Google Home, Alexa, etc.
These devices record and upload voice recordings to the cloud (with a percentage apparently shared with Google contractors to process). The audio clips can be recordings of conversations of people who have not consented to being recorded. And I can't imagine it's possible for Amazon or Google to reasonably comply with Art. 15 or 17 (right of access by data subject, and right to erasure).
Is there something I'm overlooking? Or is this just a known risk that Amazon/Google accept in providing the service?
Click on Voice & Audio Activity, then Manage Activity. You can listen to and delete every recording Google has of your voice. It's a little trippy to be honest. They've had this since long before the GDPR.
>> The audio clips can be recordings of conversations of people who have not consented to being recorded
My guess would be that the person who does consent to the terms of service of the device and plugs it in will be the one responsible for ensuring that it is used in a way that does not invade the privacy of anyone else. It would be similar to me buying an IP camera, agreeing to the terms of service then putting it in my neighbours house without their knowledge.
In the context of the GDPR, I think the question comes down to what rights a third party person has when a user puts their personal information on the company's servers. If someone uploads a video of me without my consent to YouTube, can I get it removed under the GDPR?
next time you will tell me they listen to voice input data from assistant...
just joking, I had actually access to them when working on Google project, if you use gboard, voice input and Google assistant, Google has basically anything you input on your phone "for research purposes"
I think this is the last straw for me. I have already tinkered with the idea and built enough for POC. But I am going to have to replace all my google home and amazon stuff with home built systems that I control both the hardware (to a extent) and the software.
What's most interesting to me is that an army of contractors is required to get even the low accuracy levels of voice recognition we have today. The "AI" revolution is pure smoke and mirrors designed entirely to bilk investors out of their dollars. There's been no improvement in "AI" in the last 50 years, except that we have a lot more data to push through the same useless models.
100 bucks for anyone who uses a 50 year model/method and get within twice the word-error-rate of today's speech recognition systems.
Also, how do you explain the progress on ImageNet challenge over the last 10 years? Each method there uses the same dataset, yet error rates (top5) have gone from 30% in 2011 to around 3% in 2016.
Not going to deny that faster GPUs has helped a lot.
But it is not the full picture either. DenseNet-121 has higher accuracy than VGG16, yet requires only 1/15 the storage space, and 1/5 the number of operations during inference. I'm pretty sure training time is faster also.
I very much believe the only reason smartphones aren't listening to us and recording us constantly is due to battery purposes. When you plug something in, yeah, I have no trouble believing they're recording everything and storing it somewhere.
1) Don't understand ML; not concerned - "I have nothing to hide."
2) Don't understand ML; concerned - "I bought this device and now people are spying on me!"
3) Understand ML; not concerned - "Of course, Google needs to label its training data."
4) Understand ML; concerned - "How can we train models/collect data in an ethical way?"
To me, category 3 is the most dangerous. Tech workers have a responsibility not just understand the technologies that they work with, but also educate themselves on the societal implications of those technologies. And as others have pointed out, this extends beyond home speakers to any voice-enabled device in general.
In conversations about this with engineers the response I've gotten is essentially: "Just trust that we [Google/Amazon/etc.] handle the data correctly." This is worrying.