A newly revealed patent application filed by Amazon is raising privacy concerns over an envisaged upgrade to the company's smart speaker systems. This change would mean that, by default, the devices end up listening to and recording everything you say in their presence.
Alexa, Amazon's virtual assistant system that runs on the company's Echo series of smart speakers, works by listening out for a 'wakeword' that tells the device to turn on its extended speech recognition systems in order to respond to spoken commands.
On Amazon's devices, the wakeword is 'Alexa', but similar systems control how Apple devices work ('Hey Siri') and also Google's ('Hey Google'), not to mention products from other tech companies.
In theory, Alexa-enabled devices will only record what you say directly after the wakeword, which is then uploaded to Amazon, where remote servers use speech recognition to deduce your meaning, then relay commands back to your local speaker.
But one issue in this flow of events, as Amazon's recently revealed patent application argues, is it means that anything you say before the wakeword isn't actually heard.
"A user may not always structure a spoken command in the form of a wakeword followed by a command (eg. 'Alexa, play some music')," the Amazon authors explain in their patent application, which was filed back in January, but only became public last week.
"Instead, a user may include the command before the wakeword (eg. 'Play some music, Alexa') or even insert the wakeword in the middle of a command (eg. 'Play some music, Alexa, the Beatles please'). While such phrasings may be natural for a user, current speech processing systems are not configured to handle commands that are not preceded by a wakeword."
To overcome this barrier, Amazon is proposing an effective workaround: simply record everything the user says all the time, and figure it out later.
Rather than only record what is said after the wakeword is spoken, the system described in the patent application would effectively continuously record all speech, then look for instances of commands issued by a person.
"The [proposed] system is configured to capture speech that precedes and/or follows a wakeword," the application explains, "such that the speech associated with the command and wakeword can be included together and considered part of a single utterance that may be processed by a system."
It's actually a clever idea, similar as others have noted to Apple's introduction of its Live Photos feature in the iPhone in 2015.
In that implementation, as soon as you open the iPhone's Camera app, the camera starts surreptitiously filming footage, even before you hit the shutter button icon to take your photo.
In fact, even once you've hit the shutter button, the camera keeps recording, and you ultimately end up with a mini movie (aka 'Live Photo') that extends for a moment on either side of the still image you manually snapped.
The proposed Alexa upgrade – which isn't necessarily something Amazon will ever roll out in its products – brings the same kind of thinking to recorded audio, ostensibly just so it never misunderstands you when you say something like, "Play some music, Alexa, the Beatles please".
It's worth noting, too, that the envisaged feature wouldn't send everything it records to Amazon's servers.
In the patent application, the authors explain that your Echo device would only ever record between 10–30 seconds of audio at a time, before wiping it from the local memory buffer, and recording a new 10–30 seconds of audio over it (again and again).
In each of these 10–30 second recordings, the device would continuously scan looking for commands involving the wakeword, and if it didn't find any, they'd get deleted forever – in theory, at least.
But because of the potential privacy implications of having a device that records you all the time, it's understandable that some people might not be thrilled about what this patent application represents, especially since Amazon has a mixed track record with Alexa recording things it wasn't ever supposed to.
As for whether this always-recording feature will ever see the light of day, it's too early to tell right now, and Amazon isn't giving anything away.
"The technology in this patent is not in use, and referring to the potential use of patents is highly speculative," a spokesperson for the company told Engadget.
"Like many companies, we file a number of forward-looking patent applications that explore new scientific ideas that may not make it into customer-facing products."
If you want to read more about the patent, the full listing is here.