Key takeaways
- Siri, Apple's personal assistant, was launched in 2011 and assists users with various tasks on their devices.
- A speech recognition unit powered by Deep Neural Networks constantly runs in the background to detect the 'Hey Siri' command.
- Temporal Integration is used to compute a confidence score that determines if Siri should be activated based on voice input.
“Hey Siri” -- this is the catchphrase that you use on your Apple devices when you feel helpless or bored. Siri is a built-in personal assistant chatbot, introduced in 2011 for the smartphone. Siri can help with the user with tasks such as getting information from the internet, scheduling events, setting a timer and making phone calls, among other things.
Astonishingly, Siri is powered by a speech recognition unit present in the phone which runs in the background all the time. This speech recogniser uses a Deep Neural Network (DNN) to correspond your voice patterns which are then generated as a probability distribution for those voice sounds. A process called Temporal Integration is used to compute a confidence score to check whether your voice contained the words ‘Hey Siri’. If the words are close, then Siri is activated, otherwise it isn’t. This article gives a brief look into the machine learning aspects behind Siri.
Behind the scenes
DNN And The Hardware
The microphone or detector in an iPhone or any Apple products such as iPad, iPod Touch and Apple Watch, turns the detected voice into a stream of instantaneous waveform samples which are created at a rate of 16,000 per second. A spectrum analysis stage converts the waveform sample stream to a sequence of frames splitting the voice into a spectrum of 0.01 second. About 20 of these frames at a time (0.2 sec of audio) are fed to the acoustic model, a Deep Neural Network (DNN) is now set to work, which converts each of these voice pattern models into a probability distribution over a set of speech sound classes which are used in the “Hey Siri” phrase (among other voice patterns) for a total of close to 20 sound classes categorised by Apple Inc. The DNN consists of matrix multiplications and logistic nonlinearities. Each separate layer is an intermediate representation identified by the DNN during its training to convert the filter bank inputs to sound classes. The final nonlinearity is primarily a Softmax function (also known as general logistic or normalized exponential), the reason is to choose logarithmic probabilities over linear ones to make the computation easier.
- Fi,t is the accumulated score for state i of the model
- qi,t is the output of the acoustic model — the log score for the phonetic class associated with the ith state given the acoustic pattern around time t
- si is a cost associated with staying in state i
- mi is a cost for moving on from state
