GLaDOS Voice Assistant

Henri Rantanen 7

I turned the most evil thing in the universe into a lightswitch.
I did not want Google or Amazon listening to me, so I created my own passive-aggressive voice assistant based on the main villain from Portal 2 to control my smart home.

Introduction

This is an ongoing DIY project by a Portal fan, not a product available commercially. Code is distributed as is to work as an example for others.

About the DIY voice assistant project

I wanted to control my smart home with voice, but having Amazon or Google devices around creeped me out because their devices and AI are just black boxes. I have no idea what they are doing in the background, and it’s out of my control.

So, I created my own voice assistant that runs locally with Python and Raspberry Pi. This project is still ongoing, and this is a short overview of what I’ve done so far. You can follow me (@nerdaxic) on Twitter; I post updates there. This project seems to have turned into functional art.

Why GLaDOS as voice assistant?

Mostly because I found GLaDOS funny. The character’s voice is recognizable and relatively easy to generate with a female TTS engine and Melodyne.

In addition, GLaDOS is a well-written character in the portal game series, with a good back-story and personality. This is important to write the responses for the assistant to stay in character.

Main features

  1. Local Trigger word detection using PocketSphinx
  2. Speech to text processing using Google’s API (for now)
  3. GLaDOS Text-to-Speech generation using https://glados.c-net.org/
  4. Local TTS cache for instant replys
  5. Animatronic eye control using servos
  6. Round LCD for an eye to display textures

GLaDOS Text-to-speech engine

The voice assistant uses an online TTS API to generat the voice samples and caches them indefinitely on the Raspberry Pi.

The first time takes time to generate, but next time using the same words can be fetched from local storage.

Works with Home Assistant

Device communicates locally with Home Assistant server to speak out notifications, get sensor data and control lights, scenes and devices.

  • “Turn on livingroom ac”
  • “When is the sauna ready”
  • “What is the bedroom humidity”

Infrared face recognition

There is Raspberry Pi NoIR camera built into the bezel of the eye.

Idea is to mount the head on a neck mechanism, and use the face coordinates from camera to turn the head towards you when you speak to GLaDOS. Even in the dark!

Animatronic eye

Large animatronic eye can move in and out, and open and close eyelids.

Eye has a round LCD and neopixel-ring to provide the light and texture.

What can it do?

Basic functionalityNoveltyHome Assistant
Tell the current timeMagic 8 ball answersControl lights and scenes
TimersLore friendly jokesControl devices and AC
Add things to shopping listAnswer to pleasantriesTell the temperature / humidity
Current weatherMove eye aroundSpeak notifications aloud
Weekly weather forecastIR-camera for face recognitionEstimate when sauna is ready

Skills in developement

  • Local GLaDOS TTS engine for instant replies
  • Local speech recognition for added data security
  • Wolfram alpha integration for math and numeric data
    How much sodium in 25 grams of salt”
    “What was the oil price in 1972”

GLaDOS Animatronic Hardware

Animatronic GLaDOS body is mostly 3D-printed, based on YouTuber Mr. Volt’s open source design.
Check out his original video on YouTube.

The main difference between our projects is that Mr. Volt used Amazon Echo as the voice assistant, where I made my own. His published design was not complete, so I had to re-design some of the internal mechanics of my own.

ItemDescription
Main boardRaspberry Pi 4 Model B 8GB
Power supply for Digital + AudioRaspberry Pi 15W USB-C Power supply
Memory cardClass 10 64 GB microSDXC U3
CameraRaspberry Pi NoIR camera
MicrocontrollerTeensy 4, to control the eye LCD and NeoPixels
Eye lightsAdafruit NeoPixel Diffused 5mm Through-Hole for the “REC” light
Eye lightsAdafruit 16 x 5050 NeoPixel Ring
Eye LCD1.28 Inch TFT LCD Display Module Round, GC9A01 Driver SPI Interface 240 x 240

Audio hardware

Adafruit stereo amplifier

GLaDOS has two paper-cone speakers built into her hears giving her stereo sound.

These are powered by 3,7W Class D amplifier. Audio in, audio out. Set volume by jumper.

ReSpeaker Mic Array

Far-field microphone array device capable of detecting voices up to 5m away. Detects sound direction and improves audio quality by onboard processing.

Audio amp is powered from Raspberry GPIO 5V line and ReSpeaker board from USB to avoid ground loops and noise issues.

ItemDescription
Audio amplifierAdafruit Stereo 3.7W Class D Audio Amplifier
SpeakersVisaton FRS 7
Microphone & Audio interfaceReSpeaker Mic Array V2.0

Mechanics

Mechanics are powered from their own power supply to allow more power for the servos and prevent brown-outs.

ItemDescription
Power supplyMeanWell LRS-50-5 5V
Servo controllerPololu Micro Maestro
Servo: Eye movement35 kg DS3235 (Control Angle 180)
Servo: Eyelids25 kg DS3225 (Control Angle 180)
ScrewsVarious M3 and M4 screws
Jumper wires0.32 mm²/22 AWG assortment

Software & AI-Voice Assistant pipeline

Software works in few main parts built into glados.py and gladosTTS.py files.

1) Keyword detection with Pocketsphinx LiveSpeech

Main script hooks into the microphone plugged into the Raspberry Pi and is constantly listening for the selected trigger word. If Pocketsphinx detects the triggerword it is passed to the speech-to-text engine.

2) Python speech_recognition library

Once triggerword is detected, speech_recognition starts to listen for input from the microphone. After a small timeout the data is then passed to recognizer built in the library. This sends the recorded audio to Google for speech-to-text recognition and the service returns the spoken input as text.

This is not optimal and I’m looking for reliable solutions that can work locally, but with this solution I know what is being sent to Google and that it’s not being tied to any Google account.

3) Natural language processing & voice assistant skills

Once the command has been gotten from the speech regonition API, command is parsed. Currently this is done with a long if-else statement:

if 'cancel' in command:
	return
elif 'time' in command:
	readTime()
elif 'turn on' in command and 'daylight' in command:
	activateScene("scene.daylight")
else:
	# tell user about an error and log the failed command

4) GLaDOS Text-to-Speech

In the Portal games GLaDOS has a distinctive and calm female voice. Originally the voice was generated by recording opera singer Ellen McLain speak the lines and pushing the audio through an autotune process to throw off the intonation to make her sound more robotic.

Online GLaDOS Voice Generator

I used to use glados.c-net.org to generate the voice samples over the API. Service uses the more traditional female TTS which is then pushed through Melodyne, but the process is really slow. With the que it can take half an hour to generate a simple voice sample!

This is why I have created local TTS cache of the audio samples and tried to break down the sentences into short segments. If the script finds the TTS sample from the memory card, play it. If not, generate it.

Most of the tasks and responses are repetitive, like time, weather and shoppinglist items. Device rarely needs to generate new samples.

Local GLaDOS Text-to-speech (TTS) Voice Generator

Later I have moved to a neural network based GLaDOS TTS engine that can generate GLaDOS voice locally on the computer that is running the voice assistant. This makes the whole process more robust and secure. Things like weather forecasts take about 10 seconds to generate on an old laptop.

The TTS model was originally developed and published by Reddit user u/R2D2_FISH. Code was adapted to work with GLaDOS Voice Assistant, you can get it to play around with on my GitHub.

Free and Open source (FOSS) Python voice assistant

You can download the code for free from GitHub.
You can use it for reference or build your own system for free!

Code:
GLaDOS Voice Assistant:
https://github.com/nerdaxic/glados-voice-assistant/

Old version GLaDOS Voice Assistant that works on Raspberry Pi:
https://github.com/nerdaxic/glados-voice-assistant/tree/raspberry

AI-based stand-alone GLaDOS Text-to-speech engine:
https://github.com/nerdaxic/glados-tts

Mr. Volt’s CAD-files:
https://www.dropbox.com/sh/xq9coq4962qgot6/AAA-V3E4OgzChqhunnZEnEvWa?dl=0

What do you think of this post?
  • Awesome (100%)
  • Like (0%)
  • Dislike (0%)
  • Surprising (0%)
  • Interesting (0%)
  • Useful (0%)
  • Boring (0%)
  • Sucks (0%)
  1. Who needs Alexa when you can have GLaDOS from Portal?

    If there’s one thing that the Raspberry Pi is known for, it’s the versatility it provides in home automation projects. The most exciting aspect of the project is the GLaDOS voice feature.

  2. Onpas hieno! Suomalainen YouTube-käyttäjä teki Portal-pelin GlaDOS-tekoälyrobotista huiman hienon ääniavustajan.

    Rantanen kertoi, että hän halusi muuttaa ”universumin pahimman asian valokatkaisijaksi”. Pitkäjänteisellä työllä ja 3D-tulostimella Rantanen onnistuikin luomaan hienon GLaDOS-pään, jonka sisällä laitteen sydämenä hyrrää Raspberry Pi 4 -tietokone.

  3. Henri Rantanen is constructing a being of malice and resentment to control his thermostat and tell him the weather.

    A self-programmed home assistant is a neat enough idea, but Rantanen really sells the fantasy of having GLaDOS in your home with the primarily 3D printed mechanical build surrounding the computer.

  4. GLaDOS Voice Assistant Passive-Aggressively Automates Home.

    Like many of us, nerdaxic gave in to the convenience and perceived simplicity of various home automation products made by Google and Amazon. Also like many of us, he found it a bit difficult to accept the privacy implications that surround such cloud connected devices.

    Instead of giving in and buying back into the closed ecosystems he’d left behind, and decided to open his home to a murderous, passive aggressive, sarcastic, slightly insane AI: GLaDOS

  5. Henri rakensi älykotiinsa avustajan – yli tonnin maksanut robotti osaa kertoa kun sauna on valmis.

    Google ja Amazon arveluttivat liikaa, joten suomalaismies rakensi Portal-pelien GLaDOS-robotin omaksi ääniavustajakseen.

  6. Fredrik Buchanan Fredrik Buchanan

    Hi, Epic work. so impressed.
    Have you managed to install the GLaDOS TTS into Home assistant. As a TTS-service for notifications or do you know any one that have?

    sincearly
    Fredrik

    • Thank you!

      I have GLaDOS Voice Assistant as notification service in my Home Assistant. In theory it should be not-too-big-of-a-job to make the GLaDOS TTS engine run stand-alone in Home Assistant, but current model does not run on Raspberry Pi due to missing instruction set in the CPU.

Leave a Reply

Your email address will not be published.