go2rtc/internal/wyoming/README.md

Wyoming

This module provide Wyoming Protocol support to create local voice assistants using Home Assistant.

• go2rtc can act as Wyoming Satellite
• go2rtc can act as Wyoming External Microphone
• go2rtc can act as Wyoming External Sound
• any supported audio source with PCM codec can be used as audio input
• any supported two-way audio source with PCM codec can be used as audio output
• any desktop/server microphone/speaker can be used as two-way audio source
  • supported any OS via FFmpeg or any similar software
  • supported Linux via alsa source

Typical Voice Pipeline

1. Audio stream (MIC)
   • any audio source with PCM codec support (include PCMA/PCMU)
2. Voice Activity Detector (VAD)
3. Wake Word (WAKE)
   • OpenWakeWord
4. Speech-to-Text (STT)
   • Whisper
   • Vosk
5. Conversation agent (INTENT)
   • Home Assistant
6. Text-to-speech (TTS)
   • Google Translate
   • Piper
7. Audio stream (SND)
   • any source with two-way audio (backchannel) and PCM codec support (include PCMA/PCMU)

You can use a large number of different projects for WAKE, STT, INTENT and TTS thanks to the Home Assistant.

And you can use a large number of different technologies for MIC and SND thanks to Go2rtc.

Configuration

You can optionally specify WAKE service. So go2rtc will start transmitting audio to Home Assistant only after WAKE word. If the WAKE service cannot be connected to or not specified - go2rtc will pass all audio to Home Assistant. In this case WAKE service must be configured in your Voice Assistant pipeline.

You can optionally specify VAD threshold. So go2rtc will start transmitting audio to WAKE service only after some audio noise.

Your stream must support audio transmission in PCM codec (include PCMA/PCMU).

wyoming:
  stream_name_from_streams_section:
    listen: :10700 
    name: "My Satellite"                # optional name
    wake_uri: tcp://192.168.1.23:10400  # optional WAKE service
    vad_threshold: 1                    # optional VAD threshold (from 0.1 to 3.5)

Home Assistant -> Settings -> Integrations -> Add -> Wyoming Protocol -> Host + Port from go2rtc.yaml

Select one or multiple wake words:

wake_uri: tcp://192.168.1.23:10400?name=alexa_v0.1&name=hey_jarvis_v0.1&name=hey_mycroft_v0.1&name=hey_rhasspy_v0.1&name=ok_nabu_v0.1

Config examples

Satellite on Windows server using FFmpeg and FFplay.

streams:
  satellite_win:
    - exec:ffmpeg -hide_banner -f dshow -i "audio=Microphone (High Definition Audio Device)" -c pcm_s16le -ar 16000 -ac 1 -f wav -
    - exec:ffplay -hide_banner -nodisp -probesize 32 -f s16le -ar 22050 -#backchannel=1#audio=s16le/22050

wyoming:
  satellite_win:
    listen: :10700
    name: "Windows Satellite"
    wake_uri: tcp://192.168.1.23:10400
    vad_threshold: 1

Satellite on Dahua camera with two-way audio support.

streams:
  dahua_camera:
    - rtsp://admin:password@192.168.1.123/cam/realmonitor?channel=1&subtype=1&unicast=true&proto=Onvif

wyoming:
  dahua_camera:
    listen: :10700
    name: "Dahua Satellite"
    wake_uri: tcp://192.168.1.23:10400
    vad_threshold: 1

Satellite on Dahua camera with two-way audio support.

streams:
  wyoming_external:
     - wyoming://192.168.1.23:10600                # wyoming-mic-external
     - wyoming://192.168.1.23:10601?backchannel=1  # wyoming-snd-external

wyoming:
   wyoming_external:
    listen: :10700
    name: "Wyoming Satellite"
    wake_uri: tcp://192.168.1.23:10400
    vad_threshold: 1

Wyoming External Microphone and Sound

Advanced users, who want to enjoy the Wyoming Satellite project, can use go2rtc as a Wyoming External Microphone or Wyoming External Sound.

go2rtc.yaml

streams:
  wyoming_mic_external:
    - exec:ffmpeg -hide_banner -f dshow -i "audio=Microphone (High Definition Audio Device)" -c pcm_s16le -ar 16000 -ac 1 -f wav -
  wyoming_snd_external:
    - exec:ffplay -hide_banner -nodisp -probesize 32 -f s16le -ar 22050 -#backchannel=1#audio=s16le/22050

wyoming:
  wyoming_mic_external:
    listen: :10600
    mode: mic
  wyoming_snd_external:
    listen: :10601
    mode: snd

docker-compose.yml

version: "3.8"
services:
  satellite:
    build: wyoming-satellite  # https://github.com/rhasspy/wyoming-satellite
    ports:
      - "10700:10700"
    command:
      - "--name"
      - "my satellite"
      - "--mic-uri"
      - "tcp://192.168.1.23:10600"
      - "--snd-uri"
      - "tcp://192.168.1.23:10601"
      - "--debug"

Wyoming External Source

go2rtc.yaml

streams:
  wyoming_external:
    - wyoming://192.168.1.23:10600
    - wyoming://192.168.1.23:10601?backchannel=1

docker-compose.yml

version: "3.8"
services:
   microphone:
      build: wyoming-mic-external  # https://github.com/rhasspy/wyoming-mic-external
      ports:
         - "10600:10600"
      devices:
         - /dev/snd:/dev/snd
      group_add:
         - audio
      command:
         - "--device"
         - "sysdefault"
         - "--debug"
   playback:
      build: wyoming-snd-external  # https://github.com/rhasspy/wyoming-snd-external
      ports:
         - "10601:10601"
      devices:
         - /dev/snd:/dev/snd
      group_add:
         - audio
      command:
         - "--device"
         - "sysdefault"
         - "--debug"

Debug

log:
  wyoming: trace