A very popular NVR solution that integrates well with Home Assistant is Frigate. It provides fantastic object and person detection from your camera streams. The Frigate project is a Docker container, so it’s easy to deploy. You can run Frigate within Home Assistant, but I prefer to run it in a separate dedicated VM on Proxmox. This blog post will walk you through building a Ubuntu VM on Proxmox dedicated to Frigate and optionally use a Google Coral PCIe TPU for object detection.
In order to boost the performance of Frigate I purchased an A+E Key m.2 PCIe Google Coral TPU. I pulled the m.2 WiFi card from my Beelink SEi12 Pro and replaced it with the Coral TPU. The m.2 Coral TPU is preferred to the USB version for a couple of reasons. First, PCIe passthrough can be more stable than trying to pass through a USB device. Second, the PCIe version has built-in thermal throttling which the USB version omits. Performance wise, USB and m.2 seem very similar from numbers I’ve seen.
It is possible to also use your GPU with Frigate to help offload some ffmpeg CPU load. However, in my testing with a single camera the CPU load was unchanged on my 12th Gen CPU when adding ffmpeg GPU offload. My VM has 1 vCPU and at the Proxmox level it uses about 15% CPU utilization with one camera detection stream at 1280×720. Per Frigate System page the ffmpeg process is only using 3% of the VM’s CPU.
In addition, doing GPU passthrough and trying to get the very latest Intel drivers on Ubuntu to work is exceptionally challenging and seems very brittle even if you can get it to work. So for the purpose of this series, I won’t touch GPU passthrough.
This blog post will be using the following stack:
- Frigate 0.12.0
- Ubuntu 22.04
- Proxmox 7.4 with 6.2 Kernel
- Google Coral m.2 PCIe TPU
- EmpireTech 4MP IP PoE Camera
- Intel 12th Generation i5-1240P CPU
This post is not a Frigate camera configuration tutorial. The camera configuration within Frigate is very environment specific, and Frigate has a ton of configuration options. At the time of this post I am using the latest Frigate version (0.12.0) which can leverage built-in go2rtc, which I use in my example. So if you look at Frigate config tutorials on other sites, make sure it’s for 0.12 (or later) and uses go2rtc.
\t\t\t\t\t\t\n\t\t\t
Tutorial Sections
\t\t\n\t\t\t\tThis tutorial is pretty long as it covers a lot of ground:
- Promox Host PCIe Passthrough
- Deploy Ubuntu 22.04 VM for Frigate
- Mount Second Drive for Media (Optional)
- Install Docker and QUMU Agent
- Coral PCIe TPU Passthrough (Optional)
- Home Assistant MQTT Setup
- Home Assistant Frigate Integration Install
- Camera Configuration (IPC-T5442T-ZE)
- Frigate Configuration File
- Frigate Docker Compose
- Verify Frigate Functionality
- Home Assistant Frigate Integration
\n\t\t\t
Proxmox Host PCIe Passthrough
\t\t\n\t\t\t\tFirst we need to modify the configuration of the Proxmox host where the Frigate VM will be running. Login to Proxmox and open a shell.
1. Modify the GRUB configuration by running the following command:
\t\t\t\t\t\t2. Comment out the existing GRUB_CMDLINE_LINUX_DEFAULT and paste the following line under the commented out line. Save and exit Nano.
\n\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-04-25_18-40-24.jpg\")
3. Run the following command to update GRUB.
\n\t\t\t\t4. Let’s modify the kernel modules file.
\n\t\t\t\t5. Add the following lines to the file, then save and exit nano.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_19-00-41.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
6. Run the following command to update the Kernel.
\n\t\t\t\t7. Now we need to blacklist the Coral devices so the Proxmox host won’t load the drivers:
\n\t\t\t\t8. Add the following lines to the blacklist file:
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_19-01-52.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
9. Save and exit nano, then run the following command:
\n\t\t\t\t10. Reboot the Proxmox host. After it reboots open a shell again and run the following command. It should NOT print anything.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-04-25_20-13-44.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t
Deploy Ubuntu 22.04 VM for Frigate
\t\t\n\t\t\t\t1. Login to your Proxmox server. Create a new VM with the following General settings. Use any Name you wish. Check the Start at Boot box.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-04-25_19-07-24.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
2. Download the Ubuntu 22.04 ISO. Upload it to a Proxmox datastore. On the OS page mount your Ubuntu 22.04 ISO.
\n\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-04-25_16-15-11-1024x728.jpg\")
\t\t\t\t\t\t\t\t\t\t\t\t\t
3. On the System page configure:
- Graphic Card: Standard VGA
- BIOS: OVMF (UEFI)
- EFI Storage: <pick your storage>
- Qemu Agent: Check
- Pre-Enroll keys: UNCHECK (SUPER CRITICAL)
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-05-01_20-55-42b.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
4. On Disks configure the following:
- Disk size: 16GB (or more if you wish)
- SSD emulation: Check
- Cache: Write through
- Discard: Check
- IO thread: Check
Note: If you have separate NVR storage add that virtual disk now. I highly recommend a spinning disk that’s designed for video surveillance like Seagate SkyHawk. 2.5″ SkyHawk drives are no longer made (just 3.5″), but Ebay may have factory sealed 2TB 2.5″ SkyHawk drives. I added a 1.7TB virtual drive.
\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-04-25_18-54-29-1024x725.jpg\")
\t\t\t\t\t\t\t\t\t\t\t\t\t
5. On the CPU page configure:
- Cores: 1
- Type: host
![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-05-04_20-10-32-1024x720.jpg\")
\t\t\t\t\t\t\t\t\t\t\t\t\t
6. On the Memory page configure:
- Memory: 2048
- Ballooning Device: un-check
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-05-01_21-04-03.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
7. On the Network page configure as needed (usually defaults are OK).
\n\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-04-25_16-20-30-1024x733.jpg\")
\t\t\t\t\t\t\t\t\t\t\t\t\t
8. Check the summary page and tick the box to Start after created.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-05-04_20-15-14.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
9. Once the VM powers on, connect to the console in the Proxmox UI. Wait for the Language selection window to appear. Select your language.
10. If you are prompted to update the installer, do so.
11. Select your keyboard language.
12. Select Ubuntu Server (NOT minimized)
13. On the Network connections screen tab to your network interface name (e.g. ens18). Press enter on the interface name and select Edit IPv4.
14. Select Manual and enter the appropriate network info. Tab to Save and press Enter. If you want to setup IPv6, repeat the network configuration process a second time. I use IPv6 DHCP. Once back on the main Network connections screen tab to Done and press Enter.
15. Skip the proxy address, unless you need it.
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_19-13-57.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_19-12-21.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
16. On the Configure Ubuntu archive mirror press Enter on Done.
17. On the Guided Storage configuration screen tab to Done and press Enter.
18. On the Storage configuration tab to ubuntu-lv under USED DEVICES and press ENTER. Tab to Edit and press ENTER.
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-03_10-35-48.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
19. Change the Size value to match the max stated value. Tab to Save and press ENTER.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-03_10-38-34.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_19-16-41.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
20. Tab to Done on the Storage configuration page.
21. On the Confirm Destructive action windfow tab to Continue and press ENTER.
22. On the Profile setup enter the appropriate information.
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_19-18-44.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
23. Skip the Upgrade to Ubuntu Pro.
24. On the SSH Setup screen tick the box to install OpenSSH server.
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_19-20-19.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
25. Skip the Featured Server Snaps.
26. Once the install is complete reboot.
\n\t\t\t
Mount Second Drive for Media (Optional)
\t\t\n\t\t\t\tIn my Beelink Mini PC I’ve installed a dedicated spinning NVR drive. It’s a (now discontinued) 2TB Seagate Skyhawk that I got factory fresh from ebay. The Skyhawk line is purpose built for video surveillance. It is strongly recommended to NOT store your Frigate video on a SSD.
I added a second virtual disk to the NVR VM in Proxmox using the following configuration. I specifically un-checked the Backup box, so that my Proxmox backup server would not backup my videos.
\t\t\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_19-38-14.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
1. Login to your new Ubuntu NVR VM.
2. Run the following command to list all disks mounted to the Frigate VM. Locate the device name for the new disk (/dev/sdb in my case).
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_19-41-26.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
3. Now we need to partition the raw disk by running the following command:
\n\t\t\t\t- Press any key when prompted.
- Press ENTER on New
- Press ENTER on First Sector
- Press ENTER on Size in sectors
- Press ENTER on Current Type
- Enter a Partition Name (e.g. frigate)
- Arrow over to Write, press enter, then confirm
- Arrow over to Quit and press enter
4. Let’s format the drive (note the addition of the 1 in the device name). Press ENTER to confirm.
\n\t\t\t\t5. Create a mount point, using whatever name and location you want.
\n\t\t\t\t6. Edit the fstab so the drive will auto mount at boot.
\n\t\t\t\t7. At the end of the file add the following line, save and exit nano.
\n\t\t\t\t9. Verify the drive is mounted and has the expected size:
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_19-47-08.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t
Install Docker and QEMU Agent
\t\t\n\t\t\t\t1. SSH into your new Ubuntu NVR VM.
2. Run the following commands setup the Docker repos.
3. After the repos are setup let’s install Docker. The hello-world test will verify that Docker is running and pulled down an image.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-04-25_20-00-24.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
4. I like setting up Docker to store the app config files under /home/docker.
\n\t\t\t\t5. Install the QEMU agent and update all the packages:
\n\t\t\t\tOptional: Connect this Docker instance to your Portainer server by installing a Protainer agent. I really like Portainer (free) to manage Docker containers. But many like Docker CLI or Docker compose. Any of these options work.
\n\t\t\t\t\n\t\t\t
Coral PCIe TPU Passthrough (Optional)
\t\t\n\t\t\t\tIf you have a Google Coral TPU on a PCIe card, we now need to pass that through to the Frigate VM. If you do not have a TPU, skip this section.
1. Power down the Frigate VM. In the Proxmox console locate the Frigate VM, click on it, then click on Hardware, then Add -> PCI Device. Locate your Coral TPU and add it. Power on the VM.
\t\t\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-05-01_16-42-53.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
2. After the VM boots, SSH in and run the following command. Verify that you see the TPU listed.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_19-59-20.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
3. Let’s start the driver installation:
\n\t\t\t\t4. Install the PCIe drivers and TPU runtime:
\n\t\t\t\t5. If you see a Secure Boot Config window STOP! Something is wrong…secure boot will disable the needed drivers. Re-check your VM configuration.
6. Reboot the VM.
7. SSH into the NVR VM and run the following commands:
10. Reboot the NVR VM again.
11. Run the following command and verify you still see the TPU PCIe device:
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-05-01_16-51-28.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
12. Run the following command and verify /dev/apex_0 is listed. If it is not, you have a problem and need to troubleshoot. One of the first things I’d look at is making sure secure boot is NOT enabled for the VM. Secure boot will prevent the apex drivers from loading. To verify secure boot is disabled, reboot the NVR VM and on the Proxmox VM boot screen press ESC. Go into the virtual BIOS and ensure Secure Boot is disabled.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-05-01_21-24-55.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t
Home Assistant MQTT Setup
\t\t\n\t\t\t\tIf you have already installed and configured a MQTT broker in Home Assistant, you can skip this section.
\n\t\t\t\t- In Home Assistant go to Settings -> Add-ons.
- Click on Add-on Store. Search for MQTT and click on Mosquito broker.
- Click on Install.
- After it installs, enable Watchdog.
- Click on Start.
- Click on Settings in the lower left, then Devices & Services.
- MQTT should be auto-discovered. Click CONFIGURE.
- Click Submit.
- Click on Settings in the lower left, then People.
- Click on ADD PERSON in the lower right.
- Enter a name (e.g. mqtt), allow the person to login, enter a password and click CREATE twice.
- Save this password in a safe place. This will be used by Frigate and any other MQTT service.
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-11_16-11-22.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t
Home Assistant Frigate Integration Install
\t\t\n\t\t\t\t- In Home Assistant ensure HACS is already installed.
- Click on this Frigate Add-ons: The official repository link then click on ADD REPOSITORY.
- Add the repository.
- Search for Frigate in the HACS Integrations and install it.
- Add the Frigate Lovelace card: HACS -> Frontend -> Explore & Add Integrations -> Frigate Card
- Open configuration.yaml and add the following:
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-05-02_21-28-06.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
7. Restart Home Assistant.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-11_16-12-37.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t
Camera Configuration (IPC-T5442T-ZE)
\t\t\n\t\t\t\tEvery camera will have their own unique optimal settings. In my case the following settings for my Dahua work very well. Use the best practices for your particular camera. Just for reference, my IP camera is at 10.13.2.231. I also configured a user named ‘feed’ with a complex password to use for the video feed.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-05-02_20-51-47a.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t
Frigate Configuration File
\t\t\n\t\t\t\tBefore we deploy the Docker container we need a valid Frigate configuration file. Much of the configuration will be unique to your camera(s), what you want to detect, etc. It’s not within the scope of this post to go into depth on Frigate configuration files. But my configuration should at least get you pointed in the right direction. Note that the IP of my Frigate VM is 10.13.2.217. As noted in the post introduction, I’m using Frigate 0.12 and its newly supported built-in go2rtc feature. Many Frigate tutorials on the internet are for older versions, so watch out. Use tutorials for 0.12 (or later).
\n\t\t\t\t\n\t\t\t
Frigate Docker Compose
\t\t\n\t\t\t\tAfter you have a valid Frigate config file, we need to setup the Docker compose file. It is pretty straight forward and you should only really need to change a few parameters:
- shm_size (depends on number of cameras)
- config.yml Frigate config file path location in your host VM
- Media Frigate media folder location in your VM
- Frigate RTSP password
If you have followed this guide exactly, no changes should be needed except the RTSP password. If you are using Portainer you can just copy and paste your modified file into a new stack and then deploy the stack. If you prefer Docker compose via CLI, just save the docker compose file as frigate-compose.yml in /home/docker/frigate.
Note: If you aren’t using the Coral PCIe TPU, remove the devices section in the compose file (lines 11-12).
\t\t\t\t\t\tIf you want to use Docker compose run the command below to start the container. Be patient for the container to start the first time as the docker image needs to be downloaded. Give it a few minutes to start up.
\n\t\t\t\t\n\t\t\t
Verify Frigate Functionality
\t\t\n\t\t\t\tAssuming your Frigate Docker container has started and your configuration is correct, you can now access the Frigate web UI via:
http://<yourVMIP>:5000
If you don’t see a working camera feed in Frigate, go to the Logs view and review for any errors. You can make config changes directly in Frigate now, and restart the service all within the UI. If everything is working and the TPU is detected, your logs will look similar to this:
\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_21-19-08-1024x260.jpg\")
\t\t\t\t\t\t\t\t\t\t\t\t\t
You can also review the System page and review TPU inference speed and camera CPU utilization stats.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_20-58-59.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t
Home Assistant Frigate Integration
\t\t\n\t\t\t\t- In Home Assistant go to Settings -> Devices & Services -> Add Integration -> Frigate.
- Enter the IP address of your NVR VM and use port 5000.
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_21-57-36.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
3. If everything works as planned, you will now see your camera added to HA. You can also modify the exposed entities and view some diagnostic information for Frigate in HA as well.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_21-58-58.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-10_22-07-17.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
4. Now you can add the Frigate card to your dashboard. It has a huge amount of configuration options. So you can really go to town on customizing your camera dashboard views.
\n\t\t\t\t\n\t\t\t\t\t\t\t![\"\"](\"https://www.derekseaman.com/wp-content/uploads/2023-06-11_16-30-19.jpg\")
\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t\t\t\t\t
\n\t\t\t
Summary
\t\t\n\t\t\t\tFrigate is a great NVR solution that has deep Home Assistant integration. Although you can run Frigate within HAOS, I prefer using a dedicated VM. This post covered deploying an Ubuntu 22.04 VM on Proxmox with Google Coral TPU passthrough. Although this process is a big long, it has been super stable. GPU passthrough could theoretically help lower CPU utilization, however in my configuration it did not. Plus, GPU passthrough was very fragile. Trying to get the right version of Ubuntu, Linux kernel and Intel drivers to all work together was too tedious for me. And even when I was testing it, ffmpeg would randomly die, likely due to Intel driver issues.
\n\t\t\t\t