Using ESPHome Without the Home Assistant Addon

The "blessed" flow for using ESPHome is the Home Assistant ESPHome Addon. This works well! It has a nice editor and it takes care of some housekeeping tasks for you. If you don't already have a comfortable development workflow it's a very nice way to start.

If you do already have a working style that doesn't involve a web UI and a browser editor you can still use ESPHome, you just have to handle those housekeeping tasks yourself.

Fundamentals

I think it can be helpful to step back and take a look at the fundamentals of a piece of software like ESPHome before diving headlong into the deep pool of non-standard workflows.

At it's base level ESPHome is a microcontroller firmware generator. That is, it reads your YAML config, generates a bunch of C++ files and config files, and then using a compiler and some helper programs it generates a binary program that your microcontroller (usually but not always an ESP32) can run.

ESPHome also has a few very useful helpers. First, it can do seamless over the air (OTA) updates once any ESPHome firmware has been installed on a device.

Second, it has a pretty powerful web-based UI and configuration editor.

Third, ESPHome ships with a "native" binary protocol it can use to talk to Home Assistant (previously) complete with Noise-based symmetric key encryption.

Lastly, it can be used as a Home Assistant addon, which as I said earlier takes care of a few things for you. The OTA update functionality requires a pre-shared key to validate that updates are coming from a known source. The addon takes care of generating that and the Noise secret and sharing these keys with Home Assistant so you don't have to care about them.

ESPHome on a Macbook?!

My ESPHome workflow doesn't involve the web UI or the addon at all. Instead, I install ESPHome on my Macbook with homebrew and manage the OTA and HA secret keys with 1Password and a small helper script. The script and all of my ESPHome configs live in this public GitHub repo.

This is the script as it exists today:

#!/bin/bash

set -x
set -eo pipefail

trap "rm common/device_base.yaml" EXIT

op inject --in-file common/device_base.yaml.in --out-file common/device_base.yaml

command=$1
shift

if [ $# -eq 0 ]; then
    configs="*.yaml"
else
    configs="$@"
fi

esphome $command $configs

All this is really doing is using 1Password's op inject tool to generate a file with my configured secrets, runs esphome, and makes sure to clean up the generated file with that trap line. The top of device_base.yaml.in looks like this:

substitutions:
  wifi_ssid: "op://keen.land secrets/ESPHome Secrets/ESPHOME_WIFI_SSID"
  wifi_password: "op://keen.land secrets/ESPHome Secrets/ESPHOME_WIFI_PASSWORD"
  fallback_ssid_password: "op://keen.land secrets/ESPHome Secrets/ESPHOME_FALLBACK_SSID_PASSWORD"
  home_assistant_encryption_key: "op://keen.land secrets/ESPHome Secrets/ESPHOME_HOME_ASSISTANT_ENCRYPTION_KEY"
  ota_password: "op://keen.land secrets/ESPHome Secrets/ESPHOME_OTA_PASSWORD"

All of those are just text entries in the ESPHome Secrets rich text item, but again they can be whatever you want. If you decide to make them password type entries I believe you'd need to add --reveal to the op inject command, but I'm not 100% certain on that.

This differs in kind of a fundamental way from the way the web UI / addon work, in so far as the addon will create and manage unique OTA and HA keys for each device. My setup instead uses two keys shared among all of my devices. I don't see this as a significant risk because I don't use esphome devices in higher security contexts (i.e. my door locks are not running esphome), but your threat model is likely different than mine so you should make your own decisions. Nothing is stopping you from using unique keys for every device with this setup, you just have more secrets to manage in 1Password.

Workflow

My workflow looks like:

$ <edit whatever.yaml>
$ ./build.sh run whatever.yaml
# a bunch of compiler output and then logging from the device itself
$ git add whatever.yaml && git commit -m 'updates' && git push origin main

There aren't many hard edges with this setup. You can put whatever you want into common and you can organize your devices however you want.

One exception is that secrets.yaml has some confusing implicit behavior, so I just commit an empty one and use a different file for my secrets.

Updating ESPHome is not something I do on a regular basis, but when I do it's basically just brew upgrade esphome && ./build.sh.

The process to add a new ESPHome device to Home Assistant is also fairly streamlined. All you do is attach the ESP32 device to your computer with USB, create a new yaml file, and run ./build.sh run <your new file>.yaml. ESPHome will pick up that there's a serially attached device without firmware and handle flashing the new firmware to it.

Once ESPHome is running on the new device it should show up in the Home Assistant integrations page and something that can be added. Clicking the accept button will open a config flow where you can paste your Home Assistant key, and then it should work just like any other device.