ESP-Now integration

[megane999]

Describe the problem you have/What new integration you would like
Integration of ESP-NOW protocol

Please describe your use case for this integration and alternatives you've tried:
Just use separated esp on this protocol, but didn't know how to use esp-now with esp-home

Additional context

[OttoWinter]

Similar to #124

New communication channels are a lot of work to add - and probably you will not see this any time soon natively supported.

Also, I don't think ESP-NOW offers many benefits for ESPHome directly - yes it (ab)uses an optimized and slimmer WiFi stack with smaller connection times etc, but with ESPHome you need a wifi connection anyway for integration with HA.

[rradar]

@OttoWinter

with smaller connection times

much⭐ smaller connection times should be possible which should enable long running battery powered esp devices

but with ESPHome you need a wifi connection anyway for integration with HA.

Depends. One esphome node could work as a gateway for all the battery powered esphome nodes using esp-now to communicate

Why esp-now?

• Works with or without WiFi connection
• Works with or without HomeAssistant
• Instant startup time if not using wifi
• No addition hardware needed
• Easy to use pub/sub event model
• Send and receive from multiple devices

@iphong did a nice start here: https://github.com/iphong/esphome-espnow

🌟 In this video the time measured using esp-now instead of wifi was a run time of total 280ms. The same hardware build using 'classic' wifi with mqtt had a run time of around 7 seconds. This is a factor of 25 ⚠️

[OttoWinter]

@rradar Yeah I see ESP-NOW is an interesting protocol for cross-device communication. However, bringing this into ESPHome as an official component is a bit difficult. Of course you can create a quick component that can send/broadcast some arbitrary bytes to devices in range, but the user then is exposed very much to the inner workings of C and has to deal with bytes/etc. Also selecting which devices to send something to, or getting some sort of acknowledgement that data was received would be necessary for an official integration.

[iphong]

One thing that make esp-now so great is that it can work along side with home assistant. So you get the best of both worlds with no trade offs.
In my opinion, espnow should not be a used as a main comunication transport. Native API and home assistant does it so well already. Since espnow doesn't rely on wifi router connection, it can be used for sensors update, or direct remote control switches.
A perfect application for espnow I can think of is a 2-way or 3-way switches. Your light connect to home assistant as a usual via local wifi connection, while receiving on off command from all switches using esp-now. And your switches can get state update of that light and display it accordingly. In case home assistant somehow disconnected, your switches still work.

[glmnet]

This is cool. I believe we need another abstraction for these

• Espnow
• LoRa
• Rs485
• can bus
• http May be, much like what web server and http components currently do
• ir, ok this one is odd

Those I believe can be target for a lightweight node to node api.

Ideally one would change the backend comm layer and node to node comm will keep working.

Api should support sending and receiving events, if the api is crafted maybe non core esphome developers (not Otto) can support those.

[iphong]

I'm working on a personal project that will push the esp8266 to its limit to see how good the esp-now protocol performs. The EspRC library used in my esphome integration example was part of this project.

Check it out at https://github.com/iphong/esp-visual-led

[OttoWinter]

@glmnet Yeah I agree we need some abstraction to enable strutured communication over all those interfaces (I've been working on some prototypes for a while based on the current native API, but it's hard to create one that works well for all base communication layers).

One major problem with ESP-Now definitely is the missing acknowledgement support, so you can broadcast something but you never know if it arrived. which means for a real protocol we will have to invent some sort of our own TCP

[rradar]

One major problem with ESP-Now definitely is the missing acknowledgement support, so you can broadcast something but you never know if it arrived.

@OttoWinter actually esp-now supports encryption which should include an acknowledgement.

From https://www.programmersought.com/article/90502088859:

ESP-NOW CCMP encryption method used VSA frame. CCMP can be found inIEEE Std. 802.11-2012standard.

WiFiDevice holds a PMK (Primary Master Key) with several LMK (Local Master Key), the length of these are 16-bit Key. PWK for AES-128 encryption algorithm by LMK, call interface esp_now_set_pmk () is provided. If the PWM is not set by the user, it will use the default PWK. LMK is arranged to encrypt the VSA frame. Support for up to six LMK, VSA does not support multicast encryption frame.

But as it looks like it's not only bad documented but actually also has a hard limitation of nodes (maximum 10 or 6 depends of SoftAP/Station mode).

A very interesting approach to overcome this limitation is the EnigmaIOT protocol:

EnigmaIoT is an open source solution for wireless multi sensor systems. It has two main components, multiple nodes and one gateway.

A number of nodes with one or more sensors each one communicate in a secure way to a central gateway in a star network using EnigmaIoT protocol.

This protocol has been designed with security on mind. All node data is encrypted with a random key that changes periodically. Key is unique for each node and dynamically negotiated, so user do not have to enter any key. Indeed, all encryption and key agreement is transparent to user.

Actually the protocol only focus on the link layer which would perfectly fit with esphome which would act as the application layer:

• Application layer is not controlled by EnigmaIoT protocol but main program. User may choose whatever data format or final destination of payload. [...] The only limit is the maximum packet length that, for ESP-NOW is around 200 bytes.
• Link layer is the one that add privacy and security. It manages connection between nodes and gateway in a transparent way. It does key agreement and node registration and checks the correctness of data messages. In case of any error it automatically start a new registration process. On this layer, data packets are encrypted using calculated symmetric key.
• Physical layer currently uses connectionless ESP-NOW. But a hardware abstraction layer has been designed so it is possible to develop interfaces for any other layer 1 technology like LoRa or nRF24F01 radios.

This actually could act like the abstraction @glmnet suggested just three posts up

[motwok]

I am working on a ESPHome component that will use ESP-Now supporting ESP8266 and ESP32 including ESP-Now based Crypto.

Basic communication is already working great... i am just in the phase to make ist usable with yaml and a sensor integration.

May i am able to do the first release next weekend.

Emmo

[motwok]

After struggle with some resource problem connected to the ESP32, crypto and wifi_component, i gave up to fix this, might be there is a complete overhauling of the wifi component needed.
So i release the component with the restriction that ESP32 with Wifi enabled can't communicate with ESP8266 and vice versa.

Emmo

[motwok]

If some one likes to check out my Wifi-Now component:

https://github.com/motwok/esphome/releases/tag/0.0.1.0

Have fun.

Also it would be great if somone does a review of the code.

Emmo

[fabir-git]

Would be great to see this functionality in ESPHome. Thanks for your work, motwok.

[rarroyo6]

Even if ESP-Now is not integrated, it would be nice to have ESPHome-ESPHome communications, bypassing the HA.
@iphong gives a good example using 2 and 3 way switches. I have the same situation, I have 2 Sonoff IFan-03 and I have 2 Sonoff switches to control them via HA. When the HA goes down, which it has, I loose control of the fans and the lights in the room.
Even a simple request server, that can be accessed with the "HTTP Request" component would be nice, and probably much simpler to integrate. That way HA can still control the devices, but if it goes down, the devices can still function manually on their own.

[motwok]

Even if ESP-Now is not integrated, it would be nice to have ESPHome-ESPHome communications, bypassing the HA.
This is exactly one of the reasons i created the wifi-Now module

@iphong gives a good example using 2 and 3 way switches. I have the same situation, I have 2 Sonoff IFan-03 and I have 2 Sonoff switches to control them via HA. When the HA goes down, which it has, I loose control of the fans and the lights in the room.
You can do similar things with Wifi-Now Module.

Even a simple request server, that can be accessed with the "HTTP Request" component would be nice, and probably much simpler to integrate. That way HA can still control the devices, but if it goes down, the devices can still function manually on their own.
An HTTP request requires at least a functioning Access Point... Wifi-Now requieres only that both modules are in range of each other.

Try it even when there is the word Alpha... it is stable enougth to be used.

Emmo

[jkdobro]

Interesting Esp-Now Flooding Mesh project:
https://github.com/arttupii/EspNowFloodingMesh

[motwok]

Interesting Esp-Now Flooding Mesh project: https://github.com/arttupii/EspNowFloodingMesh

I made the "intresting" Esp now implementation... and i had also mesh in mind... but after wasting many hours for an Pullrequest that is completly ignored (there was a request to remove tests, no comment) like allmost all other PR with Wifi in name... i have better things to do than obeying dumb changing CI Rules and maintain PR for futhur ignorance over the next years.

The reason: the "community" is not intrested, at least the intrested people in the community do not show intrest...

mo

[ybarigou]

@motwok I tried your fork (with 'wifi' and 'wifi_now') as external components, unfortunately, I kept getting a bunch of compilation errors one after the other, seems like it does not fit with the current ESPHome internal structure anymore. Is this the case or am I doing something wrong ?!

[motwok]

Is this the case or am I doing something wrong ?!

I am sorry... due to an currently overvelming job i have no time to investigate this in near future.

[racingpassion]

Has this project died :( was looking forward to some low powered esp.

[motwok]

Nope... near future there will be an overhaul dividing it into 2 parts... one is meant as shared library and the other part is the yaml stuff

[gmag11]

One major problem with ESP-Now definitely is the missing acknowledgement support, so you can broadcast something but you never know if it arrived.

@OttoWinter actually esp-now supports encryption which should include an acknowledgement.

From https://www.programmersought.com/article/90502088859:

ESP-NOW CCMP encryption method used VSA frame. CCMP can be found inIEEE Std. 802.11-2012standard.
WiFiDevice holds a PMK (Primary Master Key) with several LMK (Local Master Key), the length of these are 16-bit Key. PWK for AES-128 encryption algorithm by LMK, call interface esp_now_set_pmk () is provided. If the PWM is not set by the user, it will use the default PWK. LMK is arranged to encrypt the VSA frame. Support for up to six LMK, VSA does not support multicast encryption frame.

But as it looks like it's not only bad documented but actually also has a hard limitation of nodes (maximum 10 or 6 depends of SoftAP/Station mode).

A very interesting approach to overcome this limitation is the EnigmaIOT protocol:

EnigmaIoT is an open source solution for wireless multi sensor systems. It has two main components, multiple nodes and one gateway.
A number of nodes with one or more sensors each one communicate in a secure way to a central gateway in a star network using EnigmaIoT protocol.
This protocol has been designed with security on mind. All node data is encrypted with a random key that changes periodically. Key is unique for each node and dynamically negotiated, so user do not have to enter any key. Indeed, all encryption and key agreement is transparent to user.

Actually the protocol only focus on the link layer which would perfectly fit with esphome which would act as the application layer:

• Application layer is not controlled by EnigmaIoT protocol but main program. User may choose whatever data format or final destination of payload. [...] The only limit is the maximum packet length that, for ESP-NOW is around 200 bytes.
• Link layer is the one that add privacy and security. It manages connection between nodes and gateway in a transparent way. It does key agreement and node registration and checks the correctness of data messages. In case of any error it automatically start a new registration process. On this layer, data packets are encrypted using calculated symmetric key.
• Physical layer currently uses connectionless ESP-NOW. But a hardware abstraction layer has been designed so it is possible to develop interfaces for any other layer 1 technology like LoRa or nRF24F01 radios.

This actually could act like the abstraction @glmnet suggested just three posts up

Hi,

I did not see this before.

Actually I guess it could be feasible to integrate EnigmaIOT as a EspHome component. I've just started reading about ESPHome, but I am not ready to start a development yet.

If anyone who is familiar with EspHome developint wants to try I'm open to give my help. I guess some changes would be necessary to do it but it is possible.

EnigmaIOT is quite stable now. As it solves my use case I'm not developing new features until needed.

This integration would enable EspHome to run in battery powered devices for long time, just like ZigBee but with a simple ESP32 as a gateway.

This gateway could be even modified to do a more straight integration with HomeAssistant.

Let me know any idea.

Regards

[jkdobro]

@gmag11 EnigmaIOT has a star topology. Can you expand so that it can work like Mesh Network?

[gmag11]

This is one of my target for long term. I could use MDF sdk, but it is only compatible with ESP32. As EnigmaIOT pretends to keep compatibility with ESP8266 this will not happen soon.

[Kimotu]

@evlo unfortunately it's ESP32 only and does not compile for 8266. I already contacted George for a 8266 version. He will grab some of his old 8266 and try to adapt it if somebody will test it.

[evlo]

To conclude, and for my future reference (as quite often I found my own posts when searching for solutions :) ) i succeeded using BLE, I used this https://github.com/wifwucite/esphome-ble-controller for the server component.

Only real issue was that the BLE and WiFi MAC are different and I did not know that.

[Ulrar]

That's an interesting idea if all you're using is esp32, but it won't work with esp8266 unfortunately since they don't have bluetooth, ESPNow would still be very useful.

Sadly given that all the no neutral switches seem to use esp8266, that doesn't help me :(

[ripnetuk]

Hi... i made the easynow component - i am planning on doing 8266 support, but real life keeps me busy.
I see that someone has already been working on it here
https://gitlab.com/dbuggz/ripnetuk-esphome-easynow
i would be delighted to get a pull request if you get it working, otherwise it will be looked at when the kids are back at school after easter :)

[Ulrar]

I wonder if you could (ab)use this for switches, by transmitting a specific value on a sensor for on and another for off, and reacting on those values on the other side to trigger lights and such

[RubenKelevra]

I think supporting ESP-Mesh is much more interesting. It creates an self-organizing mesh network which supports more than 1000 nodes (10 connections per node). In comparison ESP-Now is pretty basic, as it only allows point to point connections which needs to be known first.

I think prioritizing ESP-Mesh over this, makes sense.

[afarago]

Development of BTHome + ESPNow combination in progress - any comments/ideas are more than welcome.
https://github.com/afarago/esphome_component_bthome

Unfortunately ESPNow does not yet support secure communication (there is a flaw in the current implementation).

[Ulrar]

This seems focused on forwarding BTHome, is there any way to use the ESPNow bits independently so they can be used to pass arbitrary stuff from one esp to another ?

Now that ESPHome has a way to turn wifi off, it'd be really cool to be able to use ESPNow instead to control devices that aren't actively connected. Like a passive light bulb controlled by a shelly behind the switch, to use a random example I'm not at all very much looking for

[afarago]

Most of the esp-now implementations need to accomplish several round-trips as they convey complex data.

This solution focuses on creating packets on one side and consuming data on a central hub.

This allows one to transmit approx 50-70 individuals measurements with a single packet.
That means less risks, faster operation and less Coulombs consumed...

[Kimotu]

@afarago BT = Bluetooth. So bluetooth is required for its use? Or does it also compile on esp8266 (which do not have bluetooth) and can be used to transmit data via esp-now from sensor to hub?

[afarago]

The beauty of Beethoven is that it works on ESP8266 and ESP32 as well.
You might want to check out the provided example yamls.

I do need to add more emphasis to that in the documentation ;)

[evlo]

I just skimmed it for now, but I do not see any option to pass click or other events, probably could be send as state "clicked" maybe?

I did migrate most of my stuff to esp32 so I can communicate using BLE, but I will try to find out some time to try your stuff, at least i could use all the esp8266s that are left somehow.

Would be pretty awesome for libre tuya stuff, that supports only wifi and being able to send message from cheap switch to cheap bulb directly might be useful, although maybe eespnow is really a thing and bekens do not support that?

use case for the switch to light control over bluetooth is shown here https://github.com/wifwucite/esphome-ble-controller#integration-with-ble_client-light-and-switch-example so something similar with virtual binary switch should be doable with yours, but i understand that your focus is low power sensors

[Kimotu]

@afarago where can I open bug reports or report issues concerning your component? Neither do I know how to manually send values (beethowen_transmitter.send) compiles without any errors, but does not seem to send something. I have to enable auto_send: true and even when using connect_persistent: true the component searches continously all channels for bth receiver and prints warnings about long time operations and blocks of 20-30ms.

[afarago]

@afarago where can I open bug reports or report issues concerning your component? Neither do I know how to manually send values (beethowen_transmitter.send) compiles without any errors, but does not seem to send something. I have to enable auto_send: true and even when using connect_persistent: true the component searches continously all channels for bth receiver and prints warnings about long time operations and blocks of 20-30ms.

Thanks for taking the time to try and test it. I have enabled the issues tab for the repo, feel free to add it there as well for easier tracking. During these weeks I could spend less time with the project, so there will be delay in my reaction.
If you are tech savvy, and find the problem, you can also create a fix and a PR.

Potential idea to check: the send action takes the complete_only config variable into account - therefore if enabled only sends packet when all inputs receive a value.
I have tested it with success on ESP8266, have had very little time to check ESP32 yet.

[gmag11]

Development of BTHome + ESPNow combination in progress - any comments/ideas are more than welcome.
https://github.com/afarago/esphome_component_bthome

Unfortunately ESPNow does not yet support secure communication (there is a flaw in the current implementation).

You can always encrypt payload before building espnow packet. Using shared secret key is not that difficult.

Using chachapoly algorithm is light enough for esp8266

[mastersf]

good idea

[tresdesarrollo]

hi, I have ESPHome 2024.9.2 and when write wifi_now logs drops Component not found: wifi_now.
I try a lot of things but any else. What can i do?

[nielsnl68]

I'm working on a official version for esphome esphome/esphome#7141 . I would love to know what you think about it.

[ripnetuk]

I'm working on a official version for esphome esphome/esphome#7141 . I would love to know what you think about it.

Hi,
I think its awesome that you are doing this. Looks like a much better design than I came up with, not to mention that I have lost interest in it since BLE achieves better battery life with no fuss for what im using it for (temperature sensors).

Good luck with it :)
cheers, George

[nielsnl68]

Thanks @ripnetuk

[tresdesarrollo]

Really good but, i cant use the normal espnow and i dont know where i can take a version with espnow component, the component or any solution to use espnow whith esphome

[nielsnl68]

what do you mean @tresdesarrollo? what kind of version are you need?

[tresdesarrollo]

sorry if i don´t write well, my english is´t very good. First, thank you about the help and interest.
What i need is simply sistem like esp_now to use in esphome. One or two esp's send and recive data for activate triggers, switchs, ... and other with wifi conected to home assisistant too.
The problem, in first, is I cant find espnow component than i can use, is no componet usable anywhere?

[tresdesarrollo]

Just now i realized that i can found in nvds-new-espnow branch, Iḿ newbie. Now i cant try but later i do it. thanks

[RubenKelevra]

@tresdesarrollo as it's not yet part of the ESPHome project you need to declare an external component, so ESPHome will fetch the new component from a different location.

The readme for this can be found here:

https://esphome.io/components/external_components.html

[ripnetuk]

@tresdesarrollo as it's not yet part of the ESPHome project you need to declare an external component, so ESPHome will fetch the new component from a different location.

The readme for this can be found here:

https://esphome.io/components/external_components.html

to add to this, I (original external EasyNow component author) have lost interest, so going forward, the one by @tresdesarrollo will be the one to watch.

[tresdesarrollo]

wow, few days with problems with the espcam and the real matter was the wifi antena, the resist was fited on on internal antena and the extarnal do not catch the wifi, that is why i want espnow. Finaly I change the resist position and implement espnow too, incredible. I use the nvds-new-espnow branch and I do not try because now is runing all funcion over wifi but y work more with it because I want they comunication if the router drops down. thanks for the help, whithout you i don't know what time i can loose

[nielsnl68]

the resistor was fitted on on internal antenna and the external do not catch the wifi, that is why i want espnow.

When you use ESPNOW you will get the same issues because espnow is based on the wifi framework layer.

[tresdesarrollo]

yes, I know but one of them is in range of the router and the other lost it for a few cm and is in range with the pther esp signal. But now two ones are coneccted to the router because I fit the resist in the good place and system is running well.
Now I need work whith other things but in few day I believe continue with espnow, thatś good in the sistem to run if router fails

[nagyrobi]

Making progress in esphome/esphome#8187, and first implementation is for UART: esphome/esphome#8214. This is a generalised approach, which will make possible to develop simpler components for exchanging data directly between nodes by any means, like TCP, ESP-Now, LoRa, etc.