Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compiling does not include usermods #4597

Open
1 task done
C0oL3r opened this issue Mar 16, 2025 · 23 comments · May be fixed by #4622
Open
1 task done

Compiling does not include usermods #4597

C0oL3r opened this issue Mar 16, 2025 · 23 comments · May be fixed by #4622

Comments

@C0oL3r
Copy link

C0oL3r commented Mar 16, 2025

What happened?

When i'm trying to compile WLED usermods are not included in the firmware.

To Reproduce Bug

If i clone the repository and try to build it like it is it will compile good with audioreactive. But if i add for example Temperature usermod and compile it again it will add Temperature usermod but it will not include audioreactive.
Then if i remove again Temperature it will not include any usermod anymore. I tried to delete .buildcache, .pio but no luck. Searching for string in the firmware file first compile it shows me about audioreactive, second about the Temperature and the third compile it will not find anything about any usermod.
Also even adding ${common.debug_flags} it will not change anything, the firmware will not be recompiled

Expected Behavior

It should add the usermods i set to custom_usermods in platformio.ini. And also recompile if anything changes in platformio.ini.

Install Method

Self-Compiled

What version of WLED?

Nightly Release 20250316 0.16.0

Which microcontroller/board are you seeing the problem on?

ESP32

Relevant log/trace output

Anything else?

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@C0oL3r C0oL3r added the bug label Mar 16, 2025
@blazoncek blazoncek removed the bug label Mar 16, 2025
@blazoncek
Copy link
Collaborator

None of the usermods (except Audioreactive) are automatically compiled with WLED.
If you want to include usermods in your compile, you need to modify platformio.ini (or better override) as specified in usermod's readme file.

@blazoncek blazoncek closed this as not planned Won't fix, can't repro, duplicate, stale Mar 16, 2025
@C0oL3r
Copy link
Author

C0oL3r commented Mar 16, 2025

i did say that i am adding them in platformio.ini, first compile works with audioreactive, second is with audioreactive and Temperature but only Temperature is added, after that none will be added.

@blazoncek
Copy link
Collaborator

Then perhaps @willmmiles may know more. Reopen if needed.

@willmmiles
Copy link
Member

I can't replicate this in my development environment; changes to platformio.ini are always picked up on the next build, and seem to build correctly with the specified usermods.

@C0oL3r , are you using an IDE or building with platformio at the command prompt? What platform is your build system (Linux, Windows, macOS, GitHub codespaces, etc.)?

@willmmiles
Copy link
Member

Actually, let me hazard another guess.

For this custom environment:

[env:my_esp32dev]
extends = env:esp32dev

you will get the default usermods with esp32dev (ie. audioreactive).

For this environment spec:

[env:my_esp32dev_v2]
extends = env:esp32dev
custom_usermods = Temperature

you will get only Temperature; the custom_usermods settings has been overridden with the new value specified there. The inherited value is lost.

For this environment spec:

[env:my_esp32dev_v3]
extends = env:esp32dev
custom_usermods = 

you will get no usermods at all (it's being overriden with the empty string).

I think maybe you want:

[env:my_esp32dev_dwiw]
extends = env:esp32dev
custom_usermods = ${env:esp32dev.usermods} Temperature

...where it keeps the inherited value (as it is explicitly referenced), plus adds the extra module you want.

@DedeHai
Copy link
Collaborator

DedeHai commented Mar 17, 2025

@willmmiles can confirm that building different configurations of usermods using VScode can lead to some or all of them not being added properly: after changing usermod configuration, a full clean must be executed prior to building, otherwise the usermods don't get pulled in properly. Can you confirm?

@C0oL3r
Copy link
Author

C0oL3r commented Mar 17, 2025

@willmmiles i am building it in VS Code with platformio, and i am using windows.
And yes i understand that creating a new environment and not setting the usermods from the original esp32dev will not include audioreactive, but i’m not using an override, i just add or delete usermods from the esp32dev environment. As @DedeHai said it’s very unstable.
If he confirms that even adding or removing flags in the configuration will not rebuild it correctly

@DedeHai
Copy link
Collaborator

DedeHai commented Mar 17, 2025

It is not unstable, if you do a full clean it works just fine. The question is just: is that expected or not.

@C0oL3r
Copy link
Author

C0oL3r commented Mar 17, 2025

It works only for the first compile, after that things starts to get weird. Maybe it’s a problem with the platform on Windows

@C0oL3r
Copy link
Author

C0oL3r commented Mar 17, 2025

I can confirm that compiling it in a Docker container works well, compiling is done everytime with success and everything is added.

@willmmiles
Copy link
Member

It's meant to work correctly every build with up-to-date PlatformIO; I do know there's some bugs with older releases that cause problems with library building.

I'll retest some cases -- I do recall seeing an issue with removed modules not actually being removed, but I thought it'd been resolved with a PlatformIO update. Could be there's still something lurking there.

@willmmiles
Copy link
Member

I couldn't reproduce this on my build system (Win11, VSCode 1.98.2, PlatformIO 6.1.18). I ran a series of verbose builds of a test environment:

custom_usermods = audioreactive
custom_usermods = audioreactive Temperature
custom_usermods = Temperature
custom_usermods = audioreactive
custom_usermods = animartrix

For each build the linker invocation included the correct files and only those for the specified usermods.

@C0oL3r could you please check your PlatformIO version on your windows machine? pio --version? If we can confirm there's an issue with a specific version, I can add a validation to the usermod inclusion script.

@DedeHai DedeHai reopened this Mar 19, 2025
@DedeHai
Copy link
Collaborator

DedeHai commented Mar 19, 2025

I ran a few tests, there is definitely something weird going on.
Here is my setup:
Win10, VSCode 1.98.2, PlatformIO 6.1.18 compiling and direct uploading to ESP32 C3

this is the sequence I ran as a test with comments what I observed:

;-> full clean
;custom_usermods = deep_sleep ; -> not added
;custom_usermods = deep_sleep Battery; -> battery is added correctly, deep_sleep is not
;custom_usermods = audioreactive Temperature deep_sleep Internal_Temperature_v2 Battery -> all but deep_sleep work
;custom_usermods = deep_sleep Battery ;-> no usermods are available (also LED config changed to 30 LEDs...)
;custom_usermods = audioreactive Temperature deep_sleep Internal_Temperature_v2 Battery -> no usermods available
;custom_usermods = audioreactive; -> not available
;custom_usermods = PWM_fan ;-> error as temperatur UM is missing
;custom_usermods = PWM_fan Temperature; -> compiles, no usermod config available though
;-> full clean
;custom_usermods = PWM_fan Temperature -> not seeing any usermod config
;custom_usermods = audioreactive -> added correctly
;custom_usermods = audioreactive Temperature Internal_Temperature_v2 -> LED numbers are reset again... also temperature and internal temperature are shown, AR is missing in config
;custom_usermods = deep_sleep; -> is added correctly
;custom_usermods = audioreactive; -> AR is not added but I still see deep sleep in UM config, it is not working though

what I checked was the UM config menu, always with full browser refresh (ctrl+F5), in the last instance where deep_sleep showed up but was not actually added, I checked if it was still working and it was not, just the menu showed up (where the menu should have shown AR)

@blazoncek
Copy link
Collaborator

FYI sometimes you need to save usermod configuration for it to appear. This is due to the fact that usermods settings page reads cfg.json file directly and if there is no configuration saved for a usermod then it will not be present in UI.

@DedeHai
Copy link
Collaborator

DedeHai commented Mar 19, 2025

thanks for that info. I checked again:

  • the issue is still the same, saving UM config does remove "deep_sleep" if it is shown but was actually removed, still does not show the other.

continuing the sequence from above:
custom_usermods = audioreactive ; -> AR is not added but I still see deep sleep in UM config -> saving UM settings removes deep sleep settings but still does not show AR
custom_usermods = audioreactive Temperature Internal_Temperature_v2 ; -> AR is not showing up, temperature and internal temperature are shown
custom_usermods = audioreactive deep_sleep Temperature Internal_Temperature_v2 ; -> only deep_sleep is showing

@blazoncek
Copy link
Collaborator

That would mean they are not linked in.

@DedeHai
Copy link
Collaborator

DedeHai commented Mar 19, 2025

correct, which is what this issue is about.
when cleaning and adding a UM for the first time after that, I see it getting pulled in in the compile log. When then compiling again, its a hit and miss, as my log shows.

@netmindz
Copy link
Member

Thanks for the bug report @C0oL3r

Can I confirm, were you waiting to try out the "bleeding edge" unreleased version of WLED or just wanting to do a regular build with certain usermods enabled. If it's the latter, please checkout a release tag or the 0_15 stable release branch

@willmmiles
Copy link
Member

I instrumented UsermodManager::setup process and I always see the correct module list being loaded for each build. I can reproduce some settings page jankiness, but I haven't been able to reproduce linker issues. The settings pages do seem more reliable for me than others are reporting. I'm using an ESP32-WROVER with debug output enabled, so it boots twice for each debug build: once after upload, and again when the serial monitor connects, and it looks like cfg.json is getting rewritten with the updated usermod settings somewhere in between before I can get to the settings page.

@DedeHai , when you say "full clean", are you removing ~/.buildcache or just WLED/.pio? I'm wondering if the global build cache could have been poisoned somehow, so that it's using bad cached linker output. If you haven't been clearing ~/.buildcache, and you want to try that, please back up your old one first -- if this does turn out to be a cache poisoning issue, it'd be good to have one to debug from.

I'm going to see about fixing the settings page to always use the live state instead, so we can test cleanly without all the jankiness from outdated configurations.

@DedeHai
Copy link
Collaborator

DedeHai commented Mar 22, 2025

I did not manually delete folders for the test, just the button 'full clean' in pio

@willmmiles
Copy link
Member

Thanks. I've found the 'full clean' button does basically nothing, since the "real" storage is ~/.buildcache. Everything just gets re-copied from the cache instead of actually being rebuilt if platformio thinks nothing has changed since the last build.

@willmmiles
Copy link
Member

Live cfg fix currently pushed to #4601. I'll break it out to a separate PR later I think.

@C0oL3r
Copy link
Author

C0oL3r commented Mar 24, 2025

When i was testing i tried to delete .pio, .buildcache, even reinstalled platformio entirely, the compiling was fixed only for a new clone of the repository, i didn’t tried anymore as i used docker to compile it but when i will have some time i will send you the versions (it should be the latest platformio as i reinstalled it).
I also tried to save the usermods page as @blazoncek told, that is why i have come to the conclusion that platformio is not including the usermods.

Even in my docker environment there are some sketchy things happening as sometimes i need to remove .buildcache in order for the compiler to rebuild my firmware file, especially when i modify only some defines in platformio.ini

@C0oL3r C0oL3r closed this as completed Mar 24, 2025
@C0oL3r C0oL3r reopened this Mar 24, 2025
@C0oL3r C0oL3r closed this as completed Mar 24, 2025
@C0oL3r C0oL3r reopened this Mar 24, 2025
willmmiles added a commit to willmmiles/WLED that referenced this issue Mar 29, 2025
Write the current usermod list to a file, and add it as a dependency to
the linked output file.  This should hopefully overcome potential
poisoning of SCons' build cache with incorrect matches.

Meant to fix wled#4597
willmmiles added a commit to willmmiles/WLED that referenced this issue Mar 29, 2025
Write the current usermod list to a file, and add it as a dependency to
the linked output file.  This should hopefully overcome potential
poisoning of SCons' build cache with incorrect matches.

Meant to fix wled#4597
@willmmiles willmmiles linked a pull request Mar 29, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants