Add hardware accel support under Linux

[tytan652]

Description

Requires:

• GS/GL Shenannigans (Prep for obs-browser hwaccel on Linux ?) obs-studio#11219

Required by:

• frontend: Enable browser hw accel toggle under Linux obs-studio#11262

Add hwaccel support on Linux with CEF 6533 and later

Caution

NVIDIA might need to be blacklisted due to their inconsistency on supporting GBM and incomplete DRI3.
TLDR: Only latest generations with latest driver has a likelihood to work
NVIDIA driver has been blacklisted

Motivation and Context

Just wanted to try to make it somehow work for fun, ended up making it for real…

How Has This Been Tested?

Tested under Wayland and X11.

Types of changes

• New feature (non-breaking change which adds functionality)

Checklist:

• My code has been run through clang-format.
• I have read the contributing document.
• My code is not on the master branch.
• The code has been tested.
• All commit messages are properly formatted and commits squashed where appropriate.
• I have included updates to all appropriate documentation.

[kkartaltepe]

Seems fine overall, i really wish we could pin down the modifier issue on x11 to something in cef's codebase... at least then we could see if it ever got fixed in the future.

[kkartaltepe]

Doesnt compile with ENABLE_BROWSER_SHARED_TEXTURE set to 0 because this is used to avoid compiling the drm-formats stuff, but they are still referenced. Just remove the conditionals and always build these functions.

[kkartaltepe]

Maybe i'm missing something on how this was tested, but current master sets the BrowserHwaccel option to false so nothing here takes effect until https://github.com/obsproject/obs-studio/blob/master/UI/obs-app.cpp#L1268-L1278 is flipped.
--- edit
I see this is the dependent PR you mention.

[kkartaltepe]

https://gist.github.com/kkartaltepe/ebf66527a38f188c8b07eb43b2d37de6 I dont remember if this happened as well on my original test months ago. But on my steamdeck tier setup there is significant rendering performance loss so rendering is twice as slow for a scene with 10 browser sources.

[tytan652]

Doesnt compile with ENABLE_BROWSER_SHARED_TEXTURE set to 0 because this is used to avoid compiling the drm-formats stuff, but they are still referenced. Just remove the conditionals and always build these functions.

#if ENABLE_BROWSER_SHARED_TEXTURE should have been #ifdef ENABLE_BROWSER_SHARED_TEXTURE, I fixed it.
The code relies on whether it's define or not, not its value.

I dont remember if this happened as well on my original test months ago. But on my steamdeck tier setup there is significant rendering performance loss so rendering is twice as slow for a scene with 10 browser sources.

This is why we test, I don't know I would be able to reproduce…

[tytan652]

Changes:

• Avoid linking libdrm but expose it's include directory since we only need DRM_FORMAT_
• Blacklist NVIDIA driver since it does not work well on both Wayland and X11

[fatalgoth]

Just streamed for six hours with these patches, minus the NVIDIA blacklisting and it solved some framedrops when certain browser overlays would draw to the screen for me. Instead of a blacklist, can it be a warning or something? It worked really well for me.

[RytoEX]

https://gist.github.com/kkartaltepe/ebf66527a38f188c8b07eb43b2d37de6 I dont remember if this happened as well on my original test months ago. But on my steamdeck tier setup there is significant rendering performance loss so rendering is twice as slow for a scene with 10 browser sources.

@kkartaltepe If the original test was done with CEF 103 and the test in October was done with CEF 126/127, then could this be the issue with Qt message loop (obsproject/obs-studio#11485 et al)? Or is it a separate issue?

Would still like to know if this is good enough to merge, or if there's something else we need to address first.

[kkartaltepe]

https://gist.github.com/kkartaltepe/ebf66527a38f188c8b07eb43b2d37de6 I dont remember if this happened as well on my original test months ago. But on my steamdeck tier setup there is significant rendering performance loss so rendering is twice as slow for a scene with 10 browser sources.

@kkartaltepe If the original test was done with CEF 103 and the test in October was done with CEF 126/127, then could this be the issue with Qt message loop (obsproject/obs-studio#11485 et al)? Or is it a separate issue?

Would still like to know if this is good enough to merge, or if there's something else we need to address first.

No I think its unrelated to that. When i profiled it, graphics stuff was just really slow.