Proposal for a CSS2DRenderer

[JakobHock]

Motivation

TresJS is currently missing a clean way to integrate the CSS2DRenderer into the scene. The standard way to use the CSS2DRenderer currently would to do something like the following:

<script setup lang="ts">
import { TresCanvas } from '@tresjs/core'
import CustomCSS2DRenderer  from "@/components/CustomCSS2DRenderer "
import Label2D from '@/components/Label2D.vue'

const meshRef = ref<THREE.Mesh | null>(null)
</script>

<template>
  <TresCanvas clear-color="#c0ffee" window-size>
    <CustomCSS2DRenderer>
     <TresPerspectiveCamera :position="[3, 3, 3]" />
     <TresMesh>
        <TresTorusGeometry :args="[1, 0.5, 16, 32]" />
        <TresMeshBasicMaterial color="orange" />

      </TresMesh>
      <Label2D :mesh="meshRef!" />
    </CustomCSS2DRenderer>
  </TresCanvas>
</template>

The CustomCSS2DRenderer would provide the renderer to the tree and the user would have to inject in into every place they would want to create a label as. Additional there is no nice way to handle the UI part as HTML elements are not allowed in a TresCanvas. This is an unmaintainable way of handling the 2D part of a 3D scene.

Proposal

To address these difficulties, i propose tres could provide a TresCSS2DRenderer component that would handle the rendering of the 2D part of the scene. With the registration of said component the user would be able to use a new TresLabel component inside meshes to create 2D labels in the scene. TresLabel is a wrapper around the CSS2DObject class from Threejs and supports props to control the underlying object. The TresLabel provides a slot for the user to define the content of the label with standard HTML and CSS. The Mesh to which the TresLabel is assigned to would be provided in the template as a slotprop to enable the user to access it's information in the label. The proposed API would look like the following:

<script setup lang="ts">
import { TresCanvas, TresCSS2DRenderer, TresLabel } from '@tresjs/core'
</script>

<template>
  <TresCanvas clear-color="#c0ffee" window-size>
    <TresCSS2DRenderer>
      <TresPerspectiveCamera :position="[3, 3, 3]" />
      <TresMesh>
        <TresTorusGeometry :args="[1, 0.5, 16, 32]" />
        <TresMeshBasicMaterial color="orange" />
        <TresLabel>
          <template #default="{ mesh }">
            <div>
              <p>Label for mesh with position: {{ mesh.position.toArray() }}</p>
            </div>
          </template>
        </TresLabel>
      </TresMesh>
    </TresCSS2DRenderer>
  </TresCanvas>
</template>

Changes

For this proposal to be implemented the following changes would have to be made:

• Add a TresCSS2DRenderer component to the core package.
• Register a CSS2DRenderer on creation of the scene if a TresCSS2DRenderer is present in the tree.
• Only Allow HTML elements inside the TresLabel component if a TresCSS2DRenderer is present in the tree.
• Expose the underlying Mesh of a TresMesh as a slotprop in the TresLabel component.

Benefits

• Ease of use: The user would have a clean and intuitive vue-like way to integrate 2D elements into the scene.
• Flexibility: The user would be able to use standard HTML and CSS to style the labels.
• Maintainability: The user is able to use the TresLabel component in every TresMesh without having to inject the renderer into the tree.
• Future Compatibility: Support for 2D elements in a 3d scene are necessary for future TresJS packages like a game engine or a 3D editor.

Conclusion

Support for 2D elements in a 3D scene is a necessary feature for TresJS to be a complete 3D framework. The proposed API would provide a clean and maintainable way to integrate 2D elements into the scene. The user would be able to use standard HTML and CSS to style the labels and would not have to fiddle around with rendering labels in a javascript native way.

As this is my first proposal/contribution to TresJS i would be happy to hear your feedback on the proposal and be able to discuss it with you.

[andretchen0]

Thanks for opening the discussion @JakobHock .

Personally, I'd like to be able to allow users to swap renderers eventually, but there are a number of changes that would need to happen in order to allow that.

We're a really small team working on the core and at the moment and other issues are taking priority. But that could change if there is feedback from the community indicating a strong desire to swap renderers.

Feel free to open a feature request on Tres or discuss on the Discord. (For Tres, Github discussions are sometimes overlooked. Those other avenues might give a better idea of how much community interest there is.)

[JakobHock]

Thank you for you response.

Is there a roadmap that is open to the public?
I would like to take a look and help out where i can.

[andretchen0]

Is there a roadmap that is open to the public?
I would like to take a look and help out where i can.

I don't think there's a public document with issues at the moment. But maybe @alvarosabu knows?

Otherwise, the Tres Cientos package is probably a good place to get started with issues, if that sounds interesting: https://github.com/Tresjs/cientos/issues

[alvarosabu]

Hi @JakobHock @andretchen0 I just made the project public https://github.com/orgs/Tresjs/projects/3

[lorenwe]

Hello, I have also encountered this problem and I am very excited to see your solution here. However, I am not sure how the CustomsCSS2Drenderer and Label2D.vue are written. Can you provide me with some examples? Thank you very much

[JakobHock]

You could implement it something like this:
https://stackblitz.com/edit/tresjs-minimal-reproduction-336ene?file=src%2FApp.vue

Note that to get the Mesh and use it you have to use ref as useTemplateRef introduced in 3.5 makes the mesh readonly.
Slots are sadly not available for the label as html breaks the tres injected vue renderer.