How do I get direct access to the Three.js scene?

[fselcukcan]

Is it possible to add a new texture and colors to the model on the fly?
How to find the Three.js Engine, Scene etc.?

For this maybe it is required to access the underlying engine, Three.js, and from there on go with three.js api to do these. The question for this is that does model-viewer exposes three.js and how?

[cdata]

@fselcukcan 🚨this is currently not supported by the public API, so be advised that even if you access the internal scene, <model-viewer> might change in a way that breaks you unexpectedly and without migration notes in a future release.

That said, you can get access to internal details using Object.getOwnPropertySymbols and looking for a symbol with the description scene, which is the internal Three.js scene.

We plan to expose access to the scene explicitly in a future release. Please comment on #738 if you would like to voice your support for this feature!

[fselcukcan]

Thank you very much! Commented there as well.
Probably I will try manually until the support comes then adapt to that.

[fselcukcan]

I have accessed the scene property through the described Symbol of the model-viewer.

const mw = document.querySelector("model-viewer");
const sceneSym = Object.getOwnPropertySymbols(mw).find(x => x.description === "scene");

I wonder if it is possible to access the three.js instance used if possible, as well. Since I will make use of materials to change materials and also textures on the fly I will need it. Though I could include Three.js in a module script in the page I will author, I may make use of th existing one avoiding downloading again. Is it possible to access that too. I have only looked and could not see it between the own symbol properties of model-viewer.

[cdata]

If you look up mw[sceneSym] it should be an instance of ModelScene which inherits from Three.js' Scene.

[cdata]

See related implementation here:

https://github.com/GoogleWebComponents/model-viewer/blob/a725fda6c0367563155e5a774137dc3d7d2869f0/src/model-viewer-base.ts#L171-L172

And here:

https://github.com/GoogleWebComponents/model-viewer/blob/a725fda6c0367563155e5a774137dc3d7d2869f0/src/three-components/ModelScene.ts#L53

[fselcukcan]

Thank you I have checked them. What I want to do is to create new material instances and attach them to a mesh in model replacing the existing one. It looks like I need THREE in the scope. I may need the MeshStandardMaterial to create materials to achieve this. For example, new THREE.MeshStandardMaterial( { color: 0xff4400, envMap: envMap, metalness: 0.9, roughness: 0.2, name: 'orange' } )
Is ModelScene instance capable of creating materials? I guess not.

[cdata]

@fselcukcan this is where it gets a little tricky (in case it did not already feel tricky enough :P). You will probably be able to get access to some constructors using JavaScript trickery, but TBH since this is totally unsupported territory YMMV.

For example, I quickly applied the above hack to access the scene on our Glitch demo, which shows the Astronaut model. In that case, I got a reference to the scene. Then, I did this:

const MeshStandardMaterial =
    scene.model.children[0].children[0].children[0].material.constructor;
const newMaterial = new MeshStandardMaterial(/* options here */);

In the above snippet, I found a material that was already in the scene and used it to get access to the constructor. You can do this for other classes too, like Object3D and Mesh.

That said, please keep in mind 🔥 this is totally unsupported 🔥 and your strategy will depend on the model you load.

[fselcukcan]

I can import the three.js with a script tag as well to saty on the safe and I actually did it. Thanks for the above trick as well. However I cannot happen to realize my original aim of updating the material/s in model viewer on runtime. It throws as follows:

Uncaught TypeError: Cannot read property 'toArray' of undefined
    at flatten (model-viewer.js:17482)
    at PureArrayUniform.setValueV3fArray [as setValue] (model-viewer.js:17931)
    at Function.WebGLUniforms.upload (model-viewer.js:18214)
    at setProgram (model-viewer.js:25982)
    at WebGLRenderer.renderBufferDirect (model-viewer.js:24754)
    at renderObject (model-viewer.js:25467)
    at renderObjects (model-viewer.js:25437)
    at WebGLRenderer.render (model-viewer.js:25239)
    at Renderer.render (model-viewer.js:56467)
    at model-viewer.js:56407

I have an event handler function like this to update the selected material of selected mesh in two select html menus, scene I got by the method you have explained previously, and materials lib is an array of materials:

function updateMaterial (event) {
      console.log("matMenu.selectedIndex: ", matMenu.selectedIndex, "meshMenu.selectedIndex: ", meshMenu.selectedIndex);
      var meshName = meshNames[meshMenu.selectedIndex]
      const mesh = scene.model.getObjectByName(meshName);
      // mesh.material.color.setHex(colors[matMenu.selectedIndex].value);
      mesh.material = materialsLib[matMenu.selectedIndex];
      mesh.material.needsUpdate = true;
}

const materialsLib = [
        new THREE.MeshStandardMaterial( { color: 0xff4400, metalness: 0.9, roughness: 0.2, name: 'orange' } ),
    	new THREE.MeshStandardMaterial( { color: 0x001166, metalness: 0.9, roughness: 0.2, name: 'blue' } ),
    	new THREE.MeshStandardMaterial( { color: 0x990000, metalness: 0.9, roughness: 0.2, name: 'red' } ),
    	new THREE.MeshStandardMaterial( { color: 0x000000, metalness: 0.9, roughness: 0.5, name: 'black' } ),
    	new THREE.MeshStandardMaterial( { color: 0xffffff, metalness: 0.9, roughness: 0.5, name: 'white' } ),
        new THREE.MeshStandardMaterial( { color: 0x555555, metalness: 1.0, roughness: 0.2, name: 'metallic' } )
];

[cdata]

@fselcukcan I'm not 100% sure, but I'm pretty sure that using another instance of Three.js will not work. You will have to use a trick like the one I showed you in order to access the constructors for Three.js classes.

[fselcukcan]

Actually it did not work. And I have got the constructor as you mentioned and it worked when I have added emissive color to material I have created with it, not only color. But if the model does not have a constructor then this method is doomed.

[cdata]

@fselcukcan yes, unfortunately this approach is a total hack (and did I mention not supported? 😅).

If you are able to clone the project and build the repo for yourself, another thing you can do is write your own mixin to extend the element class that gets defined. Your mixin can import the Symbol properties and also Three.js modules needed from the same source modules, and this would allow you to avoid all of the hacking. This is harder than just loading from a CDN, but it's a much cleaner way to do fancy modifications.

[deers]

@cdata @fselcukcan I think it's working, you can see my demo here.
But I need to copy the envMap from the existing material otherwise the model will show black, and if we move the model and then change the material, the model doesn't update automatically so I need to modified exposure twice to trigger the update, if there's an event to trigger render manually it will be much easier.

[ArvindSharda78]

Thanks @deers , i was also able to use the code and adapt it to my use case.
But I am facing a challenge that when i go to AR view, the changes are lost, they are not visible in the AR, do you encounter this issue.
@cdata @fselcukcan can you please help!

[elalish]

@ArvindSharda78 Your changes can only be maintained in the WebXR AR mode, since this stays in the browser. SceneViewer and QuickLook (the only option on iOS) are natives apps that must re-download your model, and hence cannot access the changes you've made in the browser.

[mdemirci-exar]

Hello, I can access the scene with the following code.

let scene =
modelViewer[
Object.getOwnPropertySymbols(modelViewer).find(
(x) => x.description === "scene"
)
];

Though I'm aware this is not officially supported, I'm integrating ModelViewer into my personal project as a framework, and therefore I require more direct access to the THREE instances. is there any way to do that?

[elalish]

You probably want to import model-viewer-module.js and import map your own three.js, as described here: https://modelviewer.dev/examples/postprocessing/#setup