GLTF Usage

[garthhenning]

Can you provide an example for how to use the decoder with GLTF buffers, ideally with jGLTF buffers?

[javagl]

This might be a bit out of scope of ~"an issue in this repository".

More generally, it's not clear who is "responsible" for creating examples, so to speak.

Of course, I'd like to add support for Draco to JglTF. And as mentioned (even somewhat recently) in the corresponding issue, a first attempt of decompressing a Draco-compressed glTF with this library (and writing it out as a valid glTF file again) seemed to work. But for now, this was just an experiment (meaning the code is an absolute mess 🙂 ).

The baseline of using the decoder is actually fairly trivial. You can obtain the BufferViewModel with the Draco-decoded data from the GltfModel and then create a DracoMesh from that:

ByteBuffer bufferViewData = bufferViewModel.getBufferViewData();
byte array[] = new byte[bufferViewData.capacity()];
bufferViewData.get(array);
DracoMesh dm = (DracoMesh)Draco.decode(array);

But then, in order to actually use that, there are a few things to do. Some lines from my experiments, just to illustrate some points:

// Fetch the POSITION attribute
PointAttribute attrPos = dm.getNamedAttribute(AttributeType.POSITION);

// (other attributes can be fetched similarly)

// Temporary arrays for face indices and positions
int[] face = new int[3];
float p3[] = new float[3];

for (int i = 0; i < dm.getNumFaces(); i++)
{
    // Fetch and print the face indices
    dm.readFace(i, face);
    int j0 = face[0];
    int j1 = face[1];
    int j2 = face[2];
    System.out.println(String.format("%d %d %d\n", j0, j1, j2));
    
    // Fetch and print the position for index "j0"        
    attrPos.getValue(attrPos.mappedIndex(j0), p3);
    System.out.println(Arrays.toString(p3));

    // (same for j1 and j2, there may be duplicates)
}

Maybe that already helps. I think this mappedIndex part was what was not immediately obvious for me when I did the first tests. But in my local experiments, I wrote this data into new accessors and pragmatically assembled a new model for that, and that was actually valid and could be rendered, so that approach in general should be not completely wrong. But proper support for Draco in JglTF will still take some time - I'm juggling with priorities.

[garthhenning]

Thank you. That example did help and the meshes now decode smoothly.

[javagl]

Are the results publicly available somewhere? Depending in the scope and goal, there may be some duplicate efforts.

I do have a local state of JglTF with draco experiments (that the given snippet was taken from), but ... I also have local states with meshopt support, and three other open PRs with improvements for the extension support in general and things that should go into a 'version 3.0.0'. I hope that I can somehow consolidate that, although there are many loose ends to tie up, and little (spare) time to do all this.

[garthhenning]

Here is what I did. It's a bit rough, but it worked. I have a simpler use case than you in that I have a single upstream source of GLTF models, all of which have one mesh, with no corner cases, and can all be checked before deployment. Also, I'm copying into a Java Fxyz PolygonMesh because I'm using this for a JavaFX scene editor. So it doesn't have to be fast and I can live with inefficient extra buffer copies so long as the user can see and position the model.

    public PolygonMesh convertDracoMeshToPolygonMesh(MeshPrimitiveModel primitive, GltfModel gltfModel) {
        PolygonMesh polyMesh = new PolygonMesh();

        try {
            var dracoConfig = (HashMap<String, Object>) primitive.getExtensions().get("KHR_draco_mesh_compression");

            int bufferViewIndex = (Integer) dracoConfig.get("bufferView");

            ByteBuffer bufferViewData = gltfModel.getBufferViewModels().get(bufferViewIndex).getBufferViewData();
            if (bufferViewData == null || gltfModel.getBufferViewModels().get(bufferViewIndex).getByteLength()==0)
                throw new IllegalStateException("No mesh data for Draco-compressed mesh.");

            byte[] bytes = new byte[bufferViewData.remaining()];
            bufferViewData.get(bytes);

            DracoMesh dm = (DracoMesh) Draco.decode(bytes);
            if (dm == null)
                throw new IllegalStateException("Draco-compressed mesh can not be decoded.");

            PointAttribute attrPos = dm.getNamedAttribute(AttributeType.POSITION);
            PointAttribute attrNrm = dm.getNamedAttribute(AttributeType.NORMAL);
            PointAttribute attrUv  = dm.getNamedAttribute(AttributeType.TEX_COORD);

            if (attrPos == null)
                throw new IllegalStateException("Draco-compressed mesh has no POSITION attribute.");

            final int numPoints = dm.getNumPoints();
            final int numFaces  = dm.getNumFaces();

            // --- points (x,y,z) per Draco point ---
            float[] points = new float[numPoints * 3];
            float[] p3 = new float[3];

            for (int p = 0; p < numPoints; p++) {
                int posIndex = attrPos.mappedIndex(p);
                attrPos.getValue(posIndex, p3);

                int base = p * 3;
                points[base]     = p3[0];
                points[base + 1] = p3[1];
                points[base + 2] = p3[2];
            }
            polyMesh.getPoints().setAll(points);

            float[] texCoords;
            if (attrUv != null) {
                texCoords = new float[numPoints * 2];
                float[] uv2 = new float[2];

                for (int p = 0; p < numPoints; p++) {
                    int uvIndex = attrUv.mappedIndex(p);
                    attrUv.getValue(uvIndex, uv2);

                    int base = p * 2;
                    texCoords[base]     = uv2[0];
                    texCoords[base + 1] = uv2[1];
                }
            } else {
                texCoords = new float[] { 0f, 0f };
            }
            polyMesh.getTexCoords().setAll(texCoords);

            int[][] faces = new int[numFaces][6];
            int[] tri = new int[3];

            for (int f = 0; f < numFaces; f++) {
                dm.readFace(f, tri);
                int j0 = tri[0], j1 = tri[1], j2 = tri[2];

                // point indices
                faces[f][0] = j0;
                faces[f][2] = j1;
                faces[f][4] = j2;

                // texCoord indices
                int t0 = (attrUv != null) ? j0 : 0;
                int t1 = (attrUv != null) ? j1 : 0;
                int t2 = (attrUv != null) ? j2 : 0;

                faces[f][1] = t0;
                faces[f][3] = t1;
                faces[f][5] = t2;
            }
            polyMesh.setFaces(faces);
            polyMesh.getFaceSmoothingGroups().resize(polyMesh.getFaces().length);
        } catch (DrakoException e) {
            e.printStackTrace();
        }
        
        return polyMesh;
    }

[javagl]

I see. Well, when everything is (and can be) hard-wired, things tend to be a bit easier. (And to bring FXyz/FXyz#198 into some reasonable state - maybe just as a draft PR with a new package of JglTF - is still on my TODO list. That list is too long, though. I'll post updates in the JglTF repo and the respective PRs when things are "ready"...)