This is neat but Splats are not really mean to be edited in this way.
Splats are sort of like byte code, they are the compiled and optimized representation of reflected light as semi-transparent guassians.
Or you can think of them as the PDF equivalent of a Google or Word Doc. All the logic is gone, and you just have final optimized results.
Generally when you edit PDFs, the results are not great and you cannot make major edits because the layout won't reflow, etc.
So while this is cool, I don't think it will take off unless there is another innovation in terms of either using AI to "reflow" the lighting and surfaces after an edit, or inferring more directly the underlying representations (true surface properties and the light sources.)
Hi Ben! I would argue that it is very useful for splats to be edited in this way. I couldn't have built this application without SuperSplat for isolating, cleaning, transforming and optimizing/compressing the PLY:
- cleaning up noisy GuassianSplats is useful. There are often stragglers floating around in space that need to get deleted.
- compression/optimizing them is useful.
This being a cleanup and compression tool makes sense, but I guess I don't call that an "editor."
I guess I was more arguing against the idea that this is a viable "editor" where one can combine and manipulate in more radical ways Gaussian Splats. The current technological approach doesn't make this a feasible use case.
- Copy & Paste: e.g. delete a tree and fill the hole with a copied patch of grass
- Color Adjustments: tinting, brightness, etc.
If these aren't editing ops, I don't know what is. :)
Sure, you _could_ go back and recapture photogrammetry or rerun training, but that's super costly in terms of time. SuperSplat lets you make simple edits quickly and easily.
In theory if you delete something you have to recompute global illumination and remove cast shadows in the immediate environment of the removed object, but that information is baked in the gaussian splats. I think that's the kind of limitation the parent comment is talking about.
To be as accurate as possible, yes, you need to consider lighting/shadows. But trust me, in many circumstances, you can copy+paste gaussians and it looks 'good enough'. It depends on the scene and the edit you want to make.
I guess in theory what you say could be correct, however in practice this tool has been very helpful for client work of editing, cleaning, cropping and even slight modification of Gaussian splats. I could see a similar argument for raster images in general -- they are hard to edit as you're modifying individual pixels and it's not efficient, but we've seen tools grow from MS Paint to modern Photoshop to become very useful. I think the same could be said here -- it's just early and we're at the "bytecode" level as you say.
Any tips for an app to use on iOS to capture the necessary .ply data?
Scaniverse is a great app by Niantic that can do this on-device, but it isn't very customizable and can't export its raw scanning data (exported .plys do not have the data this editor requires).
Not sure if I'm missing something, but the submission title says "open-source" and in the tool's help menu there's a link to the repo (https://github.com/playcanvas/supersplat), the tool runs in your browser, there's no server involved besides a Web-Server hosting the files.
Those were the times when you could actually rely on Google to give you the right results for such a query near the top, and more importantly, give you the same results it would give to the person you're telling to Google it.
Is there a reading guide to the maths behind Gaussian splats? All the resources I could find either assumed lots of knowledge (including what a "3D Gaussian" even is), or were written for complete lay-person (and probably includes some AI grift).
I wish all these guides used better notation. They all use the scariest possible Greek symbols (pi epsilon feta etc) that are hard to make sense of without a degree in anthropology, instead of nicely named variables like programmers use.
This is neat but Splats are not really mean to be edited in this way.
Splats are sort of like byte code, they are the compiled and optimized representation of reflected light as semi-transparent guassians.
Or you can think of them as the PDF equivalent of a Google or Word Doc. All the logic is gone, and you just have final optimized results.
Generally when you edit PDFs, the results are not great and you cannot make major edits because the layout won't reflow, etc.
So while this is cool, I don't think it will take off unless there is another innovation in terms of either using AI to "reflow" the lighting and surfaces after an edit, or inferring more directly the underlying representations (true surface properties and the light sources.)
Hi Ben! I would argue that it is very useful for splats to be edited in this way. I couldn't have built this application without SuperSplat for isolating, cleaning, transforming and optimizing/compressing the PLY:
https://playcanv.as/e/p/cLkf99ZV/
Integrating AI is an interesting topic and something that certainly has potential.
I 100% agree with:
- cleaning up noisy GuassianSplats is useful. There are often stragglers floating around in space that need to get deleted.
- compression/optimizing them is useful.
This being a cleanup and compression tool makes sense, but I guess I don't call that an "editor."
I guess I was more arguing against the idea that this is a viable "editor" where one can combine and manipulate in more radical ways Gaussian Splats. The current technological approach doesn't make this a feasible use case.
Coming very soon is:
- Copy & Paste: e.g. delete a tree and fill the hole with a copied patch of grass
- Color Adjustments: tinting, brightness, etc.
If these aren't editing ops, I don't know what is. :) Sure, you _could_ go back and recapture photogrammetry or rerun training, but that's super costly in terms of time. SuperSplat lets you make simple edits quickly and easily.
In theory if you delete something you have to recompute global illumination and remove cast shadows in the immediate environment of the removed object, but that information is baked in the gaussian splats. I think that's the kind of limitation the parent comment is talking about.
To be as accurate as possible, yes, you need to consider lighting/shadows. But trust me, in many circumstances, you can copy+paste gaussians and it looks 'good enough'. It depends on the scene and the edit you want to make.
Wow, the fade-in animation is most excellent! Mind sharing how you created it?
I guess in theory what you say could be correct, however in practice this tool has been very helpful for client work of editing, cleaning, cropping and even slight modification of Gaussian splats. I could see a similar argument for raster images in general -- they are hard to edit as you're modifying individual pixels and it's not efficient, but we've seen tools grow from MS Paint to modern Photoshop to become very useful. I think the same could be said here -- it's just early and we're at the "bytecode" level as you say.
I'm not really sure what you mean. Think of SuperSplat as the photoshop of gaussian splats?
- SuperSplat dev :)
That's a ridiculous take. Generated splats almost always have garbage parts that need to be truncated. An editor is absolutely needed for that.
Relightable gaussian splats - https://junxuan-li.github.io/urgca-website/
There are already approaches to infer bidirectional reflectance distribution functions (BRDFs, the "true surface properties") and lights: https://nju-3dv.github.io/projects/Relightable3DGaussian/
great metaphor, thanks!
I could imagine this as a clean-up tool for splats. In any case, beautiful interface and the sample model made me smile. Thanks for sharing.
There's an app for Quest 3 called Gracia, which allows you to see these in 3D space:
- https://www.meta.com/en-gb/experiences/gracia/25784099001234...
- https://www.gracia.ai
Whats the performance like?
This is cool!
Any tips for an app to use on iOS to capture the necessary .ply data?
Scaniverse is a great app by Niantic that can do this on-device, but it isn't very customizable and can't export its raw scanning data (exported .plys do not have the data this editor requires).
PolyCam I think is the most popular?
Impressive! Does anyone know if this is open source? Or perhaps can be run locally as a server?
Not sure if I'm missing something, but the submission title says "open-source" and in the tool's help menu there's a link to the repo (https://github.com/playcanvas/supersplat), the tool runs in your browser, there's no server involved besides a Web-Server hosting the files.
I found its repository after searching google. The license is MIT.
[1] https://github.com/playcanvas/supersplat
I remember a time when it was considered unpolite to ask a question without googling first. Is it still the case?
> I remember a time when it was considered unpolite to ask a question without googling first. Is it still the case?
Yes
As I recall it, the asshole move was to reply to a question with a LMGTFY link.
Those were the times when you could actually rely on Google to give you the right results for such a query near the top, and more importantly, give you the same results it would give to the person you're telling to Google it.
Is there a reading guide to the maths behind Gaussian splats? All the resources I could find either assumed lots of knowledge (including what a "3D Gaussian" even is), or were written for complete lay-person (and probably includes some AI grift).
The blog posts from Aras are a really good starting point: https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pr...
I wish all these guides used better notation. They all use the scariest possible Greek symbols (pi epsilon feta etc) that are hard to make sense of without a degree in anthropology, instead of nicely named variables like programmers use.
I do hope they don't use brined white cheese as a symbol.
Oh, they do. They most definitely do. And salads. ∪= Feta hasn't made into Unicode yet, so I'm substituting it for generic cheese, but just you wait.
It would be interesting to have a 3d version of a mesh warp / puppet warp