It looks like LEGO provides 3D models for their components (e.g: via Bricklink [1]).
Wouldn't it be easier to generate the training data with a rendering pipeline of some sort that randomizes the brick position in a 3D scene with different lighting?
Of course you can still collect the user submissions for the test set
From personal experience, it doesn't work that well. The available 3D models [1] are not detailed enough and the effort to make realistic and diverse renders is huge. It will kinda work but it won't generalize well into real images.
That is something I don't really understand exactly why. 3D Models provide in one dataset the whole geometry information of each part, making it easier to learn and recognize that part in the real world, wouldn't it?
Is it because these ML algos lack a way to internally interpret 3D models for learning?
On top of that you reduce effort in data labeling, as each model would come with the relevant part ID, shape and color.
Of course, finetuning can be done afterwards with real world photos to increase robustness.
I think it's because we classify single images which lose 3d information. We expect them to work with single image but not even humans do that (without stereoscopic vision) if we can help it.
I’ve tried different part detectors, and nothing has been as good as the Google Image search (download the Google App if you’re on an iPhone.)
I’ve found it surprisingly good at identifying LEGO parts.
I built a rudimentary brick sorting machine with the Lego color sensor many years ago, and I've seen some awesome brick sorting machines pop up over the years [1]. I started planning a sorting machine with a larger array of buckets in CAD but never got around to building it, hopefully the intrepid Lego fans over time will keep advancing the state of the art.
Neat, I happpened to rediscover and sort my childhood lego collection two weeks ago, and had good success for the "no so common parts" with the brickowl camera search (I don't know what kind of engine is behind)
RebrickNet is an abandoned project that supports only 300 parts. I recommend Brickognize [1], which recognizes all 85k Lego parts, minifigures, and sets.
That's a weirdly minimalistic site. I mean it's nice that the front page is exactly what you want to see but I wish the About page would show some more information about the what the site can do, some examples, technology bits etc.
(I'm not a LEGO expert.)
It looks like LEGO provides 3D models for their components (e.g: via Bricklink [1]). Wouldn't it be easier to generate the training data with a rendering pipeline of some sort that randomizes the brick position in a 3D scene with different lighting?
Of course you can still collect the user submissions for the test set
[1]: https://www.bricklink.com/v3/studio/download.page
There is some code to generate Lego images from Lego 3d files here and use it to train models:
https://github.com/jtheiner/LegoBrickClassification
It is based on this post (which others have mentioned)
https://jacquesmattheij.com/sorting-lego-the-software-side/
From personal experience, it doesn't work that well. The available 3D models [1] are not detailed enough and the effort to make realistic and diverse renders is huge. It will kinda work but it won't generalize well into real images.
[1] https://www.ldraw.org/
That is something I don't really understand exactly why. 3D Models provide in one dataset the whole geometry information of each part, making it easier to learn and recognize that part in the real world, wouldn't it?
Is it because these ML algos lack a way to internally interpret 3D models for learning?
On top of that you reduce effort in data labeling, as each model would come with the relevant part ID, shape and color.
Of course, finetuning can be done afterwards with real world photos to increase robustness.
I think it's because we classify single images which lose 3d information. We expect them to work with single image but not even humans do that (without stereoscopic vision) if we can help it.
I’ve tried different part detectors, and nothing has been as good as the Google Image search (download the Google App if you’re on an iPhone.) I’ve found it surprisingly good at identifying LEGO parts.
Nice.
Reminds me of this: https://jacquesmattheij.com/sorting-two-metric-tons-of-lego/
I built a rudimentary brick sorting machine with the Lego color sensor many years ago, and I've seen some awesome brick sorting machines pop up over the years [1]. I started planning a sorting machine with a larger array of buckets in CAD but never got around to building it, hopefully the intrepid Lego fans over time will keep advancing the state of the art.
1. https://www.youtube.com/watch?v=04JkdHEX3Yk
Neat, I happpened to rediscover and sort my childhood lego collection two weeks ago, and had good success for the "no so common parts" with the brickowl camera search (I don't know what kind of engine is behind)
No mention of license on contributed data, or the model.
Does it use brickognize underneath?
RebrickNet is an abandoned project that supports only 300 parts. I recommend Brickognize [1], which recognizes all 85k Lego parts, minifigures, and sets.
Disclosure: I'm the creator.
[1] https://brickognize.com/
That's a weirdly minimalistic site. I mean it's nice that the front page is exactly what you want to see but I wish the About page would show some more information about the what the site can do, some examples, technology bits etc.
Sometimes a hammer can just be a hammer. I wish the whole web was so clean.
Piotr has an amazing video on the inner workings of his model https://www.youtube.com/watch?v=bzyG4Wf1Nkc
Which works really well! Thanks for a great project!
Is there any more information available on how it works?