RebrickNet – Lego Part Detector

(rebrickable.com)

46 points | by wonger_ 4 days ago ago

16 comments

  • denysvitali 4 hours ago

    (I'm not a LEGO expert.)

    It looks like LEGO provides 3D models for their components (e.g: via Bricklink [1]). Wouldn't it be easier to generate the training data with a rendering pipeline of some sort that randomizes the brick position in a 3D scene with different lighting?

    Of course you can still collect the user submissions for the test set

    [1]: https://www.bricklink.com/v3/studio/download.page

    • juliangamble 31 minutes ago

      There is some code to generate Lego images from Lego 3d files here and use it to train models:

      https://github.com/jtheiner/LegoBrickClassification

      It is based on this post (which others have mentioned)

      https://jacquesmattheij.com/sorting-lego-the-software-side/

    • prybak 4 hours ago

      From personal experience, it doesn't work that well. The available 3D models [1] are not detailed enough and the effort to make realistic and diverse renders is huge. It will kinda work but it won't generalize well into real images.

      [1] https://www.ldraw.org/

      • tirant 2 hours ago

        That is something I don't really understand exactly why. 3D Models provide in one dataset the whole geometry information of each part, making it easier to learn and recognize that part in the real world, wouldn't it?

        Is it because these ML algos lack a way to internally interpret 3D models for learning?

        On top of that you reduce effort in data labeling, as each model would come with the relevant part ID, shape and color.

        Of course, finetuning can be done afterwards with real world photos to increase robustness.

        • loa_in_ 24 minutes ago

          I think it's because we classify single images which lose 3d information. We expect them to work with single image but not even humans do that (without stereoscopic vision) if we can help it.

  • juliangamble 28 minutes ago

    I’ve tried different part detectors, and nothing has been as good as the Google Image search (download the Google App if you’re on an iPhone.) I’ve found it surprisingly good at identifying LEGO parts.

  • bigiain 5 hours ago
  • primitivesuave 4 hours ago

    I built a rudimentary brick sorting machine with the Lego color sensor many years ago, and I've seen some awesome brick sorting machines pop up over the years [1]. I started planning a sorting machine with a larger array of buckets in CAD but never got around to building it, hopefully the intrepid Lego fans over time will keep advancing the state of the art.

    1. https://www.youtube.com/watch?v=04JkdHEX3Yk

  • shagrath 4 hours ago

    Neat, I happpened to rediscover and sort my childhood lego collection two weeks ago, and had good success for the "no so common parts" with the brickowl camera search (I don't know what kind of engine is behind)

  • Aissen 3 hours ago

    No mention of license on contributed data, or the model.

  • byrolew 4 hours ago

    Does it use brickognize underneath?

  • prybak 4 hours ago

    RebrickNet is an abandoned project that supports only 300 parts. I recommend Brickognize [1], which recognizes all 85k Lego parts, minifigures, and sets.

    Disclosure: I'm the creator.

    [1] https://brickognize.com/

    • pragmatick an hour ago

      That's a weirdly minimalistic site. I mean it's nice that the front page is exactly what you want to see but I wish the About page would show some more information about the what the site can do, some examples, technology bits etc.

      • rkagerer an hour ago

        Sometimes a hammer can just be a hammer. I wish the whole web was so clean.

    • majke an hour ago

      Piotr has an amazing video on the inner workings of his model https://www.youtube.com/watch?v=bzyG4Wf1Nkc

    • cgeier 2 hours ago

      Which works really well! Thanks for a great project!

      Is there any more information available on how it works?