High Performance Image Sensor Processing Using FPGAs [pdf]

(oda.uni-obuda.hu)

87 points | by teleforce 6 days ago ago

13 comments

  • mikewarot 5 days ago

    That's quite an extensive tour through the world of image signal processing.

    I can't quite grok the filter added to the DDS to generate twiddle factors for FFTs. I'll have to re-read that section a few time for it to sink in.

  • kvemkon 5 days ago

    > 3.5 Hardware Implementation of Flat-Field Correction

    Shouldn't one use some hardware description language (HDL) in such chapters? Or I've overlooked where the code is placed?

    • CamperBob2 5 days ago

      It's left as an exercise to the student^H^H^H^H^H^H^H LLM.

  • qoez 5 days ago

    FPGAs? This is something I assumed was processed long ago with ASICs

    • 0xfedcafe 5 days ago

      what do you mean? in image processing or in general? in general they're used a lot where performance is important, especially in high frequency trading, crypto-mining, etc. also they're quite important for image processing in defence/medical imaging.

    • 15155 5 days ago

      Every digital ASIC design is simulated on FPGAs first.

      • FirmwareBurner 4 days ago

        Every? On which FPGA did Nvidia simulate the 5090?

        • Traster 4 days ago

          If you go to nvidia's jobs website today you'll find they're actively hiring FPGA developers for ASIC prototyping. Obviously they're not dumping their 5090 RTL straight into some 10 metre wide FPGA chip. First, they grab the largest FPGA you can get your hands on - the FPGA vendors tend to have a couple of comically expensive comically large SKUs for specifically this purpose. Then you pop a few of them onto a development board and partition your design across the cluster of FPGAs with some custom interconnect, orchestration and DFT stuff. FPGAs offer quite a compelling way of getting test mileage vs simuluation/emulation in software.

        • Kirby64 4 days ago

          Every. If I was to guess, NVidia probably uses Cadence Palladium/Protium solutions[1]. They're basically industry standard, and essentially everyone uses Cadence design tools for circuit design.

          [1]: https://www.cadence.com/en_US/home/tools/system-design-and-v...

        • 15155 4 days ago

          You don't need to simulate the design completely/simultaneously. The FPGA sim implementation might only contain 1 CUDA SM, for instance.

          For large ASIC designs like this, companies often use numerous (12+) FPGAs connected via transceivers on dedicated simulation boards.

    • wmf 5 days ago

      Yes, many SoCs already have an ISP core. Maybe FPGAs are used in low-volume specialized cameras.

    • fennecbutt 5 days ago

      But asics are a big $$$ ask.

    • meindnoch 5 days ago

      Sure. If you buy like a million units.