Iván Hernández Dalas: Ubicept replaces blurry CMOS with sharp SPAD imaging

Image showing how noise is handled differently on a SPAD sensor, such as one from Ubicept.

SPAD sensors handle noise differently than a CMOS sensor, reducing blur in video images. Credit: Ubicept

Ubicept last month released the Ubicept Toolkit, which it said brings physics-based imaging to modern computer vision systems. The company claimed that its approach delivers high-quality, trustworthy data.

The toolkit allows offline and live processing of complementary metal-oxide-semiconductor (CMOS) videos and single-photon avalanche diode (SPAD) cameras, aiding in object recognition and spatial awareness. Ubicept asserted that its SPAD sensors offer superior low-light and high-contrast imaging in comparison with conventional cameras.

Industrial cameras in use for the past four decades have used CMOS sensors. This is also the same technology used in digital photography cameras and security cameras, so their production volume makes CMOS sensors relatively cheap.

The problem with CMOS cameras is that low-light images can be noisy, and videos captured with a CMOS sensor are blurry when looking at a single frame.

The invention: better low-light resolution

Ubicept said its novel approach is to use SPAD sensors with a new image-processing pipeline to improve the contrast and clarity of the resulting images and/or videos. This also offers immediate image quality improvements with existing CMOS hardware, according to the Boston-based company.

For vision perception applications in low-light situations, such as autonomous driving at night, Ubicept said it system can effectively improve several edge-case scenarios that confuse existing camera-based navigation and perception pipelines.

The SPAD technology, which captures each photon with sub-nanosecond resolution, can enhance image quality in various environments, the company claimed.

Ubicept plans to integrate its software with existing hardware, such as Qualcomm Snapdragon and NXP chips, to improve perception models in robotics and automotive applications. The technology is expected to mature and become more affordable within the next one to two years.

graph illustrating how a SPAD sensor captures light photons.

Simplified illustration of the weighted photon counting technique. The earlier the arrival of the first incident photon, the brighter the incident light. Credit: Canon

“Ubicept’s technology revolutionizes how machines see the world by unlocking the full potential of today’s and tomorrow’s image sensors,” stated Sebastian Bauer, CEO of Ubicept. “Our physics-based approach captures the full complexity of motion, even in low-light or high-dynamic-range conditions, providing more trustworthy data than AI-based video enhancement.”

“With the Ubicept Toolkit, we’re now making our advanced single-photon imaging more accessible for a broad range of applications from robotics to automotive to industrial sensing,” he added.

Hardware versus software

When ask about the product roadmap, Bauer replied, “What we’ve released is a piece of software that people can run with the GUI [graphical user interface], but also it has an API [application programming interface]. And what it does is you can do offline processing of existing CMOS videos.”

“You can also hook up CMOS cameras live and have live processed outputs,” he said. “You can connect your computer to a SPAD camera and really see the quality difference with our imaging, evaluate the quality of improve improvement of CMOS versus SPAD, and build your own computer vision application.”

Ubicept said its system is designed for the most advanced sensors to maximize image data quality and reliability. It noted that the Ubicept Toolkit can support any widely available CMOS camera with raw uncompressed output, giving perception developers immediate quality gains.

The company said it wants to work with robotics developers and camera manufacturers to develop a new generation of smart perception.

“We’re not planning on making sensors ourselves, but we have really good IP [intellectual property] for 3D stacked sensors, meaning doing some processing directly on the sensor chip, and then essentially all the way up to the perception,” said Bauer. “If you think about cameras, you have a camera that’s a sensor, plus an ISP or image signal processing chip, and then you have a perception model.”

“Our thinking is to to build all that full stack, including a full-blown ISP/processing pipeline by using physically optimal single full-term perception and converting that into into high-quality perception output, like object detection, simultaneous localization and mapping [SLAM], visual odometry,” he explained.

Ubicept partners with Pi Imaging Technology

Pi Imaging Technology is a leader in the field with its SPAD Alpha, a next-generation 1-megapixel camera that it said delivers zero read noise, nanosecond-level exposure control, and frame rates up to 73,000 fps. Designed for demanding scientific applications, the camera offers researchers and developers extreme temporal precision and light sensitivity.

The companies said the Ubicept Toolkit builds on these strengths by transforming the SPAD Alpha’s raw photon data into clear, ready-to-use imagery for perception and analysis.

“Ubicept shares our deep commitment to advancing perception technology,” said Michel Antolović, CEO of Pi Imaging Technology. “By combining our SPAD Alpha’s state-of-the-art hardware with Ubicept’s real-time processing, perception engineers can get the most from what single-photon imaging has to offer.”

Ubicept said its toolkit provides engineering teams with everything they need to visualize, capture, and process video data efficiently with the Ubicept Photon Fusion (UPF) algorithm. The SPAD Toolkit also includes Ubicept’s FLARE (Flexible Light Acquisition and Representation Engine) firmware for optimized photon capture.

In addition, the Ubicept Toolkit comes with “white-glove support to early adopters for a highly personalized and premium experience,” said the company.