#PhotonInc | Explore Tumblr Posts and Blogs

fastcompression · 4 years

Text

Software for Hamamatsu ORCA Processing on GPU

Author: Fyodor Serzhenko

Scientific research demands modern cameras with low noise, high resolution, frame rate and bit depth. Such imaging solutions are indispensable in microscopy, experiments with cold atom gases, astronomy, photonics, etc. Apart from outstanding hardware there is a need for high performance software to process streams in realtime with high precision.

Hamamatsu Photonincs company is a world leader in scientific cameras, light sources, photo diodes and advanced imaging applications. For high performance scientific cameras and advanced imaging applications, Hamamatsu introduced ORCA cameras with outstanding features. ORCA cameras are high precision instruments for scientific imaging due to on-board FPGA processing enabling intelligent data reduction, pixel-level calibrations, increased USB 3.0 frame rates, purposeful and innovative triggering capabilities, patented lightsheet read out modes and individual camera noise characterization.

ORCA-Flash4.0 cameras have always provided the advantage of low camera noise. In quantitative applications, like single molecule imaging and super resolution microscopy imaging, fully understanding camera noise is also important. Every ORCA-Flash4.0 V3 is carefully calibrated to deliver outstanding linearity, especially at low light, to offer improved photo response non-uniformity (PRNU) and dark signal non-uniformity (DSNU), to minimize pixel differences and to reduce fixed pattern noise (FPN).

The ORCA-Flash4.0 V3 includes patented Lightsheet Readout Mode, which takes advantage of sCMOS rolling shutter readout to enhance the quality of lightsheet images. When paired with W-VIEW GEMINI image splitting optics, a single ORCA-Flash4.0 V3 camera becomes a powerful dual wavelength imaging device. In "W-VIEW Mode" each half of the image sensor can be exposed independently, facilitating balanced dual color imaging with a single camera. And this feature can be combined with the new and patented "Dual Lightsheet Mode" to offer simultaneous dual wavelength lightsheet microscopy.

Applications for Hamamatsu ORCA cameras

There are quite a lot of scientific imaging tasks which could be solved with Hamamatsu ORCA cameras:

Digital Microscopy

Light Sheet Fluorescence Microscopy

Live-Cell Microscopy and Live-Cell Imaging

Laser Scanning Confocal Microscopy

Biophysics and Biophotonics

Biological and Biomedical Sciences

Bioimaging and Biosensing

Neuroimaging

Hamamatsu ORCA-Flash4.0 V3 Digital CMOS camera (image from https://camera.hamamatsu.com/jp/en/product/search/C13440-20CU/index.html)

Hamamatsu ORCA-Flash4.0 V3 Digital CMOS camera: C13440-20CU

Image processing for Hamamatsu ORCA-Flash4.0 V3 Digital CMOS camera

That camera generates quite high data rate. Maximum performance for Hamamatsu ORCA-Flash4.0 V3 could be evaluated as 100 fps * 4 MPix * 2 Byte/Pix = 800 MByte/s. As soon as these are 16-bit monochrome frames, that high data rate could be a bottleneck to save such streams to SSD for two-camera system for long-term recording, which is quite usual for microscopy applications.

If we consider one-day recoding duration, storage for such a stream could be a problem. That two-camera system generates 5.76 TB data per hour and it could be a good idea to implement realtime compression to cut storage cost. To compress 16-bit frames, we can't utilize either JPEG or H.265 encoding algorithms because they don't support more than 12-bit data. The best choice here is JPEG2000 compression algorithm which is working natively with 16-bit images. On NVIDIA GeForce GTX 1080 we've got the performance around 240 fps for lossy JPEG2000 encoding with compression ratio around 20. This is the result that we can't achieve on CPU because corresponding JPEG2000 implementations (OpenJPEG, Jasper, J2K, Kakadu) are much slower. Here you can see JPEG2000 benchmark comparison for widespread J2K encoders.

JPEG2000 lossless compression algorithm is also available, but it offers much less compression ratio, usually in the range of 2-2.5 times. Still, it's useful option to store original compressed data without any losses which could be mandatory for particular image processing workflow. In any way, lossless compression makes data rate less, so it's always good for storage and performance issues.

Optimal compression ratio for lossy JPEG2000 encoding should be defined by checking different quality metrics and their correspondence to a particular task to be solved. Still, there is no good alternative for fast JPEG2000 compression for 16-bit data, so JPEG2000 looks as the best fit. We would also recommend to add the following image processing modules to the full pipeline to get better image quality:

Dynamic Bad Pixel Correction

Data linearization with 1D LUT

Dark Frame Subtraction

Flat Field Correction (vignette removal)

White/Black Points

Exposure Correction

Curves and Levels

Denoising

Crop, Flip/Flop, Rotate 90/180/270, Resize

Geometric transforms, Rotation to an arbitrary angle

Sharp

Gamma Correction

Realtime Histogram and Parade

Mapping and monitor output

Output JPEG2000 encoding (lossless or lossy)

The above image processing pipeline could be fully implemented on GPU to achieve realtime performance or even faster. It could be done with Fastvideo SDK and NVIDIA GPU. That SDK is supplied with sample applications in source codes, so user can create his own GPU-based application very fast. Fastvideo SDK is avalilable for Windows, Linux, L4T.

There is also a gpu-camera-sample application which is based on Fastvideo SDK. You can download source codes and/or binaries for Windows from the following link on Github - gpu camera sample. Binaries are able to work with raw images in PGM format (8/12/16-bit), even without a camera. User can add support for Hamamatsu cameras to process images in realtime on NVIDIA GPU.

Fastvideo SDK to process on GPU raw images from Hamamatsu ORCA sCMOS cameras

The performance of JPEG2000 codec strongly depends on GPU, image content, encoding parameters and complexity of the full image processing pipeline. To scale the performance, user can also utilize several GPUs for image processing at the same time. Multiple GPU processing option is the part of Fastvideo SDK.

If you have any questions, please fill the form below with your task description and send us your sample images for evaluation.

Links

Hamamatsu ORCA-Flash4.0 V3 Digital sCMOS camera

GPU Software for camera applications

JPEG2000 Codec on NVIDIA GPU

Image and Video Processing SDK for NVIDIA GPUs

GPU Software for machine vision and industrial cameras

Original article see at: https://www.fastcompression.com/blog/hamamatsu-orca-gpu-image-processing.htm

Subscribe to our mail list: https://mailchi.mp/fb5491a63dff/fastcompression

#Hamamatsu #ORCA #camera #software #GPU

0 notes