Re: AW: AW: Measuring noise characteristics of sCMOS cameras

Posted by Zdenek Svindrych-2 on
URL: http://confocal-microscopy-list.275.s1.nabble.com/Measuring-noise-characteristics-of-sCMOS-cameras-tp7585913p7585924.html

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Good point,
as a matter of fact, even in standard photon transfer curve measurements, the fixed pattern noise becomes apparent at several thousand photons per pixel. We're talking about couple hundreds here.
But it's hard to tell how large frame-to-frame and longer term fluctuations are still OK given the large number of frames analyzed (tens of thousands) and the non-trivial method of gain determination in the Joerg's paper (least square estimation). On the other hand, given the photon noise standard deviation is 5% at 400 photons, 1% (max) illumination intensity fluctuation does not sound like a gross overkill...
Best, zdenek
Btw, while my PC monitor LED backlight shows nice PWM at 1.2 kHz, my smartphones produced steady illumination (no fluctuations up to 50 kHz).

---------- Původní zpráva ----------
Od: Gerhard Holst <[hidden email]>
Komu: [hidden email]
Datum: 26. 10. 2016 11:14:42
Předmět: AW: AW: Measuring noise characteristics of sCMOS cameras


Hi Kyle,

 

may I ask 1% of what? Further, including poisson noise? I mean the fluctuations go with the square root of the number of photons.

 

with best regards,

 

Gerhard

___________________________

Dr. Gerhard Holst

Science & Research

PCO AG

Donaupark 11

93309 Kelheim, Germany

fon +49 9441 2005 36

fax +49 9441 2005 20

mob +49 172 711 6049

[hidden email]

www.pco.de

 

Von: Confocal Microscopy List [mailto:[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 17:01
An: [hidden email]
Betreff: Re: AW: Measuring noise characteristics of sCMOS cameras

 

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Thanks to everyone who has thus far replied to my original e-mail. I realize after doing some more reading that I was essentially conflating two different "noise" sources into one: fixed pattern noise, for which one certainly wants a uniform illumination to measure, and the per pixel gain, which is what is measured in the Huang/Bewersdorf paper.

My next question is: what kind of light source is going to provide <1% temporal intensity fluctuations? A CW LED pumped into an integrating sphere?

Zdenek: Thanks for the heads up about the PWM driver in the smart phone LED backlight. I wouldn't be surprised if this were true. Interestingly, I can see aliasing in a live stream from the camera when using a computer monitor as illumination (60 Hz refresh rate) but I can not see it with the smart phone.

Gerhard: The EMVA 1288 document is absolutely wonderful. Thanks for this.

Seamus: I actually have the book next to me on my desk. Thanks for the reminder about the chapter :)

Cheers,
Kyle


On 10/26/2016 03:37 PM, [hidden email] wrote:

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Honestly, I hope my Orca Flash does not have microlenses!

But back to Kyle's question.

Joerg Bewersdorf's approach is strictly on pixel basis, the spatial uniformity is not critical (97% is more than sufficient), but temporal uniformity is important and < 1% intensity fluctuations are not trivial to achieve (I could not find this figure in my smartphone's specs :-). Slow drift can be easily seen in your images (if you average every 1000 images and look at the trends), but short-time fluctuations may be harder to detect (beware, there is often some sort of PWM driver for the LED backlight)!

As mentioned in the ref 17 (thanks Seamus for the link) there are many sources of "fixed pattern noise" (I don't like the term noise, it's just non-uniformity), including different light sensitivity of individual pixels or dust on the optics. Joerg's approach does not account for these effects, he just intended to bring SCMOSes on par with CCDs. There are more critical aspects of localization superresolution microscopy (such as illumination uniformity) than dust on the camera window... (but another word of caution, smudges on the camera won't be visible during the calibration, but may be visible when coupled to the microscope, because the light has fairly low etendue / cone angle / numerical aperture at the detector).

Btw, has anyone characterized the new back-illuminated SCMOS cameras (like Photometrics Prime 95B)? How do they compare with standard SCMOSes (Zyla, Orca)?

Best, zdenek


--
Zdenek Svindrych, Ph.D.
W.M. Keck Center for Cellular Imaging (PLSB 003)
University of Virginia, Charlottesville, VA
http://www.kcci.virginia.edu/
tel: 434-982-4869
Annual FRET Workshop: http://kcci.virginia.edu/workshop-2017

---------- Původní zpráva ----------
Od: Gerhard Holst [hidden email]
Komu: [hidden email]
Datum: 26. 10. 2016 4:49:59
Předmět: AW: Measuring noise characteristics of sCMOS cameras

 

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Kyle,

your question is absolutely reasonable, and in my opinion it doesn't have to be. If you google you will find the EMVA1288 standard for measuring and representing quality parameters of image sensors and cameras. There is described how the gain can be measured. If a linear camera or image sensor model can be assumed, and our experience as manufacturer of sCMOS cameras proves that, for ease of measurement a relatively homogenous illumination should be ok. The standard suggests a diffuse illumination. The homogeneous illumination is more important for getting information on the homogeneous reaction of the image sensor, means to determine the so called photo-response-non-uniformity. To determine the gain the photon transfer curve is usually measured, and this is the variance versus mean signal, and the variance is usually calculated from the difference of two images, therefore if the pixel have more or less the same brightness, that's good enough.
Alternatively you might use the Fe+55 method. Here the knowledge about the charge generation if silicon is hit be x-ray quants is used. We have done that and in case of the sCMOS image sensors in delivers the same results like the PTC curve approach.

If you take an integrating sphere with diffuse reflection and scattering, like suggested in the EMVA1288 that would be good enough in my opinion. Since the image sensors have micro lenses, directed radiation is not such a good idea.

with best regards,

Gerhard
___________________________
Dr. Gerhard Holst
Science & Research
PCO AG
Donaupark 11
93309 Kelheim, Germany
fon +49 9441 2005 36
fax +49 9441 2005 20
mob +49 172 711 6049
[hidden email]
www.pco.de

-----Ursprüngliche Nachricht-----
Von: Confocal Microscopy List [[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 09:55
An: [hidden email]
Betreff: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few questions, so I am providing a "too long; didn't read" first to summarize. I'm hoping that some of you will find this topic interesting and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon response curve of an sCMOS camera? Why should the illumination pattern be so uniform when each sCMOS pixel can be thought of as an independent sensor?

I am returning to work on a minor problem that has interested me for some time. I work in localization microscopy (STORM/PALM/PAINT) and have been using sCMOS cameras for the past two years with good results. To precisely localize the single molecule emissions, we take into account the pixel-dependent noise characteristics of our sensors, incorporating the measured characteristics of the sensor into the maximum likelihood estimation of a fluorescent molecule's position. This estimation procedure was--as far as I know--first described in Huang et al., Nature Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each pixel of our cameras:

1. the offset (average ADU count under zero illumination) 2. the read noise (variance of the ADU counts under zero illumination) 3. the gain (the number of photoelectrons per ADU when the camera is in the linear response regime)

The offset and read noise are trivial to measure. To measure the gain, however, we capture a few tens of thousands of camera frames with the camera chip under uniform illumination at different light intensities and follow the mathematical operations described in the supplement to the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the gain by observing fluctuations in the pixels' ADU counts in time, not in space? I can understand why illumination non-uniformities would lead to errors when measuring the noise of a CCD chip. For CCD's, I believe that one typically treats each pixel as an independent sample of the noise from the entire chip, so one inherently assumes that the photon shot noise is uniform across the sensor. However, each pixel is only compared to itself when measuring the gain of an sCMOS sensor in the manner described above, so why does it matter that each pixel receives the same light intensity?

2. How flat is "flat enough" for this calibration procedure? With a smart phone screen set an optimum distance from the bare camera port and carefully rotated into position, I can get about 97% uniformity across the whole chip by simply by displaying gray scale images. Most of the non-uniformity appears at the corners of the chip where I think shadowing from the opening in the camera's housing is decreasing the light intensity slightly. The calibrations I get from this method allow me to obtain a localization precision that I independently measured from sparsely distributed dye molecules to be between 8 and 12 nm, which is in line with published STORM results. When measuring tiny clusters of proteins, the scatter plots of the localizations match the overall shapes of their widefield images quite well.

However, a recent paper by Li et al., J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better than 99% uniformity to avoid introducing significant bias into the noise measurements. Furthermore, the engineers at one of the big camera manufacturers once told me I shouldn't even bother trying to do the noise characterization myself since I wouldn't be able to get the required level of uniformity for an accurate characterization. (In fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these questions. So far, my results seem to suggest that my calibration is good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland http://kmdouglass.github.io http://leb.epfl.ch



--  
Kyle M. Douglass, PhD 
Post-doctoral researcher 
The Laboratory of Experimental Biophysics 
EPFL, Lausanne, Switzerland 
http://kmdouglass.github.io 
http://leb.epfl.ch