Measuring noise characteristics of sCMOS cameras

[Kyle Michael Douglass]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few
questions, so I am providing a "too long; didn't read" first to
summarize. I'm hoping that some of you will find this topic interesting
and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon
response curve of an sCMOS camera? Why should the illumination pattern
be so uniform when each sCMOS pixel can be thought of as an independent
sensor?

I am returning to work on a minor problem that has interested me for
some time. I work in localization microscopy (STORM/PALM/PAINT) and have
been using sCMOS cameras for the past two years with good results. To
precisely localize the single molecule emissions, we take into account
the pixel-dependent noise characteristics of our sensors, incorporating
the measured characteristics of the sensor into the maximum likelihood
estimation of a fluorescent molecule's position. This estimation
procedure was--as far as I know--first described in Huang et al., Nature
Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each
pixel of our cameras:

1. the offset (average ADU count under zero illumination)
2. the read noise (variance of the ADU counts under zero illumination)
3. the gain (the number of photoelectrons per ADU when the camera is in
the linear response regime)

The offset and read noise are trivial to measure. To measure the gain,
however, we capture a few tens of thousands of camera frames with the
camera chip under uniform illumination at different light intensities
and follow the mathematical operations described in the supplement to
the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the
gain by observing fluctuations in the pixels' ADU counts in time, not in
space? I can understand why illumination non-uniformities would lead to
errors when measuring the noise of a CCD chip. For CCD's, I believe that
one typically treats each pixel as an independent sample of the noise
from the entire chip, so one inherently assumes that the photon shot
noise is uniform across the sensor. However, each pixel is only compared
to itself when measuring the gain of an sCMOS sensor in the manner
described above, so why does it matter that each pixel receives the same
light intensity?

2. How flat is "flat enough" for this calibration procedure? With a
smart phone screen set an optimum distance from the bare camera port and
carefully rotated into position, I can get about 97% uniformity across
the whole chip by simply by displaying gray scale images. Most of the
non-uniformity appears at the corners of the chip where I think
shadowing from the opening in the camera's housing is decreasing the
light intensity slightly. The calibrations I get from this method allow
me to obtain a localization precision that I independently measured from
sparsely distributed dye molecules to be between 8 and 12 nm, which is
in line with published STORM results. When measuring tiny clusters of
proteins, the scatter plots of the localizations match the overall
shapes of their widefield images quite well.

However, a recent paper by Li et al.,  J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs
better than 99% uniformity to avoid introducing significant bias into
the noise measurements. Furthermore, the engineers at one of the big
camera manufacturers once told me I shouldn't even bother trying to do
the noise characterization myself since I wouldn't be able to get the
required level of uniformity for an accurate characterization. (In
fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these
questions. So far, my results seem to suggest that my calibration is
good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics
EPFL, Lausanne, Switzerland
http://kmdouglass.github.io
http://leb.epfl.ch

[Seamus Holden-2]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Kyle

Ref 17 in that paper you refer to is very interesting:

T. J. Lambert, J. C. Waters, \Assessing camera
performance for quantitative microscopy," Quantitative
Imaging in Cell Biology, Chap. 3, J. C. Waters,
T. Wittman, Eds., pp. 35–53, Elsevier, NewYork
(2013).
https://www.researchgate.net/publication/263515014_Assessing_camera_performance_for_quantitative_microscopy

The most thorough description of how to characterise a camera that I have seen.

Read noise and poisson noise I think can be treated on a per-pixel basis.  But after reading that article, my impression is that the contribution of fixed pattern noise may be hard to account for without uniform illumination. But I would have thought that contribution would be fairly small.

Best wishes
Seamus

Dr Seamus Holden
University Research Fellow

Centre for Bacterial Cell Biology
Baddiley-Clark Building
Newcastle University
Richardson Road
Newcastle upon Tyne
NE2 4AX, United Kingdom

Phone: [phone]

-----Original Message-----
From: Confocal Microscopy List [mailto:[hidden email]] On Behalf Of Kyle Douglass
Sent: 26 October 2016 08:55
To: [hidden email]
Subject: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few questions, so I am providing a "too long; didn't read" first to summarize. I'm hoping that some of you will find this topic interesting and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon response curve of an sCMOS camera? Why should the illumination pattern be so uniform when each sCMOS pixel can be thought of as an independent sensor?

I am returning to work on a minor problem that has interested me for some time. I work in localization microscopy (STORM/PALM/PAINT) and have been using sCMOS cameras for the past two years with good results. To precisely localize the single molecule emissions, we take into account the pixel-dependent noise characteristics of our sensors, incorporating the measured characteristics of the sensor into the maximum likelihood estimation of a fluorescent molecule's position. This estimation procedure was--as far as I know--first described in Huang et al., Nature Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each pixel of our cameras:

1. the offset (average ADU count under zero illumination) 2. the read noise (variance of the ADU counts under zero illumination) 3. the gain (the number of photoelectrons per ADU when the camera is in the linear response regime)

The offset and read noise are trivial to measure. To measure the gain, however, we capture a few tens of thousands of camera frames with the camera chip under uniform illumination at different light intensities and follow the mathematical operations described in the supplement to the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the gain by observing fluctuations in the pixels' ADU counts in time, not in space? I can understand why illumination non-uniformities would lead to errors when measuring the noise of a CCD chip. For CCD's, I believe that one typically treats each pixel as an independent sample of the noise from the entire chip, so one inherently assumes that the photon shot noise is uniform across the sensor. However, each pixel is only compared to itself when measuring the gain of an sCMOS sensor in the manner described above, so why does it matter that each pixel receives the same light intensity?

2. How flat is "flat enough" for this calibration procedure? With a smart phone screen set an optimum distance from the bare camera port and carefully rotated into position, I can get about 97% uniformity across the whole chip by simply by displaying gray scale images. Most of the non-uniformity appears at the corners of the chip where I think shadowing from the opening in the camera's housing is decreasing the light intensity slightly. The calibrations I get from this method allow me to obtain a localization precision that I independently measured from sparsely distributed dye molecules to be between 8 and 12 nm, which is in line with published STORM results. When measuring tiny clusters of proteins, the scatter plots of the localizations match the overall shapes of their widefield images quite well.

However, a recent paper by Li et al.,  J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better than 99% uniformity to avoid introducing significant bias into the noise measurements. Furthermore, the engineers at one of the big camera manufacturers once told me I shouldn't even bother trying to do the noise characterization myself since I wouldn't be able to get the required level of uniformity for an accurate characterization. (In fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these questions. So far, my results seem to suggest that my calibration is good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland http://kmdouglass.github.io http://leb.epfl.ch

[Gerhard Holst]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Kyle,

your question is absolutely reasonable, and in my opinion it doesn't have to be. If you google you will find the EMVA1288 standard for measuring and representing quality parameters of image sensors and cameras. There is described how the gain can be measured. If a linear camera or image sensor model can be assumed, and our experience as manufacturer of sCMOS cameras proves that, for ease of measurement a relatively homogenous illumination should be ok. The standard suggests a diffuse illumination. The homogeneous illumination is more important for getting information on the homogeneous reaction of the image sensor, means to determine the so called photo-response-non-uniformity. To determine the gain the photon transfer curve is usually measured, and this is the variance versus mean signal, and the variance is usually calculated from the difference of two images, therefore if the pixel have more or less the same brightness, that's good enough.
Alternatively you might use the Fe+55 method. Here the knowledge about the charge generation if silicon is hit be x-ray quants is used. We have done that and in case of the sCMOS image sensors in delivers the same results like the PTC curve approach.

If you take an integrating sphere with diffuse reflection and scattering, like suggested in the EMVA1288 that would be good enough in my opinion. Since the image sensors have micro lenses, directed radiation is not such a good idea.

with best regards,

Gerhard
___________________________
Dr. Gerhard Holst
Science & Research
PCO AG
Donaupark 11
93309 Kelheim, Germany
fon [phone]
fax [phone]
mob [phone]
[hidden email]
www.pco.de

-----Ursprüngliche Nachricht-----
Von: Confocal Microscopy List [mailto:[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 09:55
An: [hidden email]
Betreff: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few questions, so I am providing a "too long; didn't read" first to summarize. I'm hoping that some of you will find this topic interesting and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon response curve of an sCMOS camera? Why should the illumination pattern be so uniform when each sCMOS pixel can be thought of as an independent sensor?

I am returning to work on a minor problem that has interested me for some time. I work in localization microscopy (STORM/PALM/PAINT) and have been using sCMOS cameras for the past two years with good results. To precisely localize the single molecule emissions, we take into account the pixel-dependent noise characteristics of our sensors, incorporating the measured characteristics of the sensor into the maximum likelihood estimation of a fluorescent molecule's position. This estimation procedure was--as far as I know--first described in Huang et al., Nature Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each pixel of our cameras:

1. the offset (average ADU count under zero illumination) 2. the read noise (variance of the ADU counts under zero illumination) 3. the gain (the number of photoelectrons per ADU when the camera is in the linear response regime)

The offset and read noise are trivial to measure. To measure the gain, however, we capture a few tens of thousands of camera frames with the camera chip under uniform illumination at different light intensities and follow the mathematical operations described in the supplement to the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the gain by observing fluctuations in the pixels' ADU counts in time, not in space? I can understand why illumination non-uniformities would lead to errors when measuring the noise of a CCD chip. For CCD's, I believe that one typically treats each pixel as an independent sample of the noise from the entire chip, so one inherently assumes that the photon shot noise is uniform across the sensor. However, each pixel is only compared to itself when measuring the gain of an sCMOS sensor in the manner described above, so why does it matter that each pixel receives the same light intensity?

2. How flat is "flat enough" for this calibration procedure? With a smart phone screen set an optimum distance from the bare camera port and carefully rotated into position, I can get about 97% uniformity across the whole chip by simply by displaying gray scale images. Most of the non-uniformity appears at the corners of the chip where I think shadowing from the opening in the camera's housing is decreasing the light intensity slightly. The calibrations I get from this method allow me to obtain a localization precision that I independently measured from sparsely distributed dye molecules to be between 8 and 12 nm, which is in line with published STORM results. When measuring tiny clusters of proteins, the scatter plots of the localizations match the overall shapes of their widefield images quite well.

However, a recent paper by Li et al.,  J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better than 99% uniformity to avoid introducing significant bias into the noise measurements. Furthermore, the engineers at one of the big camera manufacturers once told me I shouldn't even bother trying to do the noise characterization myself since I wouldn't be able to get the required level of uniformity for an accurate characterization. (In fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these questions. So far, my results seem to suggest that my calibration is good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland http://kmdouglass.github.io http://leb.epfl.ch

[Zdenek Svindrych-2]

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Honestly, I hope my Orca Flash does not have microlenses!

But back to Kyle's question.

Joerg Bewersdorf's approach is strictly on pixel basis, the spatial uniformity is not critical (97% is more than sufficient), but temporal uniformity is important and < 1% intensity fluctuations are not trivial to achieve (I could not find this figure in my smartphone's specs :-). Slow drift can be easily seen in your images (if you average every 1000 images and look at the trends), but short-time fluctuations may be harder to detect (beware, there is often some sort of PWM driver for the LED backlight)!

As mentioned in the ref 17 (thanks Seamus for the link) there are many sources of "fixed pattern noise" (I don't like the term noise, it's just non-uniformity), including different light sensitivity of individual pixels or dust on the optics. Joerg's approach does not account for these effects, he just intended to bring SCMOSes on par with CCDs. There are more critical aspects of localization superresolution microscopy (such as illumination uniformity) than dust on the camera window... (but another word of caution, smudges on the camera won't be visible during the calibration, but may be visible when coupled to the microscope, because the light has fairly low etendue / cone angle / numerical aperture at the detector).

Btw, has anyone characterized the new back-illuminated SCMOS cameras (like Photometrics Prime 95B)? How do they compare with standard SCMOSes (Zyla, Orca)?

Best, zdenek

--
Zdenek Svindrych, Ph.D.
W.M. Keck Center for Cellular Imaging (PLSB 003)
University of Virginia, Charlottesville, VA
http://www.kcci.virginia.edu/
tel: [phone]
Annual FRET Workshop: http://kcci.virginia.edu/workshop-2017

---------- Původní zpráva ----------
Od: Gerhard Holst <[hidden email]>
Komu: [hidden email]
Datum: 26. 10. 2016 4:49:59
Předmět: AW: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Kyle,

your question is absolutely reasonable, and in my opinion it doesn't have to be. If you google you will find the EMVA1288 standard for measuring and representing quality parameters of image sensors and cameras. There is described how the gain can be measured. If a linear camera or image sensor model can be assumed, and our experience as manufacturer of sCMOS cameras proves that, for ease of measurement a relatively homogenous illumination should be ok. The standard suggests a diffuse illumination. The homogeneous illumination is more important for getting information on the homogeneous reaction of the image sensor, means to determine the so called photo-response-non-uniformity. To determine the gain the photon transfer curve is usually measured, and this is the variance versus mean signal, and the variance is usually calculated from the difference of two images, therefore if the pixel have more or less the same brightness, that's good enough.
Alternatively you might use the Fe+55 method. Here the knowledge about the charge generation if silicon is hit be x-ray quants is used. We have done that and in case of the sCMOS image sensors in delivers the same results like the PTC curve approach.

If you take an integrating sphere with diffuse reflection and scattering, like suggested in the EMVA1288 that would be good enough in my opinion. Since the image sensors have micro lenses, directed radiation is not such a good idea.

with best regards,

Gerhard
___________________________
Dr. Gerhard Holst
Science & Research
PCO AG
Donaupark 11
93309 Kelheim, Germany
fon [phone]
fax [phone]
mob [phone]
[hidden email]
www.pco.de

-----Ursprüngliche Nachricht-----
Von: Confocal Microscopy List [mailto:[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 09:55
An: [hidden email]
Betreff: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few questions, so I am providing a "too long; didn't read" first to summarize. I'm hoping that some of you will find this topic interesting and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon response curve of an sCMOS camera? Why should the illumination pattern be so uniform when each sCMOS pixel can be thought of as an independent sensor?

I am returning to work on a minor problem that has interested me for some time. I work in localization microscopy (STORM/PALM/PAINT) and have been using sCMOS cameras for the past two years with good results. To precisely localize the single molecule emissions, we take into account the pixel-dependent noise characteristics of our sensors, incorporating the measured characteristics of the sensor into the maximum likelihood estimation of a fluorescent molecule's position. This estimation procedure was--as far as I know--first described in Huang et al., Nature Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each pixel of our cameras:

1. the offset (average ADU count under zero illumination) 2. the read noise (variance of the ADU counts under zero illumination) 3. the gain (the number of photoelectrons per ADU when the camera is in the linear response regime)

The offset and read noise are trivial to measure. To measure the gain, however, we capture a few tens of thousands of camera frames with the camera chip under uniform illumination at different light intensities and follow the mathematical operations described in the supplement to the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the gain by observing fluctuations in the pixels' ADU counts in time, not in space? I can understand why illumination non-uniformities would lead to errors when measuring the noise of a CCD chip. For CCD's, I believe that one typically treats each pixel as an independent sample of the noise from the entire chip, so one inherently assumes that the photon shot noise is uniform across the sensor. However, each pixel is only compared to itself when measuring the gain of an sCMOS sensor in the manner described above, so why does it matter that each pixel receives the same light intensity?

2. How flat is "flat enough" for this calibration procedure? With a smart phone screen set an optimum distance from the bare camera port and carefully rotated into position, I can get about 97% uniformity across the whole chip by simply by displaying gray scale images. Most of the non-uniformity appears at the corners of the chip where I think shadowing from the opening in the camera's housing is decreasing the light intensity slightly. The calibrations I get from this method allow me to obtain a localization precision that I independently measured from sparsely distributed dye molecules to be between 8 and 12 nm, which is in line with published STORM results. When measuring tiny clusters of proteins, the scatter plots of the localizations match the overall shapes of their widefield images quite well.

However, a recent paper by Li et al., J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better than 99% uniformity to avoid introducing significant bias into the noise measurements. Furthermore, the engineers at one of the big camera manufacturers once told me I shouldn't even bother trying to do the noise characterization myself since I wouldn't be able to get the required level of uniformity for an accurate characterization. (In fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these questions. So far, my results seem to suggest that my calibration is good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland http://kmdouglass.github.io http://leb.epfl.ch

[Gerhard Holst]

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Dear Zdenek,

 

„Honestly, I hope my Orca Flash does not have microlenses!”

I am afraid I have to disappoint you, all the front illuminated sCMOS image sensors have to have microlenses, because otherwise the quantum efficiency would be a lot lower. You have to compensate for the space all the nice transistors use in the pixel area. Since we did co-develop the chip and we use the chip in the Orca as well, we know.

 

“As mentioned in the ref 17 (thanks Seamus for the link) there are many sources of "fixed pattern noise" (I don't like the term noise, it's just non-uniformity), including different light sensitivity of individual pixels or dust on the optics.”

That’s why we from the camera side use temporal and spatial noise as terms, therefore the dark signal non uniformity and the photo response non-uniformity are expressions of spatial noise, while the readout noise and variance is temporal noise. But based on the experiences with the CCDs to shorten the measuring process sometimes the average an all pixels of an image is used as well to access the temporal noise (by assuming that the spatial noise is negligible).

 

“Btw, has anyone characterized the new back-illuminated SCMOS cameras (like Photometrics Prime 95B)? How do they compare with standard SCMOSes (Zyla, Orca)?”

May I add to your standard enumeration the pco.edge family? It would be interesting to see comparisons, but if you do so, it would be nice not just to exchange the camera and be surprised that the signal is so large, because in case just the camera is exchanged, now the light formerly falling on 4 pixels (pitch 6.5um) now falls an on 1 pixel (pitch 11um), which means that a single pixel now sees nearly 4 times the signal, just to consider this if the camera is compared. Otheriwse I would expect from the image sensor a comparable result.

 

with best regards,

 

Gerhard

___________________________

Dr. Gerhard Holst

Science & Research

PCO AG

Donaupark 11

93309 Kelheim, Germany

fon [phone]

fax [phone]

mob [phone]

[hidden email]

www.pco.de

 

Von: Confocal Microscopy List [mailto:[hidden email]] Im Auftrag von [hidden email]
Gesendet: Mittwoch, 26. Oktober 2016 15:37
An: [hidden email]
Betreff: Re: AW: Measuring noise characteristics of sCMOS cameras

 

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Honestly, I hope my Orca Flash does not have microlenses!

But back to Kyle's question.

Joerg Bewersdorf's approach is strictly on pixel basis, the spatial uniformity is not critical (97% is more than sufficient), but temporal uniformity is important and < 1% intensity fluctuations are not trivial to achieve (I could not find this figure in my smartphone's specs :-). Slow drift can be easily seen in your images (if you average every 1000 images and look at the trends), but short-time fluctuations may be harder to detect (beware, there is often some sort of PWM driver for the LED backlight)!

As mentioned in the ref 17 (thanks Seamus for the link) there are many sources of "fixed pattern noise" (I don't like the term noise, it's just non-uniformity), including different light sensitivity of individual pixels or dust on the optics. Joerg's approach does not account for these effects, he just intended to bring SCMOSes on par with CCDs. There are more critical aspects of localization superresolution microscopy (such as illumination uniformity) than dust on the camera window... (but another word of caution, smudges on the camera won't be visible during the calibration, but may be visible when coupled to the microscope, because the light has fairly low etendue / cone angle / numerical aperture at the detector).

Btw, has anyone characterized the new back-illuminated SCMOS cameras (like Photometrics Prime 95B)? How do they compare with standard SCMOSes (Zyla, Orca)?

Best, zdenek

--
Zdenek Svindrych, Ph.D.
W.M. Keck Center for Cellular Imaging (PLSB 003)
University of Virginia, Charlottesville, VA
http://www.kcci.virginia.edu/
tel: [phone]
Annual FRET Workshop: http://kcci.virginia.edu/workshop-2017

---------- Původní zpráva ----------
Od: Gerhard Holst <[hidden email]>
Komu: [hidden email]
Datum: 26. 10. 2016 4:49:59
Předmět: AW: Measuring noise characteristics of sCMOS cameras

 

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Kyle,

your question is absolutely reasonable, and in my opinion it doesn't have to be. If you google you will find the EMVA1288 standard for measuring and representing quality parameters of image sensors and cameras. There is described how the gain can be measured. If a linear camera or image sensor model can be assumed, and our experience as manufacturer of sCMOS cameras proves that, for ease of measurement a relatively homogenous illumination should be ok. The standard suggests a diffuse illumination. The homogeneous illumination is more important for getting information on the homogeneous reaction of the image sensor, means to determine the so called photo-response-non-uniformity. To determine the gain the photon transfer curve is usually measured, and this is the variance versus mean signal, and the variance is usually calculated from the difference of two images, therefore if the pixel have more or less the same brightness, that's good enough.
Alternatively you might use the Fe+55 method. Here the knowledge about the charge generation if silicon is hit be x-ray quants is used. We have done that and in case of the sCMOS image sensors in delivers the same results like the PTC curve approach.

If you take an integrating sphere with diffuse reflection and scattering, like suggested in the EMVA1288 that would be good enough in my opinion. Since the image sensors have micro lenses, directed radiation is not such a good idea.

with best regards,

Gerhard
___________________________
Dr. Gerhard Holst
Science & Research
PCO AG
Donaupark 11
93309 Kelheim, Germany
fon [phone]
fax [phone]
mob [phone]
[hidden email]
www.pco.de

-----Ursprüngliche Nachricht-----
Von: Confocal Microscopy List [[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 09:55
An: [hidden email]
Betreff: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few questions, so I am providing a "too long; didn't read" first to summarize. I'm hoping that some of you will find this topic interesting and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon response curve of an sCMOS camera? Why should the illumination pattern be so uniform when each sCMOS pixel can be thought of as an independent sensor?

I am returning to work on a minor problem that has interested me for some time. I work in localization microscopy (STORM/PALM/PAINT) and have been using sCMOS cameras for the past two years with good results. To precisely localize the single molecule emissions, we take into account the pixel-dependent noise characteristics of our sensors, incorporating the measured characteristics of the sensor into the maximum likelihood estimation of a fluorescent molecule's position. This estimation procedure was--as far as I know--first described in Huang et al., Nature Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each pixel of our cameras:

1. the offset (average ADU count under zero illumination) 2. the read noise (variance of the ADU counts under zero illumination) 3. the gain (the number of photoelectrons per ADU when the camera is in the linear response regime)

The offset and read noise are trivial to measure. To measure the gain, however, we capture a few tens of thousands of camera frames with the camera chip under uniform illumination at different light intensities and follow the mathematical operations described in the supplement to the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the gain by observing fluctuations in the pixels' ADU counts in time, not in space? I can understand why illumination non-uniformities would lead to errors when measuring the noise of a CCD chip. For CCD's, I believe that one typically treats each pixel as an independent sample of the noise from the entire chip, so one inherently assumes that the photon shot noise is uniform across the sensor. However, each pixel is only compared to itself when measuring the gain of an sCMOS sensor in the manner described above, so why does it matter that each pixel receives the same light intensity?

2. How flat is "flat enough" for this calibration procedure? With a smart phone screen set an optimum distance from the bare camera port and carefully rotated into position, I can get about 97% uniformity across the whole chip by simply by displaying gray scale images. Most of the non-uniformity appears at the corners of the chip where I think shadowing from the opening in the camera's housing is decreasing the light intensity slightly. The calibrations I get from this method allow me to obtain a localization precision that I independently measured from sparsely distributed dye molecules to be between 8 and 12 nm, which is in line with published STORM results. When measuring tiny clusters of proteins, the scatter plots of the localizations match the overall shapes of their widefield images quite well.

However, a recent paper by Li et al., J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better than 99% uniformity to avoid introducing significant bias into the noise measurements. Furthermore, the engineers at one of the big camera manufacturers once told me I shouldn't even bother trying to do the noise characterization myself since I wouldn't be able to get the required level of uniformity for an accurate characterization. (In fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these questions. So far, my results seem to suggest that my calibration is good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland http://kmdouglass.github.io http://leb.epfl.ch

[Kyle Michael Douglass]

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Thanks to everyone who has thus far replied to my original e-mail. I realize after doing some more reading that I was essentially conflating two different "noise" sources into one: fixed pattern noise, for which one certainly wants a uniform illumination to measure, and the per pixel gain, which is what is measured in the Huang/Bewersdorf paper.

My next question is: what kind of light source is going to provide <1% temporal intensity fluctuations? A CW LED pumped into an integrating sphere?

Zdenek: Thanks for the heads up about the PWM driver in the smart phone LED backlight. I wouldn't be surprised if this were true. Interestingly, I can see aliasing in a live stream from the camera when using a computer monitor as illumination (60 Hz refresh rate) but I can not see it with the smart phone.

Gerhard: The EMVA 1288 document is absolutely wonderful. Thanks for this.

Seamus: I actually have the book next to me on my desk. Thanks for the reminder about the chapter :)

Cheers,
Kyle

On 10/26/2016 03:37 PM, [hidden email] wrote:

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Honestly, I hope my Orca Flash does not have microlenses!

But back to Kyle's question.

Joerg Bewersdorf's approach is strictly on pixel basis, the spatial uniformity is not critical (97% is more than sufficient), but temporal uniformity is important and < 1% intensity fluctuations are not trivial to achieve (I could not find this figure in my smartphone's specs :-). Slow drift can be easily seen in your images (if you average every 1000 images and look at the trends), but short-time fluctuations may be harder to detect (beware, there is often some sort of PWM driver for the LED backlight)!

As mentioned in the ref 17 (thanks Seamus for the link) there are many sources of "fixed pattern noise" (I don't like the term noise, it's just non-uniformity), including different light sensitivity of individual pixels or dust on the optics. Joerg's approach does not account for these effects, he just intended to bring SCMOSes on par with CCDs. There are more critical aspects of localization superresolution microscopy (such as illumination uniformity) than dust on the camera window... (but another word of caution, smudges on the camera won't be visible during the calibration, but may be visible when coupled to the microscope, because the light has fairly low etendue / cone angle / numerical aperture at the detector).

Btw, has anyone characterized the new back-illuminated SCMOS cameras (like Photometrics Prime 95B)? How do they compare with standard SCMOSes (Zyla, Orca)?

Best, zdenek

--
Zdenek Svindrych, Ph.D.
W.M. Keck Center for Cellular Imaging (PLSB 003)
University of Virginia, Charlottesville, VA
http://www.kcci.virginia.edu/
tel: [phone]
Annual FRET Workshop: http://kcci.virginia.edu/workshop-2017

---------- Původní zpráva ----------
Od: Gerhard Holst [hidden email]
Komu: [hidden email]
Datum: 26. 10. 2016 4:49:59
Předmět: AW: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Kyle,

your question is absolutely reasonable, and in my opinion it doesn't have to be. If you google you will find the EMVA1288 standard for measuring and representing quality parameters of image sensors and cameras. There is described how the gain can be measured. If a linear camera or image sensor model can be assumed, and our experience as manufacturer of sCMOS cameras proves that, for ease of measurement a relatively homogenous illumination should be ok. The standard suggests a diffuse illumination. The homogeneous illumination is more important for getting information on the homogeneous reaction of the image sensor, means to determine the so called photo-response-non-uniformity. To determine the gain the photon transfer curve is usually measured, and this is the variance versus mean signal, and the variance is usually calculated from the difference of two images, therefore if the pixel have more or less the same brightness, that's good enough.
Alternatively you might use the Fe+55 method. Here the knowledge about the charge generation if silicon is hit be x-ray quants is used. We have done that and in case of the sCMOS image sensors in delivers the same results like the PTC curve approach.

If you take an integrating sphere with diffuse reflection and scattering, like suggested in the EMVA1288 that would be good enough in my opinion. Since the image sensors have micro lenses, directed radiation is not such a good idea.

with best regards,

Gerhard
___________________________
Dr. Gerhard Holst
Science & Research
PCO AG
Donaupark 11
93309 Kelheim, Germany
fon [phone]
fax [phone]
mob [phone]
[hidden email]
www.pco.de

-----Ursprüngliche Nachricht-----
Von: Confocal Microscopy List [[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 09:55
An: [hidden email]
Betreff: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few questions, so I am providing a "too long; didn't read" first to summarize. I'm hoping that some of you will find this topic interesting and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon response curve of an sCMOS camera? Why should the illumination pattern be so uniform when each sCMOS pixel can be thought of as an independent sensor?

I am returning to work on a minor problem that has interested me for some time. I work in localization microscopy (STORM/PALM/PAINT) and have been using sCMOS cameras for the past two years with good results. To precisely localize the single molecule emissions, we take into account the pixel-dependent noise characteristics of our sensors, incorporating the measured characteristics of the sensor into the maximum likelihood estimation of a fluorescent molecule's position. This estimation procedure was--as far as I know--first described in Huang et al., Nature Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each pixel of our cameras:

1. the offset (average ADU count under zero illumination) 2. the read noise (variance of the ADU counts under zero illumination) 3. the gain (the number of photoelectrons per ADU when the camera is in the linear response regime)

The offset and read noise are trivial to measure. To measure the gain, however, we capture a few tens of thousands of camera frames with the camera chip under uniform illumination at different light intensities and follow the mathematical operations described in the supplement to the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the gain by observing fluctuations in the pixels' ADU counts in time, not in space? I can understand why illumination non-uniformities would lead to errors when measuring the noise of a CCD chip. For CCD's, I believe that one typically treats each pixel as an independent sample of the noise from the entire chip, so one inherently assumes that the photon shot noise is uniform across the sensor. However, each pixel is only compared to itself when measuring the gain of an sCMOS sensor in the manner described above, so why does it matter that each pixel receives the same light intensity?

2. How flat is "flat enough" for this calibration procedure? With a smart phone screen set an optimum distance from the bare camera port and carefully rotated into position, I can get about 97% uniformity across the whole chip by simply by displaying gray scale images. Most of the non-uniformity appears at the corners of the chip where I think shadowing from the opening in the camera's housing is decreasing the light intensity slightly. The calibrations I get from this method allow me to obtain a localization precision that I independently measured from sparsely distributed dye molecules to be between 8 and 12 nm, which is in line with published STORM results. When measuring tiny clusters of proteins, the scatter plots of the localizations match the overall shapes of their widefield images quite well.

However, a recent paper by Li et al., J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better than 99% uniformity to avoid introducing significant bias into the noise measurements. Furthermore, the engineers at one of the big camera manufacturers once told me I shouldn't even bother trying to do the noise characterization myself since I wouldn't be able to get the required level of uniformity for an accurate characterization. (In fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these questions. So far, my results seem to suggest that my calibration is good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland http://kmdouglass.github.io http://leb.epfl.ch

[Gerhard Holst]

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Hi Kyle,

 

may I ask 1% of what? Further, including poisson noise? I mean the fluctuations go with the square root of the number of photons.

 

with best regards,

 

Gerhard

___________________________

Dr. Gerhard Holst

Science & Research

PCO AG

Donaupark 11

93309 Kelheim, Germany

fon [phone]

fax [phone]

mob [phone]

[hidden email]

www.pco.de

 

Von: Confocal Microscopy List [mailto:[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 17:01
An: [hidden email]
Betreff: Re: AW: Measuring noise characteristics of sCMOS cameras

 

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Thanks to everyone who has thus far replied to my original e-mail. I realize after doing some more reading that I was essentially conflating two different "noise" sources into one: fixed pattern noise, for which one certainly wants a uniform illumination to measure, and the per pixel gain, which is what is measured in the Huang/Bewersdorf paper.

My next question is: what kind of light source is going to provide <1% temporal intensity fluctuations? A CW LED pumped into an integrating sphere?

Zdenek: Thanks for the heads up about the PWM driver in the smart phone LED backlight. I wouldn't be surprised if this were true. Interestingly, I can see aliasing in a live stream from the camera when using a computer monitor as illumination (60 Hz refresh rate) but I can not see it with the smart phone.

Gerhard: The EMVA 1288 document is absolutely wonderful. Thanks for this.

Seamus: I actually have the book next to me on my desk. Thanks for the reminder about the chapter :)

Cheers,
Kyle

On 10/26/2016 03:37 PM, [hidden email] wrote:

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Honestly, I hope my Orca Flash does not have microlenses!

But back to Kyle's question.

Joerg Bewersdorf's approach is strictly on pixel basis, the spatial uniformity is not critical (97% is more than sufficient), but temporal uniformity is important and < 1% intensity fluctuations are not trivial to achieve (I could not find this figure in my smartphone's specs :-). Slow drift can be easily seen in your images (if you average every 1000 images and look at the trends), but short-time fluctuations may be harder to detect (beware, there is often some sort of PWM driver for the LED backlight)!

As mentioned in the ref 17 (thanks Seamus for the link) there are many sources of "fixed pattern noise" (I don't like the term noise, it's just non-uniformity), including different light sensitivity of individual pixels or dust on the optics. Joerg's approach does not account for these effects, he just intended to bring SCMOSes on par with CCDs. There are more critical aspects of localization superresolution microscopy (such as illumination uniformity) than dust on the camera window... (but another word of caution, smudges on the camera won't be visible during the calibration, but may be visible when coupled to the microscope, because the light has fairly low etendue / cone angle / numerical aperture at the detector).

Btw, has anyone characterized the new back-illuminated SCMOS cameras (like Photometrics Prime 95B)? How do they compare with standard SCMOSes (Zyla, Orca)?

Best, zdenek

--
Zdenek Svindrych, Ph.D.
W.M. Keck Center for Cellular Imaging (PLSB 003)
University of Virginia, Charlottesville, VA
http://www.kcci.virginia.edu/
tel: [phone]
Annual FRET Workshop: http://kcci.virginia.edu/workshop-2017

---------- Původní zpráva ----------
Od: Gerhard Holst [hidden email]
Komu: [hidden email]
Datum: 26. 10. 2016 4:49:59
Předmět: AW: Measuring noise characteristics of sCMOS cameras

 

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Kyle,

your question is absolutely reasonable, and in my opinion it doesn't have to be. If you google you will find the EMVA1288 standard for measuring and representing quality parameters of image sensors and cameras. There is described how the gain can be measured. If a linear camera or image sensor model can be assumed, and our experience as manufacturer of sCMOS cameras proves that, for ease of measurement a relatively homogenous illumination should be ok. The standard suggests a diffuse illumination. The homogeneous illumination is more important for getting information on the homogeneous reaction of the image sensor, means to determine the so called photo-response-non-uniformity. To determine the gain the photon transfer curve is usually measured, and this is the variance versus mean signal, and the variance is usually calculated from the difference of two images, therefore if the pixel have more or less the same brightness, that's good enough.
Alternatively you might use the Fe+55 method. Here the knowledge about the charge generation if silicon is hit be x-ray quants is used. We have done that and in case of the sCMOS image sensors in delivers the same results like the PTC curve approach.

If you take an integrating sphere with diffuse reflection and scattering, like suggested in the EMVA1288 that would be good enough in my opinion. Since the image sensors have micro lenses, directed radiation is not such a good idea.

with best regards,

Gerhard
___________________________
Dr. Gerhard Holst
Science & Research
PCO AG
Donaupark 11
93309 Kelheim, Germany
fon [phone]
fax [phone]
mob [phone]
[hidden email]
www.pco.de

-----Ursprüngliche Nachricht-----
Von: Confocal Microscopy List [[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 09:55
An: [hidden email]
Betreff: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few questions, so I am providing a "too long; didn't read" first to summarize. I'm hoping that some of you will find this topic interesting and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon response curve of an sCMOS camera? Why should the illumination pattern be so uniform when each sCMOS pixel can be thought of as an independent sensor?

I am returning to work on a minor problem that has interested me for some time. I work in localization microscopy (STORM/PALM/PAINT) and have been using sCMOS cameras for the past two years with good results. To precisely localize the single molecule emissions, we take into account the pixel-dependent noise characteristics of our sensors, incorporating the measured characteristics of the sensor into the maximum likelihood estimation of a fluorescent molecule's position. This estimation procedure was--as far as I know--first described in Huang et al., Nature Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each pixel of our cameras:

1. the offset (average ADU count under zero illumination) 2. the read noise (variance of the ADU counts under zero illumination) 3. the gain (the number of photoelectrons per ADU when the camera is in the linear response regime)

The offset and read noise are trivial to measure. To measure the gain, however, we capture a few tens of thousands of camera frames with the camera chip under uniform illumination at different light intensities and follow the mathematical operations described in the supplement to the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the gain by observing fluctuations in the pixels' ADU counts in time, not in space? I can understand why illumination non-uniformities would lead to errors when measuring the noise of a CCD chip. For CCD's, I believe that one typically treats each pixel as an independent sample of the noise from the entire chip, so one inherently assumes that the photon shot noise is uniform across the sensor. However, each pixel is only compared to itself when measuring the gain of an sCMOS sensor in the manner described above, so why does it matter that each pixel receives the same light intensity?

2. How flat is "flat enough" for this calibration procedure? With a smart phone screen set an optimum distance from the bare camera port and carefully rotated into position, I can get about 97% uniformity across the whole chip by simply by displaying gray scale images. Most of the non-uniformity appears at the corners of the chip where I think shadowing from the opening in the camera's housing is decreasing the light intensity slightly. The calibrations I get from this method allow me to obtain a localization precision that I independently measured from sparsely distributed dye molecules to be between 8 and 12 nm, which is in line with published STORM results. When measuring tiny clusters of proteins, the scatter plots of the localizations match the overall shapes of their widefield images quite well.

However, a recent paper by Li et al., J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better than 99% uniformity to avoid introducing significant bias into the noise measurements. Furthermore, the engineers at one of the big camera manufacturers once told me I shouldn't even bother trying to do the noise characterization myself since I wouldn't be able to get the required level of uniformity for an accurate characterization. (In fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these questions. So far, my results seem to suggest that my calibration is good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland http://kmdouglass.github.io http://leb.epfl.ch

-- 

Kyle M. Douglass, PhD

Post-doctoral researcher

The Laboratory of Experimental Biophysics

EPFL, Lausanne, Switzerland

http://kmdouglass.github.io

http://leb.epfl.ch

[Kyle Michael Douglass]

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Hi Gerhard,

Thanks again for the response.

On 10/26/2016 05:12 PM, Gerhard Holst wrote:

may I ask 1% of what? Further, including poisson noise? I mean the fluctuations go with the square root of the number of photons.

 

I was referring to the number that Zdenek had suggested for temporal stability of the light source. I understand that there will be photon shot noise in any light source; how I interpreted Zdenek's comment was that the **mean** irradiance produced by the source on the detector should vary by less than 1% over the time of the measurement. In other words, the light source should be wide sense stationary.


To be honest, I don't have a good feel for what kinds of sources are going to be this stable, so I am hoping that someone could suggest one.

Cheers,
Kyle

with best regards,

 

Gerhard

___________________________

Dr. Gerhard Holst

Science & Research

PCO AG

Donaupark 11

93309 Kelheim, Germany

fon [phone]

fax [phone]

mob [phone]

[hidden email]

www.pco.de

 

Von: Confocal Microscopy List [[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 17:01
An: [hidden email]
Betreff: Re: AW: Measuring noise characteristics of sCMOS cameras

 

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Thanks to everyone who has thus far replied to my original e-mail. I realize after doing some more reading that I was essentially conflating two different "noise" sources into one: fixed pattern noise, for which one certainly wants a uniform illumination to measure, and the per pixel gain, which is what is measured in the Huang/Bewersdorf paper.

My next question is: what kind of light source is going to provide <1% temporal intensity fluctuations? A CW LED pumped into an integrating sphere?

Zdenek: Thanks for the heads up about the PWM driver in the smart phone LED backlight. I wouldn't be surprised if this were true. Interestingly, I can see aliasing in a live stream from the camera when using a computer monitor as illumination (60 Hz refresh rate) but I can not see it with the smart phone.

Gerhard: The EMVA 1288 document is absolutely wonderful. Thanks for this.

Seamus: I actually have the book next to me on my desk. Thanks for the reminder about the chapter :)

Cheers,
Kyle

On 10/26/2016 03:37 PM, [hidden email] wrote:

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Honestly, I hope my Orca Flash does not have microlenses!

But back to Kyle's question.

Joerg Bewersdorf's approach is strictly on pixel basis, the spatial uniformity is not critical (97% is more than sufficient), but temporal uniformity is important and < 1% intensity fluctuations are not trivial to achieve (I could not find this figure in my smartphone's specs :-). Slow drift can be easily seen in your images (if you average every 1000 images and look at the trends), but short-time fluctuations may be harder to detect (beware, there is often some sort of PWM driver for the LED backlight)!

As mentioned in the ref 17 (thanks Seamus for the link) there are many sources of "fixed pattern noise" (I don't like the term noise, it's just non-uniformity), including different light sensitivity of individual pixels or dust on the optics. Joerg's approach does not account for these effects, he just intended to bring SCMOSes on par with CCDs. There are more critical aspects of localization superresolution microscopy (such as illumination uniformity) than dust on the camera window... (but another word of caution, smudges on the camera won't be visible during the calibration, but may be visible when coupled to the microscope, because the light has fairly low etendue / cone angle / numerical aperture at the detector).

Btw, has anyone characterized the new back-illuminated SCMOS cameras (like Photometrics Prime 95B)? How do they compare with standard SCMOSes (Zyla, Orca)?

Best, zdenek

--
Zdenek Svindrych, Ph.D.
W.M. Keck Center for Cellular Imaging (PLSB 003)
University of Virginia, Charlottesville, VA
http://www.kcci.virginia.edu/
tel: [phone]
Annual FRET Workshop: http://kcci.virginia.edu/workshop-2017

---------- Původní zpráva ----------
Od: Gerhard Holst [hidden email]
Komu: [hidden email]
Datum: 26. 10. 2016 4:49:59
Předmět: AW: Measuring noise characteristics of sCMOS cameras

 

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Kyle,

your question is absolutely reasonable, and in my opinion it doesn't have to be. If you google you will find the EMVA1288 standard for measuring and representing quality parameters of image sensors and cameras. There is described how the gain can be measured. If a linear camera or image sensor model can be assumed, and our experience as manufacturer of sCMOS cameras proves that, for ease of measurement a relatively homogenous illumination should be ok. The standard suggests a diffuse illumination. The homogeneous illumination is more important for getting information on the homogeneous reaction of the image sensor, means to determine the so called photo-response-non-uniformity. To determine the gain the photon transfer curve is usually measured, and this is the variance versus mean signal, and the variance is usually calculated from the difference of two images, therefore if the pixel have more or less the same brightness, that's good enough.
Alternatively you might use the Fe+55 method. Here the knowledge about the charge generation if silicon is hit be x-ray quants is used. We have done that and in case of the sCMOS image sensors in delivers the same results like the PTC curve approach.

If you take an integrating sphere with diffuse reflection and scattering, like suggested in the EMVA1288 that would be good enough in my opinion. Since the image sensors have micro lenses, directed radiation is not such a good idea.

with best regards,

Gerhard
___________________________
Dr. Gerhard Holst
Science & Research
PCO AG
Donaupark 11
93309 Kelheim, Germany
fon [phone]
fax [phone]
mob [phone]
[hidden email]
www.pco.de

-----Ursprüngliche Nachricht-----
Von: Confocal Microscopy List [[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 09:55
An: [hidden email]
Betreff: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few questions, so I am providing a "too long; didn't read" first to summarize. I'm hoping that some of you will find this topic interesting and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon response curve of an sCMOS camera? Why should the illumination pattern be so uniform when each sCMOS pixel can be thought of as an independent sensor?

I am returning to work on a minor problem that has interested me for some time. I work in localization microscopy (STORM/PALM/PAINT) and have been using sCMOS cameras for the past two years with good results. To precisely localize the single molecule emissions, we take into account the pixel-dependent noise characteristics of our sensors, incorporating the measured characteristics of the sensor into the maximum likelihood estimation of a fluorescent molecule's position. This estimation procedure was--as far as I know--first described in Huang et al., Nature Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each pixel of our cameras:

1. the offset (average ADU count under zero illumination) 2. the read noise (variance of the ADU counts under zero illumination) 3. the gain (the number of photoelectrons per ADU when the camera is in the linear response regime)

The offset and read noise are trivial to measure. To measure the gain, however, we capture a few tens of thousands of camera frames with the camera chip under uniform illumination at different light intensities and follow the mathematical operations described in the supplement to the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the gain by observing fluctuations in the pixels' ADU counts in time, not in space? I can understand why illumination non-uniformities would lead to errors when measuring the noise of a CCD chip. For CCD's, I believe that one typically treats each pixel as an independent sample of the noise from the entire chip, so one inherently assumes that the photon shot noise is uniform across the sensor. However, each pixel is only compared to itself when measuring the gain of an sCMOS sensor in the manner described above, so why does it matter that each pixel receives the same light intensity?

2. How flat is "flat enough" for this calibration procedure? With a smart phone screen set an optimum distance from the bare camera port and carefully rotated into position, I can get about 97% uniformity across the whole chip by simply by displaying gray scale images. Most of the non-uniformity appears at the corners of the chip where I think shadowing from the opening in the camera's housing is decreasing the light intensity slightly. The calibrations I get from this method allow me to obtain a localization precision that I independently measured from sparsely distributed dye molecules to be between 8 and 12 nm, which is in line with published STORM results. When measuring tiny clusters of proteins, the scatter plots of the localizations match the overall shapes of their widefield images quite well.

However, a recent paper by Li et al., J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better than 99% uniformity to avoid introducing significant bias into the noise measurements. Furthermore, the engineers at one of the big camera manufacturers once told me I shouldn't even bother trying to do the noise characterization myself since I wouldn't be able to get the required level of uniformity for an accurate characterization. (In fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these questions. So far, my results seem to suggest that my calibration is good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland http://kmdouglass.github.io http://leb.epfl.ch

-- 

Kyle M. Douglass, PhD

Post-doctoral researcher

The Laboratory of Experimental Biophysics

EPFL, Lausanne, Switzerland

http://kmdouglass.github.io

http://leb.epfl.ch

[Zdenek Svindrych-2]

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Good point,

as a matter of fact, even in standard photon transfer curve measurements, the fixed pattern noise becomes apparent at several thousand photons per pixel. We're talking about couple hundreds here.
But it's hard to tell how large frame-to-frame and longer term fluctuations are still OK given the large number of frames analyzed (tens of thousands) and the non-trivial method of gain determination in the Joerg's paper (least square estimation). On the other hand, given the photon noise standard deviation is 5% at 400 photons, 1% (max) illumination intensity fluctuation does not sound like a gross overkill...

Best, zdenek

Btw, while my PC monitor LED backlight shows nice PWM at 1.2 kHz, my smartphones produced steady illumination (no fluctuations up to 50 kHz).

---------- Původní zpráva ----------
Od: Gerhard Holst <[hidden email]>
Komu: [hidden email]
Datum: 26. 10. 2016 11:14:42
Předmět: AW: AW: Measuring noise characteristics of sCMOS cameras

Hi Kyle,

 

may I ask 1% of what? Further, including poisson noise? I mean the fluctuations go with the square root of the number of photons.

 

with best regards,

 

Gerhard

___________________________

Dr. Gerhard Holst

Science & Research

PCO AG

Donaupark 11

93309 Kelheim, Germany

fon [phone]

fax [phone]

mob [phone]

[hidden email]

www.pco.de

 

Von: Confocal Microscopy List [mailto:[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 17:01
An: [hidden email]
Betreff: Re: AW: Measuring noise characteristics of sCMOS cameras

 

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Thanks to everyone who has thus far replied to my original e-mail. I realize after doing some more reading that I was essentially conflating two different "noise" sources into one: fixed pattern noise, for which one certainly wants a uniform illumination to measure, and the per pixel gain, which is what is measured in the Huang/Bewersdorf paper.

My next question is: what kind of light source is going to provide <1% temporal intensity fluctuations? A CW LED pumped into an integrating sphere?

Zdenek: Thanks for the heads up about the PWM driver in the smart phone LED backlight. I wouldn't be surprised if this were true. Interestingly, I can see aliasing in a live stream from the camera when using a computer monitor as illumination (60 Hz refresh rate) but I can not see it with the smart phone.

Gerhard: The EMVA 1288 document is absolutely wonderful. Thanks for this.

Seamus: I actually have the book next to me on my desk. Thanks for the reminder about the chapter :)

Cheers,
Kyle

On 10/26/2016 03:37 PM, [hidden email] wrote:

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Honestly, I hope my Orca Flash does not have microlenses!

But back to Kyle's question.

Joerg Bewersdorf's approach is strictly on pixel basis, the spatial uniformity is not critical (97% is more than sufficient), but temporal uniformity is important and < 1% intensity fluctuations are not trivial to achieve (I could not find this figure in my smartphone's specs :-). Slow drift can be easily seen in your images (if you average every 1000 images and look at the trends), but short-time fluctuations may be harder to detect (beware, there is often some sort of PWM driver for the LED backlight)!

As mentioned in the ref 17 (thanks Seamus for the link) there are many sources of "fixed pattern noise" (I don't like the term noise, it's just non-uniformity), including different light sensitivity of individual pixels or dust on the optics. Joerg's approach does not account for these effects, he just intended to bring SCMOSes on par with CCDs. There are more critical aspects of localization superresolution microscopy (such as illumination uniformity) than dust on the camera window... (but another word of caution, smudges on the camera won't be visible during the calibration, but may be visible when coupled to the microscope, because the light has fairly low etendue / cone angle / numerical aperture at the detector).

Btw, has anyone characterized the new back-illuminated SCMOS cameras (like Photometrics Prime 95B)? How do they compare with standard SCMOSes (Zyla, Orca)?

Best, zdenek

--
Zdenek Svindrych, Ph.D.
W.M. Keck Center for Cellular Imaging (PLSB 003)
University of Virginia, Charlottesville, VA
http://www.kcci.virginia.edu/
tel: [phone]
Annual FRET Workshop: http://kcci.virginia.edu/workshop-2017

---------- Původní zpráva ----------
Od: Gerhard Holst [hidden email]
Komu: [hidden email]
Datum: 26. 10. 2016 4:49:59
Předmět: AW: Measuring noise characteristics of sCMOS cameras

 

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Kyle,

your question is absolutely reasonable, and in my opinion it doesn't have to be. If you google you will find the EMVA1288 standard for measuring and representing quality parameters of image sensors and cameras. There is described how the gain can be measured. If a linear camera or image sensor model can be assumed, and our experience as manufacturer of sCMOS cameras proves that, for ease of measurement a relatively homogenous illumination should be ok. The standard suggests a diffuse illumination. The homogeneous illumination is more important for getting information on the homogeneous reaction of the image sensor, means to determine the so called photo-response-non-uniformity. To determine the gain the photon transfer curve is usually measured, and this is the variance versus mean signal, and the variance is usually calculated from the difference of two images, therefore if the pixel have more or less the same brightness, that's good enough.
Alternatively you might use the Fe+55 method. Here the knowledge about the charge generation if silicon is hit be x-ray quants is used. We have done that and in case of the sCMOS image sensors in delivers the same results like the PTC curve approach.

If you take an integrating sphere with diffuse reflection and scattering, like suggested in the EMVA1288 that would be good enough in my opinion. Since the image sensors have micro lenses, directed radiation is not such a good idea.

with best regards,

Gerhard
___________________________
Dr. Gerhard Holst
Science & Research
PCO AG
Donaupark 11
93309 Kelheim, Germany
fon [phone]
fax [phone]
mob [phone]
[hidden email]
www.pco.de

-----Ursprüngliche Nachricht-----
Von: Confocal Microscopy List [[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 09:55
An: [hidden email]
Betreff: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few questions, so I am providing a "too long; didn't read" first to summarize. I'm hoping that some of you will find this topic interesting and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon response curve of an sCMOS camera? Why should the illumination pattern be so uniform when each sCMOS pixel can be thought of as an independent sensor?

I am returning to work on a minor problem that has interested me for some time. I work in localization microscopy (STORM/PALM/PAINT) and have been using sCMOS cameras for the past two years with good results. To precisely localize the single molecule emissions, we take into account the pixel-dependent noise characteristics of our sensors, incorporating the measured characteristics of the sensor into the maximum likelihood estimation of a fluorescent molecule's position. This estimation procedure was--as far as I know--first described in Huang et al., Nature Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each pixel of our cameras:

1. the offset (average ADU count under zero illumination) 2. the read noise (variance of the ADU counts under zero illumination) 3. the gain (the number of photoelectrons per ADU when the camera is in the linear response regime)

The offset and read noise are trivial to measure. To measure the gain, however, we capture a few tens of thousands of camera frames with the camera chip under uniform illumination at different light intensities and follow the mathematical operations described in the supplement to the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the gain by observing fluctuations in the pixels' ADU counts in time, not in space? I can understand why illumination non-uniformities would lead to errors when measuring the noise of a CCD chip. For CCD's, I believe that one typically treats each pixel as an independent sample of the noise from the entire chip, so one inherently assumes that the photon shot noise is uniform across the sensor. However, each pixel is only compared to itself when measuring the gain of an sCMOS sensor in the manner described above, so why does it matter that each pixel receives the same light intensity?

2. How flat is "flat enough" for this calibration procedure? With a smart phone screen set an optimum distance from the bare camera port and carefully rotated into position, I can get about 97% uniformity across the whole chip by simply by displaying gray scale images. Most of the non-uniformity appears at the corners of the chip where I think shadowing from the opening in the camera's housing is decreasing the light intensity slightly. The calibrations I get from this method allow me to obtain a localization precision that I independently measured from sparsely distributed dye molecules to be between 8 and 12 nm, which is in line with published STORM results. When measuring tiny clusters of proteins, the scatter plots of the localizations match the overall shapes of their widefield images quite well.

However, a recent paper by Li et al., J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better than 99% uniformity to avoid introducing significant bias into the noise measurements. Furthermore, the engineers at one of the big camera manufacturers once told me I shouldn't even bother trying to do the noise characterization myself since I wouldn't be able to get the required level of uniformity for an accurate characterization. (In fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these questions. So far, my results seem to suggest that my calibration is good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland http://kmdouglass.github.io http://leb.epfl.ch

--  

Kyle M. Douglass, PhD 

Post-doctoral researcher 

The Laboratory of Experimental Biophysics 

EPFL, Lausanne, Switzerland 

http://kmdouglass.github.io 

http://leb.epfl.ch 

[James Pawley]

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** I am enjoying this discussion but am unclear as to why some people are recommending the use of an integrating hemisphere versus a small remote source. It would seem to me that having a "large source" and a finite camera opening would just increase the problem of vignetting. 

The importance of the microlenses has been mentioned but again, it would seem to me that these would become less of a factor with paraxial illumination, as long as the source was close to the optical axis of the lenses. And of course a small-ish (i.e, NOT a Laser) remote source in an otherwise dark room would be a good way to get paraxial illumination. 

Perhaps we also need to remember that we are trying to characterize the performance of a particular camera when it is being used to view a small specimen on a microscope, surely this is closer to paraxial illumination?

While we are on the topic: in the mists of time, Bob Allen and Shinya Inoue developed the video-enhanced DIC microscope (the technique that first "discovered" the action of Kinesin?). This technique involved averaging a number of video frames from a vidicon camera to reduce Poisson noise and then using the contrast and brightness controls to increase the visibility of a small segment (1-3%?) of the intensity response in order to reveal the very small contrast produced by sub-resolution filamentous structures such as micro-tubules or even actin filaments. A major problem was that this slight optical contrast was swamped by the presence of what was called "mottle". Mottle was actually the low contrast "image noise" apparently produced by dirt and/or optical inhomogeneities in the microscope. The problem was reduced by storing an averaged image recorded with the (thin) specimen out of focus and then subtracting this stored image from the averaged in-focus image. 

If we assume that modern microscopes may suffer from the same problems (air bubbles in glass, dust, reflections etc) that plagued scopes in the 1970 and also that the success of the more advanced versions of  PALM/STORM really do depend on recording the data from the in-focus plane with the accuracy implied by this discussion, then it would seem to me that the measurements of the uniformity and noise performance of the recording system must include not only the performance of the camera itself but of the all the optics as well, particularly the optics after the specimen. 

How about mounting a small (but spatially incoherent and temporally stable) source some distance in front of the objective lens?

Best,

Jim Pawley

              ****************************************

James and Christine Pawley, 5446 Burley Place (PO Box 2348), Sechelt, BC, Canada, V0N3A0,

Phone [phone], email <[hidden email]>

NEW! NEW! AND DIFFERENT Cell (when I remember to turn it on!) 1-604-989-6146

On Oct 26, 2016, at 9:58 AM, [hidden email]

 wrote:

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Good point,

as a matter of fact, even in standard photon transfer curve measurements, the fixed pattern noise becomes apparent at several thousand photons per pixel. We're talking about couple hundreds here.
But it's hard to tell how large frame-to-frame and longer term fluctuations are still OK given the large number of frames analyzed (tens of thousands) and the non-trivial method of gain determination in the Joerg's paper (least square estimation). On the other hand, given the photon noise standard deviation is 5% at 400 photons, 1% (max) illumination intensity fluctuation does not sound like a gross overkill...

Best, zdenek

Btw, while my PC monitor LED backlight shows nice PWM at 1.2 kHz, my smartphones produced steady illumination (no fluctuations up to 50 kHz).

---------- Původní zpráva ----------
Od: Gerhard Holst <[hidden email]>
Komu: [hidden email]
Datum: 26. 10. 2016 11:14:42
Předmět: AW: AW: Measuring noise characteristics of sCMOS cameras

Hi Kyle,

 

may I ask 1% of what? Further, including poisson noise? I mean the fluctuations go with the square root of the number of photons.

 

with best regards,

 

Gerhard

___________________________

Dr. Gerhard Holst

Science & Research

PCO AG

Donaupark 11

93309 Kelheim, Germany

fon [phone]

fax [phone]

mob [phone]

[hidden email]

www.pco.de

 

Von: Confocal Microscopy List [mailto:[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 17:01
An: [hidden email]
Betreff: Re: AW: Measuring noise characteristics of sCMOS cameras

 

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Thanks to everyone who has thus far replied to my original e-mail. I realize after doing some more reading that I was essentially conflating two different "noise" sources into one: fixed pattern noise, for which one certainly wants a uniform illumination to measure, and the per pixel gain, which is what is measured in the Huang/Bewersdorf paper.

My next question is: what kind of light source is going to provide <1% temporal intensity fluctuations? A CW LED pumped into an integrating sphere?

Zdenek: Thanks for the heads up about the PWM driver in the smart phone LED backlight. I wouldn't be surprised if this were true. Interestingly, I can see aliasing in a live stream from the camera when using a computer monitor as illumination (60 Hz refresh rate) but I can not see it with the smart phone.

Gerhard: The EMVA 1288 document is absolutely wonderful. Thanks for this.

Seamus: I actually have the book next to me on my desk. Thanks for the reminder about the chapter :)

Cheers,
Kyle

On 10/26/2016 03:37 PM, [hidden email] wrote:

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Honestly, I hope my Orca Flash does not have microlenses!

But back to Kyle's question.

Joerg Bewersdorf's approach is strictly on pixel basis, the spatial uniformity is not critical (97% is more than sufficient), but temporal uniformity is important and < 1% intensity fluctuations are not trivial to achieve (I could not find this figure in my smartphone's specs :-). Slow drift can be easily seen in your images (if you average every 1000 images and look at the trends), but short-time fluctuations may be harder to detect (beware, there is often some sort of PWM driver for the LED backlight)!

As mentioned in the ref 17 (thanks Seamus for the link) there are many sources of "fixed pattern noise" (I don't like the term noise, it's just non-uniformity), including different light sensitivity of individual pixels or dust on the optics. Joerg's approach does not account for these effects, he just intended to bring SCMOSes on par with CCDs. There are more critical aspects of localization superresolution microscopy (such as illumination uniformity) than dust on the camera window... (but another word of caution, smudges on the camera won't be visible during the calibration, but may be visible when coupled to the microscope, because the light has fairly low etendue / cone angle / numerical aperture at the detector).

Btw, has anyone characterized the new back-illuminated SCMOS cameras (like Photometrics Prime 95B)? How do they compare with standard SCMOSes (Zyla, Orca)?

Best, zdenek

--
Zdenek Svindrych, Ph.D.
W.M. Keck Center for Cellular Imaging (PLSB 003)
University of Virginia, Charlottesville, VA
http://www.kcci.virginia.edu/
tel: [phone]
Annual FRET Workshop: http://kcci.virginia.edu/workshop-2017

---------- Původní zpráva ----------
Od: Gerhard Holst [hidden email]
Komu: [hidden email]
Datum: 26. 10. 2016 4:49:59
Předmět: AW: Measuring noise characteristics of sCMOS cameras

 

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Kyle,

your question is absolutely reasonable, and in my opinion it doesn't have to be. If you google you will find the EMVA1288 standard for measuring and representing quality parameters of image sensors and cameras. There is described how the gain can be measured. If a linear camera or image sensor model can be assumed, and our experience as manufacturer of sCMOS cameras proves that, for ease of measurement a relatively homogenous illumination should be ok. The standard suggests a diffuse illumination. The homogeneous illumination is more important for getting information on the homogeneous reaction of the image sensor, means to determine the so called photo-response-non-uniformity. To determine the gain the photon transfer curve is usually measured, and this is the variance versus mean signal, and the variance is usually calculated from the difference of two images, therefore if the pixel have more or less the same brightness, that's good enough.
Alternatively you might use the Fe+55 method. Here the knowledge about the charge generation if silicon is hit be x-ray quants is used. We have done that and in case of the sCMOS image sensors in delivers the same results like the PTC curve approach.

If you take an integrating sphere with diffuse reflection and scattering, like suggested in the EMVA1288 that would be good enough in my opinion. Since the image sensors have micro lenses, directed radiation is not such a good idea.

with best regards,

Gerhard
___________________________
Dr. Gerhard Holst
Science & Research
PCO AG
Donaupark 11
93309 Kelheim, Germany
fon [phone]
fax [phone]
mob [phone]
[hidden email]
www.pco.de

-----Ursprüngliche Nachricht-----
Von: Confocal Microscopy List [[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 09:55
An: [hidden email]
Betreff: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few questions, so I am providing a "too long; didn't read" first to summarize. I'm hoping that some of you will find this topic interesting and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon response curve of an sCMOS camera? Why should the illumination pattern be so uniform when each sCMOS pixel can be thought of as an independent sensor?

I am returning to work on a minor problem that has interested me for some time. I work in localization microscopy (STORM/PALM/PAINT) and have been using sCMOS cameras for the past two years with good results. To precisely localize the single molecule emissions, we take into account the pixel-dependent noise characteristics of our sensors, incorporating the measured characteristics of the sensor into the maximum likelihood estimation of a fluorescent molecule's position. This estimation procedure was--as far as I know--first described in Huang et al., Nature Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each pixel of our cameras:

1. the offset (average ADU count under zero illumination) 2. the read noise (variance of the ADU counts under zero illumination) 3. the gain (the number of photoelectrons per ADU when the camera is in the linear response regime)

The offset and read noise are trivial to measure. To measure the gain, however, we capture a few tens of thousands of camera frames with the camera chip under uniform illumination at different light intensities and follow the mathematical operations described in the supplement to the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the gain by observing fluctuations in the pixels' ADU counts in time, not in space? I can understand why illumination non-uniformities would lead to errors when measuring the noise of a CCD chip. For CCD's, I believe that one typically treats each pixel as an independent sample of the noise from the entire chip, so one inherently assumes that the photon shot noise is uniform across the sensor. However, each pixel is only compared to itself when measuring the gain of an sCMOS sensor in the manner described above, so why does it matter that each pixel receives the same light intensity?

2. How flat is "flat enough" for this calibration procedure? With a smart phone screen set an optimum distance from the bare camera port and carefully rotated into position, I can get about 97% uniformity across the whole chip by simply by displaying gray scale images. Most of the non-uniformity appears at the corners of the chip where I think shadowing from the opening in the camera's housing is decreasing the light intensity slightly. The calibrations I get from this method allow me to obtain a localization precision that I independently measured from sparsely distributed dye molecules to be between 8 and 12 nm, which is in line with published STORM results. When measuring tiny clusters of proteins, the scatter plots of the localizations match the overall shapes of their widefield images quite well.

However, a recent paper by Li et al., J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better than 99% uniformity to avoid introducing significant bias into the noise measurements. Furthermore, the engineers at one of the big camera manufacturers once told me I shouldn't even bother trying to do the noise characterization myself since I wouldn't be able to get the required level of uniformity for an accurate characterization. (In fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these questions. So far, my results seem to suggest that my calibration is good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland http://kmdouglass.github.io http://leb.epfl.ch

--  

Kyle M. Douglass, PhD 

Post-doctoral researcher 

The Laboratory of Experimental Biophysics 

EPFL, Lausanne, Switzerland 

http://kmdouglass.github.io 

http://leb.epfl.ch 

[Gerhard Holst]

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Hi Jim,

 

„I am enjoying this discussion but am unclear as to why some people are recommending the use of an integrating hemisphere versus a small remote source. It would seem to me that having a "large source" and a finite camera opening would just increase the problem of vignetting.”

 

The reason for the integrating sphere is the absence of any preferred direction in the radiation, this in combination with the recommended aperture of 8 (referring to EMVA1288 this is the ratio of d the distance between the image sensor and the opening spot of the sphere and the diameter of the opening of the sphere D, f = d/D), this allows a good and practical illumination of the image sensor in the camera without additional optics (except the microlenses). Nevertheless you could use the far field of a “point light source” as well.

 

But I absolutely agree that for each application the whole system made of optics and camera has to be analysed.

 

with best regards,

 

Gerhard

___________________________

Dr. Gerhard Holst

Science & Research

PCO AG

Donaupark 11

93309 Kelheim, Germany

fon [phone]

fax [phone]

mob [phone]

[hidden email]

www.pco.de

 

Von: Confocal Microscopy List [mailto:[hidden email]] Im Auftrag von JAMES B PAWLEY
Gesendet: Mittwoch, 26. Oktober 2016 19:41
An: [hidden email]
Betreff: Re: AW: AW: Measuring noise characteristics of sCMOS cameras

 

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

I am enjoying this discussion but am unclear as to why some people are recommending the use of an integrating hemisphere versus a small remote source. It would seem to me that having a "large source" and a finite camera opening would just increase the problem of vignetting. 

 

The importance of the microlenses has been mentioned but again, it would seem to me that these would become less of a factor with paraxial illumination, as long as the source was close to the optical axis of the lenses. And of course a small-ish (i.e, NOT a Laser) remote source in an otherwise dark room would be a good way to get paraxial illumination. 

 

Perhaps we also need to remember that we are trying to characterize the performance of a particular camera when it is being used to view a small specimen on a microscope, surely this is closer to paraxial illumination?

 

While we are on the topic: in the mists of time, Bob Allen and Shinya Inoue developed the video-enhanced DIC microscope (the technique that first "discovered" the action of Kinesin?). This technique involved averaging a number of video frames from a vidicon camera to reduce Poisson noise and then using the contrast and brightness controls to increase the visibility of a small segment (1-3%?) of the intensity response in order to reveal the very small contrast produced by sub-resolution filamentous structures such as micro-tubules or even actin filaments. A major problem was that this slight optical contrast was swamped by the presence of what was called "mottle". Mottle was actually the low contrast "image noise" apparently produced by dirt and/or optical inhomogeneities in the microscope. The problem was reduced by storing an averaged image recorded with the (thin) specimen out of focus and then subtracting this stored image from the averaged in-focus image. 

 

If we assume that modern microscopes may suffer from the same problems (air bubbles in glass, dust, reflections etc) that plagued scopes in the 1970 and also that the success of the more advanced versions of  PALM/STORM really do depend on recording the data from the in-focus plane with the accuracy implied by this discussion, then it would seem to me that the measurements of the uniformity and noise performance of the recording system must include not only the performance of the camera itself but of the all the optics as well, particularly the optics after the specimen. 

 

How about mounting a small (but spatially incoherent and temporally stable) source some distance in front of the objective lens?

 

Best,

 

Jim Pawley

 

 

 

 

 

 

              ****************************************

James and Christine Pawley, 5446 Burley Place (PO Box 2348), Sechelt, BC, Canada, V0N3A0,

Phone [phone], email <[hidden email]>

NEW! NEW! AND DIFFERENT Cell (when I remember to turn it on!) 1-604-989-6146

 

On Oct 26, 2016, at 9:58 AM, [hidden email]

 wrote:

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Good point,

as a matter of fact, even in standard photon transfer curve measurements, the fixed pattern noise becomes apparent at several thousand photons per pixel. We're talking about couple hundreds here.
But it's hard to tell how large frame-to-frame and longer term fluctuations are still OK given the large number of frames analyzed (tens of thousands) and the non-trivial method of gain determination in the Joerg's paper (least square estimation). On the other hand, given the photon noise standard deviation is 5% at 400 photons, 1% (max) illumination intensity fluctuation does not sound like a gross overkill...

Best, zdenek

Btw, while my PC monitor LED backlight shows nice PWM at 1.2 kHz, my smartphones produced steady illumination (no fluctuations up to 50 kHz).

 

---------- Původní zpráva ----------
Od: Gerhard Holst <[hidden email]>
Komu: [hidden email]
Datum: 26. 10. 2016 11:14:42
Předmět: AW: AW: Measuring noise characteristics of sCMOS cameras

 

Hi Kyle,

 

may I ask 1% of what? Further, including poisson noise? I mean the fluctuations go with the square root of the number of photons.

 

with best regards,

 

Gerhard

___________________________

Dr. Gerhard Holst

Science & Research

PCO AG

Donaupark 11

93309 Kelheim, Germany

fon [phone]

fax [phone]

mob [phone]

[hidden email]

www.pco.de

 

Von: Confocal Microscopy List [[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 17:01
An: [hidden email]
Betreff: Re: AW: Measuring noise characteristics of sCMOS cameras

 

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Thanks to everyone who has thus far replied to my original e-mail. I realize after doing some more reading that I was essentially conflating two different "noise" sources into one: fixed pattern noise, for which one certainly wants a uniform illumination to measure, and the per pixel gain, which is what is measured in the Huang/Bewersdorf paper.

My next question is: what kind of light source is going to provide <1% temporal intensity fluctuations? A CW LED pumped into an integrating sphere?

Zdenek: Thanks for the heads up about the PWM driver in the smart phone LED backlight. I wouldn't be surprised if this were true. Interestingly, I can see aliasing in a live stream from the camera when using a computer monitor as illumination (60 Hz refresh rate) but I can not see it with the smart phone.

Gerhard: The EMVA 1288 document is absolutely wonderful. Thanks for this.

Seamus: I actually have the book next to me on my desk. Thanks for the reminder about the chapter :)

Cheers,
Kyle

On 10/26/2016 03:37 PM, [hidden email] wrote:

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Honestly, I hope my Orca Flash does not have microlenses!

But back to Kyle's question.

Joerg Bewersdorf's approach is strictly on pixel basis, the spatial uniformity is not critical (97% is more than sufficient), but temporal uniformity is important and < 1% intensity fluctuations are not trivial to achieve (I could not find this figure in my smartphone's specs :-). Slow drift can be easily seen in your images (if you average every 1000 images and look at the trends), but short-time fluctuations may be harder to detect (beware, there is often some sort of PWM driver for the LED backlight)!

As mentioned in the ref 17 (thanks Seamus for the link) there are many sources of "fixed pattern noise" (I don't like the term noise, it's just non-uniformity), including different light sensitivity of individual pixels or dust on the optics. Joerg's approach does not account for these effects, he just intended to bring SCMOSes on par with CCDs. There are more critical aspects of localization superresolution microscopy (such as illumination uniformity) than dust on the camera window... (but another word of caution, smudges on the camera won't be visible during the calibration, but may be visible when coupled to the microscope, because the light has fairly low etendue / cone angle / numerical aperture at the detector).

Btw, has anyone characterized the new back-illuminated SCMOS cameras (like Photometrics Prime 95B)? How do they compare with standard SCMOSes (Zyla, Orca)?

Best, zdenek

--
Zdenek Svindrych, Ph.D.
W.M. Keck Center for Cellular Imaging (PLSB 003)
University of Virginia, Charlottesville, VA
http://www.kcci.virginia.edu/
tel: [phone]
Annual FRET Workshop: http://kcci.virginia.edu/workshop-2017

---------- Původní zpráva ----------
Od: Gerhard Holst [hidden email]
Komu: [hidden email]
Datum: 26. 10. 2016 4:49:59
Předmět: AW: Measuring noise characteristics of sCMOS cameras

 

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Kyle,

your question is absolutely reasonable, and in my opinion it doesn't have to be. If you google you will find the EMVA1288 standard for measuring and representing quality parameters of image sensors and cameras. There is described how the gain can be measured. If a linear camera or image sensor model can be assumed, and our experience as manufacturer of sCMOS cameras proves that, for ease of measurement a relatively homogenous illumination should be ok. The standard suggests a diffuse illumination. The homogeneous illumination is more important for getting information on the homogeneous reaction of the image sensor, means to determine the so called photo-response-non-uniformity. To determine the gain the photon transfer curve is usually measured, and this is the variance versus mean signal, and the variance is usually calculated from the difference of two images, therefore if the pixel have more or less the same brightness, that's good enough.
Alternatively you might use the Fe+55 method. Here the knowledge about the charge generation if silicon is hit be x-ray quants is used. We have done that and in case of the sCMOS image sensors in delivers the same results like the PTC curve approach.

If you take an integrating sphere with diffuse reflection and scattering, like suggested in the EMVA1288 that would be good enough in my opinion. Since the image sensors have micro lenses, directed radiation is not such a good idea.

with best regards,

Gerhard
___________________________
Dr. Gerhard Holst
Science & Research
PCO AG
Donaupark 11
93309 Kelheim, Germany
fon [phone]
fax [phone]
mob [phone]
[hidden email]
www.pco.de

-----Ursprüngliche Nachricht-----
Von: Confocal Microscopy List [[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 09:55
An: [hidden email]
Betreff: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few questions, so I am providing a "too long; didn't read" first to summarize. I'm hoping that some of you will find this topic interesting and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon response curve of an sCMOS camera? Why should the illumination pattern be so uniform when each sCMOS pixel can be thought of as an independent sensor?

I am returning to work on a minor problem that has interested me for some time. I work in localization microscopy (STORM/PALM/PAINT) and have been using sCMOS cameras for the past two years with good results. To precisely localize the single molecule emissions, we take into account the pixel-dependent noise characteristics of our sensors, incorporating the measured characteristics of the sensor into the maximum likelihood estimation of a fluorescent molecule's position. This estimation procedure was--as far as I know--first described in Huang et al., Nature Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each pixel of our cameras:

1. the offset (average ADU count under zero illumination) 2. the read noise (variance of the ADU counts under zero illumination) 3. the gain (the number of photoelectrons per ADU when the camera is in the linear response regime)

The offset and read noise are trivial to measure. To measure the gain, however, we capture a few tens of thousands of camera frames with the camera chip under uniform illumination at different light intensities and follow the mathematical operations described in the supplement to the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the gain by observing fluctuations in the pixels' ADU counts in time, not in space? I can understand why illumination non-uniformities would lead to errors when measuring the noise of a CCD chip. For CCD's, I believe that one typically treats each pixel as an independent sample of the noise from the entire chip, so one inherently assumes that the photon shot noise is uniform across the sensor. However, each pixel is only compared to itself when measuring the gain of an sCMOS sensor in the manner described above, so why does it matter that each pixel receives the same light intensity?

2. How flat is "flat enough" for this calibration procedure? With a smart phone screen set an optimum distance from the bare camera port and carefully rotated into position, I can get about 97% uniformity across the whole chip by simply by displaying gray scale images. Most of the non-uniformity appears at the corners of the chip where I think shadowing from the opening in the camera's housing is decreasing the light intensity slightly. The calibrations I get from this method allow me to obtain a localization precision that I independently measured from sparsely distributed dye molecules to be between 8 and 12 nm, which is in line with published STORM results. When measuring tiny clusters of proteins, the scatter plots of the localizations match the overall shapes of their widefield images quite well.

However, a recent paper by Li et al., J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better than 99% uniformity to avoid introducing significant bias into the noise measurements. Furthermore, the engineers at one of the big camera manufacturers once told me I shouldn't even bother trying to do the noise characterization myself since I wouldn't be able to get the required level of uniformity for an accurate characterization. (In fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these questions. So far, my results seem to suggest that my calibration is good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland http://kmdouglass.github.io http://leb.epfl.ch

 

--  

Kyle M. Douglass, PhD 

Post-doctoral researcher 

The Laboratory of Experimental Biophysics 

EPFL, Lausanne, Switzerland 

http://kmdouglass.github.io 

http://leb.epfl.ch 

 

[Vitaly Boyko-2]

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Hi Gerhard,

Pixels on CMOS sensors are non-linear and I expect high variability on a pixel-by-pixel basis. I assume the same is true for the sCMOS sensors, plus microlenses, plus variabilies in optical resolution of an optical system(s). I have seen huge deviations in PSF from one 63x NA 1.4 lens to another lens of the same specs and manufacturer. Linearity of sCMOS cameras seems to be "restored" ("corrected") post-acquisition. How accurate is it taking into account presumably big differences in light sensitivity within a single CMOS sensor. I haven't seen any graphs with standard error bars on pixel-by-pixel variability within a single or dozens of sCMOS chips, microlenses, any other optical hardware. On top of that we have a thin versus thick samples; undersampling (with low N.A. objectives) is a "daily routine". Can we still perform accurate quantitative image analysis on a voxel-by-voxel basis (e.g. VOI 3x3x2)? What is the main source of linearity - optical hardware or a biospecimen? The Whole Brain Projects are still struggling with the undersampled data and there are no quick image processing tools yet to handle much larger but properly sampled 3D/4D data sets (GPU based parallel computing is still at the level of startups).

Best regards,

Vitaly 

On Thursday, October 27, 2016 2:24 AM, Gerhard Holst <[hidden email]> wrote:

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Hi Jim,

 

„I am enjoying this discussion but am unclear as to why some people are recommending the use of an integrating hemisphere versus a small remote source. It would seem to me that having a "large source" and a finite camera opening would just increase the problem of vignetting.”

 

The reason for the integrating sphere is the absence of any preferred direction in the radiation, this in combination with the recommended aperture of 8 (referring to EMVA1288 this is the ratio of d the distance between the image sensor and the opening spot of the sphere and the diameter of the opening of the sphere D, f = d/D), this allows a good and practical illumination of the image sensor in the camera without additional optics (except the microlenses). Nevertheless you could use the far field of a “point light source” as well.

 

But I absolutely agree that for each application the whole system made of optics and camera has to be analysed.

 

with best regards,

 

Gerhard

___________________________

Dr. Gerhard Holst

Science & Research

PCO AG

Donaupark 11

93309 Kelheim, Germany

fon [phone]

fax [phone]

mob [phone]

[hidden email]

www.pco.de

 

Von: Confocal Microscopy List [mailto:[hidden email]] Im Auftrag von JAMES B PAWLEY
Gesendet: Mittwoch, 26. Oktober 2016 19:41
An: [hidden email]
Betreff: Re: AW: AW: Measuring noise characteristics of sCMOS cameras

 

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

I am enjoying this discussion but am unclear as to why some people are recommending the use of an integrating hemisphere versus a small remote source. It would seem to me that having a "large source" and a finite camera opening would just increase the problem of vignetting. 

 

The importance of the microlenses has been mentioned but again, it would seem to me that these would become less of a factor with paraxial illumination, as long as the source was close to the optical axis of the lenses. And of course a small-ish (i.e, NOT a Laser) remote source in an otherwise dark room would be a good way to get paraxial illumination. 

 

Perhaps we also need to remember that we are trying to characterize the performance of a particular camera when it is being used to view a small specimen on a microscope, surely this is closer to paraxial illumination?

 

While we are on the topic: in the mists of time, Bob Allen and Shinya Inoue developed the video-enhanced DIC microscope (the technique that first "discovered" the action of Kinesin?). This technique involved averaging a number of video frames from a vidicon camera to reduce Poisson noise and then using the contrast and brightness controls to increase the visibility of a small segment (1-3%?) of the intensity response in order to reveal the very small contrast produced by sub-resolution filamentous structures such as micro-tubules or even actin filaments. A major problem was that this slight optical contrast was swamped by the presence of what was called "mottle". Mottle was actually the low contrast "image noise" apparently produced by dirt and/or optical inhomogeneities in the microscope. The problem was reduced by storing an averaged image recorded with the (thin) specimen out of focus and then subtracting this stored image from the averaged in-focus image. 

 

If we assume that modern microscopes may suffer from the same problems (air bubbles in glass, dust, reflections etc) that plagued scopes in the 1970 and also that the success of the more advanced versions of  PALM/STORM really do depend on recording the data from the in-focus plane with the accuracy implied by this discussion, then it would seem to me that the measurements of the uniformity and noise performance of the recording system must include not only the performance of the camera itself but of the all the optics as well, particularly the optics after the specimen. 

 

How about mounting a small (but spatially incoherent and temporally stable) source some distance in front of the objective lens?

 

Best,

 

Jim Pawley

 

 

 

 

 

 

              ****************************************

James and Christine Pawley, 5446 Burley Place (PO Box 2348), Sechelt, BC, Canada, V0N3A0,

Phone [phone], email <[hidden email]>

NEW! NEW! AND DIFFERENT Cell (when I remember to turn it on!) 1-604-989-6146

 

On Oct 26, 2016, at 9:58 AM, [hidden email]

 wrote:

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Good point,

as a matter of fact, even in standard photon transfer curve measurements, the fixed pattern noise becomes apparent at several thousand photons per pixel. We're talking about couple hundreds here.
But it's hard to tell how large frame-to-frame and longer term fluctuations are still OK given the large number of frames analyzed (tens of thousands) and the non-trivial method of gain determination in the Joerg's paper (least square estimation). On the other hand, given the photon noise standard deviation is 5% at 400 photons, 1% (max) illumination intensity fluctuation does not sound like a gross overkill...

Best, zdenek

Btw, while my PC monitor LED backlight shows nice PWM at 1.2 kHz, my smartphones produced steady illumination (no fluctuations up to 50 kHz).

 

---------- Původní zpráva ----------
Od: Gerhard Holst <[hidden email]>
Komu: [hidden email]
Datum: 26. 10. 2016 11:14:42
Předmět: AW: AW: Measuring noise characteristics of sCMOS cameras

 

Hi Kyle,

 

may I ask 1% of what? Further, including poisson noise? I mean the fluctuations go with the square root of the number of photons.

 

with best regards,

 

Gerhard

___________________________

Dr. Gerhard Holst

Science & Research

PCO AG

Donaupark 11

93309 Kelheim, Germany

fon [phone]

fax [phone]

mob [phone]

[hidden email]

www.pco.de

 

Von: Confocal Microscopy List [[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 17:01
An: [hidden email]
Betreff: Re: AW: Measuring noise characteristics of sCMOS cameras

 

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. *****

Thanks to everyone who has thus far replied to my original e-mail. I realize after doing some more reading that I was essentially conflating two different "noise" sources into one: fixed pattern noise, for which one certainly wants a uniform illumination to measure, and the per pixel gain, which is what is measured in the Huang/Bewersdorf paper.

My next question is: what kind of light source is going to provide <1% temporal intensity fluctuations? A CW LED pumped into an integrating sphere?

Zdenek: Thanks for the heads up about the PWM driver in the smart phone LED backlight. I wouldn't be surprised if this were true. Interestingly, I can see aliasing in a live stream from the camera when using a computer monitor as illumination (60 Hz refresh rate) but I can not see it with the smart phone.

Gerhard: The EMVA 1288 document is absolutely wonderful. Thanks for this.

Seamus: I actually have the book next to me on my desk. Thanks for the reminder about the chapter :)

Cheers,
Kyle

On 10/26/2016 03:37 PM, [hidden email] wrote:

***** To join, leave or search the confocal microscopy listserv, go to: http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on http://www.imgur.com and include the link in your posting. ***** Honestly, I hope my Orca Flash does not have microlenses!

But back to Kyle's question.

Joerg Bewersdorf's approach is strictly on pixel basis, the spatial uniformity is not critical (97% is more than sufficient), but temporal uniformity is important and < 1% intensity fluctuations are not trivial to achieve (I could not find this figure in my smartphone's specs :-). Slow drift can be easily seen in your images (if you average every 1000 images and look at the trends), but short-time fluctuations may be harder to detect (beware, there is often some sort of PWM driver for the LED backlight)!

As mentioned in the ref 17 (thanks Seamus for the link) there are many sources of "fixed pattern noise" (I don't like the term noise, it's just non-uniformity), including different light sensitivity of individual pixels or dust on the optics. Joerg's approach does not account for these effects, he just intended to bring SCMOSes on par with CCDs. There are more critical aspects of localization superresolution microscopy (such as illumination uniformity) than dust on the camera window... (but another word of caution, smudges on the camera won't be visible during the calibration, but may be visible when coupled to the microscope, because the light has fairly low etendue / cone angle / numerical aperture at the detector).

Btw, has anyone characterized the new back-illuminated SCMOS cameras (like Photometrics Prime 95B)? How do they compare with standard SCMOSes (Zyla, Orca)?

Best, zdenek

--
Zdenek Svindrych, Ph.D.
W.M. Keck Center for Cellular Imaging (PLSB 003)
University of Virginia, Charlottesville, VA
http://www.kcci.virginia.edu/
tel: [phone]
Annual FRET Workshop: http://kcci.virginia.edu/workshop-2017

---------- Původní zpráva ----------
Od: Gerhard Holst [hidden email]
Komu: [hidden email]
Datum: 26. 10. 2016 4:49:59
Předmět: AW: Measuring noise characteristics of sCMOS cameras

 

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Kyle,

your question is absolutely reasonable, and in my opinion it doesn't have to be. If you google you will find the EMVA1288 standard for measuring and representing quality parameters of image sensors and cameras. There is described how the gain can be measured. If a linear camera or image sensor model can be assumed, and our experience as manufacturer of sCMOS cameras proves that, for ease of measurement a relatively homogenous illumination should be ok. The standard suggests a diffuse illumination. The homogeneous illumination is more important for getting information on the homogeneous reaction of the image sensor, means to determine the so called photo-response-non-uniformity. To determine the gain the photon transfer curve is usually measured, and this is the variance versus mean signal, and the variance is usually calculated from the difference of two images, therefore if the pixel have more or less the same brightness, that's good enough.
Alternatively you might use the Fe+55 method. Here the knowledge about the charge generation if silicon is hit be x-ray quants is used. We have done that and in case of the sCMOS image sensors in delivers the same results like the PTC curve approach.

If you take an integrating sphere with diffuse reflection and scattering, like suggested in the EMVA1288 that would be good enough in my opinion. Since the image sensors have micro lenses, directed radiation is not such a good idea.

with best regards,

Gerhard
___________________________
Dr. Gerhard Holst
Science & Research
PCO AG
Donaupark 11
93309 Kelheim, Germany
fon [phone]
fax [phone]
mob [phone]
[hidden email]
www.pco.de

-----Ursprüngliche Nachricht-----
Von: Confocal Microscopy List [[hidden email]] Im Auftrag von Kyle Douglass
Gesendet: Mittwoch, 26. Oktober 2016 09:55
An: [hidden email]
Betreff: Measuring noise characteristics of sCMOS cameras

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi everyone,

This is a rather long and technical post which comes down to a few questions, so I am providing a "too long; didn't read" first to summarize. I'm hoping that some of you will find this topic interesting and be able to reply.

tl;dr: How flat should the illumination be when measuring the photon response curve of an sCMOS camera? Why should the illumination pattern be so uniform when each sCMOS pixel can be thought of as an independent sensor?

I am returning to work on a minor problem that has interested me for some time. I work in localization microscopy (STORM/PALM/PAINT) and have been using sCMOS cameras for the past two years with good results. To precisely localize the single molecule emissions, we take into account the pixel-dependent noise characteristics of our sensors, incorporating the measured characteristics of the sensor into the maximum likelihood estimation of a fluorescent molecule's position. This estimation procedure was--as far as I know--first described in Huang et al., Nature Methods 10, 653 (2013), doi:10.1038/nmeth.2488.

To do the characterization requires measuring three quantities for each pixel of our cameras:

1. the offset (average ADU count under zero illumination) 2. the read noise (variance of the ADU counts under zero illumination) 3. the gain (the number of photoelectrons per ADU when the camera is in the linear response regime)

The offset and read noise are trivial to measure. To measure the gain, however, we capture a few tens of thousands of camera frames with the camera chip under uniform illumination at different light intensities and follow the mathematical operations described in the supplement to the paper cited above.

My questions are:

1. Why does the illumination need to be flat when we are measuring the gain by observing fluctuations in the pixels' ADU counts in time, not in space? I can understand why illumination non-uniformities would lead to errors when measuring the noise of a CCD chip. For CCD's, I believe that one typically treats each pixel as an independent sample of the noise from the entire chip, so one inherently assumes that the photon shot noise is uniform across the sensor. However, each pixel is only compared to itself when measuring the gain of an sCMOS sensor in the manner described above, so why does it matter that each pixel receives the same light intensity?

2. How flat is "flat enough" for this calibration procedure? With a smart phone screen set an optimum distance from the bare camera port and carefully rotated into position, I can get about 97% uniformity across the whole chip by simply by displaying gray scale images. Most of the non-uniformity appears at the corners of the chip where I think shadowing from the opening in the camera's housing is decreasing the light intensity slightly. The calibrations I get from this method allow me to obtain a localization precision that I independently measured from sparsely distributed dye molecules to be between 8 and 12 nm, which is in line with published STORM results. When measuring tiny clusters of proteins, the scatter plots of the localizations match the overall shapes of their widefield images quite well.

However, a recent paper by Li et al., J. Innov. Opt. Health Sci. 09,
1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better than 99% uniformity to avoid introducing significant bias into the noise measurements. Furthermore, the engineers at one of the big camera manufacturers once told me I shouldn't even bother trying to do the noise characterization myself since I wouldn't be able to get the required level of uniformity for an accurate characterization. (In fairness, they sell the characterization process as a service.)

Unfortunately, I have been unable to find satisfactory answers to these questions. So far, my results seem to suggest that my calibration is good enough, but I wonder if someone else can offer their input.

Thanks!
Kyle

--
Kyle M. Douglass, PhD
Post-doctoral researcher
The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland http://kmdouglass.github.io http://leb.epfl.ch

 

--  

Kyle M. Douglass, PhD 

Post-doctoral researcher 

The Laboratory of Experimental Biophysics 

EPFL, Lausanne, Switzerland 

http://kmdouglass.github.io 

http://leb.epfl.ch 

 

[Marc Reinig]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

I think the light source itself is not the issue for temporal
stability over the measurements, it is more likely the power source
for the light source.  For photo enlargers, a stabilizing transformer
was/is used, that kept the power constant to the bulb, regardless of
the input fluctuations.

Regards,

Marco
Marco

Marc R. Reinig
W. M. Keck Center for Adaptive Optical Microscopy
University of California Santa Cruz



On Wed, Oct 26, 2016 at 9:58 AM,  <[hidden email]> wrote:

> ***** To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on
> http://www.imgur.com and include the link in your posting. *****
> Good point,
> as a matter of fact, even in standard photon transfer curve measurements,
> the fixed pattern noise becomes apparent at several thousand photons per
> pixel. We're talking about couple hundreds here.
> But it's hard to tell how large frame-to-frame and longer term fluctuations
> are still OK given the large number of frames analyzed (tens of thousands)
> and the non-trivial method of gain determination in the Joerg's paper (least
> square estimation). On the other hand, given the photon noise standard
> deviation is 5% at 400 photons, 1% (max) illumination intensity fluctuation
> does not sound like a gross overkill...
> Best, zdenek
> Btw, while my PC monitor LED backlight shows nice PWM at 1.2 kHz, my
> smartphones produced steady illumination (no fluctuations up to 50 kHz).
>
> ---------- Původní zpráva ----------
> Od: Gerhard Holst <[hidden email]>
> Komu: [hidden email]
> Datum: 26. 10. 2016 11:14:42
> Předmět: AW: AW: Measuring noise characteristics of sCMOS cameras
>
>
> Hi Kyle,
>
>
>
> may I ask 1% of what? Further, including poisson noise? I mean the
> fluctuations go with the square root of the number of photons.
>
>
>
> with best regards,
>
>
>
> Gerhard
>
> ___________________________
>
> Dr. Gerhard Holst
>
> Science & Research
>
> PCO AG
>
> Donaupark 11
>
> 93309 Kelheim, Germany
>
> fon [phone]
>
> fax [phone]
>
> mob [phone]
>
> [hidden email]
>
> www.pco.de
>
>
>
> Von: Confocal Microscopy List [mailto:[hidden email]] Im
> Auftrag von Kyle Douglass
> Gesendet: Mittwoch, 26. Oktober 2016 17:01
> An: [hidden email]
> Betreff: Re: AW: Measuring noise characteristics of sCMOS cameras
>
>
>
> ***** To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on
> http://www.imgur.com and include the link in your posting. *****
>
> Thanks to everyone who has thus far replied to my original e-mail. I realize
> after doing some more reading that I was essentially conflating two
> different "noise" sources into one: fixed pattern noise, for which one
> certainly wants a uniform illumination to measure, and the per pixel gain,
> which is what is measured in the Huang/Bewersdorf paper.
>
> My next question is: what kind of light source is going to provide <1%
> temporal intensity fluctuations? A CW LED pumped into an integrating sphere?
>
> Zdenek: Thanks for the heads up about the PWM driver in the smart phone LED
> backlight. I wouldn't be surprised if this were true. Interestingly, I can
> see aliasing in a live stream from the camera when using a computer monitor
> as illumination (60 Hz refresh rate) but I can not see it with the smart
> phone.
>
> Gerhard: The EMVA 1288 document is absolutely wonderful. Thanks for this.
>
> Seamus: I actually have the book next to me on my desk. Thanks for the
> reminder about the chapter :)
>
> Cheers,
> Kyle
>
>
> On 10/26/2016 03:37 PM, [hidden email] wrote:
>
> ***** To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy Post images on
> http://www.imgur.com and include the link in your posting. ***** Honestly, I
> hope my Orca Flash does not have microlenses!
>
> But back to Kyle's question.
>
> Joerg Bewersdorf's approach is strictly on pixel basis, the spatial
> uniformity is not critical (97% is more than sufficient), but temporal
> uniformity is important and < 1% intensity fluctuations are not trivial to
> achieve (I could not find this figure in my smartphone's specs :-). Slow
> drift can be easily seen in your images (if you average every 1000 images
> and look at the trends), but short-time fluctuations may be harder to detect
> (beware, there is often some sort of PWM driver for the LED backlight)!
>
> As mentioned in the ref 17 (thanks Seamus for the link) there are many
> sources of "fixed pattern noise" (I don't like the term noise, it's just
> non-uniformity), including different light sensitivity of individual pixels
> or dust on the optics. Joerg's approach does not account for these effects,
> he just intended to bring SCMOSes on par with CCDs. There are more critical
> aspects of localization superresolution microscopy (such as illumination
> uniformity) than dust on the camera window... (but another word of caution,
> smudges on the camera won't be visible during the calibration, but may be
> visible when coupled to the microscope, because the light has fairly low
> etendue / cone angle / numerical aperture at the detector).
>
> Btw, has anyone characterized the new back-illuminated SCMOS cameras (like
> Photometrics Prime 95B)? How do they compare with standard SCMOSes (Zyla,
> Orca)?
>
> Best, zdenek
>
>
> --
> Zdenek Svindrych, Ph.D.
> W.M. Keck Center for Cellular Imaging (PLSB 003)
> University of Virginia, Charlottesville, VA
> http://www.kcci.virginia.edu/
> tel: [phone]
> Annual FRET Workshop: http://kcci.virginia.edu/workshop-2017
>
> ---------- Původní zpráva ----------
> Od: Gerhard Holst <[hidden email]>
> Komu: [hidden email]
> Datum: 26. 10. 2016 4:49:59
> Předmět: AW: Measuring noise characteristics of sCMOS cameras
>
>
>
> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hi Kyle,
>
> your question is absolutely reasonable, and in my opinion it doesn't have to
> be. If you google you will find the EMVA1288 standard for measuring and
> representing quality parameters of image sensors and cameras. There is
> described how the gain can be measured. If a linear camera or image sensor
> model can be assumed, and our experience as manufacturer of sCMOS cameras
> proves that, for ease of measurement a relatively homogenous illumination
> should be ok. The standard suggests a diffuse illumination. The homogeneous
> illumination is more important for getting information on the homogeneous
> reaction of the image sensor, means to determine the so called
> photo-response-non-uniformity. To determine the gain the photon transfer
> curve is usually measured, and this is the variance versus mean signal, and
> the variance is usually calculated from the difference of two images,
> therefore if the pixel have more or less the same brightness, that's good
> enough.
> Alternatively you might use the Fe+55 method. Here the knowledge about the
> charge generation if silicon is hit be x-ray quants is used. We have done
> that and in case of the sCMOS image sensors in delivers the same results
> like the PTC curve approach.
>
> If you take an integrating sphere with diffuse reflection and scattering,
> like suggested in the EMVA1288 that would be good enough in my opinion.
> Since the image sensors have micro lenses, directed radiation is not such a
> good idea.
>
> with best regards,
>
> Gerhard
> ___________________________
> Dr. Gerhard Holst
> Science & Research
> PCO AG
> Donaupark 11
> 93309 Kelheim, Germany
> fon [phone]
> fax [phone]
> mob [phone]
> [hidden email]
> www.pco.de
>
> -----Ursprüngliche Nachricht-----
> Von: Confocal Microscopy List [mailto:[hidden email]] Im
> Auftrag von Kyle Douglass
> Gesendet: Mittwoch, 26. Oktober 2016 09:55
> An: [hidden email]
> Betreff: Measuring noise characteristics of sCMOS cameras
>
> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hi everyone,
>
> This is a rather long and technical post which comes down to a few
> questions, so I am providing a "too long; didn't read" first to summarize.
> I'm hoping that some of you will find this topic interesting and be able to
> reply.
>
> tl;dr: How flat should the illumination be when measuring the photon
> response curve of an sCMOS camera? Why should the illumination pattern be so
> uniform when each sCMOS pixel can be thought of as an independent sensor?
>
> I am returning to work on a minor problem that has interested me for some
> time. I work in localization microscopy (STORM/PALM/PAINT) and have been
> using sCMOS cameras for the past two years with good results. To precisely
> localize the single molecule emissions, we take into account the
> pixel-dependent noise characteristics of our sensors, incorporating the
> measured characteristics of the sensor into the maximum likelihood
> estimation of a fluorescent molecule's position. This estimation procedure
> was--as far as I know--first described in Huang et al., Nature Methods 10,
> 653 (2013), doi:10.1038/nmeth.2488.
>
> To do the characterization requires measuring three quantities for each
> pixel of our cameras:
>
> 1. the offset (average ADU count under zero illumination) 2. the read noise
> (variance of the ADU counts under zero illumination) 3. the gain (the number
> of photoelectrons per ADU when the camera is in the linear response regime)
>
> The offset and read noise are trivial to measure. To measure the gain,
> however, we capture a few tens of thousands of camera frames with the camera
> chip under uniform illumination at different light intensities and follow
> the mathematical operations described in the supplement to the paper cited
> above.
>
> My questions are:
>
> 1. Why does the illumination need to be flat when we are measuring the gain
> by observing fluctuations in the pixels' ADU counts in time, not in space? I
> can understand why illumination non-uniformities would lead to errors when
> measuring the noise of a CCD chip. For CCD's, I believe that one typically
> treats each pixel as an independent sample of the noise from the entire
> chip, so one inherently assumes that the photon shot noise is uniform across
> the sensor. However, each pixel is only compared to itself when measuring
> the gain of an sCMOS sensor in the manner described above, so why does it
> matter that each pixel receives the same light intensity?
>
> 2. How flat is "flat enough" for this calibration procedure? With a smart
> phone screen set an optimum distance from the bare camera port and carefully
> rotated into position, I can get about 97% uniformity across the whole chip
> by simply by displaying gray scale images. Most of the non-uniformity
> appears at the corners of the chip where I think shadowing from the opening
> in the camera's housing is decreasing the light intensity slightly. The
> calibrations I get from this method allow me to obtain a localization
> precision that I independently measured from sparsely distributed dye
> molecules to be between 8 and 12 nm, which is in line with published STORM
> results. When measuring tiny clusters of proteins, the scatter plots of the
> localizations match the overall shapes of their widefield images quite well.
>
> However, a recent paper by Li et al., J. Innov. Opt. Health Sci. 09,
> 1630008 (2016), doi:10.1142/S1793545816300081, states that one needs better
> than 99% uniformity to avoid introducing significant bias into the noise
> measurements. Furthermore, the engineers at one of the big camera
> manufacturers once told me I shouldn't even bother trying to do the noise
> characterization myself since I wouldn't be able to get the required level
> of uniformity for an accurate characterization. (In fairness, they sell the
> characterization process as a service.)
>
> Unfortunately, I have been unable to find satisfactory answers to these
> questions. So far, my results seem to suggest that my calibration is good
> enough, but I wonder if someone else can offer their input.
>
> Thanks!
> Kyle
>
> --
> Kyle M. Douglass, PhD
> Post-doctoral researcher
> The Laboratory of Experimental Biophysics EPFL, Lausanne, Switzerland
> http://kmdouglass.github.io http://leb.epfl.ch
>
>
>
> --
>
> Kyle M. Douglass, PhD
>
> Post-doctoral researcher
>
> The Laboratory of Experimental Biophysics
>
> EPFL, Lausanne, Switzerland
>
> http://kmdouglass.github.io
>
> http://leb.epfl.ch