Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

[John Oreopoulos]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Earlier today a few people (including myself) brought up Doug Cromey's excellent treatise on digital imaging ethics in a related thread that dealt with training new microscope users within a research setting. Lately I've been hearing a lot about applications of machine learning and artificial intelligence to "improve", "de-noise", or "fix" images (microscopy or otherwise), extracting new information from low-resolution images, and even creating new 3D views of samples with very little information. Here is just one such example from Nvidia and MIT:

https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/

https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo

It's clear that the microscopy world will eventually come to a head with this technology. I think I've seen a few research articles on this topic now, and this month's issue of Nature Methods has a paper on this topic too:

https://www.nature.com/articles/s41592-018-0194-9

I've been wondering if and how Cromey's guide for digital imaging ethics should be altered when it comes to AI-assisted microscope imaging. Should it be allowed/accepted? Other readings of mine on AI show that machine learning algorithms can produce biased results if the training datasets are incomplete in some way, and the very nature of machine learning makes it difficult to understand why it produced a certain result, since the deep learning neural networks that are used to generate the results are essentially black boxes that can't easily be probed. But on the other hand, I'm constantly blown away by what I've seen so far online for other various applications of AI (facial recognition, translation, etc.).

I also just finished a good read about AI from the perspective of economics:

https://www.predictionmachines.ai/

https://youtu.be/5G0PbwtiMJk

The basic message of this book is that AI makes prediction cheap. When something is cheap, we use more of it. Other processes that complement prediction, like judgement (by a human or otherwise) becomes more valuable. It's easy to see how the lessons of this book could be re-framed for imaging science.

Curious to know the community's opinion on this matter. I used to laugh at the following video, but now I'm not laughing:

https://www.youtube.com/watch?v=LhF_56SxrGk

John Oreopoulos

[John Oreopoulos]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Oh yeah, and then there's this too:

https://www.youtube.com/watch?v=gg0F5JjKmhA

Will we need AI to detect AI-manipulated microscopy images?

John Oreopoulos


On 2018-11-16, at 9:32 PM, John Oreopoulos wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Earlier today a few people (including myself) brought up Doug Cromey's excellent treatise on digital imaging ethics in a related thread that dealt with training new microscope users within a research setting. Lately I've been hearing a lot about applications of machine learning and artificial intelligence to "improve", "de-noise", or "fix" images (microscopy or otherwise), extracting new information from low-resolution images, and even creating new 3D views of samples with very little information. Here is just one such example from Nvidia and MIT:
>
> https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
>
> https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo
>
> It's clear that the microscopy world will eventually come to a head with this technology. I think I've seen a few research articles on this topic now, and this month's issue of Nature Methods has a paper on this topic too:
>
> https://www.nature.com/articles/s41592-018-0194-9
>
> I've been wondering if and how Cromey's guide for digital imaging ethics should be altered when it comes to AI-assisted microscope imaging. Should it be allowed/accepted? Other readings of mine on AI show that machine learning algorithms can produce biased results if the training datasets are incomplete in some way, and the very nature of machine learning makes it difficult to understand why it produced a certain result, since the deep learning neural networks that are used to generate the results are essentially black boxes that can't easily be probed. But on the other hand, I'm constantly blown away by what I've seen so far online for other various applications of AI (facial recognition, translation, etc.).
>
> I also just finished a good read about AI from the perspective of economics:
>
> https://www.predictionmachines.ai/
>
> https://youtu.be/5G0PbwtiMJk
>
> The basic message of this book is that AI makes prediction cheap. When something is cheap, we use more of it. Other processes that complement prediction, like judgement (by a human or otherwise) becomes more valuable. It's easy to see how the lessons of this book could be re-framed for imaging science.
>
> Curious to know the community's opinion on this matter. I used to laugh at the following video, but now I'm not laughing:
>
> https://www.youtube.com/watch?v=LhF_56SxrGk
>
> John Oreopoulos

[0000001ed7f52e4a-dmarc-request]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Thanks John for bringing us up-to date on image processing, these are indeed very important developments. I think there will be great changes coming over the next years driven by the AI revolution in image and video processing. But the fundamental limit should is that one cannot increase the information content of an image beyond what was originally recorded. Of course the missing information can be replaced by knowledge derived from other images. But then the new AI algorithms will have similar flaws as human perception (optical illusions). Science should be about measuring accurately instead of guessing.
My criticism of current publications and promotional videos in image processing using AI is that they show the cases when the algorithms actually work well (which might be most of the time), the important question is when do they fail and produce the wrong results? With the fast development in the field, today's problems are often solved a few days later with a new generation of the algorithm, so detecting flaws in these algorithms is an ungrateful task. But I think it is important and we will need to come up with very good quality control standards to accept these results for medical imaging or scientific publications. 
A few years ago I was very excited about using compressed sensing in microscopy to "break the Nyquist barrier", but after looking into this in more detail, I came to the conclusion that this only works well when images are already heavily oversampled like in normal photography (more megapixels sell better). When microscopy data is taken at the resolution limit there is not much room for further compression. I would expect the same for neural network approaches, works well when you have a lot of pixels and the information content is not so high or accuracy is not so important. 
So the question is what is actually necessary for a given experiment? If one wants to track some cells in a fluorescence time laps movie, maybe noisy (low exposure) jpeg compressed data combined with the latest AI algorithm trained on this problem is better than the perfect exposure needed for current segmentation methods and raw data recording, as in the latter case the cells might be killed a short time after the experiment started by the higher light exposure.
best wishes
Andreas



-----Original Message-----
From: John Oreopoulos <[hidden email]>
To: CONFOCALMICROSCOPY <[hidden email]>
Sent: Sat, 17 Nov 2018 2:33
Subject: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Earlier today a few people (including myself) brought up Doug Cromey's excellent treatise on digital imaging ethics in a related thread that dealt with training new microscope users within a research setting. Lately I've been hearing a lot about applications of machine learning and artificial intelligence to "improve", "de-noise", or "fix" images (microscopy or otherwise), extracting new information from low-resolution images, and even creating new 3D views of samples with very little information. Here is just one such example from Nvidia and MIT:

https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/

https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo

It's clear that the microscopy world will eventually come to a head with this technology. I think I've seen a few research articles on this topic now, and this month's issue of Nature Methods has a paper on this topic too:

https://www.nature.com/articles/s41592-018-0194-9

I've been wondering if and how Cromey's guide for digital imaging ethics should be altered when it comes to AI-assisted microscope imaging. Should it be allowed/accepted? Other readings of mine on AI show that machine learning algorithms can produce biased results if the training datasets are incomplete in some way, and the very nature of machine learning makes it difficult to understand why it produced a certain result, since the deep learning neural networks that are used to generate the results are essentially black boxes that can't easily be probed. But on the other hand, I'm constantly blown away by what I've seen so far online for other various applications of AI (facial recognition, translation, etc.).

I also just finished a good read about AI from the perspective of economics:

https://www.predictionmachines.ai/

https://youtu.be/5G0PbwtiMJk

The basic message of this book is that AI makes prediction cheap. When something is cheap, we use more of it. Other processes that complement prediction, like judgement (by a human or otherwise) becomes more valuable. It's easy to see how the lessons of this book could be re-framed for imaging science.

Curious to know the community's opinion on this matter. I used to laugh at the following video, but now I'm not laughing:

https://www.youtube.com/watch?v=LhF_56SxrGk

John Oreopoulos

[Jeremy Adler-4]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

A defence against over manipulation and dubious quantification is to make it an absolute requirement that the original unprocessed images should be provided to any interested party. On the only occasion I requested access to a published image the authors refused and the journal upheld the right of the authors to protect their published image from examination.

However there are difficulties with mandating access:  Leica now offers immediate deconvolution, meaning that the unprocessed images are not necessarily retained, and some techniques can produce overwhelming volumes of data. A related issue is who should be responsible for archiving data - authors, institutions or journals.

Nonetheless the principle that original images should be accessible is an important check.

Jeremy Adler
BioVis
Uppsala U


-----Original Message-----
From: Confocal Microscopy List <[hidden email]> On Behalf Of Andreas Bruckbauer
Sent: den 18 november 2018 12:55
To: [hidden email]
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Thanks John for bringing us up-to date on image processing, these are indeed very important developments. I think there will be great changes coming over the next years driven by the AI revolution in image and video processing. But the fundamental limit should is that one cannot increase the information content of an image beyond what was originally recorded. Of course the missing information can be replaced by knowledge derived from other images. But then the new AI algorithms will have similar flaws as human perception (optical illusions). Science should be about measuring accurately instead of guessing.
My criticism of current publications and promotional videos in image processing using AI is that they show the cases when the algorithms actually work well (which might be most of the time), the important question is when do they fail and produce the wrong results? With the fast development in the field, today's problems are often solved a few days later with a new generation of the algorithm, so detecting flaws in these algorithms is an ungrateful task. But I think it is important and we will need to come up with very good quality control standards to accept these results for medical imaging or scientific publications. A few years ago I was very excited about using compressed sensing in microscopy to "break the Nyquist barrier", but after looking into this in more detail, I came to the conclusion that this only works well when images are already heavily oversampled like in normal photography (more megapixels sell better). When microscopy data is taken at the resolution limit there is not much room for further compression. I would expect the same for neural network approaches, works well when you have a lot of pixels and the information content is not so high or accuracy is not so important. So the question is what is actually necessary for a given experiment? If one wants to track some cells in a fluorescence time laps movie, maybe noisy (low exposure) jpeg compressed data combined with the latest AI algorithm trained on this problem is better than the perfect exposure needed for current segmentation methods and raw data recording, as in the latter case the cells might be killed a short time after the experiment started by the higher light exposure.
best wishes
Andreas



-----Original Message-----
From: John Oreopoulos <[hidden email]>
To: CONFOCALMICROSCOPY <[hidden email]>
Sent: Sat, 17 Nov 2018 2:33
Subject: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Earlier today a few people (including myself) brought up Doug Cromey's excellent treatise on digital imaging ethics in a related thread that dealt with training new microscope users within a research setting. Lately I've been hearing a lot about applications of machine learning and artificial intelligence to "improve", "de-noise", or "fix" images (microscopy or otherwise), extracting new information from low-resolution images, and even creating new 3D views of samples with very little information. Here is just one such example from Nvidia and MIT:

https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/

https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo

It's clear that the microscopy world will eventually come to a head with this technology. I think I've seen a few research articles on this topic now, and this month's issue of Nature Methods has a paper on this topic too:

https://www.nature.com/articles/s41592-018-0194-9

I've been wondering if and how Cromey's guide for digital imaging ethics should be altered when it comes to AI-assisted microscope imaging. Should it be allowed/accepted? Other readings of mine on AI show that machine learning algorithms can produce biased results if the training datasets are incomplete in some way, and the very nature of machine learning makes it difficult to understand why it produced a certain result, since the deep learning neural networks that are used to generate the results are essentially black boxes that can't easily be probed. But on the other hand, I'm constantly blown away by what I've seen so far online for other various applications of AI (facial recognition, translation, etc.).

I also just finished a good read about AI from the perspective of economics:

https://www.predictionmachines.ai/

https://youtu.be/5G0PbwtiMJk

The basic message of this book is that AI makes prediction cheap. When something is cheap, we use more of it. Other processes that complement prediction, like judgement (by a human or otherwise) becomes more valuable. It's easy to see how the lessons of this book could be re-framed for imaging science.

Curious to know the community's opinion on this matter. I used to laugh at the following video, but now I'm not laughing:

https://www.youtube.com/watch?v=LhF_56SxrGk

John Oreopoulos








När du har kontakt med oss på Uppsala universitet med e-post så innebär det att vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du läsa här: http://www.uu.se/om-uu/dataskydd-personuppgifter/

E-mailing Uppsala University means that we will process your personal data. For more information on how this is performed, please read here: http://www.uu.se/en/about-uu/data-protection-policy

[Benjamin Smith]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Here are my two cents, I think the simplest rubric for using an AI
algorithm in science is, "Have I ever seen a paper where this was done by
hand?"

For example, tracking objects, identifying subsets of objects, counting
objects are all the types of image processing tasks that have been done by
hand, and therefore are excellent candidates for AI.  The reason these are
valid is that a human can look at the data and immediately ascertain the
result, but the challenge is in coming up with a computational algorithm
that will robustly define the problem as clearly as we are seeing it.
Whenever I teach people computational image processing, I always point out
that there are certain tasks that are trivial for computers but are
non-trivial for humans (convolutions, counting and measuring thousands of
objects, measuring distances between all pairs of objects, etc.) and tasks
that are often trivial for humans but non-trivial for computers (image
segmentation in a complex feature space, tracking objects in a 2D image
that can cross in 3D space, etc.), and therefore sometimes the best
solution is to write an algorithm that lets the human do the things they
are good at, and the computer does the rest.

To me, the goal of AI in image processing is to replicate what we as
researchers would have done anyway, just in a manner where the computer
does it on its own.  For example, using a well designed convolutional
neural network to learn to segment an image is an excellent application.
You could find an segmentation algorithm by hand, tuning bandpass filters,
trying different convolutions, adjusting both the magnitude and order of
these parameters to get to some lowest energy state.  With convolutional
neural networks, computers can effectively perform this exact same task,
and even in relatively the same manner that we would have, allowing us to
go do something else.  On top of that, they can search a much broader
parameter space than anyone would ever want to do by hand, and therefore
possibly come up with an even more robust algorithm than we could have.
Therefore, if it will likely be faster to build a training dataset than to
try and explore the entire parameter space needed to segment an image, AI
is a good candidate for these tasks.

Using the same rubric I mentioned at the start, I have never seen a paper
where someone showed an artist a bunch of high resolution images, and then
asked them to draw high resolution interpretations of low resolution
images, and then publishing meaningful conclusions from this (for example,
scientific conclusions should not be drawn from artistic renderings of
exoplanets).  The reason we don't do this is that the task is premised on
the assumption that the low resolution images have the same feature space
as the high resolution ones.  Therefore, the only case where the assumption
that the artist's renderings are correct are if our low resolution images
have the same information as the high resolution ones, meaning the low res
images are completely redundant and useless in the sense that they offer no
additional information.

Along these lines, asking a neural network to draw in features in images
based on previous images is not science, because you are forcing the
assumption that all the data matches the hypothesis.  Google's DeepDream is
an excellent case of this, where depending on what the algorithm was
trained on, it will then put those features into any other image.  This is
a great video explaining the impact of using different network layers in
image reconstruction: https://www.youtube.com/watch?v=BsSmBPmPeYQ

Therefore, if you use a neural network in image reconstruction, what you
are really doing is having a computer or artist to draw a high resolution
version of your image that matches your hypothesis.  While this is
inherently not science (as you are changing the data to match your
hypothesis, rather than the other way around), this type of image
reconstruction still has a place in medical science.  For example, a human
body does have a well defined and constrained feature space.  As such, if
you took a low-res CT scan of a human, reconstruction that image based on a
training set of a human would create a meaningful image that could guide
diagnoses, and allow for significantly lower X-ray exposure to the
patient.  Therefore, while AI image reconstruction in science appears to be
a case of circular logic, in medicine it can have very meaningful
applications.

Just my own to cents, and looking forward to hearing other people's
insights,
   Ben Smith

On Sun, Nov 18, 2018 at 3:55 AM Andreas Bruckbauer <
[hidden email]> wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Thanks John for bringing us up-to date on image processing, these are
> indeed very important developments. I think there will be great changes
> coming over the next years driven by the AI revolution in image and video
> processing. But the fundamental limit should is that one cannot increase
> the information content of an image beyond what was originally recorded. Of
> course the missing information can be replaced by knowledge derived from
> other images. But then the new AI algorithms will have similar flaws as
> human perception (optical illusions). Science should be about measuring
> accurately instead of guessing.
> My criticism of current publications and promotional videos in image
> processing using AI is that they show the cases when the algorithms
> actually work well (which might be most of the time), the important
> question is when do they fail and produce the wrong results? With the fast
> development in the field, today's problems are often solved a few days
> later with a new generation of the algorithm, so detecting flaws in these
> algorithms is an ungrateful task. But I think it is important and we will
> need to come up with very good quality control standards to accept these
> results for medical imaging or scientific publications.
> A few years ago I was very excited about using compressed sensing in
> microscopy to "break the Nyquist barrier", but after looking into this in
> more detail, I came to the conclusion that this only works well when images
> are already heavily oversampled like in normal photography (more megapixels
> sell better). When microscopy data is taken at the resolution limit there
> is not much room for further compression. I would expect the same for
> neural network approaches, works well when you have a lot of pixels and the
> information content is not so high or accuracy is not so important.
> So the question is what is actually necessary for a given experiment? If
> one wants to track some cells in a fluorescence time laps movie, maybe
> noisy (low exposure) jpeg compressed data combined with the latest AI
> algorithm trained on this problem is better than the perfect exposure
> needed for current segmentation methods and raw data recording, as in the
> latter case the cells might be killed a short time after the experiment
> started by the higher light exposure.
> best wishes
> Andreas
>
>
>
> -----Original Message-----
> From: John Oreopoulos <[hidden email]>
> To: CONFOCALMICROSCOPY <[hidden email]>
> Sent: Sat, 17 Nov 2018 2:33
> Subject: Digital imaging ethics as pertaining to the enhancement of
> microscopy images with artificial intelligence
>
> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Earlier today a few people (including myself) brought up Doug Cromey's
> excellent treatise on digital imaging ethics in a related thread that dealt
> with training new microscope users within a research setting. Lately I've
> been hearing a lot about applications of machine learning and artificial
> intelligence to "improve", "de-noise", or "fix" images (microscopy or
> otherwise), extracting new information from low-resolution images, and even
> creating new 3D views of samples with very little information. Here is just
> one such example from Nvidia and MIT:
>
>
> https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
>
> https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo
>
> It's clear that the microscopy world will eventually come to a head with
> this technology. I think I've seen a few research articles on this topic
> now, and this month's issue of Nature Methods has a paper on this topic too:
>
> https://www.nature.com/articles/s41592-018-0194-9
>
> I've been wondering if and how Cromey's guide for digital imaging ethics
> should be altered when it comes to AI-assisted microscope imaging. Should
> it be allowed/accepted? Other readings of mine on AI show that machine
> learning algorithms can produce biased results if the training datasets are
> incomplete in some way, and the very nature of machine learning makes it
> difficult to understand why it produced a certain result, since the deep
> learning neural networks that are used to generate the results are
> essentially black boxes that can't easily be probed. But on the other hand,
> I'm constantly blown away by what I've seen so far online for other various
> applications of AI (facial recognition, translation, etc.).
>
> I also just finished a good read about AI from the perspective of
> economics:
>
> https://www.predictionmachines.ai/
>
> https://youtu.be/5G0PbwtiMJk
>
> The basic message of this book is that AI makes prediction cheap. When
> something is cheap, we use more of it. Other processes that complement
> prediction, like judgement (by a human or otherwise) becomes more valuable.
> It's easy to see how the lessons of this book could be re-framed for
> imaging science.
>
> Curious to know the community's opinion on this matter. I used to laugh at
> the following video, but now I'm not laughing:
>
> https://www.youtube.com/watch?v=LhF_56SxrGk
>
> John Oreopoulos
>

--
Benjamin E. Smith, Ph. D.
Imaging Specialist, Vision Science
University of California, Berkeley
195 Life Sciences Addition
Berkeley, CA  94720-3200
Tel  [phone]
Fax [phone]
e-mail: [hidden email]
http://vision.berkeley.edu/?page_id=5635 <http://vision.berkeley.edu/>

[Kirsten Miles]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

I concur with Jeremy Adler, one safety, or "check" to AI-assisted microscopy
is to retain the original pre-processed image. As he states, some research
produces an unwieldy set of data, but more importantly in order for this to be
achievable a many-pronged approach needs to be taken. Support for data-retention
is still incomplete at best. Who is responsible for retention of current data,
of archiving data, especially for a scientist who has moved on, or of insuring
access to data? How many journals that accept data in image form have
instructive policies, and which are adhered to?
Attempts to routinize data integrity through software algorithms have never had
satisfactory results, and attempts to create standards have received little
formal support. Doug Cromey could address AI-assisted microscopy, but until a
set of standards is developed that includes original data retention (and money
to do so follows it) and more journals address the issue on their end, we'll
continue to fall behind as technology enables production without ethical
temperance.
Who is working on standards and digital ethics today?



Kirsten Miles
P.I.  Outcomes434-960-5193
[hidden email]://scienceimageintegrity.org/  





On Sun, Nov 18, 2018 2:16 PM, Jeremy Adler [hidden email]  wrote:
*****

To join, leave or search the confocal microscopy listserv, go to:

http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy 

Post images on http://www.imgur.com and include the link in your posting.

*****

 

A defence against over manipulation and dubious quantification is to make it an
absolute requirement that the original unprocessed images should be provided to
any interested party. On the only occasion I requested access to a published
image the authors refused and the journal upheld the right of the authors to
protect their published image from examination.

 

However there are difficulties with mandating access: Leica now offers immediate
deconvolution, meaning that the unprocessed images are not necessarily retained,
and some techniques can produce overwhelming volumes of data. A related issue is
who should be responsible for archiving data - authors, institutions or
journals.

 

Nonetheless the principle that original images should be accessible is an
important check.

 

Jeremy Adler

BioVis

Uppsala U

 

 

-----Original Message-----

From: Confocal Microscopy List <[hidden email]> On Behalf Of
Andreas Bruckbauer

Sent: den 18 november 2018 12:55

To: [hidden email]

Subject: Re: Digital imaging ethics as pertaining to the enhancement of
microscopy images with artificial intelligence

 

*****

To join, leave or search the confocal microscopy listserv, go to:

http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy 

Post images on http://www.imgur.com and include the link in your posting.

*****

 

Thanks John for bringing us up-to date on image processing, these are indeed
very important developments. I think there will be great changes coming over the
next years driven by the AI revolution in image and video processing. But the
fundamental limit should is that one cannot increase the information content of
an image beyond what was originally recorded. Of course the missing information
can be replaced by knowledge derived from other images. But then the new AI
algorithms will have similar flaws as human perception (optical illusions).
Science should be about measuring accurately instead of guessing.

My criticism of current publications and promotional videos in image processing
using AI is that they show the cases when the algorithms actually work well
(which might be most of the time), the important question is when do they fail
and produce the wrong results? With the fast development in the field, today's
problems are often solved a few days later with a new generation of the
algorithm, so detecting flaws in these algorithms is an ungrateful task. But I
think it is important and we will need to come up with very good quality control
standards to accept these results for medical imaging or scientific
publications. A few years ago I was very excited about using compressed sensing
in microscopy to "break the Nyquist barrier", but after looking into this in
more detail, I came to the conclusion that this only works well when images are
already heavily oversampled like in normal photography (more megapixels sell
better). When microscopy data is taken at the resolution limit there is not much
room for further compression. I would expect the same for neural network
approaches, works well when you have a lot of pixels and the information content
is not so high or accuracy is not so important. So the question is what is
actually necessary for a given experiment? If one wants to track some cells in a
fluorescence time laps movie, maybe noisy (low exposure) jpeg compressed data
combined with the latest AI algorithm trained on this problem is better than the
perfect exposure needed for current segmentation methods and raw data recording,
as in the latter case the cells might be killed a short time after the
experiment started by the higher light exposure.

best wishes

Andreas

 

 

 

-----Original Message-----

From: John Oreopoulos <[hidden email]>

To: CONFOCALMICROSCOPY <[hidden email]>

Sent: Sat, 17 Nov 2018 2:33

Subject: Digital imaging ethics as pertaining to the enhancement of microscopy
images with artificial intelligence

 

*****

To join, leave or search the confocal microscopy listserv, go to:

http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy 

Post images on http://www.imgur.com and include the link in your posting.

*****

 

Earlier today a few people (including myself) brought up Doug Cromey's excellent
treatise on digital imaging ethics in a related thread that dealt with training
new microscope users within a research setting. Lately I've been hearing a lot
about applications of machine learning and artificial intelligence to "improve",
"de-noise", or "fix" images (microscopy or otherwise), extracting new
information from low-resolution images, and even creating new 3D views of
samples with very little information. Here is just one such example from Nvidia
and MIT:

 

https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
 

 

https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo 

 

It's clear that the microscopy world will eventually come to a head with this
technology. I think I've seen a few research articles on this topic now, and
this month's issue of Nature Methods has a paper on this topic too:

 

https://www.nature.com/articles/s41592-018-0194-9 

 

I've been wondering if and how Cromey's guide for digital imaging ethics should
be altered when it comes to AI-assisted microscope imaging. Should it be
allowed/accepted? Other readings of mine on AI show that machine learning
algorithms can produce biased results if the training datasets are incomplete in
some way, and the very nature of machine learning makes it difficult to
understand why it produced a certain result, since the deep learning neural
networks that are used to generate the results are essentially black boxes that
can't easily be probed. But on the other hand, I'm constantly blown away by what
I've seen so far online for other various applications of AI (facial
recognition, translation, etc.).

 

I also just finished a good read about AI from the perspective of economics:

 

https://www.predictionmachines.ai/ 

 

https://youtu.be/5G0PbwtiMJk 

 

The basic message of this book is that AI makes prediction cheap. When something
is cheap, we use more of it. Other processes that complement prediction, like
judgement (by a human or otherwise) becomes more valuable. It's easy to see how
the lessons of this book could be re-framed for imaging science.

 

Curious to know the community's opinion on this matter. I used to laugh at the
following video, but now I'm not laughing:

 

https://www.youtube.com/watch?v=LhF_56SxrGk 

 

John Oreopoulos

 

 

 

 

 

 

 

 

När du har kontakt med oss på Uppsala universitet med e-post så innebär det att
vi behandlar dina personuppgifter. För att läsa mer om hur vi gör det kan du
läsa här: http://www.uu.se/om-uu/dataskydd-personuppgifter/ 

 

E-mailing Uppsala University means that we will process your personal data. For
more information on how this is performed, please read here:
http://www.uu.se/en/about-uu/data-protection-policy

[Ralf Palmisano]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Benjamin, thanks for pointing out all this which I fully agree with!

One more from my side.
How can I ever trust AI if I am exploring terra incognita by imaging,
which quite often is the case, that's why quite a few of us are doing
imaging.
How could (whatever background) AI ever produce data that are not known
yet. How could IT know what is unknown???

Cheers Ralf

Am 18.11.2018 um 23:19 schrieb Benjamin Smith:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Here are my two cents, I think the simplest rubric for using an AI
> algorithm in science is, "Have I ever seen a paper where this was done by
> hand?"
>
> For example, tracking objects, identifying subsets of objects, counting
> objects are all the types of image processing tasks that have been done by
> hand, and therefore are excellent candidates for AI.  The reason these are
> valid is that a human can look at the data and immediately ascertain the
> result, but the challenge is in coming up with a computational algorithm
> that will robustly define the problem as clearly as we are seeing it.
> Whenever I teach people computational image processing, I always point out
> that there are certain tasks that are trivial for computers but are
> non-trivial for humans (convolutions, counting and measuring thousands of
> objects, measuring distances between all pairs of objects, etc.) and tasks
> that are often trivial for humans but non-trivial for computers (image
> segmentation in a complex feature space, tracking objects in a 2D image
> that can cross in 3D space, etc.), and therefore sometimes the best
> solution is to write an algorithm that lets the human do the things they
> are good at, and the computer does the rest.
>
> To me, the goal of AI in image processing is to replicate what we as
> researchers would have done anyway, just in a manner where the computer
> does it on its own.  For example, using a well designed convolutional
> neural network to learn to segment an image is an excellent application.
> You could find an segmentation algorithm by hand, tuning bandpass filters,
> trying different convolutions, adjusting both the magnitude and order of
> these parameters to get to some lowest energy state.  With convolutional
> neural networks, computers can effectively perform this exact same task,
> and even in relatively the same manner that we would have, allowing us to
> go do something else.  On top of that, they can search a much broader
> parameter space than anyone would ever want to do by hand, and therefore
> possibly come up with an even more robust algorithm than we could have.
> Therefore, if it will likely be faster to build a training dataset than to
> try and explore the entire parameter space needed to segment an image, AI
> is a good candidate for these tasks.
>
> Using the same rubric I mentioned at the start, I have never seen a paper
> where someone showed an artist a bunch of high resolution images, and then
> asked them to draw high resolution interpretations of low resolution
> images, and then publishing meaningful conclusions from this (for example,
> scientific conclusions should not be drawn from artistic renderings of
> exoplanets).  The reason we don't do this is that the task is premised on
> the assumption that the low resolution images have the same feature space
> as the high resolution ones.  Therefore, the only case where the assumption
> that the artist's renderings are correct are if our low resolution images
> have the same information as the high resolution ones, meaning the low res
> images are completely redundant and useless in the sense that they offer no
> additional information.
>
> Along these lines, asking a neural network to draw in features in images
> based on previous images is not science, because you are forcing the
> assumption that all the data matches the hypothesis.  Google's DeepDream is
> an excellent case of this, where depending on what the algorithm was
> trained on, it will then put those features into any other image.  This is
> a great video explaining the impact of using different network layers in
> image reconstruction: https://www.youtube.com/watch?v=BsSmBPmPeYQ
>
> Therefore, if you use a neural network in image reconstruction, what you
> are really doing is having a computer or artist to draw a high resolution
> version of your image that matches your hypothesis.  While this is
> inherently not science (as you are changing the data to match your
> hypothesis, rather than the other way around), this type of image
> reconstruction still has a place in medical science.  For example, a human
> body does have a well defined and constrained feature space.  As such, if
> you took a low-res CT scan of a human, reconstruction that image based on a
> training set of a human would create a meaningful image that could guide
> diagnoses, and allow for significantly lower X-ray exposure to the
> patient.  Therefore, while AI image reconstruction in science appears to be
> a case of circular logic, in medicine it can have very meaningful
> applications.
>
> Just my own to cents, and looking forward to hearing other people's
> insights,
>     Ben Smith
>
> On Sun, Nov 18, 2018 at 3:55 AM Andreas Bruckbauer <
> [hidden email]> wrote:
>
>> *****
>> To join, leave or search the confocal microscopy listserv, go to:
>> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
>> Post images on http://www.imgur.com and include the link in your posting.
>> *****
>>
>> Thanks John for bringing us up-to date on image processing, these are
>> indeed very important developments. I think there will be great changes
>> coming over the next years driven by the AI revolution in image and video
>> processing. But the fundamental limit should is that one cannot increase
>> the information content of an image beyond what was originally recorded. Of
>> course the missing information can be replaced by knowledge derived from
>> other images. But then the new AI algorithms will have similar flaws as
>> human perception (optical illusions). Science should be about measuring
>> accurately instead of guessing.
>> My criticism of current publications and promotional videos in image
>> processing using AI is that they show the cases when the algorithms
>> actually work well (which might be most of the time), the important
>> question is when do they fail and produce the wrong results? With the fast
>> development in the field, today's problems are often solved a few days
>> later with a new generation of the algorithm, so detecting flaws in these
>> algorithms is an ungrateful task. But I think it is important and we will
>> need to come up with very good quality control standards to accept these
>> results for medical imaging or scientific publications.
>> A few years ago I was very excited about using compressed sensing in
>> microscopy to "break the Nyquist barrier", but after looking into this in
>> more detail, I came to the conclusion that this only works well when images
>> are already heavily oversampled like in normal photography (more megapixels
>> sell better). When microscopy data is taken at the resolution limit there
>> is not much room for further compression. I would expect the same for
>> neural network approaches, works well when you have a lot of pixels and the
>> information content is not so high or accuracy is not so important.
>> So the question is what is actually necessary for a given experiment? If
>> one wants to track some cells in a fluorescence time laps movie, maybe
>> noisy (low exposure) jpeg compressed data combined with the latest AI
>> algorithm trained on this problem is better than the perfect exposure
>> needed for current segmentation methods and raw data recording, as in the
>> latter case the cells might be killed a short time after the experiment
>> started by the higher light exposure.
>> best wishes
>> Andreas
>>
>>
>>
>> -----Original Message-----
>> From: John Oreopoulos <[hidden email]>
>> To: CONFOCALMICROSCOPY <[hidden email]>
>> Sent: Sat, 17 Nov 2018 2:33
>> Subject: Digital imaging ethics as pertaining to the enhancement of
>> microscopy images with artificial intelligence
>>
>> *****
>> To join, leave or search the confocal microscopy listserv, go to:
>> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
>> Post images on http://www.imgur.com and include the link in your posting.
>> *****
>>
>> Earlier today a few people (including myself) brought up Doug Cromey's
>> excellent treatise on digital imaging ethics in a related thread that dealt
>> with training new microscope users within a research setting. Lately I've
>> been hearing a lot about applications of machine learning and artificial
>> intelligence to "improve", "de-noise", or "fix" images (microscopy or
>> otherwise), extracting new information from low-resolution images, and even
>> creating new 3D views of samples with very little information. Here is just
>> one such example from Nvidia and MIT:
>>
>>
>> https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
>>
>> https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo
>>
>> It's clear that the microscopy world will eventually come to a head with
>> this technology. I think I've seen a few research articles on this topic
>> now, and this month's issue of Nature Methods has a paper on this topic too:
>>
>> https://www.nature.com/articles/s41592-018-0194-9
>>
>> I've been wondering if and how Cromey's guide for digital imaging ethics
>> should be altered when it comes to AI-assisted microscope imaging. Should
>> it be allowed/accepted? Other readings of mine on AI show that machine
>> learning algorithms can produce biased results if the training datasets are
>> incomplete in some way, and the very nature of machine learning makes it
>> difficult to understand why it produced a certain result, since the deep
>> learning neural networks that are used to generate the results are
>> essentially black boxes that can't easily be probed. But on the other hand,
>> I'm constantly blown away by what I've seen so far online for other various
>> applications of AI (facial recognition, translation, etc.).
>>
>> I also just finished a good read about AI from the perspective of
>> economics:
>>
>> https://www.predictionmachines.ai/
>>
>> https://youtu.be/5G0PbwtiMJk
>>
>> The basic message of this book is that AI makes prediction cheap. When
>> something is cheap, we use more of it. Other processes that complement
>> prediction, like judgement (by a human or otherwise) becomes more valuable.
>> It's easy to see how the lessons of this book could be re-framed for
>> imaging science.
>>
>> Curious to know the community's opinion on this matter. I used to laugh at
>> the following video, but now I'm not laughing:
>>
>> https://www.youtube.com/watch?v=LhF_56SxrGk
>>
>> John Oreopoulos
>>
>

--
Ralf Palmisano
Head - Optical Imaging Centre Erlangen

Fellow Royal Microscopical Society
Member Royal Society of Medicine

Speaker Scientific Advisory Board "German Society for Microscopy and Image Analysis"
Board of Directors Core Technologies for Life Science

Hartmannstr. 14
91052 Erlangen, Germany

[phone] (Office)
[phone] (Secretary)
[phone] (Fax)

www.oice.uni-erlangen.de

[0000001ed7f52e4a-dmarc-request]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Ralf, AI will one day be able to discover the “unknown” just as humans do, knowing their field, looking out for things which do not fit into the current theoretical framework, forming a new hypothesis, doing experiments and modifying the hypothesis.... the robot scientist is waiting to replace us, obviously there is long way to go
http://aiweirdness.com/post/172894792687/when-algorithms-surprise-us

Andreas



Sent from my phone

> On 19 Nov 2018, at 10:59, Ralf Palmisano OICE <[hidden email]> wrote:
>
> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Benjamin, thanks for pointing out all this which I fully agree with!
>
> One more from my side.
> How can I ever trust AI if I am exploring terra incognita by imaging, which quite often is the case, that's why quite a few of us are doing imaging.
> How could (whatever background) AI ever produce data that are not known yet. How could IT know what is unknown???
>
> Cheers Ralf
>
>> Am 18.11.2018 um 23:19 schrieb Benjamin Smith:
>> *****
>> To join, leave or search the confocal microscopy listserv, go to:
>> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
>> Post images on http://www.imgur.com and include the link in your posting.
>> *****
>>
>> Here are my two cents, I think the simplest rubric for using an AI
>> algorithm in science is, "Have I ever seen a paper where this was done by
>> hand?"
>>
>> For example, tracking objects, identifying subsets of objects, counting
>> objects are all the types of image processing tasks that have been done by
>> hand, and therefore are excellent candidates for AI.  The reason these are
>> valid is that a human can look at the data and immediately ascertain the
>> result, but the challenge is in coming up with a computational algorithm
>> that will robustly define the problem as clearly as we are seeing it.
>> Whenever I teach people computational image processing, I always point out
>> that there are certain tasks that are trivial for computers but are
>> non-trivial for humans (convolutions, counting and measuring thousands of
>> objects, measuring distances between all pairs of objects, etc.) and tasks
>> that are often trivial for humans but non-trivial for computers (image
>> segmentation in a complex feature space, tracking objects in a 2D image
>> that can cross in 3D space, etc.), and therefore sometimes the best
>> solution is to write an algorithm that lets the human do the things they
>> are good at, and the computer does the rest.
>>
>> To me, the goal of AI in image processing is to replicate what we as
>> researchers would have done anyway, just in a manner where the computer
>> does it on its own.  For example, using a well designed convolutional
>> neural network to learn to segment an image is an excellent application.
>> You could find an segmentation algorithm by hand, tuning bandpass filters,
>> trying different convolutions, adjusting both the magnitude and order of
>> these parameters to get to some lowest energy state.  With convolutional
>> neural networks, computers can effectively perform this exact same task,
>> and even in relatively the same manner that we would have, allowing us to
>> go do something else.  On top of that, they can search a much broader
>> parameter space than anyone would ever want to do by hand, and therefore
>> possibly come up with an even more robust algorithm than we could have.
>> Therefore, if it will likely be faster to build a training dataset than to
>> try and explore the entire parameter space needed to segment an image, AI
>> is a good candidate for these tasks.
>>
>> Using the same rubric I mentioned at the start, I have never seen a paper
>> where someone showed an artist a bunch of high resolution images, and then
>> asked them to draw high resolution interpretations of low resolution
>> images, and then publishing meaningful conclusions from this (for example,
>> scientific conclusions should not be drawn from artistic renderings of
>> exoplanets).  The reason we don't do this is that the task is premised on
>> the assumption that the low resolution images have the same feature space
>> as the high resolution ones.  Therefore, the only case where the assumption
>> that the artist's renderings are correct are if our low resolution images
>> have the same information as the high resolution ones, meaning the low res
>> images are completely redundant and useless in the sense that they offer no
>> additional information.
>>
>> Along these lines, asking a neural network to draw in features in images
>> based on previous images is not science, because you are forcing the
>> assumption that all the data matches the hypothesis.  Google's DeepDream is
>> an excellent case of this, where depending on what the algorithm was
>> trained on, it will then put those features into any other image.  This is
>> a great video explaining the impact of using different network layers in
>> image reconstruction: https://www.youtube.com/watch?v=BsSmBPmPeYQ
>>
>> Therefore, if you use a neural network in image reconstruction, what you
>> are really doing is having a computer or artist to draw a high resolution
>> version of your image that matches your hypothesis.  While this is
>> inherently not science (as you are changing the data to match your
>> hypothesis, rather than the other way around), this type of image
>> reconstruction still has a place in medical science.  For example, a human
>> body does have a well defined and constrained feature space.  As such, if
>> you took a low-res CT scan of a human, reconstruction that image based on a
>> training set of a human would create a meaningful image that could guide
>> diagnoses, and allow for significantly lower X-ray exposure to the
>> patient.  Therefore, while AI image reconstruction in science appears to be
>> a case of circular logic, in medicine it can have very meaningful
>> applications.
>>
>> Just my own to cents, and looking forward to hearing other people's
>> insights,
>>    Ben Smith
>>
>> On Sun, Nov 18, 2018 at 3:55 AM Andreas Bruckbauer <
>> [hidden email]> wrote:
>>
>>> *****
>>> To join, leave or search the confocal microscopy listserv, go to:
>>> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
>>> Post images on http://www.imgur.com and include the link in your posting.
>>> *****
>>>
>>> Thanks John for bringing us up-to date on image processing, these are
>>> indeed very important developments. I think there will be great changes
>>> coming over the next years driven by the AI revolution in image and video
>>> processing. But the fundamental limit should is that one cannot increase
>>> the information content of an image beyond what was originally recorded. Of
>>> course the missing information can be replaced by knowledge derived from
>>> other images. But then the new AI algorithms will have similar flaws as
>>> human perception (optical illusions). Science should be about measuring
>>> accurately instead of guessing.
>>> My criticism of current publications and promotional videos in image
>>> processing using AI is that they show the cases when the algorithms
>>> actually work well (which might be most of the time), the important
>>> question is when do they fail and produce the wrong results? With the fast
>>> development in the field, today's problems are often solved a few days
>>> later with a new generation of the algorithm, so detecting flaws in these
>>> algorithms is an ungrateful task. But I think it is important and we will
>>> need to come up with very good quality control standards to accept these
>>> results for medical imaging or scientific publications.
>>> A few years ago I was very excited about using compressed sensing in
>>> microscopy to "break the Nyquist barrier", but after looking into this in
>>> more detail, I came to the conclusion that this only works well when images
>>> are already heavily oversampled like in normal photography (more megapixels
>>> sell better). When microscopy data is taken at the resolution limit there
>>> is not much room for further compression. I would expect the same for
>>> neural network approaches, works well when you have a lot of pixels and the
>>> information content is not so high or accuracy is not so important.
>>> So the question is what is actually necessary for a given experiment? If
>>> one wants to track some cells in a fluorescence time laps movie, maybe
>>> noisy (low exposure) jpeg compressed data combined with the latest AI
>>> algorithm trained on this problem is better than the perfect exposure
>>> needed for current segmentation methods and raw data recording, as in the
>>> latter case the cells might be killed a short time after the experiment
>>> started by the higher light exposure.
>>> best wishes
>>> Andreas
>>>
>>>
>>>
>>> -----Original Message-----
>>> From: John Oreopoulos <[hidden email]>
>>> To: CONFOCALMICROSCOPY <[hidden email]>
>>> Sent: Sat, 17 Nov 2018 2:33
>>> Subject: Digital imaging ethics as pertaining to the enhancement of
>>> microscopy images with artificial intelligence
>>>
>>> *****
>>> To join, leave or search the confocal microscopy listserv, go to:
>>> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
>>> Post images on http://www.imgur.com and include the link in your posting.
>>> *****
>>>
>>> Earlier today a few people (including myself) brought up Doug Cromey's
>>> excellent treatise on digital imaging ethics in a related thread that dealt
>>> with training new microscope users within a research setting. Lately I've
>>> been hearing a lot about applications of machine learning and artificial
>>> intelligence to "improve", "de-noise", or "fix" images (microscopy or
>>> otherwise), extracting new information from low-resolution images, and even
>>> creating new 3D views of samples with very little information. Here is just
>>> one such example from Nvidia and MIT:
>>>
>>>
>>> https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
>>>
>>> https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo
>>>
>>> It's clear that the microscopy world will eventually come to a head with
>>> this technology. I think I've seen a few research articles on this topic
>>> now, and this month's issue of Nature Methods has a paper on this topic too:
>>>
>>> https://www.nature.com/articles/s41592-018-0194-9
>>>
>>> I've been wondering if and how Cromey's guide for digital imaging ethics
>>> should be altered when it comes to AI-assisted microscope imaging. Should
>>> it be allowed/accepted? Other readings of mine on AI show that machine
>>> learning algorithms can produce biased results if the training datasets are
>>> incomplete in some way, and the very nature of machine learning makes it
>>> difficult to understand why it produced a certain result, since the deep
>>> learning neural networks that are used to generate the results are
>>> essentially black boxes that can't easily be probed. But on the other hand,
>>> I'm constantly blown away by what I've seen so far online for other various
>>> applications of AI (facial recognition, translation, etc.).
>>>
>>> I also just finished a good read about AI from the perspective of
>>> economics:
>>>
>>> https://www.predictionmachines.ai/
>>>
>>> https://youtu.be/5G0PbwtiMJk
>>>
>>> The basic message of this book is that AI makes prediction cheap. When
>>> something is cheap, we use more of it. Other processes that complement
>>> prediction, like judgement (by a human or otherwise) becomes more valuable.
>>> It's easy to see how the lessons of this book could be re-framed for
>>> imaging science.
>>>
>>> Curious to know the community's opinion on this matter. I used to laugh at
>>> the following video, but now I'm not laughing:
>>>
>>> https://www.youtube.com/watch?v=LhF_56SxrGk
>>>
>>> John Oreopoulos
>>>
>>
>
> --
> Ralf Palmisano
> Head - Optical Imaging Centre Erlangen
>
> Fellow Royal Microscopical Society
> Member Royal Society of Medicine
>
> Speaker Scientific Advisory Board "German Society for Microscopy and Image Analysis"
> Board of Directors Core Technologies for Life Science
>
> Hartmannstr. 14
> 91052 Erlangen, Germany
>
> [phone] (Office)
> [phone] (Secretary)
> [phone] (Fax)
>
> www.oice.uni-erlangen.de

[Ralf Palmisano]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Andreas,

I agree in the broader picture, but I do think that will be within my
estimated time of departure as a carbon based "intelligent" life form on
this planet... :-)
And in remembrance of Philip K. Dick:
"Everything in life is just for a while."

Cheers
Ralf

Am 19.11.2018 um 14:38 schrieb Andreas Bruckbauer:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Ralf, AI will one day be able to discover the “unknown” just as humans do, knowing their field, looking out for things which do not fit into the current theoretical framework, forming a new hypothesis, doing experiments and modifying the hypothesis.... the robot scientist is waiting to replace us, obviously there is long way to go
> http://aiweirdness.com/post/172894792687/when-algorithms-surprise-us
>
> Andreas
>
>
>
> Sent from my phone
>
>> On 19 Nov 2018, at 10:59, Ralf Palmisano OICE <[hidden email]> wrote:
>>
>> *****
>> To join, leave or search the confocal microscopy listserv, go to:
>> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
>> Post images on http://www.imgur.com and include the link in your posting.
>> *****
>>
>> Benjamin, thanks for pointing out all this which I fully agree with!
>>
>> One more from my side.
>> How can I ever trust AI if I am exploring terra incognita by imaging, which quite often is the case, that's why quite a few of us are doing imaging.
>> How could (whatever background) AI ever produce data that are not known yet. How could IT know what is unknown???
>>
>> Cheers Ralf
>>
>>> Am 18.11.2018 um 23:19 schrieb Benjamin Smith:
>>> *****
>>> To join, leave or search the confocal microscopy listserv, go to:
>>> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
>>> Post images on http://www.imgur.com and include the link in your posting.
>>> *****
>>>
>>> Here are my two cents, I think the simplest rubric for using an AI
>>> algorithm in science is, "Have I ever seen a paper where this was done by
>>> hand?"
>>>
>>> For example, tracking objects, identifying subsets of objects, counting
>>> objects are all the types of image processing tasks that have been done by
>>> hand, and therefore are excellent candidates for AI.  The reason these are
>>> valid is that a human can look at the data and immediately ascertain the
>>> result, but the challenge is in coming up with a computational algorithm
>>> that will robustly define the problem as clearly as we are seeing it.
>>> Whenever I teach people computational image processing, I always point out
>>> that there are certain tasks that are trivial for computers but are
>>> non-trivial for humans (convolutions, counting and measuring thousands of
>>> objects, measuring distances between all pairs of objects, etc.) and tasks
>>> that are often trivial for humans but non-trivial for computers (image
>>> segmentation in a complex feature space, tracking objects in a 2D image
>>> that can cross in 3D space, etc.), and therefore sometimes the best
>>> solution is to write an algorithm that lets the human do the things they
>>> are good at, and the computer does the rest.
>>>
>>> To me, the goal of AI in image processing is to replicate what we as
>>> researchers would have done anyway, just in a manner where the computer
>>> does it on its own.  For example, using a well designed convolutional
>>> neural network to learn to segment an image is an excellent application.
>>> You could find an segmentation algorithm by hand, tuning bandpass filters,
>>> trying different convolutions, adjusting both the magnitude and order of
>>> these parameters to get to some lowest energy state.  With convolutional
>>> neural networks, computers can effectively perform this exact same task,
>>> and even in relatively the same manner that we would have, allowing us to
>>> go do something else.  On top of that, they can search a much broader
>>> parameter space than anyone would ever want to do by hand, and therefore
>>> possibly come up with an even more robust algorithm than we could have.
>>> Therefore, if it will likely be faster to build a training dataset than to
>>> try and explore the entire parameter space needed to segment an image, AI
>>> is a good candidate for these tasks.
>>>
>>> Using the same rubric I mentioned at the start, I have never seen a paper
>>> where someone showed an artist a bunch of high resolution images, and then
>>> asked them to draw high resolution interpretations of low resolution
>>> images, and then publishing meaningful conclusions from this (for example,
>>> scientific conclusions should not be drawn from artistic renderings of
>>> exoplanets).  The reason we don't do this is that the task is premised on
>>> the assumption that the low resolution images have the same feature space
>>> as the high resolution ones.  Therefore, the only case where the assumption
>>> that the artist's renderings are correct are if our low resolution images
>>> have the same information as the high resolution ones, meaning the low res
>>> images are completely redundant and useless in the sense that they offer no
>>> additional information.
>>>
>>> Along these lines, asking a neural network to draw in features in images
>>> based on previous images is not science, because you are forcing the
>>> assumption that all the data matches the hypothesis.  Google's DeepDream is
>>> an excellent case of this, where depending on what the algorithm was
>>> trained on, it will then put those features into any other image.  This is
>>> a great video explaining the impact of using different network layers in
>>> image reconstruction: https://www.youtube.com/watch?v=BsSmBPmPeYQ
>>>
>>> Therefore, if you use a neural network in image reconstruction, what you
>>> are really doing is having a computer or artist to draw a high resolution
>>> version of your image that matches your hypothesis.  While this is
>>> inherently not science (as you are changing the data to match your
>>> hypothesis, rather than the other way around), this type of image
>>> reconstruction still has a place in medical science.  For example, a human
>>> body does have a well defined and constrained feature space.  As such, if
>>> you took a low-res CT scan of a human, reconstruction that image based on a
>>> training set of a human would create a meaningful image that could guide
>>> diagnoses, and allow for significantly lower X-ray exposure to the
>>> patient.  Therefore, while AI image reconstruction in science appears to be
>>> a case of circular logic, in medicine it can have very meaningful
>>> applications.
>>>
>>> Just my own to cents, and looking forward to hearing other people's
>>> insights,
>>>     Ben Smith
>>>
>>> On Sun, Nov 18, 2018 at 3:55 AM Andreas Bruckbauer <
>>> [hidden email]> wrote:
>>>
>>>> *****
>>>> To join, leave or search the confocal microscopy listserv, go to:
>>>> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
>>>> Post images on http://www.imgur.com and include the link in your posting.
>>>> *****
>>>>
>>>> Thanks John for bringing us up-to date on image processing, these are
>>>> indeed very important developments. I think there will be great changes
>>>> coming over the next years driven by the AI revolution in image and video
>>>> processing. But the fundamental limit should is that one cannot increase
>>>> the information content of an image beyond what was originally recorded. Of
>>>> course the missing information can be replaced by knowledge derived from
>>>> other images. But then the new AI algorithms will have similar flaws as
>>>> human perception (optical illusions). Science should be about measuring
>>>> accurately instead of guessing.
>>>> My criticism of current publications and promotional videos in image
>>>> processing using AI is that they show the cases when the algorithms
>>>> actually work well (which might be most of the time), the important
>>>> question is when do they fail and produce the wrong results? With the fast
>>>> development in the field, today's problems are often solved a few days
>>>> later with a new generation of the algorithm, so detecting flaws in these
>>>> algorithms is an ungrateful task. But I think it is important and we will
>>>> need to come up with very good quality control standards to accept these
>>>> results for medical imaging or scientific publications.
>>>> A few years ago I was very excited about using compressed sensing in
>>>> microscopy to "break the Nyquist barrier", but after looking into this in
>>>> more detail, I came to the conclusion that this only works well when images
>>>> are already heavily oversampled like in normal photography (more megapixels
>>>> sell better). When microscopy data is taken at the resolution limit there
>>>> is not much room for further compression. I would expect the same for
>>>> neural network approaches, works well when you have a lot of pixels and the
>>>> information content is not so high or accuracy is not so important.
>>>> So the question is what is actually necessary for a given experiment? If
>>>> one wants to track some cells in a fluorescence time laps movie, maybe
>>>> noisy (low exposure) jpeg compressed data combined with the latest AI
>>>> algorithm trained on this problem is better than the perfect exposure
>>>> needed for current segmentation methods and raw data recording, as in the
>>>> latter case the cells might be killed a short time after the experiment
>>>> started by the higher light exposure.
>>>> best wishes
>>>> Andreas
>>>>
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: John Oreopoulos <[hidden email]>
>>>> To: CONFOCALMICROSCOPY <[hidden email]>
>>>> Sent: Sat, 17 Nov 2018 2:33
>>>> Subject: Digital imaging ethics as pertaining to the enhancement of
>>>> microscopy images with artificial intelligence
>>>>
>>>> *****
>>>> To join, leave or search the confocal microscopy listserv, go to:
>>>> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
>>>> Post images on http://www.imgur.com and include the link in your posting.
>>>> *****
>>>>
>>>> Earlier today a few people (including myself) brought up Doug Cromey's
>>>> excellent treatise on digital imaging ethics in a related thread that dealt
>>>> with training new microscope users within a research setting. Lately I've
>>>> been hearing a lot about applications of machine learning and artificial
>>>> intelligence to "improve", "de-noise", or "fix" images (microscopy or
>>>> otherwise), extracting new information from low-resolution images, and even
>>>> creating new 3D views of samples with very little information. Here is just
>>>> one such example from Nvidia and MIT:
>>>>
>>>>
>>>> https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
>>>>
>>>> https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo
>>>>
>>>> It's clear that the microscopy world will eventually come to a head with
>>>> this technology. I think I've seen a few research articles on this topic
>>>> now, and this month's issue of Nature Methods has a paper on this topic too:
>>>>
>>>> https://www.nature.com/articles/s41592-018-0194-9
>>>>
>>>> I've been wondering if and how Cromey's guide for digital imaging ethics
>>>> should be altered when it comes to AI-assisted microscope imaging. Should
>>>> it be allowed/accepted? Other readings of mine on AI show that machine
>>>> learning algorithms can produce biased results if the training datasets are
>>>> incomplete in some way, and the very nature of machine learning makes it
>>>> difficult to understand why it produced a certain result, since the deep
>>>> learning neural networks that are used to generate the results are
>>>> essentially black boxes that can't easily be probed. But on the other hand,
>>>> I'm constantly blown away by what I've seen so far online for other various
>>>> applications of AI (facial recognition, translation, etc.).
>>>>
>>>> I also just finished a good read about AI from the perspective of
>>>> economics:
>>>>
>>>> https://www.predictionmachines.ai/
>>>>
>>>> https://youtu.be/5G0PbwtiMJk
>>>>
>>>> The basic message of this book is that AI makes prediction cheap. When
>>>> something is cheap, we use more of it. Other processes that complement
>>>> prediction, like judgement (by a human or otherwise) becomes more valuable.
>>>> It's easy to see how the lessons of this book could be re-framed for
>>>> imaging science.
>>>>
>>>> Curious to know the community's opinion on this matter. I used to laugh at
>>>> the following video, but now I'm not laughing:
>>>>
>>>> https://www.youtube.com/watch?v=LhF_56SxrGk
>>>>
>>>> John Oreopoulos
>>>>
>> --
>> Ralf Palmisano
>> Head - Optical Imaging Centre Erlangen
>>
>> Fellow Royal Microscopical Society
>> Member Royal Society of Medicine
>>
>> Speaker Scientific Advisory Board "German Society for Microscopy and Image Analysis"
>> Board of Directors Core Technologies for Life Science
>>
>> Hartmannstr. 14
>> 91052 Erlangen, Germany
>>
>> [phone] (Office)
>> [phone] (Secretary)
>> [phone] (Fax)
>>
>> www.oice.uni-erlangen.de

--
Ralf Palmisano
Head - Optical Imaging Centre Erlangen

Fellow Royal Microscopical Society
Member Royal Society of Medicine

Speaker Scientific Advisory Board "German Society for Microscopy and Image Analysis"
Board of Directors Core Technologies for Life Science

Hartmannstr. 14
91052 Erlangen, Germany

[phone] (Office)
[phone] (Secretary)
[phone] (Fax)

www.oice.uni-erlangen.de

[Oshel, Philip Eugene]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

John,

You forgot this clip:
https://www.youtube.com/watch?v=WwnI0RS6J5A

Phil
-------------
Philip Oshel    
Imaging Facility Director
Biology Department
1304 Biosciences
1455 Calumet Ct.
Central Michigan University
Mt. Pleasant, MI 48859
989 774-3576 office
989 774-7567 lab

-----Original Message-----
From: Confocal Microscopy List <[hidden email]> on behalf of John Oreopoulos <[hidden email]>
Reply-To: Confocal Microscopy List <[hidden email]>
Date: Friday,  16November, 2018 at 21:33
To: "[hidden email]" <[hidden email]>
Subject: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

    *****
    To join, leave or search the confocal microscopy listserv, go to:
    http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    Post images on http://www.imgur.com and include the link in your posting.
    *****
   
    Earlier today a few people (including myself) brought up Doug Cromey's excellent treatise on digital imaging ethics in a related thread that dealt with training new microscope users within a research setting. Lately I've been hearing a lot about applications of machine learning and artificial intelligence to "improve", "de-noise", or "fix" images (microscopy or otherwise), extracting new information from low-resolution images, and even creating new 3D views of samples with very little information. Here is just one such example from Nvidia and MIT:
   
    https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
   
    https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo
   
    It's clear that the microscopy world will eventually come to a head with this technology. I think I've seen a few research articles on this topic now, and this month's issue of Nature Methods has a paper on this topic too:
   
    https://www.nature.com/articles/s41592-018-0194-9
   
    I've been wondering if and how Cromey's guide for digital imaging ethics should be altered when it comes to AI-assisted microscope imaging. Should it be allowed/accepted? Other readings of mine on AI show that machine learning algorithms can produce biased results if the training datasets are incomplete in some way, and the very nature of machine learning makes it difficult to understand why it produced a certain result, since the deep learning neural networks that are used to generate the results are essentially black boxes that can't easily be probed. But on the other hand, I'm constantly blown away by what I've seen so far online for other various applications of AI (facial recognition, translation, etc.).
   
    I also just finished a good read about AI from the perspective of economics:
   
    https://www.predictionmachines.ai/
   
    https://youtu.be/5G0PbwtiMJk
   
    The basic message of this book is that AI makes prediction cheap. When something is cheap, we use more of it. Other processes that complement prediction, like judgement (by a human or otherwise) becomes more valuable. It's easy to see how the lessons of this book could be re-framed for imaging science.
   
    Curious to know the community's opinion on this matter. I used to laugh at the following video, but now I'm not laughing:
   
    https://www.youtube.com/watch?v=LhF_56SxrGk
   
    John Oreopoulos
   

[Jason Swedlow-2]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi All-

On a more serious note (although any comment that cites The Simpsons is an
excellent one), see https://www.nature.com/articles/s41592-018-0195-8 for
an update and proposal for image data publication resources.

Cheers,

Jason

On Mon, Nov 19, 2018 at 7:25 PM Oshel, Philip Eugene <[hidden email]>
wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> John,
>
> You forgot this clip:
> https://www.youtube.com/watch?v=WwnI0RS6J5A
>
> Phil
> -------------
> Philip Oshel
> Imaging Facility Director
> Biology Department
> 1304 Biosciences
> 1455 Calumet Ct.
> Central Michigan University
> Mt. Pleasant, MI 48859
> 989 774-3576 office
> 989 774-7567 lab
>
> -----Original Message-----
> From: Confocal Microscopy List <[hidden email]> on
> behalf of John Oreopoulos <[hidden email]>
> Reply-To: Confocal Microscopy List <[hidden email]>
> Date: Friday,  16November, 2018 at 21:33
> To: "[hidden email]" <[hidden email]>
> Subject: Digital imaging ethics as pertaining to the enhancement of
> microscopy images with artificial intelligence
>
>     *****
>     To join, leave or search the confocal microscopy listserv, go to:
>     http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
>     Post images on http://www.imgur.com and include the link in your
> posting.
>     *****
>
>     Earlier today a few people (including myself) brought up Doug Cromey's
> excellent treatise on digital imaging ethics in a related thread that dealt
> with training new microscope users within a research setting. Lately I've
> been hearing a lot about applications of machine learning and artificial
> intelligence to "improve", "de-noise", or "fix" images (microscopy or
> otherwise), extracting new information from low-resolution images, and even
> creating new 3D views of samples with very little information. Here is just
> one such example from Nvidia and MIT:
>
>
> https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
>
>     https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo
>
>     It's clear that the microscopy world will eventually come to a head
> with this technology. I think I've seen a few research articles on this
> topic now, and this month's issue of Nature Methods has a paper on this
> topic too:
>
>     https://www.nature.com/articles/s41592-018-0194-9
>
>     I've been wondering if and how Cromey's guide for digital imaging
> ethics should be altered when it comes to AI-assisted microscope imaging.
> Should it be allowed/accepted? Other readings of mine on AI show that
> machine learning algorithms can produce biased results if the training
> datasets are incomplete in some way, and the very nature of machine
> learning makes it difficult to understand why it produced a certain result,
> since the deep learning neural networks that are used to generate the
> results are essentially black boxes that can't easily be probed. But on the
> other hand, I'm constantly blown away by what I've seen so far online for
> other various applications of AI (facial recognition, translation, etc.).
>
>     I also just finished a good read about AI from the perspective of
> economics:
>
>     https://www.predictionmachines.ai/
>
>     https://youtu.be/5G0PbwtiMJk
>
>     The basic message of this book is that AI makes prediction cheap. When
> something is cheap, we use more of it. Other processes that complement
> prediction, like judgement (by a human or otherwise) becomes more valuable.
> It's easy to see how the lessons of this book could be re-framed for
> imaging science.
>
>     Curious to know the community's opinion on this matter. I used to
> laugh at the following video, but now I'm not laughing:
>
>     https://www.youtube.com/watch?v=LhF_56SxrGk
>
>     John Oreopoulos
>
>
>

--
**************************
Centre for Gene Regulation & Expression
School of Life Sciences
University of Dundee
Dundee  DD1 5EH
United Kingdom

phone [phone]
Intl phone:  [phone]
FAX   [phone]
email: [hidden email]

Lab Page: http://www.lifesci.dundee.ac.uk/people/jason-swedlow
Open Microscopy Environment: http://openmicroscopy.org
**************************

[Oshel, Philip Eugene]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Good reference, thanks!

Note: I would like to publish an article (or two) on this issue in Microscopy Today. If someone has something they'd like to contribute, please contact me.

Phil
P.S. The clip was from Futurama.
-------------
Philip Oshel    
Technical Editor, Microscopy Today
Imaging Facility Director
Biology Department
1304 Biosciences
1455 Calumet Ct.
Central Michigan University
Mt. Pleasant, MI 48859
[phone]
www(dot)microscopy-today(dot)com

-----Original Message-----
From: Confocal Microscopy List <[hidden email]> on behalf of Jason Swedlow <[hidden email]>
Reply-To: Confocal Microscopy List <[hidden email]>
Date: Monday,  19November, 2018 at 19:16
To: "[hidden email]" <[hidden email]>
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

    *****
    To join, leave or search the confocal microscopy listserv, go to:
    http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    Post images on http://www.imgur.com and include the link in your posting.
    *****
   
    Hi All-
   
    On a more serious note (although any comment that cites The Simpsons is an
    excellent one), see https://www.nature.com/articles/s41592-018-0195-8 for
    an update and proposal for image data publication resources.
   
    Cheers,
   
    Jason
   
    On Mon, Nov 19, 2018 at 7:25 PM Oshel, Philip Eugene <[hidden email]>
    wrote:
   
    > *****
    > To join, leave or search the confocal microscopy listserv, go to:
    > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    > Post images on http://www.imgur.com and include the link in your posting.
    > *****
    >
    > John,
    >
    > You forgot this clip:
    > https://www.youtube.com/watch?v=WwnI0RS6J5A
    >
    > Phil
    > -------------
    > Philip Oshel
    > Imaging Facility Director
    > Biology Department
    > 1304 Biosciences
    > 1455 Calumet Ct.
    > Central Michigan University
    > Mt. Pleasant, MI 48859
    > 989 774-3576 office
    > 989 774-7567 lab
    >
    > -----Original Message-----
    > From: Confocal Microscopy List <[hidden email]> on
    > behalf of John Oreopoulos <[hidden email]>
    > Reply-To: Confocal Microscopy List <[hidden email]>
    > Date: Friday,  16November, 2018 at 21:33
    > To: "[hidden email]" <[hidden email]>
    > Subject: Digital imaging ethics as pertaining to the enhancement of
    > microscopy images with artificial intelligence
    >
    >     *****
    >     To join, leave or search the confocal microscopy listserv, go to:
    >     http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    >     Post images on http://www.imgur.com and include the link in your
    > posting.
    >     *****
    >
    >     Earlier today a few people (including myself) brought up Doug Cromey's
    > excellent treatise on digital imaging ethics in a related thread that dealt
    > with training new microscope users within a research setting. Lately I've
    > been hearing a lot about applications of machine learning and artificial
    > intelligence to "improve", "de-noise", or "fix" images (microscopy or
    > otherwise), extracting new information from low-resolution images, and even
    > creating new 3D views of samples with very little information. Here is just
    > one such example from Nvidia and MIT:
    >
    >
    > https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
    >
    >     https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo
    >
    >     It's clear that the microscopy world will eventually come to a head
    > with this technology. I think I've seen a few research articles on this
    > topic now, and this month's issue of Nature Methods has a paper on this
    > topic too:
    >
    >     https://www.nature.com/articles/s41592-018-0194-9
    >
    >     I've been wondering if and how Cromey's guide for digital imaging
    > ethics should be altered when it comes to AI-assisted microscope imaging.
    > Should it be allowed/accepted? Other readings of mine on AI show that
    > machine learning algorithms can produce biased results if the training
    > datasets are incomplete in some way, and the very nature of machine
    > learning makes it difficult to understand why it produced a certain result,
    > since the deep learning neural networks that are used to generate the
    > results are essentially black boxes that can't easily be probed. But on the
    > other hand, I'm constantly blown away by what I've seen so far online for
    > other various applications of AI (facial recognition, translation, etc.).
    >
    >     I also just finished a good read about AI from the perspective of
    > economics:
    >
    >     https://www.predictionmachines.ai/
    >
    >     https://youtu.be/5G0PbwtiMJk
    >
    >     The basic message of this book is that AI makes prediction cheap. When
    > something is cheap, we use more of it. Other processes that complement
    > prediction, like judgement (by a human or otherwise) becomes more valuable.
    > It's easy to see how the lessons of this book could be re-framed for
    > imaging science.
    >
    >     Curious to know the community's opinion on this matter. I used to
    > laugh at the following video, but now I'm not laughing:
    >
    >     https://www.youtube.com/watch?v=LhF_56SxrGk
    >
    >     John Oreopoulos
    >
    >
    >
   
    --
    **************************
    Centre for Gene Regulation & Expression
    School of Life Sciences
    University of Dundee
    Dundee  DD1 5EH
    United Kingdom
   
    phone [phone]
    Intl phone:  [phone]
    FAX   [phone]
    email: [hidden email]
   
    Lab Page: http://www.lifesci.dundee.ac.uk/people/jason-swedlow
    Open Microscopy Environment: http://openmicroscopy.org
    **************************
   

[Cromey, Douglas W - (dcromey)]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Wow. Fascinating discussions! John - I went to a small meeting last fall (BioImage Informatics) where one of the presenters was either from the group that published the paper you cited (https://www.nature.com/articles/s41592-018-0194-9) or did something similar. It was a fascinating presentation and it sure seemed convincing (after using a multi-channel training set, the AI was able to predict three colors of fluorescence from just a greyscale transmitted light image). At the end of the talk I raised my hand, admitted that I was much more of a microscopist than a computer scientist, and asked "what happens when the biology turns around and 'bites you in the butt'?" The presenter sputtered a bit about doing good controls, but as he was someone who was probably more of a computer scientist, I don't think he grasped my point. At lunch the next day several others who were more microscopy oriented agreed with me, we have all learned the hard way that biological samples (especially live samples) can occasionally do things that are completely unexpected and very hard to explain. Having read a bit on the topic of AI, it seems like (as others have noted here) in carefully picked use cases, AI/Deep Learning/Neural Networks have been able to do amazing things with image processing and recognition. Some of the problems within the field seem to be code written in different programming languages (hard to share and/or compare), labs unwilling to publish or share their code, etc. (http://science.sciencemag.org/content/359/6377/725.long)

One thing to cheer about with AI is the application of algorithms to the task of culling through the literature. This seems more promising, in the short term, than some of the image processing/recognition/manipulation algorithms.  

About those image ethics guidelines... Back in 1997 our local MSA affiliated society held a panel discussion to ask some local "experts" about working with digital images, since this was a new area for most of us. The panel's answers to our questions were all over the map, and some were along the lines of "trust me, I'm a scientist". The most telling, for me, was our local biomedical photographer sharing with the audience (back when Photoshop was only version 3.0) that his clients were already asking for gel bands to be selectively enhanced, or incorrectly manipulated. Since my job is to support investigators at an NIH funded center, I figured I needed to know how to explain what was appropriate to my users (especially since I was just getting into the field myself), so I set out to find some guidelines. When I didn't find them online or in the literature, I set out to learn more about the topic (a big tip of the hat to Dr. John Russ, and many others) and eventually wrote the guidelines for a local newsletter, and then published them on a webpage and ultimately was asked to turn a conference talk into a paper.

It's always been my contention that users can do just about anything to a digital image, as long as they provide a detailed protocol of what was done to the published image (and, as was pointed out, they retain the original unaltered raw data). Since the HHS/NIH/ORI definition of misconduct includes the important caveat that "honest error" and "differences of scientific opinion" do not constitute misconduct, I tell users that if the protocol is out there, they might be called an idiot based on their image processing protocol, but they cannot be called a cheat. With AI image manipulation, I think it would be much more difficult to explain all the steps, especially since (as noted in one of the YouTube videos cited earlier) the computer scientists don't always completely understand what is happening to get the end result. Also (said somewhat tongue-in-cheek), if the AI software uses what it knows about or actual parts of other images to manipulate and create a new image, isn't that a bit like plagiarism?

Probably more timely for me than the future issues with AI is helping our users understand how to appropriately interpret data from current generation microscopes (SIM, Airyscan, etc) that has been processed. I'm still on this learning curve (Fourier math is cool and complicated!), trying to get to the point where I can be a resource in this new field.

As always, I am very grateful for the shared wisdom and experiences on this listserv.
Doug

------------------------------------------------------------------------------------------
Douglas W. Cromey, M.S. - Associate Scientific Investigator
Dept. of Cellular & Molecular Medicine, University of Arizona
1501 N. Campbell Ave, Tucson, AZ  85724-5044 USA

office:  LSN 463              email: [hidden email]
voice:  520-626-2824       fax:  [phone]

http://microscopy.arizona.edu/learn/microscopy-imaging-resources-www
Home of: "Microscopy and Imaging Resources on the WWW"

UA Microscopy Alliance -  http://microscopy.arizona.edu 

-----Original Message-----
From: Confocal Microscopy List <[hidden email]> On Behalf Of Oshel, Philip Eugene
Sent: Tuesday, November 20, 2018 6:11 AM
To: [hidden email]
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Good reference, thanks!

Note: I would like to publish an article (or two) on this issue in Microscopy Today. If someone has something they'd like to contribute, please contact me.

Phil
P.S. The clip was from Futurama.
-------------
Philip Oshel    
Technical Editor, Microscopy Today
Imaging Facility Director
Biology Department
1304 Biosciences
1455 Calumet Ct.
Central Michigan University
Mt. Pleasant, MI 48859
[phone]
www(dot)microscopy-today(dot)com

-----Original Message-----
From: Confocal Microscopy List <[hidden email]> on behalf of Jason Swedlow <[hidden email]>
Reply-To: Confocal Microscopy List <[hidden email]>
Date: Monday,  19November, 2018 at 19:16
To: "[hidden email]" <[hidden email]>
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

    *****
    To join, leave or search the confocal microscopy listserv, go to:
    http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    Post images on http://www.imgur.com and include the link in your posting.
    *****
   
    Hi All-
   
    On a more serious note (although any comment that cites The Simpsons is an
    excellent one), see https://www.nature.com/articles/s41592-018-0195-8 for
    an update and proposal for image data publication resources.
   
    Cheers,
   
    Jason
   
    On Mon, Nov 19, 2018 at 7:25 PM Oshel, Philip Eugene <[hidden email]>
    wrote:
   
    > *****
    > To join, leave or search the confocal microscopy listserv, go to:
    > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    > Post images on http://www.imgur.com and include the link in your posting.
    > *****
    >
    > John,
    >
    > You forgot this clip:
    > https://www.youtube.com/watch?v=WwnI0RS6J5A
    >
    > Phil
    > -------------
    > Philip Oshel
    > Imaging Facility Director
    > Biology Department
    > 1304 Biosciences
    > 1455 Calumet Ct.
    > Central Michigan University
    > Mt. Pleasant, MI 48859
    > 989 774-3576 office
    > 989 774-7567 lab
    >
    > -----Original Message-----
    > From: Confocal Microscopy List <[hidden email]> on
    > behalf of John Oreopoulos <[hidden email]>
    > Reply-To: Confocal Microscopy List <[hidden email]>
    > Date: Friday,  16November, 2018 at 21:33
    > To: "[hidden email]" <[hidden email]>
    > Subject: Digital imaging ethics as pertaining to the enhancement of
    > microscopy images with artificial intelligence
    >
    >     *****
    >     To join, leave or search the confocal microscopy listserv, go to:
    >     http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    >     Post images on http://www.imgur.com and include the link in your
    > posting.
    >     *****
    >
    >     Earlier today a few people (including myself) brought up Doug Cromey's
    > excellent treatise on digital imaging ethics in a related thread that dealt
    > with training new microscope users within a research setting. Lately I've
    > been hearing a lot about applications of machine learning and artificial
    > intelligence to "improve", "de-noise", or "fix" images (microscopy or
    > otherwise), extracting new information from low-resolution images, and even
    > creating new 3D views of samples with very little information. Here is just
    > one such example from Nvidia and MIT:
    >
    >
    > https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
    >
    >     https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo
    >
    >     It's clear that the microscopy world will eventually come to a head
    > with this technology. I think I've seen a few research articles on this
    > topic now, and this month's issue of Nature Methods has a paper on this
    > topic too:
    >
    >     https://www.nature.com/articles/s41592-018-0194-9
    >
    >     I've been wondering if and how Cromey's guide for digital imaging
    > ethics should be altered when it comes to AI-assisted microscope imaging.
    > Should it be allowed/accepted? Other readings of mine on AI show that
    > machine learning algorithms can produce biased results if the training
    > datasets are incomplete in some way, and the very nature of machine
    > learning makes it difficult to understand why it produced a certain result,
    > since the deep learning neural networks that are used to generate the
    > results are essentially black boxes that can't easily be probed. But on the
    > other hand, I'm constantly blown away by what I've seen so far online for
    > other various applications of AI (facial recognition, translation, etc.).
    >
    >     I also just finished a good read about AI from the perspective of
    > economics:
    >
    >     https://www.predictionmachines.ai/
    >
    >     https://youtu.be/5G0PbwtiMJk
    >
    >     The basic message of this book is that AI makes prediction cheap. When
    > something is cheap, we use more of it. Other processes that complement
    > prediction, like judgement (by a human or otherwise) becomes more valuable.
    > It's easy to see how the lessons of this book could be re-framed for
    > imaging science.
    >
    >     Curious to know the community's opinion on this matter. I used to
    > laugh at the following video, but now I'm not laughing:
    >
    >     https://www.youtube.com/watch?v=LhF_56SxrGk
    >
    >     John Oreopoulos
    >
    >
    >
   
    --
    **************************
    Centre for Gene Regulation & Expression
    School of Life Sciences
    University of Dundee
    Dundee  DD1 5EH
    United Kingdom
   
    phone [phone]
    Intl phone:  [phone]
    FAX   [phone]
    email: [hidden email]
   
    Lab Page: http://www.lifesci.dundee.ac.uk/people/jason-swedlow
    Open Microscopy Environment: http://openmicroscopy.org
    **************************
   

[Sylvie Le Guyader]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

'users can do just about anything to a digital image, as long as they provide a detailed protocol of what was done to the published image (and, as was pointed out, they retain the original unaltered raw data). (...) if the protocol is out there, they might be called an idiot based on their image processing protocol, but they cannot be called a cheat.'

Thanks Doug! I totally agree and will use your 'guideline' in the future. :) Clearly if there is something a user does to an image that they would rather not write in the protocol, it is best they don't do it!

In the era of metadata, I wonder why there is no way yet to automatically add to an image metadata what type of processing was done after acquisition. Aren't processing commands very small text files? I am not talking about saving layers like in Photoshop so one can Undo but simply recording the commands. That would allow researchers to recall the pipeline they used 1 year before and also allow others to reproduce the same type of pipeline with a different software. A bit like an embedded Fiji macro recording. :)

Med vänlig hälsning / Best regards

Sylvie

@@@@@@@@@@@@@@@@@@@@@@@@
Sylvie Le Guyader, PhD
Live Cell Imaging Facility Manager
Karolinska Institutet- Bionut Dpt
Hälsovägen 7C,
Room 7362 (lab)/7840 (office)
14157 Huddinge, Sweden
mobile: [phone]
LCI website
Follow our microscopy blog!


-----Original Message-----
From: Confocal Microscopy List <[hidden email]> On Behalf Of Cromey, Douglas W - (dcromey)
Sent: den 20 november 2018 17:45
To: [hidden email]
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Wow. Fascinating discussions! John - I went to a small meeting last fall (BioImage Informatics) where one of the presenters was either from the group that published the paper you cited (https://www.nature.com/articles/s41592-018-0194-9) or did something similar. It was a fascinating presentation and it sure seemed convincing (after using a multi-channel training set, the AI was able to predict three colors of fluorescence from just a greyscale transmitted light image). At the end of the talk I raised my hand, admitted that I was much more of a microscopist than a computer scientist, and asked "what happens when the biology turns around and 'bites you in the butt'?" The presenter sputtered a bit about doing good controls, but as he was someone who was probably more of a computer scientist, I don't think he grasped my point. At lunch the next day several others who were more microscopy oriented agreed with me, we have all learned the hard way that biological samples (especially live samples) can occasionally do things that are completely unexpected and very hard to explain. Having read a bit on the topic of AI, it seems like (as others have noted here) in carefully picked use cases, AI/Deep Learning/Neural Networks have been able to do amazing things with image processing and recognition. Some of the problems within the field seem to be code written in different programming languages (hard to share and/or compare), labs unwilling to publish or share their code, etc. (http://science.sciencemag.org/content/359/6377/725.long)

One thing to cheer about with AI is the application of algorithms to the task of culling through the literature. This seems more promising, in the short term, than some of the image processing/recognition/manipulation algorithms.

About those image ethics guidelines... Back in 1997 our local MSA affiliated society held a panel discussion to ask some local "experts" about working with digital images, since this was a new area for most of us. The panel's answers to our questions were all over the map, and some were along the lines of "trust me, I'm a scientist". The most telling, for me, was our local biomedical photographer sharing with the audience (back when Photoshop was only version 3.0) that his clients were already asking for gel bands to be selectively enhanced, or incorrectly manipulated. Since my job is to support investigators at an NIH funded center, I figured I needed to know how to explain what was appropriate to my users (especially since I was just getting into the field myself), so I set out to find some guidelines. When I didn't find them online or in the literature, I set out to learn more about the topic (a big tip of the hat to Dr. John Russ, and many others) and eventually wrote the guidelines for a local newsletter, and then published them on a webpage and ultimately was asked to turn a conference talk into a paper.

It's always been my contention that users can do just about anything to a digital image, as long as they provide a detailed protocol of what was done to the published image (and, as was pointed out, they retain the original unaltered raw data). Since the HHS/NIH/ORI definition of misconduct includes the important caveat that "honest error" and "differences of scientific opinion" do not constitute misconduct, I tell users that if the protocol is out there, they might be called an idiot based on their image processing protocol, but they cannot be called a cheat. With AI image manipulation, I think it would be much more difficult to explain all the steps, especially since (as noted in one of the YouTube videos cited earlier) the computer scientists don't always completely understand what is happening to get the end result. Also (said somewhat tongue-in-cheek), if the AI software uses what it knows about or actual parts of other images to manipulate and create a new image, isn't that a bit like plagiarism?

Probably more timely for me than the future issues with AI is helping our users understand how to appropriately interpret data from current generation microscopes (SIM, Airyscan, etc) that has been processed. I'm still on this learning curve (Fourier math is cool and complicated!), trying to get to the point where I can be a resource in this new field.

As always, I am very grateful for the shared wisdom and experiences on this listserv.
Doug

------------------------------------------------------------------------------------------
Douglas W. Cromey, M.S. - Associate Scientific Investigator Dept. of Cellular & Molecular Medicine, University of Arizona
1501 N. Campbell Ave, Tucson, AZ  85724-5044 USA

office:  LSN 463              email: [hidden email]
voice:  520-626-2824       fax:  [phone]

http://microscopy.arizona.edu/learn/microscopy-imaging-resources-www
Home of: "Microscopy and Imaging Resources on the WWW"

UA Microscopy Alliance -  http://microscopy.arizona.edu

-----Original Message-----
From: Confocal Microscopy List <[hidden email]> On Behalf Of Oshel, Philip Eugene
Sent: Tuesday, November 20, 2018 6:11 AM
To: [hidden email]
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Good reference, thanks!

Note: I would like to publish an article (or two) on this issue in Microscopy Today. If someone has something they'd like to contribute, please contact me.

Phil
P.S. The clip was from Futurama.
-------------
Philip Oshel
Technical Editor, Microscopy Today
Imaging Facility Director
Biology Department
1304 Biosciences
1455 Calumet Ct.
Central Michigan University
Mt. Pleasant, MI 48859
[phone]
www(dot)microscopy-today(dot)com

-----Original Message-----
From: Confocal Microscopy List <[hidden email]> on behalf of Jason Swedlow <[hidden email]>
Reply-To: Confocal Microscopy List <[hidden email]>
Date: Monday,  19November, 2018 at 19:16
To: "[hidden email]" <[hidden email]>
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

    *****
    To join, leave or search the confocal microscopy listserv, go to:
    http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    Post images on http://www.imgur.com and include the link in your posting.
    *****

    Hi All-

    On a more serious note (although any comment that cites The Simpsons is an
    excellent one), see https://www.nature.com/articles/s41592-018-0195-8 for
    an update and proposal for image data publication resources.

    Cheers,

    Jason

    On Mon, Nov 19, 2018 at 7:25 PM Oshel, Philip Eugene <[hidden email]>
    wrote:

    > *****
    > To join, leave or search the confocal microscopy listserv, go to:
    > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    > Post images on http://www.imgur.com and include the link in your posting.
    > *****
    >
    > John,
    >
    > You forgot this clip:
    > https://www.youtube.com/watch?v=WwnI0RS6J5A
    >
    > Phil
    > -------------
    > Philip Oshel
    > Imaging Facility Director
    > Biology Department
    > 1304 Biosciences
    > 1455 Calumet Ct.
    > Central Michigan University
    > Mt. Pleasant, MI 48859
    > 989 774-3576 office
    > 989 774-7567 lab
    >
    > -----Original Message-----
    > From: Confocal Microscopy List <[hidden email]> on
    > behalf of John Oreopoulos <[hidden email]>
    > Reply-To: Confocal Microscopy List <[hidden email]>
    > Date: Friday,  16November, 2018 at 21:33
    > To: "[hidden email]" <[hidden email]>
    > Subject: Digital imaging ethics as pertaining to the enhancement of
    > microscopy images with artificial intelligence
    >
    >     *****
    >     To join, leave or search the confocal microscopy listserv, go to:
    >     http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    >     Post images on http://www.imgur.com and include the link in your
    > posting.
    >     *****
    >
    >     Earlier today a few people (including myself) brought up Doug Cromey's
    > excellent treatise on digital imaging ethics in a related thread that dealt
    > with training new microscope users within a research setting. Lately I've
    > been hearing a lot about applications of machine learning and artificial
    > intelligence to "improve", "de-noise", or "fix" images (microscopy or
    > otherwise), extracting new information from low-resolution images, and even
    > creating new 3D views of samples with very little information. Here is just
    > one such example from Nvidia and MIT:
    >
    >
    > https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
    >
    >     https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo
    >
    >     It's clear that the microscopy world will eventually come to a head
    > with this technology. I think I've seen a few research articles on this
    > topic now, and this month's issue of Nature Methods has a paper on this
    > topic too:
    >
    >     https://www.nature.com/articles/s41592-018-0194-9
    >
    >     I've been wondering if and how Cromey's guide for digital imaging
    > ethics should be altered when it comes to AI-assisted microscope imaging.
    > Should it be allowed/accepted? Other readings of mine on AI show that
    > machine learning algorithms can produce biased results if the training
    > datasets are incomplete in some way, and the very nature of machine
    > learning makes it difficult to understand why it produced a certain result,
    > since the deep learning neural networks that are used to generate the
    > results are essentially black boxes that can't easily be probed. But on the
    > other hand, I'm constantly blown away by what I've seen so far online for
    > other various applications of AI (facial recognition, translation, etc.).
    >
    >     I also just finished a good read about AI from the perspective of
    > economics:
    >
    >     https://www.predictionmachines.ai/
    >
    >     https://youtu.be/5G0PbwtiMJk
    >
    >     The basic message of this book is that AI makes prediction cheap. When
    > something is cheap, we use more of it. Other processes that complement
    > prediction, like judgement (by a human or otherwise) becomes more valuable.
    > It's easy to see how the lessons of this book could be re-framed for
    > imaging science.
    >
    >     Curious to know the community's opinion on this matter. I used to
    > laugh at the following video, but now I'm not laughing:
    >
    >     https://www.youtube.com/watch?v=LhF_56SxrGk
    >
    >     John Oreopoulos
    >
    >
    >

    --
    **************************
    Centre for Gene Regulation & Expression
    School of Life Sciences
    University of Dundee
    Dundee  DD1 5EH
    United Kingdom

    phone [phone]
    Intl phone:  [phone]
    FAX   [phone]
    email: [hidden email]

    Lab Page: http://www.lifesci.dundee.ac.uk/people/jason-swedlow
    Open Microscopy Environment: http://openmicroscopy.org
    **************************




När du skickar e-post till Karolinska Institutet (KI) innebär detta att KI kommer att behandla dina personuppgifter. Här finns information om hur KI behandlar personuppgifter<https://ki.se/medarbetare/integritetsskyddspolicy>.


Sending email to Karolinska Institutet (KI) will result in KI processing your personal data. You can read more about KI’s processing of personal data here<https://ki.se/en/staff/data-protection-policy>.

[Cromey, Douglas W - (dcromey)]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Sylvie,

Kirsten Miles (and colleagues) looked into this a while back and I don't think anything has changed. Most of the software we use for manipulating digital images is terrible at creating an "audit trail" of what was done to an image.

   In Photoshop you have to remember to turn the feature on (EDIT | PREFERENCES | HISTORY LOG), and I am pretty sure that is the case every time you start up the Photoshop software (i.e., it's not a persistent setting). And, it doesn't concatenate the information across multiple editing sessions.

   Last known ImageJ could sort of create a text file, if you knew how to ask for it.

   Paul Thompson was working on developing a feature for audit trails in GIMP, but it sounded like there were some issues in the plugin capturing all of the data you would want in an audit trail.

I tell my users that they are most likely going to need to record their steps "by hand".

Doug

------------------------------------------------------------------------------------------
Douglas W. Cromey, M.S. - Associate Scientific Investigator
Dept. of Cellular & Molecular Medicine, University of Arizona
1501 N. Campbell Ave, Tucson, AZ  85724-5044 USA

office:  LSN 463              email: [hidden email]
voice:  520-626-2824       fax:  [phone]

http://microscopy.arizona.edu/learn/microscopy-imaging-resources-www
Home of: "Microscopy and Imaging Resources on the WWW"

UA Microscopy Alliance -  http://microscopy.arizona.edu 


-----Original Message-----
From: Confocal Microscopy List <[hidden email]> On Behalf Of Sylvie Le Guyader
Sent: Tuesday, November 20, 2018 10:46 AM
To: [hidden email]
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

'users can do just about anything to a digital image, as long as they provide a detailed protocol of what was done to the published image (and, as was pointed out, they retain the original unaltered raw data). (...) if the protocol is out there, they might be called an idiot based on their image processing protocol, but they cannot be called a cheat.'

Thanks Doug! I totally agree and will use your 'guideline' in the future. :) Clearly if there is something a user does to an image that they would rather not write in the protocol, it is best they don't do it!

In the era of metadata, I wonder why there is no way yet to automatically add to an image metadata what type of processing was done after acquisition. Aren't processing commands very small text files? I am not talking about saving layers like in Photoshop so one can Undo but simply recording the commands. That would allow researchers to recall the pipeline they used 1 year before and also allow others to reproduce the same type of pipeline with a different software. A bit like an embedded Fiji macro recording. :)

Med vänlig hälsning / Best regards

Sylvie

@@@@@@@@@@@@@@@@@@@@@@@@
Sylvie Le Guyader, PhD
Live Cell Imaging Facility Manager
Karolinska Institutet- Bionut Dpt
Hälsovägen 7C,
Room 7362 (lab)/7840 (office)
14157 Huddinge, Sweden
mobile: [phone]
LCI website
Follow our microscopy blog!


-----Original Message-----
From: Confocal Microscopy List <[hidden email]> On Behalf Of Cromey, Douglas W - (dcromey)
Sent: den 20 november 2018 17:45
To: [hidden email]
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Wow. Fascinating discussions! John - I went to a small meeting last fall (BioImage Informatics) where one of the presenters was either from the group that published the paper you cited (https://www.nature.com/articles/s41592-018-0194-9) or did something similar. It was a fascinating presentation and it sure seemed convincing (after using a multi-channel training set, the AI was able to predict three colors of fluorescence from just a greyscale transmitted light image). At the end of the talk I raised my hand, admitted that I was much more of a microscopist than a computer scientist, and asked "what happens when the biology turns around and 'bites you in the butt'?" The presenter sputtered a bit about doing good controls, but as he was someone who was probably more of a computer scientist, I don't think he grasped my point. At lunch the next day several others who were more microscopy oriented agreed with me, we have all learned the hard way that biological samples (especially live samples) can occasionally do things that are completely unexpected and very hard to explain. Having read a bit on the topic of AI, it seems like (as others have noted here) in carefully picked use cases, AI/Deep Learning/Neural Networks have been able to do amazing things with image processing and recognition. Some of the problems within the field seem to be code written in different programming languages (hard to share and/or compare), labs unwilling to publish or share their code, etc. (http://science.sciencemag.org/content/359/6377/725.long)

One thing to cheer about with AI is the application of algorithms to the task of culling through the literature. This seems more promising, in the short term, than some of the image processing/recognition/manipulation algorithms.

About those image ethics guidelines... Back in 1997 our local MSA affiliated society held a panel discussion to ask some local "experts" about working with digital images, since this was a new area for most of us. The panel's answers to our questions were all over the map, and some were along the lines of "trust me, I'm a scientist". The most telling, for me, was our local biomedical photographer sharing with the audience (back when Photoshop was only version 3.0) that his clients were already asking for gel bands to be selectively enhanced, or incorrectly manipulated. Since my job is to support investigators at an NIH funded center, I figured I needed to know how to explain what was appropriate to my users (especially since I was just getting into the field myself), so I set out to find some guidelines. When I didn't find them online or in the literature, I set out to learn more about the topic (a big tip of the hat to Dr. John Russ, and many others) and eventually wrote the guidelines for a local newsletter, and then published them on a webpage and ultimately was asked to turn a conference talk into a paper.

It's always been my contention that users can do just about anything to a digital image, as long as they provide a detailed protocol of what was done to the published image (and, as was pointed out, they retain the original unaltered raw data). Since the HHS/NIH/ORI definition of misconduct includes the important caveat that "honest error" and "differences of scientific opinion" do not constitute misconduct, I tell users that if the protocol is out there, they might be called an idiot based on their image processing protocol, but they cannot be called a cheat. With AI image manipulation, I think it would be much more difficult to explain all the steps, especially since (as noted in one of the YouTube videos cited earlier) the computer scientists don't always completely understand what is happening to get the end result. Also (said somewhat tongue-in-cheek), if the AI software uses what it knows about or actual parts of other images to manipulate and create a new image, isn't that a bit like plagiarism?

Probably more timely for me than the future issues with AI is helping our users understand how to appropriately interpret data from current generation microscopes (SIM, Airyscan, etc) that has been processed. I'm still on this learning curve (Fourier math is cool and complicated!), trying to get to the point where I can be a resource in this new field.

As always, I am very grateful for the shared wisdom and experiences on this listserv.
Doug

------------------------------------------------------------------------------------------
Douglas W. Cromey, M.S. - Associate Scientific Investigator Dept. of Cellular & Molecular Medicine, University of Arizona
1501 N. Campbell Ave, Tucson, AZ  85724-5044 USA

office:  LSN 463              email: [hidden email]
voice:  520-626-2824       fax:  [phone]

http://microscopy.arizona.edu/learn/microscopy-imaging-resources-www
Home of: "Microscopy and Imaging Resources on the WWW"

UA Microscopy Alliance -  http://microscopy.arizona.edu

-----Original Message-----
From: Confocal Microscopy List <[hidden email]> On Behalf Of Oshel, Philip Eugene
Sent: Tuesday, November 20, 2018 6:11 AM
To: [hidden email]
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Good reference, thanks!

Note: I would like to publish an article (or two) on this issue in Microscopy Today. If someone has something they'd like to contribute, please contact me.

Phil
P.S. The clip was from Futurama.
-------------
Philip Oshel
Technical Editor, Microscopy Today
Imaging Facility Director
Biology Department
1304 Biosciences
1455 Calumet Ct.
Central Michigan University
Mt. Pleasant, MI 48859
[phone]
www(dot)microscopy-today(dot)com

-----Original Message-----
From: Confocal Microscopy List <[hidden email]> on behalf of Jason Swedlow <[hidden email]>
Reply-To: Confocal Microscopy List <[hidden email]>
Date: Monday,  19November, 2018 at 19:16
To: "[hidden email]" <[hidden email]>
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

    *****
    To join, leave or search the confocal microscopy listserv, go to:
    http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    Post images on http://www.imgur.com and include the link in your posting.
    *****

    Hi All-

    On a more serious note (although any comment that cites The Simpsons is an
    excellent one), see https://www.nature.com/articles/s41592-018-0195-8 for
    an update and proposal for image data publication resources.

    Cheers,

    Jason

    On Mon, Nov 19, 2018 at 7:25 PM Oshel, Philip Eugene <[hidden email]>
    wrote:

    > *****
    > To join, leave or search the confocal microscopy listserv, go to:
    > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    > Post images on http://www.imgur.com and include the link in your posting.
    > *****
    >
    > John,
    >
    > You forgot this clip:
    > https://www.youtube.com/watch?v=WwnI0RS6J5A
    >
    > Phil
    > -------------
    > Philip Oshel
    > Imaging Facility Director
    > Biology Department
    > 1304 Biosciences
    > 1455 Calumet Ct.
    > Central Michigan University
    > Mt. Pleasant, MI 48859
    > 989 774-3576 office
    > 989 774-7567 lab
    >
    > -----Original Message-----
    > From: Confocal Microscopy List <[hidden email]> on
    > behalf of John Oreopoulos <[hidden email]>
    > Reply-To: Confocal Microscopy List <[hidden email]>
    > Date: Friday,  16November, 2018 at 21:33
    > To: "[hidden email]" <[hidden email]>
    > Subject: Digital imaging ethics as pertaining to the enhancement of
    > microscopy images with artificial intelligence
    >
    >     *****
    >     To join, leave or search the confocal microscopy listserv, go to:
    >     http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    >     Post images on http://www.imgur.com and include the link in your
    > posting.
    >     *****
    >
    >     Earlier today a few people (including myself) brought up Doug Cromey's
    > excellent treatise on digital imaging ethics in a related thread that dealt
    > with training new microscope users within a research setting. Lately I've
    > been hearing a lot about applications of machine learning and artificial
    > intelligence to "improve", "de-noise", or "fix" images (microscopy or
    > otherwise), extracting new information from low-resolution images, and even
    > creating new 3D views of samples with very little information. Here is just
    > one such example from Nvidia and MIT:
    >
    >
    > https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
    >
    >     https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo
    >
    >     It's clear that the microscopy world will eventually come to a head
    > with this technology. I think I've seen a few research articles on this
    > topic now, and this month's issue of Nature Methods has a paper on this
    > topic too:
    >
    >     https://www.nature.com/articles/s41592-018-0194-9
    >
    >     I've been wondering if and how Cromey's guide for digital imaging
    > ethics should be altered when it comes to AI-assisted microscope imaging.
    > Should it be allowed/accepted? Other readings of mine on AI show that
    > machine learning algorithms can produce biased results if the training
    > datasets are incomplete in some way, and the very nature of machine
    > learning makes it difficult to understand why it produced a certain result,
    > since the deep learning neural networks that are used to generate the
    > results are essentially black boxes that can't easily be probed. But on the
    > other hand, I'm constantly blown away by what I've seen so far online for
    > other various applications of AI (facial recognition, translation, etc.).
    >
    >     I also just finished a good read about AI from the perspective of
    > economics:
    >
    >     https://www.predictionmachines.ai/
    >
    >     https://youtu.be/5G0PbwtiMJk
    >
    >     The basic message of this book is that AI makes prediction cheap. When
    > something is cheap, we use more of it. Other processes that complement
    > prediction, like judgement (by a human or otherwise) becomes more valuable.
    > It's easy to see how the lessons of this book could be re-framed for
    > imaging science.
    >
    >     Curious to know the community's opinion on this matter. I used to
    > laugh at the following video, but now I'm not laughing:
    >
    >     https://www.youtube.com/watch?v=LhF_56SxrGk
    >
    >     John Oreopoulos
    >
    >
    >

    --
    **************************
    Centre for Gene Regulation & Expression
    School of Life Sciences
    University of Dundee
    Dundee  DD1 5EH
    United Kingdom

    phone [phone]
    Intl phone:  [phone]
    FAX   [phone]
    email: [hidden email]

    Lab Page: http://www.lifesci.dundee.ac.uk/people/jason-swedlow
    Open Microscopy Environment: http://openmicroscopy.org
    **************************




När du skickar e-post till Karolinska Institutet (KI) innebär detta att KI kommer att behandla dina personuppgifter. Här finns information om hur KI behandlar personuppgifter<https://ki.se/medarbetare/integritetsskyddspolicy>.


Sending email to Karolinska Institutet (KI) will result in KI processing your personal data. You can read more about KI’s processing of personal data here<https://ki.se/en/staff/data-protection-policy>.

[Armstrong, Brian]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi All, the Forensic Tools, or Forensic Droplets, from the Office of Research Integrity at Health and Human Services (USA) will sort of take you backwards in order to show what was done to a digital image.

Cheers,
https://ori.hhs.gov/forensic-tools
 

Brian Armstrong PhD
Associate Research Professor
Developmental and Stem Cell Biology
Diabetes and Metabolic Diseases
Director, Light Microscopy Core
Beckman Research Institute, City of Hope



-----Original Message-----
From: Confocal Microscopy List [mailto:[hidden email]] On Behalf Of Cromey, Douglas W - (dcromey)
Sent: Tuesday, November 20, 2018 10:02 AM
To: [hidden email]
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

[Attention: This email came from an external source. Do not open attachments or click on links from unknown senders or unexpected emails.]





*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Sylvie,

Kirsten Miles (and colleagues) looked into this a while back and I don't think anything has changed. Most of the software we use for manipulating digital images is terrible at creating an "audit trail" of what was done to an image.

   In Photoshop you have to remember to turn the feature on (EDIT | PREFERENCES | HISTORY LOG), and I am pretty sure that is the case every time you start up the Photoshop software (i.e., it's not a persistent setting). And, it doesn't concatenate the information across multiple editing sessions.

   Last known ImageJ could sort of create a text file, if you knew how to ask for it.

   Paul Thompson was working on developing a feature for audit trails in GIMP, but it sounded like there were some issues in the plugin capturing all of the data you would want in an audit trail.

I tell my users that they are most likely going to need to record their steps "by hand".

Doug

------------------------------------------------------------------------------------------
Douglas W. Cromey, M.S. - Associate Scientific Investigator Dept. of Cellular & Molecular Medicine, University of Arizona
1501 N. Campbell Ave, Tucson, AZ  85724-5044 USA

office:  LSN 463              email: [hidden email]
voice:  520-626-2824       fax:  [phone]

http://microscopy.arizona.edu/learn/microscopy-imaging-resources-www
Home of: "Microscopy and Imaging Resources on the WWW"

UA Microscopy Alliance -  http://microscopy.arizona.edu 


-----Original Message-----
From: Confocal Microscopy List <[hidden email]> On Behalf Of Sylvie Le Guyader
Sent: Tuesday, November 20, 2018 10:46 AM
To: [hidden email]
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

'users can do just about anything to a digital image, as long as they provide a detailed protocol of what was done to the published image (and, as was pointed out, they retain the original unaltered raw data). (...) if the protocol is out there, they might be called an idiot based on their image processing protocol, but they cannot be called a cheat.'

Thanks Doug! I totally agree and will use your 'guideline' in the future. :) Clearly if there is something a user does to an image that they would rather not write in the protocol, it is best they don't do it!

In the era of metadata, I wonder why there is no way yet to automatically add to an image metadata what type of processing was done after acquisition. Aren't processing commands very small text files? I am not talking about saving layers like in Photoshop so one can Undo but simply recording the commands. That would allow researchers to recall the pipeline they used 1 year before and also allow others to reproduce the same type of pipeline with a different software. A bit like an embedded Fiji macro recording. :)

Med vänlig hälsning / Best regards

Sylvie

@@@@@@@@@@@@@@@@@@@@@@@@
Sylvie Le Guyader, PhD
Live Cell Imaging Facility Manager
Karolinska Institutet- Bionut Dpt
Hälsovägen 7C,
Room 7362 (lab)/7840 (office)
14157 Huddinge, Sweden
mobile: [phone]
LCI website
Follow our microscopy blog!


-----Original Message-----
From: Confocal Microscopy List <[hidden email]> On Behalf Of Cromey, Douglas W - (dcromey)
Sent: den 20 november 2018 17:45
To: [hidden email]
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Wow. Fascinating discussions! John - I went to a small meeting last fall (BioImage Informatics) where one of the presenters was either from the group that published the paper you cited (https://www.nature.com/articles/s41592-018-0194-9) or did something similar. It was a fascinating presentation and it sure seemed convincing (after using a multi-channel training set, the AI was able to predict three colors of fluorescence from just a greyscale transmitted light image). At the end of the talk I raised my hand, admitted that I was much more of a microscopist than a computer scientist, and asked "what happens when the biology turns around and 'bites you in the butt'?" The presenter sputtered a bit about doing good controls, but as he was someone who was probably more of a computer scientist, I don't think he grasped my point. At lunch the next day several others who were more microscopy oriented agreed with me, we have all learned the hard way that biological samples (especially live samples) can occasionally do things that are completely unexpected and very hard to explain. Having read a bit on the topic of AI, it seems like (as others have noted here) in carefully picked use cases, AI/Deep Learning/Neural Networks have been able to do amazing things with image processing and recognition. Some of the problems within the field seem to be code written in different programming languages (hard to share and/or compare), labs unwilling to publish or share their code, etc. (http://science.sciencemag.org/content/359/6377/725.long)

One thing to cheer about with AI is the application of algorithms to the task of culling through the literature. This seems more promising, in the short term, than some of the image processing/recognition/manipulation algorithms.

About those image ethics guidelines... Back in 1997 our local MSA affiliated society held a panel discussion to ask some local "experts" about working with digital images, since this was a new area for most of us. The panel's answers to our questions were all over the map, and some were along the lines of "trust me, I'm a scientist". The most telling, for me, was our local biomedical photographer sharing with the audience (back when Photoshop was only version 3.0) that his clients were already asking for gel bands to be selectively enhanced, or incorrectly manipulated. Since my job is to support investigators at an NIH funded center, I figured I needed to know how to explain what was appropriate to my users (especially since I was just getting into the field myself), so I set out to find some guidelines. When I didn't find them online or in the literature, I set out to learn more about the topic (a big tip of the hat to Dr. John Russ, and many others) and eventually wrote the guidelines for a local newsletter, and then published them on a webpage and ultimately was asked to turn a conference talk into a paper.

It's always been my contention that users can do just about anything to a digital image, as long as they provide a detailed protocol of what was done to the published image (and, as was pointed out, they retain the original unaltered raw data). Since the HHS/NIH/ORI definition of misconduct includes the important caveat that "honest error" and "differences of scientific opinion" do not constitute misconduct, I tell users that if the protocol is out there, they might be called an idiot based on their image processing protocol, but they cannot be called a cheat. With AI image manipulation, I think it would be much more difficult to explain all the steps, especially since (as noted in one of the YouTube videos cited earlier) the computer scientists don't always completely understand what is happening to get the end result. Also (said somewhat tongue-in-cheek), if the AI software uses what it knows about or actual parts of other images to manipulate and create a new image, isn't that a bit like plagiarism?

Probably more timely for me than the future issues with AI is helping our users understand how to appropriately interpret data from current generation microscopes (SIM, Airyscan, etc) that has been processed. I'm still on this learning curve (Fourier math is cool and complicated!), trying to get to the point where I can be a resource in this new field.

As always, I am very grateful for the shared wisdom and experiences on this listserv.
Doug

------------------------------------------------------------------------------------------
Douglas W. Cromey, M.S. - Associate Scientific Investigator Dept. of Cellular & Molecular Medicine, University of Arizona
1501 N. Campbell Ave, Tucson, AZ  85724-5044 USA

office:  LSN 463              email: [hidden email]
voice:  520-626-2824       fax:  [phone]

http://microscopy.arizona.edu/learn/microscopy-imaging-resources-www
Home of: "Microscopy and Imaging Resources on the WWW"

UA Microscopy Alliance -  http://microscopy.arizona.edu

-----Original Message-----
From: Confocal Microscopy List <[hidden email]> On Behalf Of Oshel, Philip Eugene
Sent: Tuesday, November 20, 2018 6:11 AM
To: [hidden email]
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Good reference, thanks!

Note: I would like to publish an article (or two) on this issue in Microscopy Today. If someone has something they'd like to contribute, please contact me.

Phil
P.S. The clip was from Futurama.
-------------
Philip Oshel
Technical Editor, Microscopy Today
Imaging Facility Director
Biology Department
1304 Biosciences
1455 Calumet Ct.
Central Michigan University
Mt. Pleasant, MI 48859
[phone]
www(dot)microscopy-today(dot)com

-----Original Message-----
From: Confocal Microscopy List <[hidden email]> on behalf of Jason Swedlow <[hidden email]>
Reply-To: Confocal Microscopy List <[hidden email]>
Date: Monday,  19November, 2018 at 19:16
To: "[hidden email]" <[hidden email]>
Subject: Re: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence

    *****
    To join, leave or search the confocal microscopy listserv, go to:
    http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    Post images on http://www.imgur.com and include the link in your posting.
    *****

    Hi All-

    On a more serious note (although any comment that cites The Simpsons is an
    excellent one), see https://www.nature.com/articles/s41592-018-0195-8 for
    an update and proposal for image data publication resources.

    Cheers,

    Jason

    On Mon, Nov 19, 2018 at 7:25 PM Oshel, Philip Eugene <[hidden email]>
    wrote:

    > *****
    > To join, leave or search the confocal microscopy listserv, go to:
    > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    > Post images on http://www.imgur.com and include the link in your posting.
    > *****
    >
    > John,
    >
    > You forgot this clip:
    > https://www.youtube.com/watch?v=WwnI0RS6J5A
    >
    > Phil
    > -------------
    > Philip Oshel
    > Imaging Facility Director
    > Biology Department
    > 1304 Biosciences
    > 1455 Calumet Ct.
    > Central Michigan University
    > Mt. Pleasant, MI 48859
    > 989 774-3576 office
    > 989 774-7567 lab
    >
    > -----Original Message-----
    > From: Confocal Microscopy List <[hidden email]> on
    > behalf of John Oreopoulos <[hidden email]>
    > Reply-To: Confocal Microscopy List <[hidden email]>
    > Date: Friday,  16November, 2018 at 21:33
    > To: "[hidden email]" <[hidden email]>
    > Subject: Digital imaging ethics as pertaining to the enhancement of
    > microscopy images with artificial intelligence
    >
    >     *****
    >     To join, leave or search the confocal microscopy listserv, go to:
    >     http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
    >     Post images on http://www.imgur.com and include the link in your
    > posting.
    >     *****
    >
    >     Earlier today a few people (including myself) brought up Doug Cromey's
    > excellent treatise on digital imaging ethics in a related thread that dealt
    > with training new microscope users within a research setting. Lately I've
    > been hearing a lot about applications of machine learning and artificial
    > intelligence to "improve", "de-noise", or "fix" images (microscopy or
    > otherwise), extracting new information from low-resolution images, and even
    > creating new 3D views of samples with very little information. Here is just
    > one such example from Nvidia and MIT:
    >
    >
    > https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/
    >
    >     https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIo
    >
    >     It's clear that the microscopy world will eventually come to a head
    > with this technology. I think I've seen a few research articles on this
    > topic now, and this month's issue of Nature Methods has a paper on this
    > topic too:
    >
    >     https://www.nature.com/articles/s41592-018-0194-9
    >
    >     I've been wondering if and how Cromey's guide for digital imaging
    > ethics should be altered when it comes to AI-assisted microscope imaging.
    > Should it be allowed/accepted? Other readings of mine on AI show that
    > machine learning algorithms can produce biased results if the training
    > datasets are incomplete in some way, and the very nature of machine
    > learning makes it difficult to understand why it produced a certain result,
    > since the deep learning neural networks that are used to generate the
    > results are essentially black boxes that can't easily be probed. But on the
    > other hand, I'm constantly blown away by what I've seen so far online for
    > other various applications of AI (facial recognition, translation, etc.).
    >
    >     I also just finished a good read about AI from the perspective of
    > economics:
    >
    >     https://www.predictionmachines.ai/
    >
    >     https://youtu.be/5G0PbwtiMJk
    >
    >     The basic message of this book is that AI makes prediction cheap. When
    > something is cheap, we use more of it. Other processes that complement
    > prediction, like judgement (by a human or otherwise) becomes more valuable.
    > It's easy to see how the lessons of this book could be re-framed for
    > imaging science.
    >
    >     Curious to know the community's opinion on this matter. I used to
    > laugh at the following video, but now I'm not laughing:
    >
    >     https://www.youtube.com/watch?v=LhF_56SxrGk
    >
    >     John Oreopoulos
    >
    >
    >

    --
    **************************
    Centre for Gene Regulation & Expression
    School of Life Sciences
    University of Dundee
    Dundee  DD1 5EH
    United Kingdom

    phone [phone]
    Intl phone:  [phone]
    FAX   [phone]
    email: [hidden email]

    Lab Page: http://www.lifesci.dundee.ac.uk/people/jason-swedlow
    Open Microscopy Environment: http://openmicroscopy.org
    **************************




När du skickar e-post till Karolinska Institutet (KI) innebär detta att KI kommer att behandla dina personuppgifter. Här finns information om hur KI behandlar personuppgifter<https://ki.se/medarbetare/integritetsskyddspolicy>.


Sending email to Karolinska Institutet (KI) will result in KI processing your personal data. You can read more about KI’s processing of personal data here<https://ki.se/en/staff/data-protection-policy>.

------------------------------------------------------------
-SECURITY/CONFIDENTIALITY WARNING-  

This message and any attachments are intended solely for the individual or entity to which they are addressed. This communication may contain information that is privileged, confidential, or exempt from disclosure under applicable law (e.g., personal health information, research data, financial information). Because this e-mail has been sent without encryption, individuals other than the intended recipient may be able to view the information, forward it to others or tamper with the information without the knowledge or consent of the sender. If you are not the intended recipient, or the employee or person responsible for delivering the message to the intended recipient, any dissemination, distribution or copying of the communication is strictly prohibited. If you received the communication in error, please notify the sender immediately by replying to this message and deleting the message and any accompanying files from your system. If, due to the security risks, you do not wish to receive further communications via e-mail, please reply to this message and inform the sender that you do not wish to receive further e-mail from the sender. (LCP301)
------------------------------------------------------------