Posted by
0000001ed7f52e4a-dmarc-request on
URL: http://confocal-microscopy-list.275.s1.nabble.com/Digital-imaging-ethics-as-pertaining-to-the-enhancement-of-microscopy-images-with-artificial-intellie-tp7588915p7588917.html
*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopyPost images on
http://www.imgur.com and include the link in your posting.
*****
Thanks John for bringing us up-to date on image processing, these are indeed very important developments. I think there will be great changes coming over the next years driven by the AI revolution in image and video processing. But the fundamental limit should is that one cannot increase the information content of an image beyond what was originally recorded. Of course the missing information can be replaced by knowledge derived from other images. But then the new AI algorithms will have similar flaws as human perception (optical illusions). Science should be about measuring accurately instead of guessing.
My criticism of current publications and promotional videos in image processing using AI is that they show the cases when the algorithms actually work well (which might be most of the time), the important question is when do they fail and produce the wrong results? With the fast development in the field, today's problems are often solved a few days later with a new generation of the algorithm, so detecting flaws in these algorithms is an ungrateful task. But I think it is important and we will need to come up with very good quality control standards to accept these results for medical imaging or scientific publications.
A few years ago I was very excited about using compressed sensing in microscopy to "break the Nyquist barrier", but after looking into this in more detail, I came to the conclusion that this only works well when images are already heavily oversampled like in normal photography (more megapixels sell better). When microscopy data is taken at the resolution limit there is not much room for further compression. I would expect the same for neural network approaches, works well when you have a lot of pixels and the information content is not so high or accuracy is not so important.
So the question is what is actually necessary for a given experiment? If one wants to track some cells in a fluorescence time laps movie, maybe noisy (low exposure) jpeg compressed data combined with the latest AI algorithm trained on this problem is better than the perfect exposure needed for current segmentation methods and raw data recording, as in the latter case the cells might be killed a short time after the experiment started by the higher light exposure.
best wishes
Andreas
-----Original Message-----
From: John Oreopoulos <
[hidden email]>
To: CONFOCALMICROSCOPY <
[hidden email]>
Sent: Sat, 17 Nov 2018 2:33
Subject: Digital imaging ethics as pertaining to the enhancement of microscopy images with artificial intelligence
*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopyPost images on
http://www.imgur.com and include the link in your posting.
*****
Earlier today a few people (including myself) brought up Doug Cromey's excellent treatise on digital imaging ethics in a related thread that dealt with training new microscope users within a research setting. Lately I've been hearing a lot about applications of machine learning and artificial intelligence to "improve", "de-noise", or "fix" images (microscopy or otherwise), extracting new information from low-resolution images, and even creating new 3D views of samples with very little information. Here is just one such example from Nvidia and MIT:
https://news.developer.nvidia.com/ai-can-now-fix-your-grainy-photos-by-only-looking-at-grainy-photos/https://www.youtube.com/watch?time_continue=84&v=pp7HdI0-MIoIt's clear that the microscopy world will eventually come to a head with this technology. I think I've seen a few research articles on this topic now, and this month's issue of Nature Methods has a paper on this topic too:
https://www.nature.com/articles/s41592-018-0194-9I've been wondering if and how Cromey's guide for digital imaging ethics should be altered when it comes to AI-assisted microscope imaging. Should it be allowed/accepted? Other readings of mine on AI show that machine learning algorithms can produce biased results if the training datasets are incomplete in some way, and the very nature of machine learning makes it difficult to understand why it produced a certain result, since the deep learning neural networks that are used to generate the results are essentially black boxes that can't easily be probed. But on the other hand, I'm constantly blown away by what I've seen so far online for other various applications of AI (facial recognition, translation, etc.).
I also just finished a good read about AI from the perspective of economics:
https://www.predictionmachines.ai/https://youtu.be/5G0PbwtiMJkThe basic message of this book is that AI makes prediction cheap. When something is cheap, we use more of it. Other processes that complement prediction, like judgement (by a human or otherwise) becomes more valuable. It's easy to see how the lessons of this book could be re-framed for imaging science.
Curious to know the community's opinion on this matter. I used to laugh at the following video, but now I'm not laughing:
https://www.youtube.com/watch?v=LhF_56SxrGkJohn Oreopoulos