Source of Richardson test slide

[JOEL B. SHEFFIELD]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

It seems to me, now that we are talking about integrity, that we
could talk about the reasons that we include images in our reports.

My general assumption is that behind every image is a set of multiple
experiments, and multiple fields of view, and that the image is just
an example of the data.  Should it be a randomly chosen example, or
should it be the best example that one can find?  And what is the
case that is to be made?  If, as one student has claimed, "all cells
are labeled", I don't want to see an image with a single cell.  I
would like to see a field of cells so that I can convince myself that
the student is correct.  Better still, of course, would be to see
some quantitation --but that is a different kettle of fish (or
cells).  When I was involved in virus research, there was an
individual who said "does one particle make an article?"

What about a situation in which only some cells are labeled, or take
a specific configuration?  I have had a student tell me that "all of
the cells have flattened out".  If quantitation shows that 70% have
done so, what is the appropriate image to show?

But then we come to a more difficult question.  Suppose that an
experiment is so expensive, or depends on such a rare event, that a
single image IS the data?  I think that especially in this
circumstance, raw images as well as processed ones should be
required.  It is just too easy, even when limited to
contrast/brightness adjustments, to eliminate minor bands on gels and
make others stand out. Similarly, one can as easukt exclude weak
fluorescence if there are some parts of the sample that are intense.  
I suppose that one could require that all images must not include
saturated pixels.

One more point.  I may be old fashioned, but I was brought up to
think that a research project involves a significant number of
experiments, some of which "work" and others don't.  As the project
progresses, the ratio of trustworthy to untrustworthy data increases
until we can become confident that we can make a statement about the
phenomenon.  This generally happens as we tweak the system to finally
arrive at more optimal conditions.  Thus, a full data set of the
experiments might be informative to a historian, but (even)
embarassing to the investigator.

Joel

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> Wow, I didn't know I was keeping company with such a manipulative bunch!
>
> I'm just replying to this last message as a general continuation of the
> thread.
>
> Where is the line? On a camera for color imaging we all do a white
> balance; and maybe there is a flatfield or background subtract applied.
> And the CCD probably has the "fix bad pixels" box checked. These have a
> well-understood reason for use, but they are systematically manipulating
> pixel values. Flatfields and "fix-bad-pixels" are not array processes in
> that they alter specific pixels in specific ways to make the image look
> better: they are altering values to nominally fix errors that would make
> the captured image not appear as the observed image. Will these
> manipulations be detected and flagged by the sleuths checking images?
>
> Dale
>
> Tina Carvalho wrote:
> > Search the CONFOCAL archive at
> > http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
> >
> >> 1. Often, in response to the referees, I have to change figures.
> >> Maybe group them differently (which means changing letters) or
> >> alter the labelling.  I suppose I should go back to the original
> >> images and re-create the entire plate, but if the changes are
> >> minor it's much easier to (for example) black out 'a' and paste
> >> in 'b', or take out an arrow and put in a letter, etc.  All these
> >> things would presumably show up as 'fraud' ....
> >
> > Yup. Sigh. This is the education part, and I guide people through this a
> > lot. The trick is to learn to use Layers in Photoshop, and save every
> > iteration (disk space is cheap) so that you can go back and move the
> > images, letters, arrows, etc. on a figure plate. Once you do this one
> > time, you realize how valuable it is!
> >
> >> 2.  A point nobody has yet raised is deconvolution imaging
> >> systems.  Clinicians love them, but they have no training in physics
> >> (and, to be honest, not much in science).  None of them has the
> >> first idea what the software is doing, just that it's giving them
> >> the pictures they want.  How does anyone handle that?
> >
> > Education. Give them a cheat sheet that lays out how it works. Give them a
> > test. Whatever. I know it can be a pain, but researchers should (in an
> > ideal world) understand what it is that they are doing. I can go on a rant
> > about labeling "kits" where you follow directions, but never have any idea
> > what is actually happening. I can go on lots of rants...
> >
> > Aloha, Tina
> >
> >> Optical Imaging Techniques in Cell Biology
> >> by Guy Cox    CRC Press / Taylor & Francis
> >>     http://www.guycox.com/optical.htm
> >> ______________________________________________
> >> Associate Professor Guy Cox, MA, DPhil(Oxon)
> >> Electron Microscope Unit, Madsen Building F09,
> >> University of Sydney, NSW 2006
> >> ______________________________________________
> >> Phone [phone]     Fax [phone]
> >> Mobile [phone]
> >> ______________________________________________
> >>      http://www.guycox.net
> >> -----Original Message-----
> >> From: Confocal Microscopy List [mailto:[hidden email]] On Behalf Of Michael Cammer
> >> Sent: Monday, 23 June 2008 2:30 AM
> >> To: [hidden email]
> >> Subject: Re: An alarming amount of image manipulation
> >>
> >> Search the CONFOCAL archive at
> >> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
> >>
> >> In my experience, when somebody shows up with micrographs that have cells pasted in or background "distractions" stamp tooled out and I suggest maybe taking more pictures or doing the experiment again, they complain that this would cost more and delay them.  "The data are what they are and I'm just showing people and if I don't make it look good, then they won't publish it."  Sometimes junior lab members will go back to their PIs and say that I wasn't helpful; I become the problem.  I will not apologize for refusing to use the lasso tool around a band on a gel and S curve adjust it in Photoshop Curves or for refusing to paint out "an anomalous bad cell off in the corner of the image".  Yes, both these requests have come through our facility along with a bunch of other egregious ones.  Some scientists really feel justified because the competition does it.  "Hey, everybody does it," they say.  Although, more often (and a search will show that we've discussed this before),

 p

> eople simply don;t understand what they are doing.  They simply misuse the tools for making figures.  As with example of film, before you can begin to alter images in an intentional manner or make even reasonably good pictures of anything, you need to be at least knowledgeable of darkroom technique.  But with the new digital tools, any dummy can be a photographer and can cut and paste and manipulate in complicated ways.
> >> -Michael
> >>
> >>> Search the CONFOCAL archive at
> >>> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
> >>>
> >>> Interesting - especially the statement that images on film cannot be
> >>> doctored. I guess it's irrelevant nowadays, but that is so far from
> >>> the truth.
> >>>
> >>> One funny story - which I've told before but some time ago.
> >>> Many years ago I (with colleagues) published a paper in a well-known
> >>> journal which included a photo of some gels - taken by the
> >>> departmental photographer.  One of the gels had cracked when it was
> >>> taken out of the tube, so there was a dark hairline across it in the
> >>> photo.  In the published paper that line had disappeared.  You can
> >>> hardly accuse me, or my two co-authors, of fraud since this was done
> >>> by the journal's art department without any reference to us!
> >>>
> >>>                                                        Guy
> >>>
> >>>
> >>> Optical Imaging Techniques in Cell Biology
> >>> by Guy Cox    CRC Press / Taylor & Francis
> >>>     http://www.guycox.com/optical.htm
> >>> ______________________________________________
> >>> Associate Professor Guy Cox, MA, DPhil(Oxon) Electron Microscope Unit,
> >>> Madsen Building F09, University of Sydney, NSW 2006
> >>> ______________________________________________
> >>> Phone [phone]     Fax [phone]
> >>> Mobile [phone]
> >>> ______________________________________________
> >>>      http://www.guycox.net
> >>> -----Original Message-----
> >>> From: Confocal Microscopy List [mailto:[hidden email]]
> >>> On Behalf Of John Oreopoulos
> >>> Sent: Sunday, 22 June 2008 10:57 PM
> >>> To: [hidden email]
> >>> Subject: An alarming amount of image manipulation
> >>>
> >>> Search the CONFOCAL archive at
> >>> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
> >>>
> >>> I was a bit surprised at the statistics cited in this article:
> >>>
> >>> http://chronicle.com/free/2008/05/3028n.htm
> >>>
> >>> Does this mean that all journals will start hiring image manipulation
> >>> detectives someday? Could be an interesting career.
> >>>
> >>>
> >>> John Oreopoulos, BSc,
> >>> PhD Candidate
> >>> University of Toronto
> >>> Institute For Biomaterials and Biomedical Engineering Centre For
> >>> Studies in Molecular Imaging
> >>>
> >>> Tel: W:416-946-5022
> >>>
> >>> No virus found in this incoming message.
> >>> Checked by AVG.
> >>> Version: 7.5.524 / Virus Database: 270.4.1/1512 - Release Date:
> >>> 21/06/2008
> >>> 9:27 AM
> >>>
> >>>
> >>> No virus found in this outgoing message.
> >>> Checked by AVG.
> >>> Version: 7.5.524 / Virus Database: 270.4.1/1512 - Release Date:
> >>> 21/06/2008
> >>> 9:27 AM
> >>>
> >>>
> >>
> >> _________________________________________
> >> Michael Cammer   http://www.aecom.yu.edu/aif/
> >>
> >> No virus found in this incoming message.
> >> Checked by AVG.
> >> Version: 7.5.524 / Virus Database: 270.4.1/1512 - Release Date: 21/06/2008 9:27 AM
> >>  
> >>
> >> No virus found in this outgoing message.
> >> Checked by AVG.
> >> Version: 7.5.524 / Virus Database: 270.4.1/1513 - Release Date: 22/06/2008 7:52 AM
> >>  
> >>
> >
> > ****************************************************************************
> > * Tina (Weatherby) Carvalho               * [hidden email]           *
> > * Biological Electron Microscope Facility * [phone]                 *
> > * University of Hawaii at Manoa           * http://www.pbrc.hawaii.edu/bemf* 
> > ****************************************************************************

--
Joel B. Sheffield, Ph.D.
Biology Department, Temple University
1900 North 12th Street
Philadelphia, PA 19122
[hidden email]  
[phone], fax [phone]
http://astro.temple.edu/~jbs

[Bill Oliver-3]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

On Mon, 30 Jun 2008, Joel Sheffield wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
>
>
> One more point.  I may be old fashioned, but I was brought up to
> think that a research project involves a significant number of
> experiments, some of which "work" and others don't.  As the project
> progresses, the ratio of trustworthy to untrustworthy data increases
> until we can become confident that we can make a statement about the
> phenomenon.  This generally happens as we tweak the system to finally
> arrive at more optimal conditions.  Thus, a full data set of the
> experiments might be informative to a historian, but (even)
> embarassing to the investigator.
>

We live in a world in which we have a limited space to demonstrate results.  Most journal articles are there to demonstrate results, not provide comprehensive data sets. Thus, the appropriate illustration is that which best demonstrates the finding.  If you have 1000 trials and of these 700 show a result and 300 show nothing, one is not obligated to provide 1000 images, or even 100 or even 10.  It is more useful to provide an illustration of the positive finding and note the statistics, assuming that the appearance of the negative finding is intuitively clear (i.e. positive shows staining and negative is dark).  If the difference between positive and negative is unclear, it may be useful to provide an image of each, but I don't think that's usually necessry.
I don't see any need to take up multiple pages of images of negative results just to demonstrate statistics that are more properly described succinctly in the text.

There is simply no way to provide all of the data in most things in a couple of pages.


billo

[ian gibbins]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

I agree, billo, but that does assume that the statistics are done
properly.

If we are talking about data integrity and reliable reporting, then a
whole squirming bag of worms, leeches and various other unlikable
beasts is opened when we bring up the "S" word. As one who does a lot
of statistical analysis for folks around here, most people do not
understand either the principles or practice of the statistics of data
sampling and its subsequent investigation. There are plenty of
excellent programs around, as well as a whole raft of texts at various
levels, that allow sophisticated data analysis with relative ease these
days. Nevertheless, most students, and most researchers, at least in my
field, have only rudimentary knowledge.

There have been studies done in the medical literature showing that a
very high proportion of papers use the wrong stats, and that many of
those papers came to the wrong conclusion as a consequence. I don't
know if that applies equally in all fields, and in some journals at
least, the standards of statistical analysis are now set pretty high.
But there is still an awful lot done badly, if at all... So we get
oversamp[ling, undersampling, biased sampling (not Nyqusit, but at the
data level), we get inappropriate "normalisation", over-use of
percentages, inappropriate use of simple tests (eg t-tests, or oneway
ANOVAs) to deal with complex experimental designs, resulting in a
tendency to get "significance" when none exists, and then the infamous
phrase "... X and Y happened leading to Z, but it did not reach
statistical significance ... "


Unfortunately, there's plenty more where that came from...

IAN

>
> We live in a world in which we have a limited space to demonstrate
> results.  Most journal articles are there to demonstrate results, not
> provide comprehensive data sets. Thus, the appropriate illustration is
> that which best demonstrates the finding.  If you have 1000 trials and
> of these 700 show a result and 300 show nothing, one is not obligated
> to provide 1000 images, or even 100 or even 10.  It is more useful to
> provide an illustration of the positive finding and note the
> statistics, assuming that the appearance of the negative finding is
> intuitively clear (i.e. positive shows staining and negative is dark).
>  If the difference between positive and negative is unclear, it may be
> useful to provide an image of each, but I don't think that's usually
> necessry. I don't see any need to take up multiple pages of images of
> negative results just to demonstrate statistics that are more properly
> described succinctly in the text.
>
> There is simply no way to provide all of the data in most things in a
> couple of pages.
>
>
> billo
>
>

* * * * * * * * * * *
Prof Ian Gibbins
Anatomy & Histology
Flinders University
GPO Box 2100
Adelaide SA 5001
AUSTRALIA

[hidden email]
voice: [phone]
fax: [phone]

http://som.flinders.edu.au/FUSA/Anatomy/
http://www.flinders.edu.au/neuroscience

[Bill Oliver-3]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

Sure.  There are lots of problems with the statistics in scientific papers in many fields.  That's not the point, though.  The point is that you have limited *space* to present your data in a peer-reviewed journal.  I recently did a survey on a professional practice issue in my primary medical specialty as part of a Master's in Public Administration.  For the project, I did a full statistical analysis of 16 of the 50 hypotheses.  The detailed report that I turned into my research advisor was 140 pages long.

When I publish the paper, I will have 5-8 pages to fit my results in.  It doesn't matter whether or not the readers want to see the original data and evaluate the statistics on their own.  They are going to get 5-8 pages of summary results.  You simply can't fit 140 pages of analysis into 6 pages.

The same prinicple applies to imagery.

billo



On Tue, 1 Jul 2008, ian gibbins wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> I agree, billo, but that does assume that the statistics are done properly.
>
> If we are talking about data integrity and reliable reporting, then a whole
> squirming bag of worms, leeches and various other unlikable beasts is opened
> when we bring up the "S" word. As one who does a lot of statistical analysis
> for folks around here, most people do not understand either the principles or
> practice of the statistics of data sampling and its subsequent investigation.
> There are plenty of excellent programs around, as well as a whole raft of
> texts at various levels, that allow sophisticated data analysis with relative
> ease these days. Nevertheless, most students, and most researchers, at least
> in my field, have only rudimentary knowledge.
>
> There have been studies done in the medical literature showing that a very
> high proportion of papers use the wrong stats, and that many of those papers
> came to the wrong conclusion as a consequence. I don't know if that applies
> equally in all fields, and in some journals at least, the standards of
> statistical analysis are now set pretty high. But there is still an awful lot
> done badly, if at all... So we get oversamp[ling, undersampling, biased
> sampling (not Nyqusit, but at the data level), we get inappropriate
> "normalisation", over-use of percentages, inappropriate use of simple tests
> (eg t-tests, or oneway ANOVAs) to deal with complex experimental designs,
> resulting in a tendency to get "significance" when none exists, and then the
> infamous phrase "... X and Y happened leading to Z, but it did not reach
> statistical significance ... "
>
>
> Unfortunately, there's plenty more where that came from...
>
> IAN
>
>
>
>
>>
>> We live in a world in which we have a limited space to demonstrate results.
>> Most journal articles are there to demonstrate results, not provide
>> comprehensive data sets. Thus, the appropriate illustration is that which
>> best demonstrates the finding.  If you have 1000 trials and of these 700
>> show a result and 300 show nothing, one is not obligated to provide 1000
>> images, or even 100 or even 10.  It is more useful to provide an
>> illustration of the positive finding and note the statistics, assuming that
>> the appearance of the negative finding is intuitively clear (i.e. positive
>> shows staining and negative is dark).  If the difference between positive
>> and negative is unclear, it may be useful to provide an image of each, but
>> I don't think that's usually necessry. I don't see any need to take up
>> multiple pages of images of negative results just to demonstrate statistics
>> that are more properly described succinctly in the text.
>>
>> There is simply no way to provide all of the data in most things in a
>> couple of pages.
>>
>>
>> billo
>>
>>
>
> * * * * * * * * * * *
> Prof Ian Gibbins
> Anatomy & Histology
> Flinders University
> GPO Box 2100
> Adelaide SA 5001
> AUSTRALIA
>
> [hidden email]
> voice: [phone]
> fax: [phone]
>
> http://som.flinders.edu.au/FUSA/Anatomy/
> http://www.flinders.edu.au/neuroscience
>

billo
http://www.billoblog.com/billoblog

[lechristophe]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

The space limitation only applies to the article by itself. As someone
said earlier, digital space is cheap, I see nothing impossible in
joining a 140 pages pdf as a supplementary data to a 5 pages article,
so anyone who wants to go deeper in the data can, whereas those who
wants to see the refined and discussed stuff read the article...

Christophe

On Tue, Jul 1, 2008 at 14:22, Bill Oliver <[hidden email]> wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> Sure.  There are lots of problems with the statistics in scientific papers
> in many fields.  That's not the point, though.  The point is that you have
> limited *space* to present your data in a peer-reviewed journal.  I recently
> did a survey on a professional practice issue in my primary medical
> specialty as part of a Master's in Public Administration.  For the project,
> I did a full statistical analysis of 16 of the 50 hypotheses.  The detailed
> report that I turned into my research advisor was 140 pages long.
>
> When I publish the paper, I will have 5-8 pages to fit my results in.  It
> doesn't matter whether or not the readers want to see the original data and
> evaluate the statistics on their own.  They are going to get 5-8 pages of
> summary results.  You simply can't fit 140 pages of analysis into 6 pages.
>
> The same prinicple applies to imagery.
>
> billo
>
>
>
> On Tue, 1 Jul 2008, ian gibbins wrote:
>
>> Search the CONFOCAL archive at
>> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>>
>> I agree, billo, but that does assume that the statistics are done
>> properly.
>>
>> If we are talking about data integrity and reliable reporting, then a
>> whole squirming bag of worms, leeches and various other unlikable beasts is
>> opened when we bring up the "S" word. As one who does a lot of statistical
>> analysis for folks around here, most people do not understand either the
>> principles or practice of the statistics of data sampling and its subsequent
>> investigation. There are plenty of excellent programs around, as well as a
>> whole raft of texts at various levels, that allow sophisticated data
>> analysis with relative ease these days. Nevertheless, most students, and
>> most researchers, at least in my field, have only rudimentary knowledge.
>>
>> There have been studies done in the medical literature showing that a very
>> high proportion of papers use the wrong stats, and that many of those papers
>> came to the wrong conclusion as a consequence. I don't know if that applies
>> equally in all fields, and in some journals at least, the standards of
>> statistical analysis are now set pretty high. But there is still an awful
>> lot done badly, if at all... So we get oversamp[ling, undersampling, biased
>> sampling (not Nyqusit, but at the data level), we get inappropriate
>> "normalisation", over-use of percentages, inappropriate use of simple tests
>> (eg t-tests, or oneway ANOVAs) to deal with complex experimental designs,
>> resulting in a tendency to get "significance" when none exists, and then the
>> infamous phrase "... X and Y happened leading to Z, but it did not reach
>> statistical significance ... "
>>
>>
>> Unfortunately, there's plenty more where that came from...
>>
>> IAN
>>
>>
>>
>>
>>>
>>> We live in a world in which we have a limited space to demonstrate
>>> results. Most journal articles are there to demonstrate results, not provide
>>> comprehensive data sets. Thus, the appropriate illustration is that which
>>> best demonstrates the finding.  If you have 1000 trials and of these 700
>>> show a result and 300 show nothing, one is not obligated to provide 1000
>>> images, or even 100 or even 10.  It is more useful to provide an
>>> illustration of the positive finding and note the statistics, assuming that
>>> the appearance of the negative finding is intuitively clear (i.e. positive
>>> shows staining and negative is dark).  If the difference between positive
>>> and negative is unclear, it may be useful to provide an image of each, but I
>>> don't think that's usually necessry. I don't see any need to take up
>>> multiple pages of images of negative results just to demonstrate statistics
>>> that are more properly described succinctly in the text.
>>>
>>> There is simply no way to provide all of the data in most things in a
>>> couple of pages.
>>>
>>>
>>> billo
>>>
>>>
>>
>> * * * * * * * * * * *
>> Prof Ian Gibbins
>> Anatomy & Histology
>> Flinders University
>> GPO Box 2100
>> Adelaide SA 5001
>> AUSTRALIA
>>
>> [hidden email]
>> voice: [phone]
>> fax: [phone]
>>
>> http://som.flinders.edu.au/FUSA/Anatomy/
>> http://www.flinders.edu.au/neuroscience
>>
>
> billo
> http://www.billoblog.com/billoblog
>

[Larry Tague]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

Christophe,

This is very true, and there are storage DBs for images that are
starting to evolve, that if standardized, could be part of the solution
to some of these problems, i.e., the Open Microscopy Environment
(http://www.openmicroscopy.org/).  I am not sure that journals will want
to take on the responsibility of data storage (maybe for the right
publication price). However, as in the past, it should probably be the
universities that shoulder the responsibility of being the "keepers of
the data" linked to appropriate publications, and not just in the labs.  
This could be a library function of sorts possibly associated with the
storage and distribution of electronic thesis and dissertations (private
research... forgive me, I am viewing this from a university
perspective), and for private research... maybe they could contract with
universities for storage of raw data once the patent pending questions
have been laid to rest. Meta data in the form of raw digital information
is very important because meta data research based on a "modern" look at
old information holds great promise. The saying "hind sight is 20 20"
takes on real meaning in view of meta data analysis. Even if a data set
is completely acceptable today to peer review and detailed analysis does
not mean it will always be that way, or that with new ways of analysis,
additional future information can be realized. My 2 cents worth...

Larry

Christophe Leterrier wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> The space limitation only applies to the article by itself. As someone
> said earlier, digital space is cheap, I see nothing impossible in
> joining a 140 pages pdf as a supplementary data to a 5 pages article,
> so anyone who wants to go deeper in the data can, whereas those who
> wants to see the refined and discussed stuff read the article...
>
> Christophe
>
> On Tue, Jul 1, 2008 at 14:22, Bill Oliver <[hidden email]> wrote:
>  
>> Search the CONFOCAL archive at
>> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>>
>> Sure.  There are lots of problems with the statistics in scientific papers
>> in many fields.  That's not the point, though.  The point is that you have
>> limited *space* to present your data in a peer-reviewed journal.  I recently
>> did a survey on a professional practice issue in my primary medical
>> specialty as part of a Master's in Public Administration.  For the project,
>> I did a full statistical analysis of 16 of the 50 hypotheses.  The detailed
>> report that I turned into my research advisor was 140 pages long.
>>
>> When I publish the paper, I will have 5-8 pages to fit my results in.  It
>> doesn't matter whether or not the readers want to see the original data and
>> evaluate the statistics on their own.  They are going to get 5-8 pages of
>> summary results.  You simply can't fit 140 pages of analysis into 6 pages.
>>
>> The same prinicple applies to imagery.
>>
>> billo
>>
>>
>>
>> On Tue, 1 Jul 2008, ian gibbins wrote:
>>
>>    
>>> Search the CONFOCAL archive at
>>> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>>>
>>> I agree, billo, but that does assume that the statistics are done
>>> properly.
>>>
>>> If we are talking about data integrity and reliable reporting, then a
>>> whole squirming bag of worms, leeches and various other unlikable beasts is
>>> opened when we bring up the "S" word. As one who does a lot of statistical
>>> analysis for folks around here, most people do not understand either the
>>> principles or practice of the statistics of data sampling and its subsequent
>>> investigation. There are plenty of excellent programs around, as well as a
>>> whole raft of texts at various levels, that allow sophisticated data
>>> analysis with relative ease these days. Nevertheless, most students, and
>>> most researchers, at least in my field, have only rudimentary knowledge.
>>>
>>> There have been studies done in the medical literature showing that a very
>>> high proportion of papers use the wrong stats, and that many of those papers
>>> came to the wrong conclusion as a consequence. I don't know if that applies
>>> equally in all fields, and in some journals at least, the standards of
>>> statistical analysis are now set pretty high. But there is still an awful
>>> lot done badly, if at all... So we get oversamp[ling, undersampling, biased
>>> sampling (not Nyqusit, but at the data level), we get inappropriate
>>> "normalisation", over-use of percentages, inappropriate use of simple tests
>>> (eg t-tests, or oneway ANOVAs) to deal with complex experimental designs,
>>> resulting in a tendency to get "significance" when none exists, and then the
>>> infamous phrase "... X and Y happened leading to Z, but it did not reach
>>> statistical significance ... "
>>>
>>>
>>> Unfortunately, there's plenty more where that came from...
>>>
>>> IAN
>>>
>>>
>>>
>>>
>>>      
>>>> We live in a world in which we have a limited space to demonstrate
>>>> results. Most journal articles are there to demonstrate results, not provide
>>>> comprehensive data sets. Thus, the appropriate illustration is that which
>>>> best demonstrates the finding.  If you have 1000 trials and of these 700
>>>> show a result and 300 show nothing, one is not obligated to provide 1000
>>>> images, or even 100 or even 10.  It is more useful to provide an
>>>> illustration of the positive finding and note the statistics, assuming that
>>>> the appearance of the negative finding is intuitively clear (i.e. positive
>>>> shows staining and negative is dark).  If the difference between positive
>>>> and negative is unclear, it may be useful to provide an image of each, but I
>>>> don't think that's usually necessry. I don't see any need to take up
>>>> multiple pages of images of negative results just to demonstrate statistics
>>>> that are more properly described succinctly in the text.
>>>>
>>>> There is simply no way to provide all of the data in most things in a
>>>> couple of pages.
>>>>
>>>>
>>>> billo
>>>>
>>>>
>>>>        
>>> * * * * * * * * * * *
>>> Prof Ian Gibbins
>>> Anatomy & Histology
>>> Flinders University
>>> GPO Box 2100
>>> Adelaide SA 5001
>>> AUSTRALIA
>>>
>>> [hidden email]
>>> voice: [phone]
>>> fax: [phone]
>>>
>>> http://som.flinders.edu.au/FUSA/Anatomy/
>>> http://www.flinders.edu.au/neuroscience
>>>
>>>      
>> billo
>> http://www.billoblog.com/billoblog
>>
>>    

--
Larry Tague
Co-Investigator BBHSL*
Co-Director of MECCA**
Research Associate (Dept. of Physiology)
University of Tennessee Health Science Center
894 Union Ave.
Memphis, TN 38163
Phone Bus.: 901-448-7152
Phone FAX:  [phone]
e-mail:[hidden email] or
       [hidden email]

*BBHSL "Building Bridges to Health Science Literacy" URL: http://bbhsl.mecca.org, a Science Education Partnership Award (SEPA). Supported by the National Center of Research Resources (NCRR) of the National Institutes of Health (NIH).
**MECCA (Memphis Educational Computer Connectivity Alliance)
URL: http://www.mecca.org/.  Originally support by the National Science Foundation's "Networking Infrastructure or Education" program.

CONFIDENTIALITY NOTICE: This email and any files transmitted with it are confidential and are intended solely for the use of the individual or entity to whom they are addressed. This communication may contain protected health information, or other legally privileged, confidential, or proprietary information. If you are not the intended recipient or the individual responsible for delivering the email to the recipient, please be advised that you have received this email in error and that any use, dissemination, forwarding, printing, or copying of this email is strictly prohibited. If you have received this email in error, please immediately notify the sender, disregard the foregoing message, and delete the message. We apologize for any inconvenience this may have caused. IMPORTANT NOTE: Confidential health   information is protected by state and federal law, including, but not limited to, the Health Insurance Portability and Accountability Act of 1996 and related regulations.

[Bill Oliver-3]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

On Tue, 1 Jul 2008, Christophe Leterrier wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> The space limitation only applies to the article by itself. As someone
> said earlier, digital space is cheap, I see nothing impossible in
> joining a 140 pages pdf as a supplementary data to a 5 pages article,
> so anyone who wants to go deeper in the data can, whereas those who
> wants to see the refined and discussed stuff read the article...
>
> Christophe
>

That's fine.  But that's not how most journals work right now, as far as
I know, and we are talking about how it works now.  If, five or ten
years from now, paper journals are no longer the standard, then the
limitations won't exist.  As long as they do, however, they do.

billo

[Bill Oliver-3]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

On Tue, 1 Jul 2008, Christophe Leterrier wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> The space limitation only applies to the article by itself. As someone
> said earlier, digital space is cheap, I see nothing impossible in
> joining a 140 pages pdf as a supplementary data to a 5 pages article,
> so anyone who wants to go deeper in the data can, whereas those who
> wants to see the refined and discussed stuff read the article...


While this doesn't apply to most research, and may rarely apply to stuff in microscopy, if folk are setting up ethics guidelines they should realize that sometimes the *unethical* thing to do is make all the data available.

Let me use my recent study as an example.  The study is one in which I looked at the effect of litigation by TASER, Inc. against Medical Examiners who call the use of TASER a cause of death (COD) or contributing factor to death.  In most cases these suits are brought only to force a change in the COD.  However, in two cases TASER has brought suit personally against the Medical Examiner and in at least one has sought compensatory and punitive damages for opinions expressed by the Medical Examiner about the dangers of the use of TASER.

I did a survey of Medical Examiners looking at if and how the threat of such litigation affected their diagnoses and will affect their diagnoses in the future.  The problem is that the number of Medical Examiners is small and even though no names are attached to the survey, it is possible to link individuals to particular answers with enough effort.  The trivial example is that of the two cases TASER has brought suit personally, one is male and the other female. Thus, if the answer to "has TASER brought suit against you" is "yes" and you know the answer to "what is your gender," you know who it is.  That is the most trivial example, but a little more work would allow a good investigator to determine individual identities and use that data for litigation.

Thus, I do not plan to release the data for public consumption, since part of that public are people who would use this for further litigation.  Instead, I have placed the data in escrow with a forensic sciences institute in Europe.  If someone has a specific question, folk will see if it can be answered in a way that will not allow the mining for individual information.

I don't know if there's something analogous for microscopy -- none of the work I did in confocal microscopy was like that.  However, broad and cut-and-dried statements about such things sometimes don't acknowledge that all circumstances are not the same.

This has also come up with the American Medical Association where, in response to the brouhaha about Guantanamo, they passed an ethical rule stating that physicians cannot observe police interviews lest they be tainted by any abuse that happens.  It is clear that no forensic medicine types had input to that ethical rule -- it would be malpractice if we did *not* observe such interviews in such things as reconstructing the circumstances of child deaths.  Thus, in their hurry to be politically correct about a dramatic circumstance that is very rare, they made "unethical" a fundamental,important and good practice in forensic pathology.  It's no surprise that only about 22% of physicians belong to the AMA any more.

Accordingly, I am always a little antsy when people start opining about how "all" of anything ought to be done "all" of the time, else it is an "ethical" violation.


billo

[lechristophe]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

> That's fine.  But that's not how most journals work right now, as far as
> I know, and we are talking about how it works now.  If, five or ten
> years from now, paper journals are no longer the standard, then the
> limitations won't exist.  As long as they do, however, they do.
>
> billo
>

This is true, however it is not uncommon to see 40-60 pages
supplementary data in Science or Nature papers nowadays. Furthermore,
as Larry pointed out, nothing keeps you today from hosting your 140
pages pdf by yourself (your university, your lab...) and link to it in
your article, as a web link only takes half a line in a paper !

[Mike Esterman]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

I have been watching this discussion with much interest.  The issue that
Bill Oliver is addressing about making "all" of the data available has an
additional pitfall.  I am retired from a major pharmaceutical company and
scientist in our drug discovery, toxicology and clinical research divisions
all publish.  No pharmaceutical company is going to allow public access to
research data for network security and legal reasons.  So while making the
all data and images from a published study available on a website might work
for a university it would not be feasible for industry like pharma.  

I would suggest that all journals have guidelines about what which image
process operations are allowable and they must be reported - software used
and operations done as part of methods and materials, I would suggest
reporting acquisition parameters, if possible.  Then if a scientist is
discovered performing undocumented process there should be some penalty. If
a person does unethical manipulations often enough they will be found out.

My thoughts on the thread.

Mike Esterman
Imaging Consultant

-----Original Message-----
From: Confocal Microscopy List [mailto:[hidden email]] On
Behalf Of Bill Oliver
Sent: Tuesday, July 01, 2008 1:14 PM
To: [hidden email]
Subject: Re: An alarming amount of (statistical) image manipulation

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

On Tue, 1 Jul 2008, Christophe Leterrier wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> The space limitation only applies to the article by itself. As someone
> said earlier, digital space is cheap, I see nothing impossible in
> joining a 140 pages pdf as a supplementary data to a 5 pages article,
> so anyone who wants to go deeper in the data can, whereas those who
> wants to see the refined and discussed stuff read the article...


While this doesn't apply to most research, and may rarely apply to stuff in
microscopy, if folk are setting up ethics guidelines they should realize
that sometimes the *unethical* thing to do is make all the data available.

Let me use my recent study as an example.  The study is one in which I
looked at the effect of litigation by TASER, Inc. against Medical Examiners
who call the use of TASER a cause of death (COD) or contributing factor to
death.  In most cases these suits are brought only to force a change in the
COD.  However, in two cases TASER has brought suit personally against the
Medical Examiner and in at least one has sought compensatory and punitive
damages for opinions expressed by the Medical Examiner about the dangers of
the use of TASER.

I did a survey of Medical Examiners looking at if and how the threat of such
litigation affected their diagnoses and will affect their diagnoses in the
future.  The problem is that the number of Medical Examiners is small and
even though no names are attached to the survey, it is possible to link
individuals to particular answers with enough effort.  The trivial example
is that of the two cases TASER has brought suit personally, one is male and
the other female. Thus, if the answer to "has TASER brought suit against
you" is "yes" and you know the answer to "what is your gender," you know who
it is.  That is the most trivial example, but a little more work would allow
a good investigator to determine individual identities and use that data for
litigation.

Thus, I do not plan to release the data for public consumption, since part
of that public are people who would use this for further litigation.
Instead, I have placed the data in escrow with a forensic sciences institute
in Europe.  If someone has a specific question, folk will see if it can be
answered in a way that will not allow the mining for individual information.

I don't know if there's something analogous for microscopy -- none of the
work I did in confocal microscopy was like that.  However, broad and
cut-and-dried statements about such things sometimes don't acknowledge that
all circumstances are not the same.

This has also come up with the American Medical Association where, in
response to the brouhaha about Guantanamo, they passed an ethical rule
stating that physicians cannot observe police interviews lest they be
tainted by any abuse that happens.  It is clear that no forensic medicine
types had input to that ethical rule -- it would be malpractice if we did
*not* observe such interviews in such things as reconstructing the
circumstances of child deaths.  Thus, in their hurry to be politically
correct about a dramatic circumstance that is very rare, they made
"unethical" a fundamental,important and good practice in forensic pathology.
It's no surprise that only about 22% of physicians belong to the AMA any
more.

Accordingly, I am always a little antsy when people start opining about how
"all" of anything ought to be done "all" of the time, else it is an
"ethical" violation.


billo

[Guy Cox]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

Well, I don't know what web space all you good folk have available to you but there's no way I have access to anywhere enough to post the several gigabytes of images which lie behind a typical confocal / multiphoton / second harmonic paper.  My University web site allows me something like 10Mb which wouldn't hold a single Z-stack or spectral series.  And it will go as soon as I leave the University - so much for being archival.  My private (paid for) site has a lot more but still nowhere near enough, and it's only there as long as I pay the bills.  

It's all very well to say you can but a terabyte drive for however many dollars, but that's just the start.  There's the power to run it (and replace it every few years), there's the backup facility without which it's not remotely archival, and most of all there's the bandwidth needed to access it.  On the scale people are talking about that's a huge operation.  A moderately popular paid site I administer, for example, has 4 GB/month allowed - and comes close to it every month.  That's not involving any confocal images!  When I put the Pawley/Cox Zeiss Diffraction Kit simulation on my personal site, and announced it to this list, it reached the site quota within 48 hours, and shut the whole thing down.

About the only way you could have a sensible archive would be to get authors to mail a few DVDs to some sort of repository.  Then anyone would wanted to check the data could pay a search fee and get copies of the disks.  The search fee would not be cheap, though.  And since most Universities have rules requiring authors to keep data for X years anyway, I can't see the point.  In any case, anyone who is being selective with the data they publish is likely to be selective in what they archive, too.

                                                                  Guy

 
Optical Imaging Techniques in Cell Biology
by Guy Cox    CRC Press / Taylor & Francis
    http://www.guycox.com/optical.htm
______________________________________________
Associate Professor Guy Cox, MA, DPhil(Oxon)
Electron Microscope Unit, Madsen Building F09,
University of Sydney, NSW 2006
______________________________________________
Phone [phone]     Fax [phone]
Mobile [phone]
______________________________________________
     http://www.guycox.net
-----Original Message-----
From: Confocal Microscopy List [mailto:[hidden email]] On Behalf Of Christophe Leterrier
Sent: Wednesday, 2 July 2008 3:16 AM
To: [hidden email]
Subject: Re: An alarming amount of (statistical) image manipulation

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

> That's fine.  But that's not how most journals work right now, as far
> as I know, and we are talking about how it works now.  If, five or ten
> years from now, paper journals are no longer the standard, then the
> limitations won't exist.  As long as they do, however, they do.
>
> billo
>

This is true, however it is not uncommon to see 40-60 pages supplementary data in Science or Nature papers nowadays. Furthermore, as Larry pointed out, nothing keeps you today from hosting your 140 pages pdf by yourself (your university, your lab...) and link to it in your article, as a web link only takes half a line in a paper !

No virus found in this incoming message.
Checked by AVG.
Version: 7.5.526 / Virus Database: 270.4.3/1527 - Release Date: 30/06/2008 6:07 PM
 

No virus found in this outgoing message.
Checked by AVG.
Version: 7.5.526 / Virus Database: 270.4.3/1529 - Release Date: 1/07/2008 7:23 PM
 

[Larry Tague]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

Guy,

You bet there would be archival problems, but no worse
(for the most part) than what we currently have with
electronic thesis and dissertations.  Many universities
are now storing 400+ very large documents/yr with images,
etc., and some of these institutions also publish
supporting data, i.e., CalTech.  However, with that said,
your statement, "And since most Universities have rules
requiring authors to keep data for X years anyway, I can't
see the point" means that the data is already stored...
make it part of the data stream for publications research.
Yes, it will take organization, etc., but why loose or
close data, unless for national security, that at least in
public institutions, and in some private institutions, the
public has already paid for with their hard earned taxes.

If the original data is already stored, and today that
means in many or most cases stored in digital format
(digital images are the only kind I collect these days),
and the authors do not want to allow others access, you
begin to wonder why. I would suggest with public data,
storage and public access should be a requirement before
receiving public funds for research.  Already with some
NIH grant programs (not bench research) there is an
external evaluation requirement that must be included in
the budget request... no if ands or buts. For bench
research, maybe accessible data storage should now be a
budget item in the proposal process..

Yes, this diatribe strays somewhat from the original image
manipulation question, but if there is no data to check or
continue using, how could you possibly know if an improper
image analysis had been applied.  Even if rules for image
analysis exist, there is no good way to be sure mistakes
were not made... especially when there are questions post
publication and no raw data to check. Cheers!

Larry
 
On Wed, 2 Jul 2008 12:14:13 +1000
  Guy Cox <[hidden email]> wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> Well, I don't know what web space all you good folk have
>available to you but there's no way I have access to
>anywhere enough to post the several gigabytes of images
>which lie behind a typical confocal / multiphoton /
>second harmonic paper.  My University web site allows me
>something like 10Mb which wouldn't hold a single Z-stack
>or spectral series.  And it will go as soon as I leave
>the University - so much for being archival.  My private
>(paid for) site has a lot more but still nowhere near
>enough, and it's only there as long as I pay the bills.  
>
> It's all very well to say you can but a terabyte drive
>for however many dollars, but that's just the start.
> There's the power to run it (and replace it every few
>years), there's the backup facility without which it's
>not remotely archival, and most of all there's the
>bandwidth needed to access it.  On the scale people are
>talking about that's a huge operation.  A moderately
>popular paid site I administer, for example, has 4
>GB/month allowed - and comes close to it every month.
> That's not involving any confocal images!  When I put
>the Pawley/Cox Zeiss Diffraction Kit simulation on my
>personal site, and announced it to this list, it reached
>the site quota within 48 hours, and shut the whole thing
>down.
>
> About the only way you could have a sensible archive
>would be to get authors to mail a few DVDs to some sort
>of repository.  Then anyone would wanted to check the
>data could pay a search fee and get copies of the disks.
> The search fee would not be cheap, though.  And since
>most Universities have rules requiring authors to keep
>data for X years anyway, I can't see the point.  In any
>case, anyone who is being selective with the data they
>publish is likely to be selective in what they archive,
>too.
>
>                                                          
>       Guy
>
>
> Optical Imaging Techniques in Cell Biology
> by Guy Cox    CRC Press / Taylor & Francis
>    http://www.guycox.com/optical.htm
> ______________________________________________
> Associate Professor Guy Cox, MA, DPhil(Oxon)
> Electron Microscope Unit, Madsen Building F09,
> University of Sydney, NSW 2006
> ______________________________________________
> Phone [phone]     Fax [phone]
> Mobile [phone]
> ______________________________________________
>     http://www.guycox.net
> -----Original Message-----
>From: Confocal Microscopy List
>[mailto:[hidden email]] On Behalf Of
>Christophe Leterrier
> Sent: Wednesday, 2 July 2008 3:16 AM
> To: [hidden email]
> Subject: Re: An alarming amount of (statistical) image
>manipulation
>
> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
>> That's fine.  But that's not how most journals work
>>right now, as far
>> as I know, and we are talking about how it works now.
>> If, five or ten
>> years from now, paper journals are no longer the
>>standard, then the
>> limitations won't exist.  As long as they do, however,
>>they do.
>>
>> billo
>>
>
> This is true, however it is not uncommon to see 40-60
>pages supplementary data in Science or Nature papers
>nowadays. Furthermore, as Larry pointed out, nothing
>keeps you today from hosting your 140 pages pdf by
>yourself (your university, your lab...) and link to it in
>your article, as a web link only takes half a line in a
>paper !
>
> No virus found in this incoming message.
> Checked by AVG.
> Version: 7.5.526 / Virus Database: 270.4.3/1527 -
>Release Date: 30/06/2008 6:07 PM
>
>
> No virus found in this outgoing message.
> Checked by AVG.
> Version: 7.5.526 / Virus Database: 270.4.3/1529 -
>Release Date: 1/07/2008 7:23 PM
>

Larry Tague
Co-Investigator BBHSL*
Co-Director of MECCA**
Research Associate                       Dept. of
Physiology
Phone Bus.: 901-448-7152                 U.T. Memphis
Phone FAX:  [phone]                 894 Union Ave.
e-mail:[hidden email]          Memphis, TN
        [hidden email]                  38163
        [hidden email]

*BBHSL (Building Bridges to Health Science Literacy). An
NIH Science Education Partnership Award (SEPA)2005-2010
URL: http://bbhsl.mecca.org
**MECCA (Memphis Educational Computer Connectivity
Alliance)
URL: http://www.mecca.org/

[Guy Cox]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

The scale is several orders of magnitude larger than the storage of
theses (which, by the way, this university does on paper - and that's
probably still the most secure archive.)  And I think few, if any,
Universities
grant Web access to theses even if they do store them digitally.

                                                                  Guy

Optical Imaging Techniques in Cell Biology
by Guy Cox    CRC Press / Taylor & Francis
    http://www.guycox.com/optical.htm
______________________________________________
Associate Professor Guy Cox, MA, DPhil(Oxon)
Electron Microscope Unit, Madsen Building F09,
University of Sydney, NSW 2006
______________________________________________
Phone [phone]     Fax [phone]
Mobile [phone]
______________________________________________
 

-----Original Message-----
From: Confocal Microscopy List [mailto:[hidden email]] On
Behalf Of Larry Tague
Sent: Wednesday, 2 July 2008 2:52 PM
To: [hidden email]
Subject: Re: An alarming amount of (statistical) image manipulation

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

Guy,

You bet there would be archival problems, but no worse
(for the most part) than what we currently have with
electronic thesis and dissertations.  Many universities
are now storing 400+ very large documents/yr with images,
etc., and some of these institutions also publish
supporting data, i.e., CalTech.  However, with that said,
your statement, "And since most Universities have rules
requiring authors to keep data for X years anyway, I can't
see the point" means that the data is already stored...
make it part of the data stream for publications research.
Yes, it will take organization, etc., but why loose or
close data, unless for national security, that at least in
public institutions, and in some private institutions, the
public has already paid for with their hard earned taxes.

If the original data is already stored, and today that
means in many or most cases stored in digital format
(digital images are the only kind I collect these days),
and the authors do not want to allow others access, you
begin to wonder why. I would suggest with public data,
storage and public access should be a requirement before
receiving public funds for research.  Already with some
NIH grant programs (not bench research) there is an
external evaluation requirement that must be included in
the budget request... no if ands or buts. For bench
research, maybe accessible data storage should now be a
budget item in the proposal process..

Yes, this diatribe strays somewhat from the original image
manipulation question, but if there is no data to check or
continue using, how could you possibly know if an improper
image analysis had been applied.  Even if rules for image
analysis exist, there is no good way to be sure mistakes
were not made... especially when there are questions post
publication and no raw data to check. Cheers!

Larry
 
On Wed, 2 Jul 2008 12:14:13 +1000
  Guy Cox <[hidden email]> wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> Well, I don't know what web space all you good folk have
>available to you but there's no way I have access to
>anywhere enough to post the several gigabytes of images
>which lie behind a typical confocal / multiphoton /
>second harmonic paper.  My University web site allows me
>something like 10Mb which wouldn't hold a single Z-stack
>or spectral series.  And it will go as soon as I leave
>the University - so much for being archival.  My private
>(paid for) site has a lot more but still nowhere near
>enough, and it's only there as long as I pay the bills.  
>
> It's all very well to say you can but a terabyte drive
>for however many dollars, but that's just the start.
> There's the power to run it (and replace it every few
>years), there's the backup facility without which it's
>not remotely archival, and most of all there's the
>bandwidth needed to access it.  On the scale people are
>talking about that's a huge operation.  A moderately
>popular paid site I administer, for example, has 4
>GB/month allowed - and comes close to it every month.
> That's not involving any confocal images!  When I put
>the Pawley/Cox Zeiss Diffraction Kit simulation on my
>personal site, and announced it to this list, it reached
>the site quota within 48 hours, and shut the whole thing
>down.
>
> About the only way you could have a sensible archive
>would be to get authors to mail a few DVDs to some sort
>of repository.  Then anyone would wanted to check the
>data could pay a search fee and get copies of the disks.
> The search fee would not be cheap, though.  And since
>most Universities have rules requiring authors to keep
>data for X years anyway, I can't see the point.  In any
>case, anyone who is being selective with the data they
>publish is likely to be selective in what they archive,
>too.
>
>                                                          
>       Guy
>
>
> Optical Imaging Techniques in Cell Biology
> by Guy Cox    CRC Press / Taylor & Francis
>    http://www.guycox.com/optical.htm
> ______________________________________________
> Associate Professor Guy Cox, MA, DPhil(Oxon)
> Electron Microscope Unit, Madsen Building F09,
> University of Sydney, NSW 2006
> ______________________________________________
> Phone [phone]     Fax [phone]
> Mobile [phone]
> ______________________________________________
>     http://www.guycox.net
> -----Original Message-----
>From: Confocal Microscopy List
>[mailto:[hidden email]] On Behalf Of
>Christophe Leterrier
> Sent: Wednesday, 2 July 2008 3:16 AM
> To: [hidden email]
> Subject: Re: An alarming amount of (statistical) image
>manipulation
>
> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
>> That's fine.  But that's not how most journals work
>>right now, as far
>> as I know, and we are talking about how it works now.
>> If, five or ten
>> years from now, paper journals are no longer the
>>standard, then the
>> limitations won't exist.  As long as they do, however,
>>they do.
>>
>> billo
>>
>
> This is true, however it is not uncommon to see 40-60
>pages supplementary data in Science or Nature papers
>nowadays. Furthermore, as Larry pointed out, nothing
>keeps you today from hosting your 140 pages pdf by
>yourself (your university, your lab...) and link to it in
>your article, as a web link only takes half a line in a
>paper !
>
> No virus found in this incoming message.
> Checked by AVG.
> Version: 7.5.526 / Virus Database: 270.4.3/1527 -
>Release Date: 30/06/2008 6:07 PM
>
>
> No virus found in this outgoing message.
> Checked by AVG.
> Version: 7.5.526 / Virus Database: 270.4.3/1529 -
>Release Date: 1/07/2008 7:23 PM
>

Larry Tague
Co-Investigator BBHSL*
Co-Director of MECCA**
Research Associate                       Dept. of
Physiology
Phone Bus.: 901-448-7152                 U.T. Memphis
Phone FAX:  [phone]                 894 Union Ave.
e-mail:[hidden email]          Memphis, TN
        [hidden email]                  38163
        [hidden email]

*BBHSL (Building Bridges to Health Science Literacy). An
NIH Science Education Partnership Award (SEPA)2005-2010
URL: http://bbhsl.mecca.org
**MECCA (Memphis Educational Computer Connectivity
Alliance)
URL: http://www.mecca.org/

No virus found in this incoming message.
Checked by AVG.
Version: 7.5.526 / Virus Database: 270.4.3/1529 - Release Date:
1/07/2008 7:23 PM
 

[lechristophe]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

I agree with Guy that finding lots of web space can be difficult.
However, things are moving slowly, and initatives such as Google Open
Source Science Data Hosting are trying to adress this problem :
http://blog.wired.com/wiredscience/2008/01/google-to-provi.html
This whole "open source" science is a very interesting topic that
generates a lot of initiatives now. Of course one has to be careful
about the open source licenses used for data, but we're lucky that
software people already thought a lot about this issues. Why not
imagine that a public-founded organism does the same kind of thing for
public-founded research than Google is doing, i.e. hosting the data at
a much larger scale than what is possible at the personal/university
level ?

Christophe

On Wed, Jul 2, 2008 at 10:47, Guy Cox <[hidden email]> wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> The scale is several orders of magnitude larger than the storage of
> theses (which, by the way, this university does on paper - and that's
> probably still the most secure archive.)  And I think few, if any,
> Universities
> grant Web access to theses even if they do store them digitally.
>
>                                                                  Guy
>
> Optical Imaging Techniques in Cell Biology
> by Guy Cox    CRC Press / Taylor & Francis
>    http://www.guycox.com/optical.htm
> ______________________________________________
> Associate Professor Guy Cox, MA, DPhil(Oxon)
> Electron Microscope Unit, Madsen Building F09,
> University of Sydney, NSW 2006
> ______________________________________________
> Phone [phone]     Fax [phone]
> Mobile [phone]
> ______________________________________________
>
>
> -----Original Message-----
> From: Confocal Microscopy List [mailto:[hidden email]] On
> Behalf Of Larry Tague
> Sent: Wednesday, 2 July 2008 2:52 PM
> To: [hidden email]
> Subject: Re: An alarming amount of (statistical) image manipulation
>
> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> Guy,
>
> You bet there would be archival problems, but no worse
> (for the most part) than what we currently have with
> electronic thesis and dissertations.  Many universities
> are now storing 400+ very large documents/yr with images,
> etc., and some of these institutions also publish
> supporting data, i.e., CalTech.  However, with that said,
> your statement, "And since most Universities have rules
> requiring authors to keep data for X years anyway, I can't
> see the point" means that the data is already stored...
> make it part of the data stream for publications research.
> Yes, it will take organization, etc., but why loose or
> close data, unless for national security, that at least in
> public institutions, and in some private institutions, the
> public has already paid for with their hard earned taxes.
>
> If the original data is already stored, and today that
> means in many or most cases stored in digital format
> (digital images are the only kind I collect these days),
> and the authors do not want to allow others access, you
> begin to wonder why. I would suggest with public data,
> storage and public access should be a requirement before
> receiving public funds for research.  Already with some
> NIH grant programs (not bench research) there is an
> external evaluation requirement that must be included in
> the budget request... no if ands or buts. For bench
> research, maybe accessible data storage should now be a
> budget item in the proposal process..
>
> Yes, this diatribe strays somewhat from the original image
> manipulation question, but if there is no data to check or
> continue using, how could you possibly know if an improper
> image analysis had been applied.  Even if rules for image
> analysis exist, there is no good way to be sure mistakes
> were not made... especially when there are questions post
> publication and no raw data to check. Cheers!
>
> Larry
>
> On Wed, 2 Jul 2008 12:14:13 +1000
>  Guy Cox <[hidden email]> wrote:
>> Search the CONFOCAL archive at
>> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>>
>> Well, I don't know what web space all you good folk have
>>available to you but there's no way I have access to
>>anywhere enough to post the several gigabytes of images
>>which lie behind a typical confocal / multiphoton /
>>second harmonic paper.  My University web site allows me
>>something like 10Mb which wouldn't hold a single Z-stack
>>or spectral series.  And it will go as soon as I leave
>>the University - so much for being archival.  My private
>>(paid for) site has a lot more but still nowhere near
>>enough, and it's only there as long as I pay the bills.
>>
>> It's all very well to say you can but a terabyte drive
>>for however many dollars, but that's just the start.
>> There's the power to run it (and replace it every few
>>years), there's the backup facility without which it's
>>not remotely archival, and most of all there's the
>>bandwidth needed to access it.  On the scale people are
>>talking about that's a huge operation.  A moderately
>>popular paid site I administer, for example, has 4
>>GB/month allowed - and comes close to it every month.
>> That's not involving any confocal images!  When I put
>>the Pawley/Cox Zeiss Diffraction Kit simulation on my
>>personal site, and announced it to this list, it reached
>>the site quota within 48 hours, and shut the whole thing
>>down.
>>
>> About the only way you could have a sensible archive
>>would be to get authors to mail a few DVDs to some sort
>>of repository.  Then anyone would wanted to check the
>>data could pay a search fee and get copies of the disks.
>> The search fee would not be cheap, though.  And since
>>most Universities have rules requiring authors to keep
>>data for X years anyway, I can't see the point.  In any
>>case, anyone who is being selective with the data they
>>publish is likely to be selective in what they archive,
>>too.
>>
>>
>>       Guy
>>
>>
>> Optical Imaging Techniques in Cell Biology
>> by Guy Cox    CRC Press / Taylor & Francis
>>    http://www.guycox.com/optical.htm
>> ______________________________________________
>> Associate Professor Guy Cox, MA, DPhil(Oxon)
>> Electron Microscope Unit, Madsen Building F09,
>> University of Sydney, NSW 2006
>> ______________________________________________
>> Phone [phone]     Fax [phone]
>> Mobile [phone]
>> ______________________________________________
>>     http://www.guycox.net
>> -----Original Message-----
>>From: Confocal Microscopy List
>>[mailto:[hidden email]] On Behalf Of
>>Christophe Leterrier
>> Sent: Wednesday, 2 July 2008 3:16 AM
>> To: [hidden email]
>> Subject: Re: An alarming amount of (statistical) image
>>manipulation
>>
>> Search the CONFOCAL archive at
>> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>>
>>> That's fine.  But that's not how most journals work
>>>right now, as far
>>> as I know, and we are talking about how it works now.
>>> If, five or ten
>>> years from now, paper journals are no longer the
>>>standard, then the
>>> limitations won't exist.  As long as they do, however,
>>>they do.
>>>
>>> billo
>>>
>>
>> This is true, however it is not uncommon to see 40-60
>>pages supplementary data in Science or Nature papers
>>nowadays. Furthermore, as Larry pointed out, nothing
>>keeps you today from hosting your 140 pages pdf by
>>yourself (your university, your lab...) and link to it in
>>your article, as a web link only takes half a line in a
>>paper !
>>
>> No virus found in this incoming message.
>> Checked by AVG.
>> Version: 7.5.526 / Virus Database: 270.4.3/1527 -
>>Release Date: 30/06/2008 6:07 PM
>>
>>
>> No virus found in this outgoing message.
>> Checked by AVG.
>> Version: 7.5.526 / Virus Database: 270.4.3/1529 -
>>Release Date: 1/07/2008 7:23 PM
>>
>
> Larry Tague
> Co-Investigator BBHSL*
> Co-Director of MECCA**
> Research Associate                       Dept. of
> Physiology
> Phone Bus.: 901-448-7152                 U.T. Memphis
> Phone FAX:  [phone]                 894 Union Ave.
> e-mail:[hidden email]          Memphis, TN
>        [hidden email]                  38163
>        [hidden email]
>
> *BBHSL (Building Bridges to Health Science Literacy). An
> NIH Science Education Partnership Award (SEPA)2005-2010
> URL: http://bbhsl.mecca.org
> **MECCA (Memphis Educational Computer Connectivity
> Alliance)
> URL: http://www.mecca.org/
>
> No virus found in this incoming message.
> Checked by AVG.
> Version: 7.5.526 / Virus Database: 270.4.3/1529 - Release Date:
> 1/07/2008 7:23 PM
>
>

[Bill Oliver-3]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

On Tue, 1 Jul 2008, Larry Tague wrote:

>
> Yes, this diatribe strays somewhat from the original image manipulation
> question, but if there is no data to check or continue using, how could you
> possibly know if an improper image analysis had been applied.  Even if rules
> for image analysis exist, there is no good way to be sure mistakes were not
> made... especially when there are questions post publication and no raw data
> to check. Cheers!
>

The traditional way that research is validated is by reproducibility rather than by combing through raw data.  The bottom line is that if "ethics" is a problem, then there's nothing to stop someone from faking the data altogether.  Further, many mistakes are not those that will lie in being able to scrutinize the images, but in the physical process of doing the experiment.  In the cases of experiments-gone-bad that I am familiar with, the errors were not in recording the data, but in the execution -- a poorly calibrated water bath, a mislabeled specimen, etc.

I kow of one study, for instance, where the data was way off base because an operator simply didn't know how to operate an oscilloscope.  The only way to figure out the error (by reproducing the results), however, was to redo the experiment.  When you did that, it was clear that the only way to get the data that was reported was by incorrectly setting the gain at one point in the process.  You couldn't see that by looking at the data itself -- the data were accurately reported.

So I would suggest that the best way to see if mistakes were made is the traditional way -- reproducibility.

billo

[Larry Tague]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

> The traditional way that research is validated is by reproducibility
> rather than by combing through raw data.
Yes... and who is going to pay for all of this reproducibility
testing... not NIH who is only funding about 1 in 10 proposals at the
present time. I do not have the time or the money to reproduce
questionable data when trying to complete my own research.  I need to be
able to quickly make a determination relative to the "acceptable" nature
of published material.  I have discovered the hard way that peer review,
even though it is the best we have, is not always sufficient. After all,
how many peer reviewers take the time to look at the original data?  
Customarily, all they see is the polished product to be published. With
the wonderful advantages of the digital age comes a mess of
disadvantages and pit-falls that will require new and innovative ways of
management and review.  There is a lot of innocent garbage being
published that is scary.  We had better be quick studies in this new
digital world, or otherwise our professions in science may quickly
develop credibility problems.

Larry

Bill Oliver wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> On Tue, 1 Jul 2008, Larry Tague wrote:
>
>>
>> Yes, this diatribe strays somewhat from the original image
>> manipulation question, but if there is no data to check or continue
>> using, how could you possibly know if an improper image analysis had
>> been applied.  Even if rules for image analysis exist, there is no
>> good way to be sure mistakes were not made... especially when there
>> are questions post publication and no raw data to check. Cheers!
>>
>
> The traditional way that research is validated is by reproducibility
> rather than by combing through raw data.  The bottom line is that if
> "ethics" is a problem, then there's nothing to stop someone from
> faking the data altogether.  Further, many mistakes are not those that
> will lie in being able to scrutinize the images, but in the physical
> process of doing the experiment.  In the cases of experiments-gone-bad
> that I am familiar with, the errors were not in recording the data,
> but in the execution -- a poorly calibrated water bath, a mislabeled
> specimen, etc.
>
> I kow of one study, for instance, where the data was way off base
> because an operator simply didn't know how to operate an
> oscilloscope.  The only way to figure out the error (by reproducing
> the results), however, was to redo the experiment.  When you did that,
> it was clear that the only way to get the data that was reported was
> by incorrectly setting the gain at one point in the process.  You
> couldn't see that by looking at the data itself -- the data were
> accurately reported.
>
> So I would suggest that the best way to see if mistakes were made is
> the traditional way -- reproducibility.
>
> billo

--
Larry Tague
Co-Investigator BBHSL*
Co-Director of MECCA**
Research Associate (Dept. of Physiology)
University of Tennessee Health Science Center
894 Union Ave.
Memphis, TN 38163
Phone Bus.: 901-448-7152
Phone FAX:  [phone]
e-mail:[hidden email] or
       [hidden email]

*BBHSL "Building Bridges to Health Science Literacy" URL: http://bbhsl.mecca.org, a Science Education Partnership Award (SEPA). Supported by the National Center of Research Resources (NCRR) of the National Institutes of Health (NIH).
**MECCA (Memphis Educational Computer Connectivity Alliance)
URL: http://www.mecca.org/.  Originally support by the National Science Foundation's "Networking Infrastructure or Education" program.

CONFIDENTIALITY NOTICE: This email and any files transmitted with it are confidential and are intended solely for the use of the individual or entity to whom they are addressed. This communication may contain protected health information, or other legally privileged, confidential, or proprietary information. If you are not the intended recipient or the individual responsible for delivering the email to the recipient, please be advised that you have received this email in error and that any use, dissemination, forwarding, printing, or copying of this email is strictly prohibited. If you have received this email in error, please immediately notify the sender, disregard the foregoing message, and delete the message. We apologize for any inconvenience this may have caused. IMPORTANT NOTE: Confidential health   information is protected by state and federal law, including, but not limited to, the Health Insurance Portability and Accountability Act of 1996 and related regulations.

[ian gibbins]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

Good point, Mike.

There also are issues with patient confidentiality in microscopy
studies using human samples - we would never be allowed to put data in
a public archive that allowed identification of the individuals...

IAN


On Wednesday, July 2, 2008, at 03:33  AM, Mike Esterman wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> I have been watching this discussion with much interest.  The issue
> that
> Bill Oliver is addressing about making "all" of the data available has
> an
> additional pitfall.  I am retired from a major pharmaceutical company
> and
> scientist in our drug discovery, toxicology and clinical research
> divisions
> all publish.  No pharmaceutical company is going to allow public
> access to
> research data for network security and legal reasons.  So while making
> the
> all data and images from a published study available on a website
> might work
> for a university it would not be feasible for industry like pharma.
>
> I would suggest that all journals have guidelines about what which
> image
> process operations are allowable and they must be reported - software
> used
> and operations done as part of methods and materials, I would suggest
> reporting acquisition parameters, if possible.  Then if a scientist is
> discovered performing undocumented process there should be some
> penalty. If
> a person does unethical manipulations often enough they will be found
> out.
>
> My thoughts on the thread.
>
> Mike Esterman
> Imaging Consultant
>
> -----Original Message-----
> From: Confocal Microscopy List [mailto:[hidden email]]
> On
> Behalf Of Bill Oliver
> Sent: Tuesday, July 01, 2008 1:14 PM
> To: [hidden email]
> Subject: Re: An alarming amount of (statistical) image manipulation
>
> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> On Tue, 1 Jul 2008, Christophe Leterrier wrote:
>
>> Search the CONFOCAL archive at
>> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>>
>> The space limitation only applies to the article by itself. As someone
>> said earlier, digital space is cheap, I see nothing impossible in
>> joining a 140 pages pdf as a supplementary data to a 5 pages article,
>> so anyone who wants to go deeper in the data can, whereas those who
>> wants to see the refined and discussed stuff read the article...
>
>
> While this doesn't apply to most research, and may rarely apply to
> stuff in
> microscopy, if folk are setting up ethics guidelines they should
> realize
> that sometimes the *unethical* thing to do is make all the data
> available.
>
> Let me use my recent study as an example.  The study is one in which I
> looked at the effect of litigation by TASER, Inc. against Medical
> Examiners
> who call the use of TASER a cause of death (COD) or contributing
> factor to
> death.  In most cases these suits are brought only to force a change
> in the
> COD.  However, in two cases TASER has brought suit personally against
> the
> Medical Examiner and in at least one has sought compensatory and
> punitive
> damages for opinions expressed by the Medical Examiner about the
> dangers of
> the use of TASER.
>
> I did a survey of Medical Examiners looking at if and how the threat
> of such
> litigation affected their diagnoses and will affect their diagnoses in
> the
> future.  The problem is that the number of Medical Examiners is small
> and
> even though no names are attached to the survey, it is possible to link
> individuals to particular answers with enough effort.  The trivial
> example
> is that of the two cases TASER has brought suit personally, one is
> male and
> the other female. Thus, if the answer to "has TASER brought suit
> against
> you" is "yes" and you know the answer to "what is your gender," you
> know who
> it is.  That is the most trivial example, but a little more work would
> allow
> a good investigator to determine individual identities and use that
> data for
> litigation.
>
> Thus, I do not plan to release the data for public consumption, since
> part
> of that public are people who would use this for further litigation.
> Instead, I have placed the data in escrow with a forensic sciences
> institute
> in Europe.  If someone has a specific question, folk will see if it
> can be
> answered in a way that will not allow the mining for individual
> information.
>
> I don't know if there's something analogous for microscopy -- none of
> the
> work I did in confocal microscopy was like that.  However, broad and
> cut-and-dried statements about such things sometimes don't acknowledge
> that
> all circumstances are not the same.
>
> This has also come up with the American Medical Association where, in
> response to the brouhaha about Guantanamo, they passed an ethical rule
> stating that physicians cannot observe police interviews lest they be
> tainted by any abuse that happens.  It is clear that no forensic
> medicine
> types had input to that ethical rule -- it would be malpractice if we
> did
> *not* observe such interviews in such things as reconstructing the
> circumstances of child deaths.  Thus, in their hurry to be politically
> correct about a dramatic circumstance that is very rare, they made
> "unethical" a fundamental,important and good practice in forensic
> pathology.
> It's no surprise that only about 22% of physicians belong to the AMA
> any
> more.
>
> Accordingly, I am always a little antsy when people start opining
> about how
> "all" of anything ought to be done "all" of the time, else it is an
> "ethical" violation.
>
>
> billo
>
>

* * * * * * * * * * *
Prof Ian Gibbins
Anatomy & Histology
Flinders University
GPO Box 2100
Adelaide SA 5001
AUSTRALIA

[hidden email]
voice: [phone]
fax: [phone]

http://som.flinders.edu.au/FUSA/Anatomy/
http://www.flinders.edu.au/neuroscience

[Jeremy Adler]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

I been refused access by the authors to images published in a recent review article about colocalisation, in a respectable journal- our declared intention was to renalyse them, there seemed to be major difference between the published measurements of colocalisation, the data as shown in a scattergram and how we believed the analysis should be performed.

I have also had no response to queries made to another set of authors about methods of data analysis. We did not even get close to a request to examine the published images.


It is very important that journals stipulate that, at the very least the published images should be considered to be in the public domain and that authors should be expected to respond to legitimate questions about poorly described methods.

A related question is how, as a community, we should respond when reasonable requests are blocked.



Jeremy Adler
Cell Biology
The Wenner-Gren Inst.
Arrhenius Laboratories E5
Stockholm University
Stockholm 106 91
Sweden



-----Original Message-----
From: Confocal Microscopy List on behalf of Larry Tague
Sent: Wed 7/2/2008 16:43
To: [hidden email]
Subject: Re: An alarming amount of (statistical) image manipulation
 
Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

> The traditional way that research is validated is by reproducibility
> rather than by combing through raw data.
Yes... and who is going to pay for all of this reproducibility
testing... not NIH who is only funding about 1 in 10 proposals at the
present time. I do not have the time or the money to reproduce
questionable data when trying to complete my own research.  I need to be
able to quickly make a determination relative to the "acceptable" nature
of published material.  I have discovered the hard way that peer review,
even though it is the best we have, is not always sufficient. After all,
how many peer reviewers take the time to look at the original data?  
Customarily, all they see is the polished product to be published. With
the wonderful advantages of the digital age comes a mess of
disadvantages and pit-falls that will require new and innovative ways of
management and review.  There is a lot of innocent garbage being
published that is scary.  We had better be quick studies in this new
digital world, or otherwise our professions in science may quickly
develop credibility problems.

Larry

Bill Oliver wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> On Tue, 1 Jul 2008, Larry Tague wrote:
>
>>
>> Yes, this diatribe strays somewhat from the original image
>> manipulation question, but if there is no data to check or continue
>> using, how could you possibly know if an improper image analysis had
>> been applied.  Even if rules for image analysis exist, there is no
>> good way to be sure mistakes were not made... especially when there
>> are questions post publication and no raw data to check. Cheers!
>>
>
> The traditional way that research is validated is by reproducibility
> rather than by combing through raw data.  The bottom line is that if
> "ethics" is a problem, then there's nothing to stop someone from
> faking the data altogether.  Further, many mistakes are not those that
> will lie in being able to scrutinize the images, but in the physical
> process of doing the experiment.  In the cases of experiments-gone-bad
> that I am familiar with, the errors were not in recording the data,
> but in the execution -- a poorly calibrated water bath, a mislabeled
> specimen, etc.
>
> I kow of one study, for instance, where the data was way off base
> because an operator simply didn't know how to operate an
> oscilloscope.  The only way to figure out the error (by reproducing
> the results), however, was to redo the experiment.  When you did that,
> it was clear that the only way to get the data that was reported was
> by incorrectly setting the gain at one point in the process.  You
> couldn't see that by looking at the data itself -- the data were
> accurately reported.
>
> So I would suggest that the best way to see if mistakes were made is
> the traditional way -- reproducibility.
>
> billo

--
Larry Tague
Co-Investigator BBHSL*
Co-Director of MECCA**
Research Associate (Dept. of Physiology)
University of Tennessee Health Science Center
894 Union Ave.
Memphis, TN 38163
Phone Bus.: 901-448-7152
Phone FAX:  [phone]
e-mail:[hidden email] or
       [hidden email]

*BBHSL "Building Bridges to Health Science Literacy" URL: http://bbhsl.mecca.org, a Science Education Partnership Award (SEPA). Supported by the National Center of Research Resources (NCRR) of the National Institutes of Health (NIH).
**MECCA (Memphis Educational Computer Connectivity Alliance)
URL: http://www.mecca.org/.  Originally support by the National Science Foundation's "Networking Infrastructure or Education" program.

CONFIDENTIALITY NOTICE: This email and any files transmitted with it are confidential and are intended solely for the use of the individual or entity to whom they are addressed. This communication may contain protected health information, or other legally privileged, confidential, or proprietary information. If you are not the intended recipient or the individual responsible for delivering the email to the recipient, please be advised that you have received this email in error and that any use, dissemination, forwarding, printing, or copying of this email is strictly prohibited. If you have received this email in error, please immediately notify the sender, disregard the foregoing message, and delete the message. We apologize for any inconvenience this may have caused. IMPORTANT NOTE: Confidential health   information is protected by state and federal law, including, but not limited to, the Health Insurance Portability and Accountability Act of 1996 and related regulations.

[rjpalmer]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

So you'd like to reanalyze published data in an attempt to prove poor
methodology because those (newly published)  data conflict with those
previously published, perhaps your own?  IMHO, this sounds like an
excellent reason for denying access to data and for forgetting about
all this "image ethics by law" stuff.  If your point is methodology,
you should complain to the journal with a letter to the editor
(published) - the reviewers are obviously not doing their job re the
methods.  Very high profile journals even have "technical comments"
sections for exactly this reason.  If your complaint is about the
scientific conclusions (the only thing that really matters unless it
is a methods journal) then refute the data with a reasoned discussion
of what is out there already and with your own analysis of your own
samples.  "Reanalysis wars" are not good for science.

>Search the CONFOCAL archive at
>http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
>I been refused access by the authors to images published in a recent
>review article about colocalisation, in a respectable journal- our
>declared intention was to renalyse them, there seemed to be major
>difference between the published measurements of colocalisation, the
>data as shown in a scattergram and how we believed the analysis
>should be performed.
>
>I have also had no response to queries made to another set of
>authors about methods of data analysis. We did not even get close to
>a request to examine the published images.
>
>
>It is very important that journals stipulate that, at the very least
>the published images should be considered to be in the public domain
>and that authors should be expected to respond to legitimate
>questions about poorly described methods.
>
>A related question is how, as a community, we should respond when
>reasonable requests are blocked.
>
>
>
>Jeremy Adler
>Cell Biology
>The Wenner-Gren Inst.
>Arrhenius Laboratories E5
>Stockholm University
>Stockholm 106 91
>Sweden
>
>
>
>-----Original Message-----
>From: Confocal Microscopy List on behalf of Larry Tague
>Sent: Wed 7/2/2008 16:43
>To: [hidden email]
>Subject: Re: An alarming amount of (statistical) image manipulation
>
>Search the CONFOCAL archive at
>http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
>>  The traditional way that research is validated is by reproducibility
>>  rather than by combing through raw data.
>Yes... and who is going to pay for all of this reproducibility
>testing... not NIH who is only funding about 1 in 10 proposals at the
>present time. I do not have the time or the money to reproduce
>questionable data when trying to complete my own research.  I need to be
>able to quickly make a determination relative to the "acceptable" nature
>of published material.  I have discovered the hard way that peer review,
>even though it is the best we have, is not always sufficient. After all,
>how many peer reviewers take the time to look at the original data?
>Customarily, all they see is the polished product to be published. With
>the wonderful advantages of the digital age comes a mess of
>disadvantages and pit-falls that will require new and innovative ways of
>management and review.  There is a lot of innocent garbage being
>published that is scary.  We had better be quick studies in this new
>digital world, or otherwise our professions in science may quickly
>develop credibility problems.
>
>Larry
>
>Bill Oliver wrote:
>>  Search the CONFOCAL archive at
>>  http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>>
>>  On Tue, 1 Jul 2008, Larry Tague wrote:
>>
>>>
>>>  Yes, this diatribe strays somewhat from the original image
>>>  manipulation question, but if there is no data to check or continue
>>>  using, how could you possibly know if an improper image analysis had
>>>  been applied.  Even if rules for image analysis exist, there is no
>>>  good way to be sure mistakes were not made... especially when there
>>>  are questions post publication and no raw data to check. Cheers!
>  >>
>>
>>  The traditional way that research is validated is by reproducibility
>>  rather than by combing through raw data.  The bottom line is that if
>>  "ethics" is a problem, then there's nothing to stop someone from
>>  faking the data altogether.  Further, many mistakes are not those that
>>  will lie in being able to scrutinize the images, but in the physical
>>  process of doing the experiment.  In the cases of experiments-gone-bad
>>  that I am familiar with, the errors were not in recording the data,
>>  but in the execution -- a poorly calibrated water bath, a mislabeled
>>  specimen, etc.
>>
>>  I kow of one study, for instance, where the data was way off base
>>  because an operator simply didn't know how to operate an
>>  oscilloscope.  The only way to figure out the error (by reproducing
>>  the results), however, was to redo the experiment.  When you did that,
>>  it was clear that the only way to get the data that was reported was
>>  by incorrectly setting the gain at one point in the process.  You
>>  couldn't see that by looking at the data itself -- the data were
>>  accurately reported.
>>
>>  So I would suggest that the best way to see if mistakes were made is
>>  the traditional way -- reproducibility.
>>
>>  billo
>
>--
>Larry Tague
>Co-Investigator BBHSL*
>Co-Director of MECCA**
>Research Associate (Dept. of Physiology)
>University of Tennessee Health Science Center
>894 Union Ave.
>Memphis, TN 38163
>Phone Bus.: 901-448-7152
>Phone FAX:  [phone]
>e-mail:[hidden email] or
>        [hidden email]
>
>*BBHSL "Building Bridges to Health Science Literacy" URL:
>http://bbhsl.mecca.org, a Science Education Partnership Award
>(SEPA). Supported by the National Center of Research Resources
>(NCRR) of the National Institutes of Health (NIH).
>**MECCA (Memphis Educational Computer Connectivity Alliance)
>URL: http://www.mecca.org/.  Originally support by the National
>Science Foundation's "Networking Infrastructure or Education"
>program.
>
>CONFIDENTIALITY NOTICE: This email and any files transmitted with it
>are confidential and are intended solely for the use of the
>individual or entity to whom they are addressed. This communication
>may contain protected health information, or other legally
>privileged, confidential, or proprietary information. If you are not
>the intended recipient or the individual responsible for delivering
>the email to the recipient, please be advised that you have received
>this email in error and that any use, dissemination, forwarding,
>printing, or copying of this email is strictly prohibited. If you
>have received this email in error, please immediately notify the
>sender, disregard the foregoing message, and delete the message. We
>apologize for any inconvenience this may have caused. IMPORTANT
>NOTE: Confidential health   information is protected by state and
>federal law, including, but not limited to, the Health Insurance
>Portability and Accountability Act of 1996 and related regulations.

--
Robert J. Palmer Jr., Ph.D.
Natl Inst Dental Craniofacial Res - Natl Insts Health
Oral Infection and Immunity Branch
Bldg 30, Room 310
30 Convent Drive
Bethesda MD 20892
ph 301-594-0025
fax [phone]

[Jeremy Adler]

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

I must admit to being completely baffled by Robert J. Palmer Jr's comments.

When an image is published, and I mean actually printed in a journal, and there appears to be a mismatch between the image that the authors' have chosen to publish and the numerical data they extract from it, it is clearly fair and reasonable, in the first instance to approach the authors.
It is possible that limitations of the printing process are to blame or that my by eye estimation is wrong or that I have misunderstood the methodology.
This is only, and easily, resolvable by examining the original image and discussion with the authors.

Science is comment based on data.
If the data is dodgy then the comments fall.
Much of the discussion of scientific papers involves technical issues about whether an experiment conducted under a (well) described set of conditions actually demonstrates what the authors claim, or whether a technical flaw renders it all spurious. This is a risk we take whenever we publish.

Robert J. Palmer Jr's position appears to be that I am allowed to comment on, but that I can't see the(his) data or ask questions about his chosen methodology.
This obviously precludes my making accurate comments either in my own publications or in a letter to a journal editor.


Jeremy Adler
Cell Biology
The Wenner-Gren Inst.
Arrhenius Laboratories E5
Stockholm University
Stockholm 106 91
Sweden



-----Original Message-----
From: Confocal Microscopy List on behalf of Robert J. Palmer Jr.
Sent: Thu 7/3/2008 13:27
To: [hidden email]
Subject: Re: An alarming amount of (statistical) image manipulation
 
Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

So you'd like to reanalyze published data in an attempt to prove poor
methodology because those (newly published)  data conflict with those
previously published, perhaps your own?  IMHO, this sounds like an
excellent reason for denying access to data and for forgetting about
all this "image ethics by law" stuff.  If your point is methodology,
you should complain to the journal with a letter to the editor
(published) - the reviewers are obviously not doing their job re the
methods.  Very high profile journals even have "technical comments"
sections for exactly this reason.  If your complaint is about the
scientific conclusions (the only thing that really matters unless it
is a methods journal) then refute the data with a reasoned discussion
of what is out there already and with your own analysis of your own
samples.  "Reanalysis wars" are not good for science.

>Search the CONFOCAL archive at
>http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
>I been refused access by the authors to images published in a recent
>review article about colocalisation, in a respectable journal- our
>declared intention was to renalyse them, there seemed to be major
>difference between the published measurements of colocalisation, the
>data as shown in a scattergram and how we believed the analysis
>should be performed.
>
>I have also had no response to queries made to another set of
>authors about methods of data analysis. We did not even get close to
>a request to examine the published images.
>
>
>It is very important that journals stipulate that, at the very least
>the published images should be considered to be in the public domain
>and that authors should be expected to respond to legitimate
>questions about poorly described methods.
>
>A related question is how, as a community, we should respond when
>reasonable requests are blocked.
>
>
>
>Jeremy Adler
>Cell Biology
>The Wenner-Gren Inst.
>Arrhenius Laboratories E5
>Stockholm University
>Stockholm 106 91
>Sweden
>
>
>
>-----Original Message-----
>From: Confocal Microscopy List on behalf of Larry Tague
>Sent: Wed 7/2/2008 16:43
>To: [hidden email]
>Subject: Re: An alarming amount of (statistical) image manipulation
>
>Search the CONFOCAL archive at
>http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
>>  The traditional way that research is validated is by reproducibility
>>  rather than by combing through raw data.
>Yes... and who is going to pay for all of this reproducibility
>testing... not NIH who is only funding about 1 in 10 proposals at the
>present time. I do not have the time or the money to reproduce
>questionable data when trying to complete my own research.  I need to be
>able to quickly make a determination relative to the "acceptable" nature
>of published material.  I have discovered the hard way that peer review,
>even though it is the best we have, is not always sufficient. After all,
>how many peer reviewers take the time to look at the original data?
>Customarily, all they see is the polished product to be published. With
>the wonderful advantages of the digital age comes a mess of
>disadvantages and pit-falls that will require new and innovative ways of
>management and review.  There is a lot of innocent garbage being
>published that is scary.  We had better be quick studies in this new
>digital world, or otherwise our professions in science may quickly
>develop credibility problems.
>
>Larry
>
>Bill Oliver wrote:
>>  Search the CONFOCAL archive at
>>  http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>>
>>  On Tue, 1 Jul 2008, Larry Tague wrote:
>>
>>>
>>>  Yes, this diatribe strays somewhat from the original image
>>>  manipulation question, but if there is no data to check or continue
>>>  using, how could you possibly know if an improper image analysis had
>>>  been applied.  Even if rules for image analysis exist, there is no
>>>  good way to be sure mistakes were not made... especially when there
>>>  are questions post publication and no raw data to check. Cheers!
>  >>
>>
>>  The traditional way that research is validated is by reproducibility
>>  rather than by combing through raw data.  The bottom line is that if
>>  "ethics" is a problem, then there's nothing to stop someone from
>>  faking the data altogether.  Further, many mistakes are not those that
>>  will lie in being able to scrutinize the images, but in the physical
>>  process of doing the experiment.  In the cases of experiments-gone-bad
>>  that I am familiar with, the errors were not in recording the data,
>>  but in the execution -- a poorly calibrated water bath, a mislabeled
>>  specimen, etc.
>>
>>  I kow of one study, for instance, where the data was way off base
>>  because an operator simply didn't know how to operate an
>>  oscilloscope.  The only way to figure out the error (by reproducing
>>  the results), however, was to redo the experiment.  When you did that,
>>  it was clear that the only way to get the data that was reported was
>>  by incorrectly setting the gain at one point in the process.  You
>>  couldn't see that by looking at the data itself -- the data were
>>  accurately reported.
>>
>>  So I would suggest that the best way to see if mistakes were made is
>>  the traditional way -- reproducibility.
>>
>>  billo
>
>--
>Larry Tague
>Co-Investigator BBHSL*
>Co-Director of MECCA**
>Research Associate (Dept. of Physiology)
>University of Tennessee Health Science Center
>894 Union Ave.
>Memphis, TN 38163
>Phone Bus.: 901-448-7152
>Phone FAX:  [phone]
>e-mail:[hidden email] or
>        [hidden email]
>
>*BBHSL "Building Bridges to Health Science Literacy" URL:
>http://bbhsl.mecca.org, a Science Education Partnership Award
>(SEPA). Supported by the National Center of Research Resources
>(NCRR) of the National Institutes of Health (NIH).
>**MECCA (Memphis Educational Computer Connectivity Alliance)
>URL: http://www.mecca.org/.  Originally support by the National
>Science Foundation's "Networking Infrastructure or Education"
>program.
>
>CONFIDENTIALITY NOTICE: This email and any files transmitted with it
>are confidential and are intended solely for the use of the
>individual or entity to whom they are addressed. This communication
>may contain protected health information, or other legally
>privileged, confidential, or proprietary information. If you are not
>the intended recipient or the individual responsible for delivering
>the email to the recipient, please be advised that you have received
>this email in error and that any use, dissemination, forwarding,
>printing, or copying of this email is strictly prohibited. If you
>have received this email in error, please immediately notify the
>sender, disregard the foregoing message, and delete the message. We
>apologize for any inconvenience this may have caused. IMPORTANT
>NOTE: Confidential health   information is protected by state and
>federal law, including, but not limited to, the Health Insurance
>Portability and Accountability Act of 1996 and related regulations.

--
Robert J. Palmer Jr., Ph.D.
Natl Inst Dental Craniofacial Res - Natl Insts Health
Oral Infection and Immunity Branch
Bldg 30, Room 310
30 Convent Drive
Bethesda MD 20892
ph 301-594-0025
fax [phone]