Re: An alarming amount of (statistical) image manipulation

Posted by rjpalmer on
URL: http://confocal-microscopy-list.275.s1.nabble.com/Source-of-Richardson-test-slide-tp593463p593508.html

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

Let's say that your reanalysis shows the original conclusions to be
nonsense.  Are you then finished and happy in your knowledge of the
truth?  Or do you write to the authors and demand a retraction under
threat of revealing your reinterpretation?  Or make announcements at
meetings?  Or e-mail colleagues?

Only the first of those alternatives sounds to me like it is NOT a
big fuss.  If you consider your area of research to negatively
impacted by published nonsense, then I assume that you are collecting
data that might refute the authors' conclusions.  Otherwise, as you
correctly note, the refutation of those data is too peripheral for
your time and effort.  Good science is rarely efficient and quick
science is even less so.

The process of science, regardless of data type, has been set for
some time.  Discuss the presented data, and the methods used to
obtain them, in public and allow the authors to defend their data
based on response to your opinions (e.g., a journal letter).  The
exception to this approach involves gross aberrations such as we have
seen with recent high-profile stem cell research.  That particular
incident indeed led to the red-faced admission by the journal that
the images were so clearly manipulated that experienced reviewers (of
which few exist for even those trivial images let alone advanced
image data) should have caught this "mistake", and that only AFTER
the original images were examined!  So ask the journal to do the work
that is, in fact, their responsibility, not yours.

Unless I felt that an outright intent to deceive exists, I would
leave the data alone and concentrate  on why the end interpretation
is unlikely to be correct.  Clearly if the interpretation is
incorrect/controversial, then other lines of data must exist to
refute or confirm the authors' interpretation.  I would hope that no
scientist publishes controversial work based solely in image data.

I have the feeling that we are very close to being told to take this
discussion elsewhere, or to cut to the chase.  I am sure that you and
Dan White have a specific example or two in mind and could provide
the citations with a brief description of what is wrong. Then those
who have experience in the matter could also comment here.  But I
guess public revelation, as I have been supporting, would be
premature without prior reanalysis?

>Search the CONFOCAL archive at
>http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
>Dear Robert
>
>you agree that there may be real differences between an image as
>published and the original.
>But then argue that is it unreasonable to ask to see the original.
>
>The original image/data may well entirely justify an author's
>conclusions and dispel any doubts that someone reading the article
>may raise. Doubts dispelled and clarity increased, surely a big plus
>for the authors.
>It is also possible that on occasions the problem may be real and
>the image/data does not bear examination.
>
>The whole point is to resolve issues quickly and efficiently,
>allowing good work to thrive.
>
>Your alternative seems to require that I either create a big fuss by
>writing to the journal editor or repeat the study myself. Both
>highly inefficient ways of resolving what maybe a trivial
>misundertanding.
>
>
>
>Jeremy Adler
>Cell Biology
>The Wenner-Gren Inst.
>Arrhenius Laboratories E5
>Stockholm University
>Stockholm 106 91
>Sweden
>
>________________________________
>
>From: Confocal Microscopy List on behalf of Robert J. Palmer Jr.
>Sent: Fri 7/4/2008 13:39
>To: [hidden email]
>Subject: Re: An alarming amount of (statistical) image manipulation
>
>
>Search the CONFOCAL archive at
>http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>Do you think it reasonable to ask for a 20-page spreadsheet of raw
>data that was used to create a table, or the mountains of raw data
>that were analyzed by a particular statistical program, with these
>requests being based on a desire to "prove" the scientific story as
>incorrect?   I am still missing the point of reanalysis - just what
>is intended here? To point out mistakes in data interpretation that
>may be inferred from the scientific content of the published image?
>As you both have noted, the published images in print and on-line
>are often of poor quality anyway - maybe therein lies your
>interpretation of the image as conveying inaccurate information?  I
>maintain that reanalysis of others' published data does not advance
>science - make your comments in the journal and let the trash-heap
>of history be the guide to scientific advances rather than turning
>research into arguments about methodological detail.  Take others'
>data as they are; show that the scientific conclusions are incorrect
>by analysis of your own data.
>
>
> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> Hi Robert, Jeremy and All,
>
> I think it is unreasonable not to give electronic access to
>images that are published in
> journal articles. The images in PDF files are ususally JPEG
>or similar compressed,
> and thus corrupted badly, and it is much better to be able to
>see the uncorrupted original images.
>
> Let us not forget that an image is just a way of visualising
>a table/matrix of numbers.
> If I had a big table of results containing thousands of numbers,
> and chose to visualise it as an image then corrupted that
>image so you can no longer
> read the numbers from the image properly, there would appear
>to be a big problem.
> This is what happens with every image published in print and in a PDF.
> If a published a table in a paper and made the numbers hard
>to read or even corrupted them,
> that would be unacceptable. Same should be true for images as
>they are the same as tables.
>
> An image is a table of numbers,
> and as a reader I expect to be able to read those numbers correctly,
> meaning the reader needs access to uncorrupted/lossy
>compressed original image data that is sent for publication
> (usually non compressed TIFF is requested by journals for images...)
>
> I have often also wanted to analyse image data from a
>published pdf file
> (where no quantitative analysis has been done to measure, for
>instance colocalisation, as is too often the case)
> but there is not point trying because the image is is so
>badly corrupted by compression that its a waste of time.
>
> Strongly  agree with Jeremy on this one.
>
> Dan White
> MPI-CBG    LMF
>
> On Jul 4, 2008, at 6:00 AM, CONFOCAL automatic digest system wrote:
>
>
>
> Date:    Thu, 3 Jul 2008 16:14:36 +0200
> From:    Jeremy Adler <[hidden email]>
> Subject: Re: An alarming amount of (statistical)
>image manipulation
>
> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=3Dconfocal
>
> I must admit to being completely baffled by Robert J.
>Palmer Jr's =
> comments.
>
> When an image is published, and I mean actually
>printed in a journal, =
> and there appears to be a mismatch between the image
>that the authors' =
> have chosen to publish and the numerical data they
>extract from it, it =
> is clearly fair and reasonable, in the first instance
>to approach the =
> authors.
> It is possible that limitations of the printing
>process are to blame or =
> that my by eye estimation is wrong or that I have
>misunderstood the =
> methodology.
> This is only, and easily, resolvable by examining the
>original image and =
> discussion with the authors.=20
>
> Science is comment based on data.
> If the data is dodgy then the comments fall.
> Much of the discussion of scientific papers involves
>technical issues =
> about whether an experiment conducted under a (well)
>described set of =
> conditions actually demonstrates what the authors
>claim, or whether a =
>
> technical flaw renders it all spurious. This is a
>risk we take whenever =
> we publish.
>
> Robert J. Palmer Jr's position appears to be that I
>am allowed to =
> comment on, but that I can't see the(his) data or ask
>questions about =
> his chosen methodology.=20
> This obviously precludes my making accurate comments
>either in my own =
> publications or in a letter to a journal editor.
>
>
> Jeremy Adler
> Cell Biology
> The Wenner-Gren Inst.
> Arrhenius Laboratories E5
> Stockholm University
> Stockholm 106 91
> Sweden
>
>
> Dr. Daniel James White BSc. (Hons.) PhD
> Senior Microscopist / Image Processing and Analysis
> Light Microscopy Facility
> Max Planck Institute of Molecular Cell Biology and Genetics
> Pfotenhauerstrasse 108
> 01307 DRESDEN
> Germany
>
>
> New Mobile Number!!!
>
> [phone] (German Mobile)
> +49  (0)351 210 2627 (Work phone at MPI-CBG)
> +49  (0)351 210 1078 (Fax MPI-CBG LMF)
>
> http://www.bioimagexd.net
> http://www.chalkie.org.uk
> [hidden email]
> ( [hidden email] )
>
>
>
>--
>Robert J. Palmer Jr., Ph.D.
>Natl Inst Dental Craniofacial Res - Natl Insts Health
>Oral Infection and Immunity Branch
>Bldg 30, Room 310
>30 Convent Drive
>Bethesda MD 20892
>ph 301-594-0025
>fax [phone]

--
Robert J. Palmer Jr., Ph.D.
Natl Inst Dental Craniofacial Res - Natl Insts Health
Oral Infection and Immunity Branch
Bldg 30, Room 310
30 Convent Drive
Bethesda MD 20892
ph 301-594-0025
fax [phone]