Source of Richardson test slide
Locked
12345
12345
 
|
Julio Vazquez |
Re: An alarming amount of (statistical) image manipulation and so much more
|
Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
-
Well, if you think there is an alarming amount of image manipulation (and bad statistics) going on, you definitely SHOULD NOT read Salas et al, (2005): A critical reassessment of the role of mitochondria in tumorigenesis. PLoS Medicine 2(11): e296. In this paper, the authors re-analyze a number of published papers dealing with somatic mitochondrial DNA mutations in tumors (actually those few papers for which a sufficient amount of primary data was available). These are (some of) their conclusions: "We have found that the vast majority (>80%) of the studies dealing with potential functional implications of the mtDNA molecule in tumorigenesis (and providing data for inspection) are based on faulty data with surreal findings. [...] Probably, we should abandon the exciting findings unleashed as the result of the many sequencing failures that accumulated during the last decade." On the positive (!) side, they attribute those findings mostly to incompetence, not malice. Scary, right? but fear not: I just found a recipe in 19 easy steps to improve the quality of scientific research: 1. take high-school students with little or no math and science background, and undeveloped analytical skills 2. give them computers with DOS 3. let them graduate in "Cult of the Mother Goddess" and "Harry Potter Mythology" studies (optional: throw in Creationism) 4. send them to grad school in a lab where they have no real project and complete lack of supervision 5. upgrade their computers to Windows 3.1 6. put a lot of pressure of them to publish in the top journals (optional): give them a PhD 7. send them to a new lab for postdoctoral "training" in a lab where they have no real project and complete lack of supervision 8. upgrade their computers to Windows 98 9. put a lot of pressure of them to publish in the top journals 10. put a lot of pressure of them to obtain funding for another year (optional): repeat 7-10 (optional): upgrade their computers to Windows 2000 11. (optional): give them a faculty position, (where they have no real project and complete lack of supervision) 12. put a lot of pressure of them to publish in the top journals 13. put a lot of pressure of them to obtain funding for another year 14. give them (undergraduate/graduate students/postdocs; choose one) with little or no math and general science background (and undeveloped analytical skills) 15. upgrade their computers to Windows XP (optional): appoint them as peer-reviewers (optional): let them delegate their peer-reviewing tasks to their undergraduate/graduate students/postdocs; (choose one), with little or no math and general science background (and undeveloped analytical skills) 16. Buy a LINUX box that no one will use ("what type of computer is this?") 17. reduce funding for schools 18. upgrade their computers to Vista 19. start over... Julio == |
|
|
Daniel James White |
Re: An alarming amount of (statistical) image manipulation
|
|
In reply to this post by ian gibbins
Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal Hi Robert, Jeremy and All, I think it is unreasonable not to give electronic access to images that are published in journal articles. The images in PDF files are ususally JPEG or similar compressed, and thus corrupted badly, and it is much better to be able to see the uncorrupted original images. Let us not forget that an image is just a way of visualising a table/ matrix of numbers. If I had a big table of results containing thousands of numbers, and chose to visualise it as an image then corrupted that image so you can no longer read the numbers from the image properly, there would appear to be a big problem. This is what happens with every image published in print and in a PDF. If a published a table in a paper and made the numbers hard to read or even corrupted them, that would be unacceptable. Same should be true for images as they are the same as tables. An image is a table of numbers, and as a reader I expect to be able to read those numbers correctly, meaning the reader needs access to uncorrupted/lossy compressed original image data that is sent for publication (usually non compressed TIFF is requested by journals for images...) I have often also wanted to analyse image data from a published pdf file (where no quantitative analysis has been done to measure, for instance colocalisation, as is too often the case) but there is not point trying because the image is is so badly corrupted by compression that its a waste of time. Strongly agree with Jeremy on this one. Dan White MPI-CBG LMF On Jul 4, 2008, at 6:00 AM, CONFOCAL automatic digest system wrote: > > Date: Thu, 3 Jul 2008 16:14:36 +0200 > From: Jeremy Adler <[hidden email]> > Subject: Re: An alarming amount of (statistical) image manipulation > > Search the CONFOCAL archive at > http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=3Dconfocal > > I must admit to being completely baffled by Robert J. Palmer Jr's = > comments. > > When an image is published, and I mean actually printed in a > journal, = > and there appears to be a mismatch between the image that the > authors' = > have chosen to publish and the numerical data they extract from it, > it = > is clearly fair and reasonable, in the first instance to approach > the = > authors. > It is possible that limitations of the printing process are to blame > or = > that my by eye estimation is wrong or that I have misunderstood the = > methodology. > This is only, and easily, resolvable by examining the original image > and = > discussion with the authors.=20 > > Science is comment based on data. > If the data is dodgy then the comments fall. > Much of the discussion of scientific papers involves technical > issues = > about whether an experiment conducted under a (well) described set > of = > conditions actually demonstrates what the authors claim, or whether > a = > technical flaw renders it all spurious. This is a risk we take > whenever = > we publish. > > Robert J. Palmer Jr's position appears to be that I am allowed to = > comment on, but that I can't see the(his) data or ask questions > about = > his chosen methodology.=20 > This obviously precludes my making accurate comments either in my > own = > publications or in a letter to a journal editor. > > > Jeremy Adler > Cell Biology > The Wenner-Gren Inst. > Arrhenius Laboratories E5 > Stockholm University > Stockholm 106 91 > Sweden Dr. Daniel James White BSc. (Hons.) PhD Senior Microscopist / Image Processing and Analysis Light Microscopy Facility Max Planck Institute of Molecular Cell Biology and Genetics Pfotenhauerstrasse 108 01307 DRESDEN Germany New Mobile Number!!! +49 (0)15114966933 (German Mobile) +49 (0)351 210 2627 (Work phone at MPI-CBG) +49 (0)351 210 1078 (Fax MPI-CBG LMF) http://www.bioimagexd.net http://www.chalkie.org.uk [hidden email] ( [hidden email] ) |
|
|
Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
Do you think it reasonable to ask for a 20-page spreadsheet of
raw data that was used to create a table, or the mountains of raw data
that were analyzed by a particular statistical program, with these
requests being based on a desire to "prove" the scientific
story as incorrect? I am still missing the point of
reanalysis - just what is intended here? To point out mistakes in data
interpretation that may be inferred from the scientific content of the
published image? As you both have noted, the published images in
print and on-line are often of poor quality anyway - maybe therein
lies your interpretation of the image as conveying inaccurate
information? I maintain that reanalysis of others' published
data does not advance science - make your comments in the journal and
let the trash-heap of history be the guide to scientific advances
rather than turning research into arguments about methodological
detail. Take others' data as they are; show that the scientific
conclusions are incorrect by analysis of your own data.
Search the CONFOCAL archive at
-- Robert J. Palmer Jr., Ph.D.
Natl Inst Dental Craniofacial Res - Natl Insts Health Oral Infection and Immunity Branch Bldg 30, Room 310 30 Convent Drive Bethesda MD 20892 ph 301-594-0025 fax 301-402-0396 |
|
|
Michael Herron |
Re: An alarming amount of (statistical) image manipulation
|
|
Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
I concur. This is the process of science.
On Jul 4, 2008, at 6:39 AM, Robert J. Palmer Jr. wrote: Search the CONFOCAL archive at http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal |
|
|
Jeremy Adler |
Re: An alarming amount of (statistical) image manipulation
|
|
In reply to this post by rjpalmer
Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal Dear Robert you agree that there may be real differences between an image as published and the original. But then argue that is it unreasonable to ask to see the original. The original image/data may well entirely justify an author's conclusions and dispel any doubts that someone reading the article may raise. Doubts dispelled and clarity increased, surely a big plus for the authors. It is also possible that on occasions the problem may be real and the image/data does not bear examination. The whole point is to resolve issues quickly and efficiently, allowing good work to thrive. Your alternative seems to require that I either create a big fuss by writing to the journal editor or repeat the study myself. Both highly inefficient ways of resolving what maybe a trivial misundertanding. Jeremy Adler Cell Biology The Wenner-Gren Inst. Arrhenius Laboratories E5 Stockholm University Stockholm 106 91 Sweden ________________________________ From: Confocal Microscopy List on behalf of Robert J. Palmer Jr. Sent: Fri 7/4/2008 13:39 To: [hidden email] Subject: Re: An alarming amount of (statistical) image manipulation Search the CONFOCAL archive at http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal Do you think it reasonable to ask for a 20-page spreadsheet of raw data that was used to create a table, or the mountains of raw data that were analyzed by a particular statistical program, with these requests being based on a desire to "prove" the scientific story as incorrect? I am still missing the point of reanalysis - just what is intended here? To point out mistakes in data interpretation that may be inferred from the scientific content of the published image? As you both have noted, the published images in print and on-line are often of poor quality anyway - maybe therein lies your interpretation of the image as conveying inaccurate information? I maintain that reanalysis of others' published data does not advance science - make your comments in the journal and let the trash-heap of history be the guide to scientific advances rather than turning research into arguments about methodological detail. Take others' data as they are; show that the scientific conclusions are incorrect by analysis of your own data. Search the CONFOCAL archive at http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal Hi Robert, Jeremy and All, I think it is unreasonable not to give electronic access to images that are published in journal articles. The images in PDF files are ususally JPEG or similar compressed, and thus corrupted badly, and it is much better to be able to see the uncorrupted original images. Let us not forget that an image is just a way of visualising a table/matrix of numbers. If I had a big table of results containing thousands of numbers, and chose to visualise it as an image then corrupted that image so you can no longer read the numbers from the image properly, there would appear to be a big problem. This is what happens with every image published in print and in a PDF. If a published a table in a paper and made the numbers hard to read or even corrupted them, that would be unacceptable. Same should be true for images as they are the same as tables. An image is a table of numbers, and as a reader I expect to be able to read those numbers correctly, meaning the reader needs access to uncorrupted/lossy compressed original image data that is sent for publication (usually non compressed TIFF is requested by journals for images...) I have often also wanted to analyse image data from a published pdf file (where no quantitative analysis has been done to measure, for instance colocalisation, as is too often the case) but there is not point trying because the image is is so badly corrupted by compression that its a waste of time. Strongly agree with Jeremy on this one. Dan White MPI-CBG LMF On Jul 4, 2008, at 6:00 AM, CONFOCAL automatic digest system wrote: Date: Thu, 3 Jul 2008 16:14:36 +0200 From: Jeremy Adler <[hidden email]> Subject: Re: An alarming amount of (statistical) image manipulation Search the CONFOCAL archive at http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=3Dconfocal I must admit to being completely baffled by Robert J. Palmer Jr's = comments. When an image is published, and I mean actually printed in a journal, = and there appears to be a mismatch between the image that the authors' = have chosen to publish and the numerical data they extract from it, it = is clearly fair and reasonable, in the first instance to approach the = authors. It is possible that limitations of the printing process are to blame or = that my by eye estimation is wrong or that I have misunderstood the = methodology. This is only, and easily, resolvable by examining the original image and = discussion with the authors.=20 Science is comment based on data. If the data is dodgy then the comments fall. Much of the discussion of scientific papers involves technical issues = about whether an experiment conducted under a (well) described set of = conditions actually demonstrates what the authors claim, or whether a = technical flaw renders it all spurious. This is a risk we take whenever = we publish. Robert J. Palmer Jr's position appears to be that I am allowed to = comment on, but that I can't see the(his) data or ask questions about = his chosen methodology.=20 This obviously precludes my making accurate comments either in my own = publications or in a letter to a journal editor. Jeremy Adler Cell Biology The Wenner-Gren Inst. Arrhenius Laboratories E5 Stockholm University Stockholm 106 91 Sweden Dr. Daniel James White BSc. (Hons.) PhD Senior Microscopist / Image Processing and Analysis Light Microscopy Facility Max Planck Institute of Molecular Cell Biology and Genetics Pfotenhauerstrasse 108 01307 DRESDEN Germany New Mobile Number!!! +49 (0)15114966933 (German Mobile) +49 (0)351 210 2627 (Work phone at MPI-CBG) +49 (0)351 210 1078 (Fax MPI-CBG LMF) http://www.bioimagexd.net http://www.chalkie.org.uk [hidden email] ( [hidden email] ) -- Robert J. Palmer Jr., Ph.D. Natl Inst Dental Craniofacial Res - Natl Insts Health Oral Infection and Immunity Branch Bldg 30, Room 310 30 Convent Drive Bethesda MD 20892 ph 301-594-0025 fax 301-402-0396 |
|
|
Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal Let's say that your reanalysis shows the original conclusions to be nonsense. Are you then finished and happy in your knowledge of the truth? Or do you write to the authors and demand a retraction under threat of revealing your reinterpretation? Or make announcements at meetings? Or e-mail colleagues? Only the first of those alternatives sounds to me like it is NOT a big fuss. If you consider your area of research to negatively impacted by published nonsense, then I assume that you are collecting data that might refute the authors' conclusions. Otherwise, as you correctly note, the refutation of those data is too peripheral for your time and effort. Good science is rarely efficient and quick science is even less so. The process of science, regardless of data type, has been set for some time. Discuss the presented data, and the methods used to obtain them, in public and allow the authors to defend their data based on response to your opinions (e.g., a journal letter). The exception to this approach involves gross aberrations such as we have seen with recent high-profile stem cell research. That particular incident indeed led to the red-faced admission by the journal that the images were so clearly manipulated that experienced reviewers (of which few exist for even those trivial images let alone advanced image data) should have caught this "mistake", and that only AFTER the original images were examined! So ask the journal to do the work that is, in fact, their responsibility, not yours. Unless I felt that an outright intent to deceive exists, I would leave the data alone and concentrate on why the end interpretation is unlikely to be correct. Clearly if the interpretation is incorrect/controversial, then other lines of data must exist to refute or confirm the authors' interpretation. I would hope that no scientist publishes controversial work based solely in image data. I have the feeling that we are very close to being told to take this discussion elsewhere, or to cut to the chase. I am sure that you and Dan White have a specific example or two in mind and could provide the citations with a brief description of what is wrong. Then those who have experience in the matter could also comment here. But I guess public revelation, as I have been supporting, would be premature without prior reanalysis? >Search the CONFOCAL archive at >http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal > >Dear Robert > >you agree that there may be real differences between an image as >published and the original. >But then argue that is it unreasonable to ask to see the original. > >The original image/data may well entirely justify an author's >conclusions and dispel any doubts that someone reading the article >may raise. Doubts dispelled and clarity increased, surely a big plus >for the authors. >It is also possible that on occasions the problem may be real and >the image/data does not bear examination. > >The whole point is to resolve issues quickly and efficiently, >allowing good work to thrive. > >Your alternative seems to require that I either create a big fuss by >writing to the journal editor or repeat the study myself. Both >highly inefficient ways of resolving what maybe a trivial >misundertanding. > > > >Jeremy Adler >Cell Biology >The Wenner-Gren Inst. >Arrhenius Laboratories E5 >Stockholm University >Stockholm 106 91 >Sweden > >________________________________ > >From: Confocal Microscopy List on behalf of Robert J. Palmer Jr. >Sent: Fri 7/4/2008 13:39 >To: [hidden email] >Subject: Re: An alarming amount of (statistical) image manipulation > > >Search the CONFOCAL archive at >http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal >Do you think it reasonable to ask for a 20-page spreadsheet of raw >data that was used to create a table, or the mountains of raw data >that were analyzed by a particular statistical program, with these >requests being based on a desire to "prove" the scientific story as >incorrect? I am still missing the point of reanalysis - just what >is intended here? To point out mistakes in data interpretation that >may be inferred from the scientific content of the published image? >As you both have noted, the published images in print and on-line >are often of poor quality anyway - maybe therein lies your >interpretation of the image as conveying inaccurate information? I >maintain that reanalysis of others' published data does not advance >science - make your comments in the journal and let the trash-heap >of history be the guide to scientific advances rather than turning >research into arguments about methodological detail. Take others' >data as they are; show that the scientific conclusions are incorrect >by analysis of your own data. > > > Search the CONFOCAL archive at > http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal > > Hi Robert, Jeremy and All, > > I think it is unreasonable not to give electronic access to >images that are published in > journal articles. The images in PDF files are ususally JPEG >or similar compressed, > and thus corrupted badly, and it is much better to be able to >see the uncorrupted original images. > > Let us not forget that an image is just a way of visualising >a table/matrix of numbers. > If I had a big table of results containing thousands of numbers, > and chose to visualise it as an image then corrupted that >image so you can no longer > read the numbers from the image properly, there would appear >to be a big problem. > This is what happens with every image published in print and in a PDF. > If a published a table in a paper and made the numbers hard >to read or even corrupted them, > that would be unacceptable. Same should be true for images as >they are the same as tables. > > An image is a table of numbers, > and as a reader I expect to be able to read those numbers correctly, > meaning the reader needs access to uncorrupted/lossy >compressed original image data that is sent for publication > (usually non compressed TIFF is requested by journals for images...) > > I have often also wanted to analyse image data from a >published pdf file > (where no quantitative analysis has been done to measure, for >instance colocalisation, as is too often the case) > but there is not point trying because the image is is so >badly corrupted by compression that its a waste of time. > > Strongly agree with Jeremy on this one. > > Dan White > MPI-CBG LMF > > On Jul 4, 2008, at 6:00 AM, CONFOCAL automatic digest system wrote: > > > > Date: Thu, 3 Jul 2008 16:14:36 +0200 > From: Jeremy Adler <[hidden email]> > Subject: Re: An alarming amount of (statistical) >image manipulation > > Search the CONFOCAL archive at > http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=3Dconfocal > > I must admit to being completely baffled by Robert J. >Palmer Jr's = > comments. > > When an image is published, and I mean actually >printed in a journal, = > and there appears to be a mismatch between the image >that the authors' = > have chosen to publish and the numerical data they >extract from it, it = > is clearly fair and reasonable, in the first instance >to approach the = > authors. > It is possible that limitations of the printing >process are to blame or = > that my by eye estimation is wrong or that I have >misunderstood the = > methodology. > This is only, and easily, resolvable by examining the >original image and = > discussion with the authors.=20 > > Science is comment based on data. > If the data is dodgy then the comments fall. > Much of the discussion of scientific papers involves >technical issues = > about whether an experiment conducted under a (well) >described set of = > conditions actually demonstrates what the authors >claim, or whether a = > > technical flaw renders it all spurious. This is a >risk we take whenever = > we publish. > > Robert J. Palmer Jr's position appears to be that I >am allowed to = > comment on, but that I can't see the(his) data or ask >questions about = > his chosen methodology.=20 > This obviously precludes my making accurate comments >either in my own = > publications or in a letter to a journal editor. > > > Jeremy Adler > Cell Biology > The Wenner-Gren Inst. > Arrhenius Laboratories E5 > Stockholm University > Stockholm 106 91 > Sweden > > > Dr. Daniel James White BSc. (Hons.) PhD > Senior Microscopist / Image Processing and Analysis > Light Microscopy Facility > Max Planck Institute of Molecular Cell Biology and Genetics > Pfotenhauerstrasse 108 > 01307 DRESDEN > Germany > > > New Mobile Number!!! > > +49 (0)15114966933 (German Mobile) > +49 (0)351 210 2627 (Work phone at MPI-CBG) > +49 (0)351 210 1078 (Fax MPI-CBG LMF) > > http://www.bioimagexd.net > http://www.chalkie.org.uk > [hidden email] > ( [hidden email] ) > > > >-- >Robert J. Palmer Jr., Ph.D. >Natl Inst Dental Craniofacial Res - Natl Insts Health >Oral Infection and Immunity Branch >Bldg 30, Room 310 >30 Convent Drive >Bethesda MD 20892 >ph 301-594-0025 >fax 301-402-0396 -- Robert J. Palmer Jr., Ph.D. Natl Inst Dental Craniofacial Res - Natl Insts Health Oral Infection and Immunity Branch Bldg 30, Room 310 30 Convent Drive Bethesda MD 20892 ph 301-594-0025 fax 301-402-0396 |
|
|
Michael Cammer |
Re: An alarming amount of (statistical) image manipulation
|
|
In reply to this post by Daniel James White
Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal We sometimes have the problem, especially with data involving small particles moving in cells, that data that are evident in raw data and really clear in deconvolved and contrast enhanced images simply disappear completely in compressed movies or figures that have to be squeezed into a single page. This is an example where journals simply make it impossible to adequately show data. > I think it is unreasonable not to give electronic access to images > that are published in > journal articles. The images in PDF files are ususally JPEG or similar > compressed, > and thus corrupted badly, and it is much better to be able to see the > uncorrupted original images. _________________________________________ Michael Cammer http://www.aecom.yu.edu/aif/ |
|
|
In reply to this post by Larry Tague
Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal Hi! All, I am developing a confocal user guideline to protect our confocal microscope in our facility. Could you share your guideline (or link on web site) in your facility if possible? Thank you very much! Tomo Kawaguchi, Ph.D. Environmental Genomics Core Facility Arnold School of Public Health University of South Carolina Columbia, SC 29208 |
«
Return to Confocal Microscopy List
|
1 view|%1 views
| Free forum by Nabble | Edit this page |