Movie Corruption Issue

classic Classic list List threaded Threaded
24 messages Options
12
heckman@bgnet.bgsu.edu heckman@bgnet.bgsu.edu
Reply | Threaded
Open this post in threaded view
|

Re: Movie Corruption Issue & Codecs

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

Steve-
"the prospect of
retroactively re-encoding 5 years worth of compiled imaging data
solely to..."

This is supposed to be what computers are good for.  Maybe you
need a good programmer.

Carol
Guy Cox Guy Cox
Reply | Threaded
Open this post in threaded view
|

Re: Movie Corruption Issue

In reply to this post by Mathieu Marchand-2
Search the CONFOCAL archive at http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal Re: Movie Corruption Issue
The message really seems extremely clear.  Do not use Quicktime
under any circumstances.  If you have old movies in QT format
convert them all on an old system (preferably isolated from the
Net) to something that has open source support.  (There is
probably a freeware Linux product that will do this as a batch
process).  But these formats are NOT designed for scientific
data - it may be fine to use a QT movie as a one-off to show
at a conference (though from the number of problems I've
seen on this front I'm not convinced!) but scientific data should
be saved as as uncompressed image series in formats that
will always be openable (even if not immediately playable as a
movie).  It's not realistic to expect entertainment software to
offer the data stability that science needs.
 
                                                                             Guy
 
Optical Imaging Techniques in Cell Biology
by Guy Cox    CRC Press / Taylor & Francis
     http://www.guycox.com/optical.htm
______________________________________________
Associate Professor Guy Cox, MA, DPhil(Oxon)
Electron Microscope Unit, Madsen Building F09,
University of Sydney, NSW 2006
______________________________________________
Phone +61 2 9351 3176     Fax +61 2 9351 7682
Mobile 0413 281 861
______________________________________________
http://www.guycox.net


From: Confocal Microscopy List on behalf of Mathieu Marchand
Sent: Sat 08/01/26 5:16 AM
To: [hidden email]
Subject: Re: Movie Corruption Issue

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

To comment the previous post,

Everybody should be aware that using any other version than the latest
available Quicktime is a major security risk, on both Mac and Windows.

The last 7.3, 7.3.1 and 7.4 updates of Quicktime corrected some CRITICAL flaws:
http://secunia.com/advisories/28502/
http://secunia.com/advisories/28092/
http://secunia.com/advisories/27755/
http://secunia.com/advisories/27523/
and these are the flaws for only the past 3 months

These flaws are rated "extremely critical" and could give an attacker
remote access on your system just by visiting a malicious website with
your favorite browser.
Since Quicktime 7.2 and later does not support Windows 2000 any more,
I recommend to uninstall Quicktime on computers running Windows 2000.

Support for old codecs like "animation" for quicktime or "cinepak" for
vista is broken or problematic and people should stop using them, and
plan to convert old movies to more recent codec. "Codec Packs" can
sometimes solve your problem, but they usually have major stability /
legality / security risks attached to them. They can introduce new
problems, too (for example: the Perian codec pack for MacOS brakes the
play out of 'ARAW' coded avi files including avi files generated by
ImageJ).

Picking a codec is a complex issue and there is no easy choice.  It is
something worth discussing with a specialist to make the best decision
for your situation.

--
Mathieu Marchand
--
Bio-Imaging Resource Center, The Rockefeller University
1230 York Avenue, box 209, New York, NY 10021
http://www.rockefeller.edu/bioimaging .
+1-212-327-7487 (7489 for fax)
http://www.pfid.org/html/ppms_agree/?fr .


On Jan 25, 2008 12:30 PM, Stephen Bunnell <[hidden email]> wrote:


> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> In follow up to my own post on the failure of old movies exported using the
> 'Apple Animation' codec:
>
> The defect is in Apple's new versions of Quicktime. Both Quicktime 7.3 and
> 7.4 fail to play these movies. After reinstalling my entire OS, with
> Quicktime 7.1.3 and OSX version 10.4.11, all of these movies now play
> perfectly. I've not tested Quicktime 7.2.
>
> As to a solution... I've heard nothing at all from Apple. I'm not going to
> upgrade Quicktime again for at least a year. My advice is to be very wary up
> performing any Quicktime upgrades before a presentation! The new versions
> (7.3 and 7.4 for the Mac) appear to have _many_ bugs, judging from the
> online discussions.
>
> Every other Mac-based player I tried failed in exactly the same way, despite
> many good suggestions from the list. Ultimately, I will need a Mac fix for
> Quicktime, or else I will have to re-export thousands of movies...
>
>      -Thanks for all your advice.
>
>
>
>
>
> On 1/18/08 2:43 PM, "Stephen C. Bunnell" <[hidden email]> wrote:
>
> > Search the CONFOCAL archive at
> > http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
> >
> > This is not technically a microscopy question, but I'm desperate:
> >
> > I use a mac. I have hundreds, if not thousands, of videos exported into .mpg
> > and .mov formats over many years of imaging.
> >
> > Of late, I have noticed that _many_ of my older .mov files are seriously
> > corrupted on the Mac. It's not the data. Archived movies are the same. It's
> > the player- Quicktime. It no longer can play the old movies. However, 2-3
> > years ago, before this problem was widespread, we exported several .mov
> > files to .avi. These .avi movies (fortunately) play just fine. However,
> > attempt to export the corrupted .mov files now yield .avi files that look
> > just like the .mov files- that is to say, they look like garbage.
> >
> > I have attempted to revert to older versions of quicktime- all the way to
> > v7.3. No luck. They're still corrupted.
> >
> > The corruption is not a computer issue. The same movies are corrupted on
> > many others Macs with current system installations.
> >
> > As you might expect, this makes for less than stellar presentations, when
> > half of your movies crash.
> >
> > Oddly, some movies exported on the same day, using the same software, will
> > play, and others will not.
> >
> > Has anyone else encountered this problem on the Mac? Any thoughts?
> >
> >     --Steve
> >
> >
> >
> > ****************************************************************************
> > Stephen C. Bunnell, Ph.D.
> > Assistant Professor
> > Tufts University Medical School
> > Department of Pathology
> > Jaharis Bldg., Room 512
> > 150 Harrison Ave.
> > Boston, MA 02111
> >
> > Phone: (617) 636-2174
> > Fax:   (617) 636-2990
> > Email: [hidden email]
> >
> > SHIPPING ADDRESS (for packages):
> > Tufts University Receiving
> > 37 Tyler St.
> > Attn: Bunnell/Pathology/Jaharis 524
> > Boston, MA 02111
>
> ****************************************************************************
> Stephen C. Bunnell, Ph.D.
> Assistant Professor
> Tufts University Medical School
> Department of Pathology
> Jaharis Bldg., Room 512
> 150 Harrison Ave.
> Boston, MA 02111
>
> Phone: (617) 636-2174
> Fax:   (617) 636-2990
> Email: [hidden email]
>
> SHIPPING ADDRESS (for packages):
> Tufts University Receiving
> 37 Tyler St.
> Attn: Bunnell/Pathology/Jaharis 524
> Boston, MA 02111
>

Bill Oliver-3 Bill Oliver-3
Reply | Threaded
Open this post in threaded view
|

Re: Movie Corruption Issue

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

On Mon, 28 Jan 2008, Guy Cox wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> The message really seems extremely clear.  Do not use Quicktime
> under any circumstances.  If you have old movies in QT format
> convert them all on an old system (preferably isolated from the
> Net) to something that has open source support.  (There is
> probably a freeware Linux product that will do this as a batch
> process).

Yes, it's the encoding command like variant of the mplayer
I wrote about.  When used for translating between codecs,
the command is "mencoder."

> But these formats are NOT designed for scientific
> data - it may be fine to use a QT movie as a one-off to show
> at a conference (though from the number of problems I've
> seen on this front I'm not convinced!) but scientific data should
> be saved as as uncompressed image series in formats that
> will always be openable (even if not immediately playable as a
> movie).  It's not realistic to expect entertainment software to
> offer the data stability that science needs.
>

Mencoder can also output as a series of stills.  I'm not sure I would agree that video *captured* as video needs to be stored as stills.  Storing as stills does not obviate file format obsolescence.

First, of course, most (though by no means all) video, even high end, incorporates a bit of lossy compression during capture.  Storing these in uncompressed image formats is closing the barn door after the horse is gone.

Second, there are well known, open source, lossless compression algorithms that are pretty stable over time and supported by multiple still formats, and uncompressed still images don't have an advantage over them.  There is, fundamentally, *no* advantage, for instance, to saving a file in uncompressed TIFF over LZW compressed TIFF.  If LZW compressed TIFF format becomes obsolete, then the TIFF format will be obsolete and you'll have to resave the images in whatever comes next anyway.  Similarly, the codec specification for DV is part of the DV format specification, and if that codec becomes obsolete, then DV is obsolete.

Third, saving images as stills in no way guarantees that it will always be available.  When was the last time you worked with Targa/Truevision graphics adapter (.tga)  files?  The bottom line is that saving images as stills may delay the issue of format obsolescence, but will not remove it.  TIFF, for instance, will become obsolete in the next few years -- it is a proprietary format owned by Adobe, and has not undergone significant development.  Eventually, it will slip into obscurity the same way that GIF is slowly doing.

The bottom line is that there is *no* image format, still or video that will save you from format obsolescence.  Rather than try to save images in file formats that will not need revision, it is probably more prudent simply to have an SOP for periodically or continuously monitoriing the status of your files and a plan for migrating them when obsolescence inevitably starts.

One good resource, by the way, for finding out what software deals with what format is the PRONOM database in the UK.  See:
http://www.nationalarchives.gov.uk/pronom/#

Another good site is the wotsit database:
http://www.wotsit.org/

One discussion of developing such an SOP can be found at:
http://www.dlib.org/dlib/november04/stanescu/11stanescu.html

See also:
http://www.clir.org/pubs/reports/pub93/pub93.pdf


billo
Guy Cox Guy Cox
Reply | Threaded
Open this post in threaded view
|

Re: Movie Corruption Issue

Search the CONFOCAL archive at http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal Re: Movie Corruption Issue
Well, I have several programs that will convert tga files.  Since all
the imaging software folks have already written their tga readers
(as, indeed, have I) they are unlikely ever to disappear, however
rarely they are used.  LZW compressed TIFF is, I agree, unlikely
to disappear but since these files can often be larger than
uncompressed ones there is little reason to use them.  Even
if a format is lost off the face of the earth, uncompressed image
data can usually be retrieved while compressed data is more
tricky.  There are readers out there even for old, strange,
confocal and image analysis files (some with non-square pixels).
Why?   Because they are scientific formats and there is a real
need to preserve scientific data.
 
Entertainment formats have a short life - technology advances
and nobody is interested in the clunky technology of 5 or 10
years ago.  Science and entertainment have very different
requirements.
 
                                                                       Guy
 
 
 
Optical Imaging Techniques in Cell Biology
by Guy Cox    CRC Press / Taylor & Francis
     http://www.guycox.com/optical.htm
______________________________________________
Associate Professor Guy Cox, MA, DPhil(Oxon)
Electron Microscope Unit, Madsen Building F09,
University of Sydney, NSW 2006
______________________________________________
Phone +61 2 9351 3176     Fax +61 2 9351 7682
Mobile 0413 281 861
______________________________________________
http://www.guycox.net


From: Confocal Microscopy List on behalf of Bill Oliver
Sent: Tue 08/01/29 7:48 AM
To: [hidden email]
Subject: Re: Movie Corruption Issue

Search the CONFOCAL archive at
http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal

On Mon, 28 Jan 2008, Guy Cox wrote:

> Search the CONFOCAL archive at
> http://listserv.acsu.buffalo.edu/cgi-bin/wa?S1=confocal
>
> The message really seems extremely clear.  Do not use Quicktime
> under any circumstances.  If you have old movies in QT format
> convert them all on an old system (preferably isolated from the
> Net) to something that has open source support.  (There is
> probably a freeware Linux product that will do this as a batch
> process).

Yes, it's the encoding command like variant of the mplayer
I wrote about.  When used for translating between codecs,
the command is "mencoder."

> But these formats are NOT designed for scientific
> data - it may be fine to use a QT movie as a one-off to show
> at a conference (though from the number of problems I've
> seen on this front I'm not convinced!) but scientific data should
> be saved as as uncompressed image series in formats that
> will always be openable (even if not immediately playable as a
> movie).  It's not realistic to expect entertainment software to
> offer the data stability that science needs.
>

Mencoder can also output as a series of stills.  I'm not sure I would agree that video *captured* as video needs to be stored as stills.  Storing as stills does not obviate file format obsolescence.

First, of course, most (though by no means all) video, even high end, incorporates a bit of lossy compression during capture.  Storing these in uncompressed image formats is closing the barn door after the horse is gone.

Second, there are well known, open source, lossless compression algorithms that are pretty stable over time and supported by multiple still formats, and uncompressed still images don't have an advantage over them.  There is, fundamentally, *no* advantage, for instance, to saving a file in uncompressed TIFF over LZW compressed TIFF.  If LZW compressed TIFF format becomes obsolete, then the TIFF format will be obsolete and you'll have to resave the images in whatever comes next anyway.  Similarly, the codec specification for DV is part of the DV format specification, and if that codec becomes obsolete, then DV is obsolete.

Third, saving images as stills in no way guarantees that it will always be available.  When was the last time you worked with Targa/Truevision graphics adapter (.tga)  files?  The bottom line is that saving images as stills may delay the issue of format obsolescence, but will not remove it.  TIFF, for instance, will become obsolete in the next few years -- it is a proprietary format owned by Adobe, and has not undergone significant development.  Eventually, it will slip into obscurity the same way that GIF is slowly doing.

The bottom line is that there is *no* image format, still or video that will save you from format obsolescence.  Rather than try to save images in file formats that will not need revision, it is probably more prudent simply to have an SOP for periodically or continuously monitoriing the status of your files and a plan for migrating them when obsolescence inevitably starts.

One good resource, by the way, for finding out what software deals with what format is the PRONOM database in the UK.  See:
http://www.nationalarchives.gov.uk/pronom/#

Another good site is the wotsit database:
http://www.wotsit.org/

One discussion of developing such an SOP can be found at:
http://www.dlib.org/dlib/november04/stanescu/11stanescu.html

See also:
http://www.clir.org/pubs/reports/pub93/pub93.pdf


billo

12