Re: Bio Image data retention

Posted by Jason Swedlow-2 on
URL: http://confocal-microscopy-list.275.s1.nabble.com/Bio-Image-data-retention-tp7580193p7580201.html

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
*****

Hi,

As many of you might guess, at the University of Dundee, we use OME's OMERO
(we contribute to OMERO development, as a participant in the Open
Microscopy Environment Consortium; http://openmicroscopy.org) to manage,
share, analyse and now publish our image data.  We hope to have examples of
the latter soon-- it means integration with Drupal, which is a bit fussy.
The OMERO instance is accessed by a separate system where we run Matlab
analysis tools for spot finding and measurements-- pretty simple stuff, but
very helpful to automate and manage.  All the analyses are stored alongside
with the original files, etc.  All data is backed up on redundant tape
libraries.

The production OMERO instance is backed with a size-limited filesystem, so
only high value data is moved there and analysed.  We also have a separate
hierachical filesystem, with about 270 TB of available on-line storage, and
redundant 2 PB LTO4 tape arrays.  The whole thing is run by Tivoli.  Most
of the images we ever collect are somewhere in this system, but we know
from over 10 years use of this system that 95% of recorded data are never
touched after 3 months have passed from collection.  That is why we use
tape-- it's cheap, reliable, space efficient and doesn't require power.

We're looking at a tape-based archiving solution for our OMERO instance.
This will take a few months, but we'll hopefully know whether this approach
works and will obviously pass that info on, once we have it.

So much of our work is funded by the Wellcome Trust, CRUK, BBSRC or MRC, we
have to follow their data retention and provenance policies.

HTH,

Cheers,

Jason






On Tue, Apr 16, 2013 at 9:58 PM, Wiegraebe, Winfried <[hidden email]>wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> *****
>
> At the Stowers Institute, at this moment users are responsible for their
> original data. We recently introduced a policy that members of the
> institute have to post all original data for their publications at our
> Original Data Repository http://odr.stowers.org/websimr/. To help them to
> organize the data, we are setting up an OMERO server (
> http://www.openmicroscopy.org/). We will automatically copy all data
> acquired at the Microscopy Center into the OMERO database. Our IT
> department and the Microscopy Center will maintain the server.
> Winfried
>
>
> Winfried Wiegraebe
> Head of Microscopy
> Stowers Institute for Medical Research
>
> [hidden email]
> [phone]
> http://research.stowers.org/microscopy/
> http://www.stowers.org/
>
>
> -----Original Message-----
> From: Confocal Microscopy List [mailto:[hidden email]]
> On Behalf Of Kurt Thorn
> Sent: Tuesday, April 16, 2013 11:41 AM
> To: [hidden email]
> Subject: Re: Bio Image data retention
>
> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> *****
>
> I'm not sure what the official policy is here, but the imaging center I
> run does not do any long term data archiving. We leave this up to the labs
> as we neither have the resources nor the expertise to provide long term
> data storage for the many terabytes of data generated here.
>
> We allow users to keep data on the microscope workstations for 30 days.
> We maintain an 8 TB file server for data transfer and temporary storage
> that we allow users to keep data on for 1 year. If we let users keep data
> on there much longer it gets full so it seems like users generate data at
> at least 6 TB/yr, which is a fair amount of data to store.
>
> Kurt
>
> On 4/16/2013 3:30 AM, Christopher Guerin wrote:
> > *****
> > To join, leave or search the confocal microscopy listserv, go to:
> > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > *****
> >
> > Hello everyone:
> >
> > Can anyone let me know if you have an official institutional policy on
> > the archiving/retention of bio imaging data? I'm particularly
> > interested in an policies defining whose responsibility it is to
> > retain unchanged raw data and for how long such data must be kept.
> > Feel free to contact me off list if you prefer.
> >
> > Many thanks,
> >
> > Chris Guérin
> >
> >
>

--
**************************
Wellcome Trust Centre for Gene Regulation & Expression
College of Life Sciences
MSI/WTB/JBC Complex
University of Dundee
Dow Street
Dundee  DD1 5EH
United Kingdom

phone [phone]
Intl phone:  [phone]
FAX   [phone]
email: [hidden email]

Lab Page: http://www.lifesci.dundee.ac.uk/gre/staff/jason-swedlow
Open Microscopy Environment: http://openmicroscopy.org
**************************