Re: Data Storage

Posted by jerie on
URL: http://confocal-microscopy-list.275.s1.nabble.com/Data-Storage-tp7585019p7585030.html

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

-commercial content warning-

Dear Listers,

it might be interesting for some of you, that the company Acquifer -an
EMBL/KIT spinoff- has developed a data solution for STORM & SPIM
applications and screening workflows.

Key features are:
- write rates of > 800MBytes/sec. for streaming from 2 external cameras
onto a RAID 5/6 array.
- 128GByte RAM 3,5 GHz HEXcore CPU,GTX 970 gfx card for GPU
processing/deconvolution/data compression.
- software defined networking router powered by OCEDO (now RIVERBED) for
secure remote viewing/remote desktop applications.
- 47TByte intermediate storage.

The system runs Windows Server 2012r2 and has been tested in
various microscope facilites, screening labs, with many open source
image/data processing packages (e.g. Fiji, ilastik, Cellprofiler, KNIME
etc.). It helps to drastically reduce network traffic.

More info is available at https://www.acquifer.de/bigdata-logistics/

I am happy to answer questions or forward you to the respective tech
specialists.

Disclaimer: I have been doing paid consultancy work for Acquifer.

Cheers, Jens

Dr. Jens Rietdorf, visiting scientist @ center for technological
development in health CDTS, Oswaldo Cruz Foundation Fiocruz, Rio de Janeiro
Brasil.

On Mon, Apr 11, 2016 at 12:27 PM, Claire Brown, Dr. <[hidden email]>
wrote:
>
> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> I'm working on some numbers for cyberinfrastructure for Compute Canada. I
am not currently doing single point localization microscopy but we plan to
get into it.
>
> I just wonder is there a consensus in the field if the raw images for
each point localization have to be retained for 7 years or are people just
keeping the localization data? When I do these calculations I get the
following so it seems impossible with existing infrastructure to retain the
raw data.
>
> 2000 x 2000 pixel camera and 16-bit images = 2000x2000x16 = 64 MB per
image
>
> 10,000 frames for single molecule imaging and two colour super resolution
> 10,000x2x64 = 1.28 TB
>
> 4 conditions, 10 images per condition, experiment done in triplicate
> 1.28 TBx4x10x3 = 154 TB per experiment
>
> So I'm guessing people are not keeping the raw data or I made a mistake
in my calculations.
>
> Looking forward to some feedback!
>
> Sincerely,
>
> Claire