Posted by
Reece, Jeff (NIH/NIDDK) [E] on
URL: http://confocal-microscopy-list.275.s1.nabble.com/Data-Storage-tp7585019p7585029.html
*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopyPost images on
http://www.imgur.com and include the link in your posting.
*****
For very large data sets, consider investing into SSD HDs and 10 Gb ethernet on your computers, and Cat6 ethernet cabling.
Latest technology is roughly an order of magnitude faster than the USB3 thumb drives.
But I like the USB external HDs too.
Cheers,
Jeff
________________________________________
From: Smith, Benjamin E. [
[hidden email]]
Sent: Monday, April 11, 2016 1:48 PM
To:
[hidden email]
Subject: Re: Data Storage
*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopyPost images on
http://www.imgur.com and include the link in your posting.
*****
One other consideration, if the data transfer is the bottleneck, then it will likely be faster if you compress the data before transferring it. Also, depending on the computer and server, FTP can be significantly faster than SFTP (keeping in mind that your login and password will be transmitted unencrypted). One other trick we do for VERY large data sets that we want to move quickly is to use a dedicated USB3 thumb drive (or for TB+ data, an internal hard drive) to move data.
-Ben Smith
________________________________________
From: Confocal Microscopy List <
[hidden email]> on behalf of Claire Brown <
[hidden email]>
Sent: Monday, April 11, 2016 12:27 PM
To:
[hidden email]
Subject: Re: Data Storage
*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopyPost images on
http://www.imgur.com and include the link in your posting.
*****
Thank you everyone for the feedback. I'll update my calculations for 16-bit
= 2 byte. That helps a bit.
We will propose to put fiber optic on all connections but it remains to be
seen who will pay for that.
I remember when I started in my facility and we had a brand new $750k
confocal but no one could pay $100 to replace the beat up old lab stool they
had at the microscope!
We have always had a policy that the core assumes no responsibility for
image data but we also try to offer a service and make sure our users and
our institution keeps up with the needs. Not so easy in a field that is
changing so quickly.
I am going to keep the 2000x2000 pixels in the calculations. I'm also doing
similar calculations for light sheet, HCS and slide scanners so I'll stick
to one image size. I like worst case scenarios too. Everyone is always
pleased if things are a bit better than the calculations. Same as budgets. I
always over estimate expenses and over estimate revenues.
I'm happy to share my final document with anyone who might be interested.
Sincerely,
Claire