Posted by
Sylvie Le Guyader on
URL: http://confocal-microscopy-list.275.s1.nabble.com/IT-infrastructure-tp7590206p7590213.html
*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopyPost images on
http://www.imgur.com and include the link in your posting.
*****
My experience is that many cloud infrastructures do not allow to upload files that are larger than 4 GB.
Not sure if it is true for all of them.
On top of that, there is of course the problem of speed to upload and need to download again to analyse the data.
With a local infrastructure, you skip this time which is considerable if one has a light sheet system.
Med vänlig hälsning / Best regards
Sylvie
@@@@@@@@@@@@@@@@@@@@@@@@
Sylvie Le Guyader, PhD
Live Cell Imaging Facility Manager
Karolinska Institutet- Bionut Dpt
Blickagången 16,
Room 7362 (lab)/7840 (office)
14157 Huddinge, Sweden
mobile: +46 (0) 73 733 5008
LCI website
Follow our microscopy blog!
-----Original Message-----
From: Confocal Microscopy List <
[hidden email]> On Behalf Of Feinstein, Timothy N
Sent: 22 November 2019 17:01
To:
[hidden email]
Subject: Re: IT infrastructure
*****
To join, leave or search the confocal microscopy listserv, go to:
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.umn.edu%2Fcgi-bin%2Fwa%3FA0%3Dconfocalmicroscopy&data=02%7C01%7Csylvie.le.guyader%40KI.SE%7C72b07d3b21974b1c317a08d76f653775%7Cbff7eef1cf4b4f32be3da1dda043c05d%7C0%7C0%7C637100352821333002&sdata=1zwM2QnH%2BG%2FZxpFUIyIjAu2cB%2FOR3a3EQs1xjMpfAfw%3D&reserved=0Post images on
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.imgur.com&data=02%7C01%7Csylvie.le.guyader%40KI.SE%7C72b07d3b21974b1c317a08d76f653775%7Cbff7eef1cf4b4f32be3da1dda043c05d%7C0%7C0%7C637100352821333002&sdata=%2BFmGOVWZjVg8TKAeHDQYE8m6fBUXw0kv9wJ2zivKpqE%3D&reserved=0 and include the link in your posting.
*****
My current institution mostly uses cloud solutions, due to University-wide agreements with cloud providers. As pointed out this isn't great for lab-wide aggregating and archiving. My previous institution also had a centralized PB-scale Isilon server for rapid-access needs, plus an offline tape backup service for long term archiving. This has a great economy of scale - and it made it very easy for us to implement OMERO, which is a fantastic image data management package! However not many institutions can support that kind of centralized data handling.
T
On 11/22/19, 8:40 AM, "Confocal Microscopy List on behalf of Gary Laevsky" <
[hidden email] on behalf of
[hidden email]> wrote:
*****
To join, leave or search the confocal microscopy listserv, go to:
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.umn.edu%2Fcgi-bin%2Fwa%3FA0%3Dconfocalmicroscopy&data=02%7C01%7Csylvie.le.guyader%40KI.SE%7C72b07d3b21974b1c317a08d76f653775%7Cbff7eef1cf4b4f32be3da1dda043c05d%7C0%7C0%7C637100352821333002&sdata=1zwM2QnH%2BG%2FZxpFUIyIjAu2cB%2FOR3a3EQs1xjMpfAfw%3D&reserved=0 Post images on
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.imgur.com&data=02%7C01%7Csylvie.le.guyader%40KI.SE%7C72b07d3b21974b1c317a08d76f653775%7Cbff7eef1cf4b4f32be3da1dda043c05d%7C0%7C0%7C637100352821333002&sdata=%2BFmGOVWZjVg8TKAeHDQYE8m6fBUXw0kv9wJ2zivKpqE%3D&reserved=0 and include the link in your posting.
*****
Hi Matthieu,
We just purchased 5 PB and migrated to an Isilon.
We do run some applications on the server, but time there is at a premium.
We also have a few fairly beefy PCs to crunch data locally.
Right now, this is our short-medium-long-term solution. Not sure if we
will migrate to something like Glacier for long-term eventually.
We are collecting cryo and light-sheet, so the numbers are getting up
there. We are also updating applicable systems to 10GB fiber for transfer.
Interested to hear other solutions as well.
Big Data is always a favorite topic at our NAMS meetings, and it's also the
name of our light-sheet workshop next summer as well (plug intended!)!
On Fri, Nov 22, 2019 at 7:54 AM George McNamara <
[hidden email]>
wrote:
> *****
> To join, leave or search the confocal microscopy listserv, go to:
>
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.umn.edu%2Fcgi-bin%2Fwa%3FA0%3Dconfocalmicroscopy&data=02%7C01%7Csylvie.le.guyader%40KI.SE%7C72b07d3b21974b1c317a08d76f653775%7Cbff7eef1cf4b4f32be3da1dda043c05d%7C0%7C0%7C637100352821333002&sdata=1zwM2QnH%2BG%2FZxpFUIyIjAu2cB%2FOR3a3EQs1xjMpfAfw%3D&reserved=0 > Post images on
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.imgur.com&data=02%7C01%7Csylvie.le.guyader%40KI.SE%7C72b07d3b21974b1c317a08d76f653775%7Cbff7eef1cf4b4f32be3da1dda043c05d%7C0%7C0%7C637100352821333002&sdata=%2BFmGOVWZjVg8TKAeHDQYE8m6fBUXw0kv9wJ2zivKpqE%3D&reserved=0 and include the link in your posting.
> *****
>
> Hi Matthieu,
>
> 20 TB WD My Book Duo USB 3.1 are $700 ... orWD 20TB My Cloud EX2 Ultra
> Network Attached Storage $999 ...
> <
>
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.amazon.com%2FCloud-Ultra-Network-Attached-Storage%2Fdp%2FB07179ZYH2%2Fref%3Dsr_1_3%3Fcrid%3D3ILUXD0SCQ70Z%26keywords%3Dwd%2B20tb%2Bmy%2Bbook%2Bduo%26qid%3D1574426442%26sprefix%3Dwd%2B20%2Btb%252Caps%252C129%26sr%3D8-3&data=02%7C01%7Csylvie.le.guyader%40KI.SE%7C72b07d3b21974b1c317a08d76f653775%7Cbff7eef1cf4b4f32be3da1dda043c05d%7C0%7C0%7C637100352821333002&sdata=82rqObMNv1MAOuQMw46Txk7ZXI%2B%2FnUGXizCjzh89OEw%3D&reserved=0>you
>
> may want to have one or more in your facility for backup (move month old
> data) AND encourage P.I.'s to have in for their labs (and most PI's
> think about their research, not about I.T. and need for data backups).
>
> In a previous thread I posted about what we have ... some 10 Gbe
> computers, 10 Gbe server (40 Tb HDD RAID array), and a switch 10 Gbe
> Ethernet (CAT7 copper wire, others pointed out SFP+ fiber optic is less
> expensive ... future: connect our image core rooms to the fiber optic
> switch room), some PCs with ASUS Hyper M.2 PCIe card with 4 Silicon
> Power 2 Tb NVMe SSDs, ~$1050 for 8Tb fast local storage (PCIe x16 slot,
> BIOS configure the slot as x4x4x4x4) ... goal is fast local acquisition
> saving --> fast network --> best wishes.
>
> JHU now provides 5 Terabytes Microsoft OneDrive cloud storage per
> employee/student ... and in principle could call I.T. and get more. So
> far, not easily "aggregated" by lab ... I hope this gets changed so a PI
> of a 20 person lab gets 100 Tb to start with, organized by user (say 1
> Tb private to that user, visible to the PI) and mostly organized by
> projects.
>
> enjoy,
>
> George
>
> On 11/22/2019 6:43 AM, VERMEREN Matthieu wrote:
> > *****
> > To join, leave or search the confocal microscopy listserv, go to:
> >
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.umn.edu%2Fcgi-bin%2Fwa%3FA0%3Dconfocalmicroscopy&data=02%7C01%7Csylvie.le.guyader%40KI.SE%7C72b07d3b21974b1c317a08d76f653775%7Cbff7eef1cf4b4f32be3da1dda043c05d%7C0%7C0%7C637100352821342994&sdata=vGYVviElhyn989IOKopV0pxFmgergEsCPkEiVMtC4aw%3D&reserved=0 > > Post images on
https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.imgur.com&data=02%7C01%7Csylvie.le.guyader%40KI.SE%7C72b07d3b21974b1c317a08d76f653775%7Cbff7eef1cf4b4f32be3da1dda043c05d%7C0%7C0%7C637100352821342994&sdata=xx1yWO%2FXv7JOXdqaVIyzwUnwdoAU6F4YwRckbQl6XDo%3D&reserved=0 and include the link in your
> posting.
> > *****
> >
> > Dear all,
> >
> > We are producing massive amount of data that legally needs to be stored
> for up to ten years. This data also needs to be analysed fairly quickly
> -hence storage on local servers. Furthermore, some software companies
> (Huygens, Imaris...) are now offering server-based applications for image
> analysis -all the hard work is done on the server and your computer is just
> a terminal.
> >
> >
> > Currently, we tell users to transfer their data to NAS immediately after
> imaging, so the microscope computers are not clogged. Image files found on
> these computers are fair game after a month. We expect users to then back
> up their files from the NAS to cloud. All our image analysis is done on
> fairly beefy workstations and not server based.
> >
> > As I'd like to future proof our IT, it would be great to know what
> servers/storage space do you have as part of your facilities/institutes?
> What solutions do you have for long term storage vs short to medium terms?
> Do you have server based applications?
> >
> > Matthieu
> > The University of Edinburgh is a charitable body, registered in
> Scotland, with registration number SC005336.
>
--
Best,
Gary Laevsky, Ph.D.
Director, Confocal Imaging Facility
Nikon Center of Excellence
Co-Founder, North Atlantic Microscopy Society (NAMS)
https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnamsmicroscopy.com%2F&data=02%7C01%7Csylvie.le.guyader%40KI.SE%7C72b07d3b21974b1c317a08d76f653775%7Cbff7eef1cf4b4f32be3da1dda043c05d%7C0%7C0%7C637100352821342994&sdata=xSlx6AxMMoHUEqLLM%2Fex2nR7kh2dHDnwdjnH1AUexOI%3D&reserved=0 Dept. of Molecular Biology
Washington Rd.
Princeton University
Princeton, New Jersey, 08544-1014
(O) 609 258 5432
(C) 508 507 1310
North Atlantic Microscopy Society Spring Meeting at UPENN, April 23, 2020.
När du skickar e-post till Karolinska Institutet (KI) innebär detta att KI kommer att behandla dina personuppgifter. Här finns information om hur KI behandlar personuppgifter<
https://ki.se/medarbetare/integritetsskyddspolicy>.
Sending email to Karolinska Institutet (KI) will result in KI processing your personal data. You can read more about KI’s processing of personal data here<
https://ki.se/en/staff/data-protection-policy>.