Lightsheet imaging analysis Workstation Specs

[Niyanta Kumar]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi all,
I plan to get a workstation that can handle confocal and lightsheet data analysis using Imaris and/or Arivis. Lightsheet data clearly has the higher bar/needs in terms of specs. Below are the specs I am considering. Can you please let me know if you have any recommendations?

Lenovo Think Station P920
3.7 GHz CPUs (Intel) 48 cores
Graphics: NVIDIA Quadro RTX 6000 or 8000
4 X 3 TB of local SSD (multiple drives - one for the OS, one for writing, one for reading)
RAM: 500 GB
Monitor: dual 1920 x 1200
300 TB NAS via 10 GB ethernet – file sharing
Mouse: 3 button wheel

Monitor: thinking of a dual Lenovo Think Vision P32u-10 but it doesn’t have a full array LED. Any suggestions? I hear the curved monitors can cause issues with display when sharing screens over Webex etc.
Thanks,
Niyanta

[Arvonn Tully]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

***Commercial Response***
Dear Niyanta
That sounds like it will be fine for Arivis Vision4d.
If anything,  you can probably  save a bit of money as you do need that
much ram for Vision4d.
In order to support large data sets on any hardware, Vision4d does not try
to cache the entire data set in Ram, only the current portion that is
required for processing.
As such, typically light sheet workstations are fine with 64-128 gb of ram.
Running with 256-512 gb won't be a problem, but we suggest you buy some
more storage instead. :)



Best Regards
arvonn

----

*Arvonn Tully*
Application Engineer
*arivis AG - Office US*
1875 Connecticut Ave NW
10th Floor
Washington, DC 20009
USA
Phone: [phone]
E-mail : [hidden email]
Web : http://www.arivis.com

On Tue, Jan 14, 2020 at 4:26 PM Niyanta Kumar <
[hidden email]> wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hi all,
> I plan to get a workstation that can handle confocal and lightsheet data
> analysis using Imaris and/or Arivis. Lightsheet data clearly has the higher
> bar/needs in terms of specs. Below are the specs I am considering. Can you
> please let me know if you have any recommendations?
>
> Lenovo Think Station P920
> 3.7 GHz CPUs (Intel) 48 cores
> Graphics: NVIDIA Quadro RTX 6000 or 8000
> 4 X 3 TB of local SSD (multiple drives - one for the OS, one for writing,
> one for reading)
> RAM: 500 GB
> Monitor: dual 1920 x 1200
> 300 TB NAS via 10 GB ethernet – file sharing
> Mouse: 3 button wheel
>
> Monitor: thinking of a dual Lenovo Think Vision P32u-10 but it doesn’t
> have a full array LED. Any suggestions? I hear the curved monitors can
> cause issues with display when sharing screens over Webex etc.
> Thanks,
> Niyanta
>

[Trevor Lancon]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

***Commercial Response***

Hello Niyanta,

If you haven't heard of Aivia, I invite you to also try it for processing your lightsheet data.

Our highest-tier system requirement (https://www.drvtechnologies.com/system-requirements) recommends a GeForce RTX 2080 Ti, which currently looks to be about 4x cheaper than the Quadro RTX 6000.

Any commercial software you use for lightsheet data should not require 500GB of RAM and would benefit more from ~64GB of RAM, plus the fastest I/O speed you can achieve with your hard storage. Your 3TB of SSDs and 10G ethernet look great for that.

Best regards,

Trevor

---
Trevor Lancon
Aivia Applications Specialist – Eastern US
[hidden email] | [phone]

-----Original Message-----
From: Confocal Microscopy List <[hidden email]> On Behalf Of Niyanta Kumar
Sent: Tuesday, January 14, 2020 3:26 PM
To: [hidden email]
Subject: Lightsheet imaging analysis Workstation Specs

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi all,
I plan to get a workstation that can handle confocal and lightsheet data analysis using Imaris and/or Arivis. Lightsheet data clearly has the higher bar/needs in terms of specs. Below are the specs I am considering. Can you please let me know if you have any recommendations?

Lenovo Think Station P920
3.7 GHz CPUs (Intel) 48 cores
Graphics: NVIDIA Quadro RTX 6000 or 8000
4 X 3 TB of local SSD (multiple drives - one for the OS, one for writing, one for reading)
RAM: 500 GB
Monitor: dual 1920 x 1200
300 TB NAS via 10 GB ethernet – file sharing
Mouse: 3 button wheel

Monitor: thinking of a dual Lenovo Think Vision P32u-10 but it doesn’t have a full array LED. Any suggestions? I hear the curved monitors can cause issues with display when sharing screens over Webex etc.
Thanks,
Niyanta  

[PAVAK SHAH]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Niyanta,

What lightsheet instrument and what is the size of your typical dataset? Is
your processing pipeline primarily for segmentation or multiview
deconvolution? This will dictate, to a large extent, the kind of memory and
CPU you need.

An RTX 2080ti or Titan RTX will be competitively performant but much
cheaper than a Quadro card, which offers no benefits for an image analysis
workflow unless you're running virtual machines. If your workflow is
primarily single view deconvolution, multiple GPUs can also be a huge boon
in terms of processing throughput, especially if individual volumes can fit
in 11 GB of VRAM since you can fill a large workstation to bursting with
2080ti's for the price of 1 Quadro card. Even 2x Titan RTX can be had for
less than the price of 1x Quadro 6000.

Depending on the specific piece of software, many will not scale
efficiently to >16 cores and a faster clocked CPU with fewer cores may be
advantageous. If you can benchmark it on a high core count system, that
should show whether your pipelines are able to keep that many cores fed
before running into algorithmic, disk access or memory throughput
bottlenecks.

Best,
Pavak

On Tue, Jan 14, 2020, 1:27 PM Niyanta Kumar <
[hidden email]> wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hi all,
> I plan to get a workstation that can handle confocal and lightsheet data
> analysis using Imaris and/or Arivis. Lightsheet data clearly has the higher
> bar/needs in terms of specs. Below are the specs I am considering. Can you
> please let me know if you have any recommendations?
>
> Lenovo Think Station P920
> 3.7 GHz CPUs (Intel) 48 cores
> Graphics: NVIDIA Quadro RTX 6000 or 8000
> 4 X 3 TB of local SSD (multiple drives - one for the OS, one for writing,
> one for reading)
> RAM: 500 GB
> Monitor: dual 1920 x 1200
> 300 TB NAS via 10 GB ethernet – file sharing
> Mouse: 3 button wheel
>
> Monitor: thinking of a dual Lenovo Think Vision P32u-10 but it doesn’t
> have a full array LED. Any suggestions? I hear the curved monitors can
> cause issues with display when sharing screens over Webex etc.
> Thanks,
> Niyanta
>

[Travis Beck]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Perhaps this is already known but changing from sata SSD drives to NVME
ssd's mounted on PCIE cards provides about a 10x increase in IO. Read write
speeds are ~300mbps r/w for sata  to ~3000mbps r/w via pcie. This makes a
dramatic difference in IO limited applications. Saw a good boost in imaging
analysis but several fold better in our genetic work.
Cheers!
Travis

[j.springfield]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

We’re using PCIe RAID cards with multiple NVME 2TB drives to get around 12GB/s read/write speeds
Things to be mindful of as already mentioned, do you require multiple GPUs for deconvolution and do the GPUs have enough VRAM to store a stack
Do you have enough RAM to store duplicates of the data when transferring from say Imaris to FIJI and back?
Does the PC/Server have enough PCIe lanes for the GPUs, RAID cards and NIC etc

You may find (if you have suitable IT support) a server grade system is more appropriate and possibly cheaper, whilst allowing multiple users
Keep in mind you may require a dedicated GPU for rendering of the session and separate CUDA capable GPUs for deconvolution

Best
James

On 15 Jan 2020, at 9:41 am, Travis Beck <[hidden email]<mailto:[hidden email]>> wrote:

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Perhaps this is already known but changing from sata SSD drives to NVME
ssd's mounted on PCIE cards provides about a 10x increase in IO. Read write
speeds are ~300mbps r/w for sata  to ~3000mbps r/w via pcie. This makes a
dramatic difference in IO limited applications. Saw a good boost in imaging
analysis but several fold better in our genetic work.
Cheers!
Travis

[Francesco Pasqualini]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Pavak et al,
This is very interesting and helpful.

Do I understand you correctly that multiple GPU won't help with multiview
deconvolution?

If there is a reference for deconvolution performances as a function of
various hardware options can anyone point it out?

Thanks
Francesco



On Wed, Jan 15, 2020, 12:57 AM PAVAK SHAH <[hidden email]> wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hi Niyanta,
>
> What lightsheet instrument and what is the size of your typical dataset? Is
> your processing pipeline primarily for segmentation or multiview
> deconvolution? This will dictate, to a large extent, the kind of memory and
> CPU you need.
>
> An RTX 2080ti or Titan RTX will be competitively performant but much
> cheaper than a Quadro card, which offers no benefits for an image analysis
> workflow unless you're running virtual machines. If your workflow is
> primarily single view deconvolution, multiple GPUs can also be a huge boon
> in terms of processing throughput, especially if individual volumes can fit
> in 11 GB of VRAM since you can fill a large workstation to bursting with
> 2080ti's for the price of 1 Quadro card. Even 2x Titan RTX can be had for
> less than the price of 1x Quadro 6000.
>
> Depending on the specific piece of software, many will not scale
> efficiently to >16 cores and a faster clocked CPU with fewer cores may be
> advantageous. If you can benchmark it on a high core count system, that
> should show whether your pipelines are able to keep that many cores fed
> before running into algorithmic, disk access or memory throughput
> bottlenecks.
>
> Best,
> Pavak
>
> On Tue, Jan 14, 2020, 1:27 PM Niyanta Kumar <
> [hidden email]> wrote:
>
> > *****
> > To join, leave or search the confocal microscopy listserv, go to:
> > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > Post images on http://www.imgur.com and include the link in your
> posting.
> > *****
> >
> > Hi all,
> > I plan to get a workstation that can handle confocal and lightsheet data
> > analysis using Imaris and/or Arivis. Lightsheet data clearly has the
> higher
> > bar/needs in terms of specs. Below are the specs I am considering. Can
> you
> > please let me know if you have any recommendations?
> >
> > Lenovo Think Station P920
> > 3.7 GHz CPUs (Intel) 48 cores
> > Graphics: NVIDIA Quadro RTX 6000 or 8000
> > 4 X 3 TB of local SSD (multiple drives - one for the OS, one for writing,
> > one for reading)
> > RAM: 500 GB
> > Monitor: dual 1920 x 1200
> > 300 TB NAS via 10 GB ethernet – file sharing
> > Mouse: 3 button wheel
> >
> > Monitor: thinking of a dual Lenovo Think Vision P32u-10 but it doesn’t
> > have a full array LED. Any suggestions? I hear the curved monitors can
> > cause issues with display when sharing screens over Webex etc.
> > Thanks,
> > Niyanta
> >
>

[Gary Laevsky]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Niyanta,

We've been through three iterations of this.

What we have found critical is to try and determine what software you are
going to be using for analysis.

Some are all CPU intensive.
Some are GPU intensive.
Some will use multiple GPU's
Some will not use multiple GPU's, so we had to get one beasty GPU (RTX 8000
48Gb).

And of course HD and RAM are important.

And as state, read/write is a rate limiter on transfer.  SSD good.  We have
10 Gb fiber cards, but never get that speed ...

I now have three different light-sheet analysis PC's, depending on the size
of the dataset and software.

On Wed, Jan 15, 2020 at 3:32 AM Francesco Pasqualini <
[hidden email]> wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hi Pavak et al,
> This is very interesting and helpful.
>
> Do I understand you correctly that multiple GPU won't help with multiview
> deconvolution?
>
> If there is a reference for deconvolution performances as a function of
> various hardware options can anyone point it out?
>
> Thanks
> Francesco
>
>
>
> On Wed, Jan 15, 2020, 12:57 AM PAVAK SHAH <[hidden email]> wrote:
>
> > *****
> > To join, leave or search the confocal microscopy listserv, go to:
> > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > Post images on http://www.imgur.com and include the link in your
> posting.
> > *****
> >
> > Hi Niyanta,
> >
> > What lightsheet instrument and what is the size of your typical dataset?
> Is
> > your processing pipeline primarily for segmentation or multiview
> > deconvolution? This will dictate, to a large extent, the kind of memory
> and
> > CPU you need.
> >
> > An RTX 2080ti or Titan RTX will be competitively performant but much
> > cheaper than a Quadro card, which offers no benefits for an image
> analysis
> > workflow unless you're running virtual machines. If your workflow is
> > primarily single view deconvolution, multiple GPUs can also be a huge
> boon
> > in terms of processing throughput, especially if individual volumes can
> fit
> > in 11 GB of VRAM since you can fill a large workstation to bursting with
> > 2080ti's for the price of 1 Quadro card. Even 2x Titan RTX can be had for
> > less than the price of 1x Quadro 6000.
> >
> > Depending on the specific piece of software, many will not scale
> > efficiently to >16 cores and a faster clocked CPU with fewer cores may be
> > advantageous. If you can benchmark it on a high core count system, that
> > should show whether your pipelines are able to keep that many cores fed
> > before running into algorithmic, disk access or memory throughput
> > bottlenecks.
> >
> > Best,
> > Pavak
> >
> > On Tue, Jan 14, 2020, 1:27 PM Niyanta Kumar <
> > [hidden email]> wrote:
> >
> > > *****
> > > To join, leave or search the confocal microscopy listserv, go to:
> > > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > > Post images on http://www.imgur.com and include the link in your
> > posting.
> > > *****
> > >
> > > Hi all,
> > > I plan to get a workstation that can handle confocal and lightsheet
> data
> > > analysis using Imaris and/or Arivis. Lightsheet data clearly has the
> > higher
> > > bar/needs in terms of specs. Below are the specs I am considering. Can
> > you
> > > please let me know if you have any recommendations?
> > >
> > > Lenovo Think Station P920
> > > 3.7 GHz CPUs (Intel) 48 cores
> > > Graphics: NVIDIA Quadro RTX 6000 or 8000
> > > 4 X 3 TB of local SSD (multiple drives - one for the OS, one for
> writing,
> > > one for reading)
> > > RAM: 500 GB
> > > Monitor: dual 1920 x 1200
> > > 300 TB NAS via 10 GB ethernet – file sharing
> > > Mouse: 3 button wheel
> > >
> > > Monitor: thinking of a dual Lenovo Think Vision P32u-10 but it doesn’t
> > > have a full array LED. Any suggestions? I hear the curved monitors can
> > > cause issues with display when sharing screens over Webex etc.
> > > Thanks,
> > > Niyanta
> > >
> >
>

--
Best,

Gary Laevsky, Ph.D.
Director, Confocal Imaging Facility
Nikon Center of Excellence
Co-Founder, North Atlantic Microscopy Society (NAMS)
https://namsmicroscopy.com/
Dept. of Molecular Biology
Washington Rd.
Princeton University
Princeton, New Jersey, 08544-1014
(O) 609 258 5432
(C) 508 507 1310

North Atlantic Microscopy Society Spring Meeting at UPENN, April 23, 2020.

[Vincent Schoonderwoert]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

**vendor reply**

Dear Niyanta,

Although your initial question is not address directly, we feel our response is
appropriate considering the type of imaging data and the response/questions of
the others.

First, any type of fusion of multiview light sheet data and optional
deconvolution needs loading of at least the complete data of two different
views in RAM and additional space for the resulting image.
You may consider to keep at least the option open to expand RAM in the future.
NVME SSDs can be very useful for extra virtual memory.

Second, regarding the question of Francesco:  Deconvolution of light sheet data
with Huygens allows using multiple GPU cards simultaneously. There is even
support for files that have multiview data
embedded as subimages, such as with Zeiss Z1 czi files.

For those of you that are interested, one of our Light Sheet Fuser
software developers
will present a webinar on January 28th about Huygens Fuser,
deconvolution of light sheet data, and CPU and GPU use. You can join via our
website or contact me offline.

Kind regards,
Vincent

---------------------------------------------
Vincent Schoonderwoert, Dr
Director Marketing & Science
Scientific Volume Imaging
www.svi.nl
---------------------------------------------

Op wo 15 jan. 2020 11:13 schreef Gary Laevsky <[hidden email]>:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hi Niyanta,
>
> We've been through three iterations of this.
>
> What we have found critical is to try and determine what software you are
> going to be using for analysis.
>
> Some are all CPU intensive.
> Some are GPU intensive.
> Some will use multiple GPU's
> Some will not use multiple GPU's, so we had to get one beasty GPU (RTX 8000
> 48Gb).
>
> And of course HD and RAM are important.
>
> And as state, read/write is a rate limiter on transfer.  SSD good.  We have
> 10 Gb fiber cards, but never get that speed ...
>
> I now have three different light-sheet analysis PC's, depending on the size
> of the dataset and software.
>
> On Wed, Jan 15, 2020 at 3:32 AM Francesco Pasqualini <
> [hidden email]> wrote:
>
> > *****
> > To join, leave or search the confocal microscopy listserv, go to:
> > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > Post images on http://www.imgur.com and include the link in your
> posting.
> > *****
> >
> > Hi Pavak et al,
> > This is very interesting and helpful.
> >
> > Do I understand you correctly that multiple GPU won't help with multiview
> > deconvolution?
> >
> > If there is a reference for deconvolution performances as a function of
> > various hardware options can anyone point it out?
> >
> > Thanks
> > Francesco
> >
> >
> >
> > On Wed, Jan 15, 2020, 12:57 AM PAVAK SHAH <[hidden email]> wrote:
> >
> > > *****
> > > To join, leave or search the confocal microscopy listserv, go to:
> > > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > > Post images on http://www.imgur.com and include the link in your
> > posting.
> > > *****
> > >
> > > Hi Niyanta,
> > >
> > > What lightsheet instrument and what is the size of your typical
> dataset?
> > Is
> > > your processing pipeline primarily for segmentation or multiview
> > > deconvolution? This will dictate, to a large extent, the kind of memory
> > and
> > > CPU you need.
> > >
> > > An RTX 2080ti or Titan RTX will be competitively performant but much
> > > cheaper than a Quadro card, which offers no benefits for an image
> > analysis
> > > workflow unless you're running virtual machines. If your workflow is
> > > primarily single view deconvolution, multiple GPUs can also be a huge
> > boon
> > > in terms of processing throughput, especially if individual volumes can
> > fit
> > > in 11 GB of VRAM since you can fill a large workstation to bursting
> with
> > > 2080ti's for the price of 1 Quadro card. Even 2x Titan RTX can be had
> for
> > > less than the price of 1x Quadro 6000.
> > >
> > > Depending on the specific piece of software, many will not scale
> > > efficiently to >16 cores and a faster clocked CPU with fewer cores may
> be
> > > advantageous. If you can benchmark it on a high core count system, that
> > > should show whether your pipelines are able to keep that many cores fed
> > > before running into algorithmic, disk access or memory throughput
> > > bottlenecks.
> > >
> > > Best,
> > > Pavak
> > >
> > > On Tue, Jan 14, 2020, 1:27 PM Niyanta Kumar <
> > > [hidden email]> wrote:
> > >
> > > > *****
> > > > To join, leave or search the confocal microscopy listserv, go to:
> > > > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > > > Post images on http://www.imgur.com and include the link in your
> > > posting.
> > > > *****
> > > >
> > > > Hi all,
> > > > I plan to get a workstation that can handle confocal and lightsheet
> > data
> > > > analysis using Imaris and/or Arivis. Lightsheet data clearly has the
> > > higher
> > > > bar/needs in terms of specs. Below are the specs I am considering.
> Can
> > > you
> > > > please let me know if you have any recommendations?
> > > >
> > > > Lenovo Think Station P920
> > > > 3.7 GHz CPUs (Intel) 48 cores
> > > > Graphics: NVIDIA Quadro RTX 6000 or 8000
> > > > 4 X 3 TB of local SSD (multiple drives - one for the OS, one for
> > writing,
> > > > one for reading)
> > > > RAM: 500 GB
> > > > Monitor: dual 1920 x 1200
> > > > 300 TB NAS via 10 GB ethernet – file sharing
> > > > Mouse: 3 button wheel
> > > >
> > > > Monitor: thinking of a dual Lenovo Think Vision P32u-10 but it
> doesn’t
> > > > have a full array LED. Any suggestions? I hear the curved monitors
> can
> > > > cause issues with display when sharing screens over Webex etc.
> > > > Thanks,
> > > > Niyanta
> > > >
> > >
> >
>
>
> --
> Best,
>
> Gary Laevsky, Ph.D.
> Director, Confocal Imaging Facility
> Nikon Center of Excellence
> Co-Founder, North Atlantic Microscopy Society (NAMS)
> https://namsmicroscopy.com/
> Dept. of Molecular Biology
> Washington Rd.
> Princeton University
> Princeton, New Jersey, 08544-1014
> (O) 609 258 5432
> (C) 508 507 1310
>
> North Atlantic Microscopy Society Spring Meeting at UPENN, April 23, 2020.
>

[PAVAK SHAH]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi Francesco,

I should clarify that it depends a lot on the software pipeline being used.
The ones I am most familiar with for diSPIM data (MIPAV, Shroff lab
software, and Stephan Preibisch's Multiview Reconstruction) don't
inherently support parallel processing of a single dataset on multiple
GPU's but multiple instances can be run separately on different datasets
(or different segments of the same dataset). I have not kept up as closely
with what folks use for fusing data from Z1, openSPIM, MuVi SPIM, or
bespoke multiview systems.

With enough memory, multiple fusions can also be performed on the same GPU,
albeit with sub-linear gains in performance. In the case of MVR, my
understanding is that GPU support may have become deprecated (at least I
haven't succeeded in running the CUDA libraries it depends on in several
years) and most users run it on CPU clusters now. I've been meaning to take
a stab at recompiling the CUDA libraries to see if they'll run as is on
10.1, but haven't found the time to do so.

Best,
Pavak

On Wed, Jan 15, 2020, 12:33 AM Francesco Pasqualini <
[hidden email]> wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hi Pavak et al,
> This is very interesting and helpful.
>
> Do I understand you correctly that multiple GPU won't help with multiview
> deconvolution?
>
> If there is a reference for deconvolution performances as a function of
> various hardware options can anyone point it out?
>
> Thanks
> Francesco
>
>
>
> On Wed, Jan 15, 2020, 12:57 AM PAVAK SHAH <[hidden email]> wrote:
>
> > *****
> > To join, leave or search the confocal microscopy listserv, go to:
> > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > Post images on http://www.imgur.com and include the link in your
> posting.
> > *****
> >
> > Hi Niyanta,
> >
> > What lightsheet instrument and what is the size of your typical dataset?
> Is
> > your processing pipeline primarily for segmentation or multiview
> > deconvolution? This will dictate, to a large extent, the kind of memory
> and
> > CPU you need.
> >
> > An RTX 2080ti or Titan RTX will be competitively performant but much
> > cheaper than a Quadro card, which offers no benefits for an image
> analysis
> > workflow unless you're running virtual machines. If your workflow is
> > primarily single view deconvolution, multiple GPUs can also be a huge
> boon
> > in terms of processing throughput, especially if individual volumes can
> fit
> > in 11 GB of VRAM since you can fill a large workstation to bursting with
> > 2080ti's for the price of 1 Quadro card. Even 2x Titan RTX can be had for
> > less than the price of 1x Quadro 6000.
> >
> > Depending on the specific piece of software, many will not scale
> > efficiently to >16 cores and a faster clocked CPU with fewer cores may be
> > advantageous. If you can benchmark it on a high core count system, that
> > should show whether your pipelines are able to keep that many cores fed
> > before running into algorithmic, disk access or memory throughput
> > bottlenecks.
> >
> > Best,
> > Pavak
> >
> > On Tue, Jan 14, 2020, 1:27 PM Niyanta Kumar <
> > [hidden email]> wrote:
> >
> > > *****
> > > To join, leave or search the confocal microscopy listserv, go to:
> > > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > > Post images on http://www.imgur.com and include the link in your
> > posting.
> > > *****
> > >
> > > Hi all,
> > > I plan to get a workstation that can handle confocal and lightsheet
> data
> > > analysis using Imaris and/or Arivis. Lightsheet data clearly has the
> > higher
> > > bar/needs in terms of specs. Below are the specs I am considering. Can
> > you
> > > please let me know if you have any recommendations?
> > >
> > > Lenovo Think Station P920
> > > 3.7 GHz CPUs (Intel) 48 cores
> > > Graphics: NVIDIA Quadro RTX 6000 or 8000
> > > 4 X 3 TB of local SSD (multiple drives - one for the OS, one for
> writing,
> > > one for reading)
> > > RAM: 500 GB
> > > Monitor: dual 1920 x 1200
> > > 300 TB NAS via 10 GB ethernet – file sharing
> > > Mouse: 3 button wheel
> > >
> > > Monitor: thinking of a dual Lenovo Think Vision P32u-10 but it doesn’t
> > > have a full array LED. Any suggestions? I hear the curved monitors can
> > > cause issues with display when sharing screens over Webex etc.
> > > Thanks,
> > > Niyanta
> > >
> >
>

[Francesco Pasqualini]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Pavak, Vincent Gary, et al - thanks for the clarification. Best, Francesco


On Wed, Jan 15, 2020 at 4:13 PM PAVAK SHAH <[hidden email]> wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hi Francesco,
>
> I should clarify that it depends a lot on the software pipeline being used.
> The ones I am most familiar with for diSPIM data (MIPAV, Shroff lab
> software, and Stephan Preibisch's Multiview Reconstruction) don't
> inherently support parallel processing of a single dataset on multiple
> GPU's but multiple instances can be run separately on different datasets
> (or different segments of the same dataset). I have not kept up as closely
> with what folks use for fusing data from Z1, openSPIM, MuVi SPIM, or
> bespoke multiview systems.
>
> With enough memory, multiple fusions can also be performed on the same GPU,
> albeit with sub-linear gains in performance. In the case of MVR, my
> understanding is that GPU support may have become deprecated (at least I
> haven't succeeded in running the CUDA libraries it depends on in several
> years) and most users run it on CPU clusters now. I've been meaning to take
> a stab at recompiling the CUDA libraries to see if they'll run as is on
> 10.1, but haven't found the time to do so.
>
> Best,
> Pavak
>
> On Wed, Jan 15, 2020, 12:33 AM Francesco Pasqualini <
> [hidden email]> wrote:
>
> > *****
> > To join, leave or search the confocal microscopy listserv, go to:
> > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > Post images on http://www.imgur.com and include the link in your
> posting.
> > *****
> >
> > Hi Pavak et al,
> > This is very interesting and helpful.
> >
> > Do I understand you correctly that multiple GPU won't help with multiview
> > deconvolution?
> >
> > If there is a reference for deconvolution performances as a function of
> > various hardware options can anyone point it out?
> >
> > Thanks
> > Francesco
> >
> >
> >
> > On Wed, Jan 15, 2020, 12:57 AM PAVAK SHAH <[hidden email]> wrote:
> >
> > > *****
> > > To join, leave or search the confocal microscopy listserv, go to:
> > > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > > Post images on http://www.imgur.com and include the link in your
> > posting.
> > > *****
> > >
> > > Hi Niyanta,
> > >
> > > What lightsheet instrument and what is the size of your typical
> dataset?
> > Is
> > > your processing pipeline primarily for segmentation or multiview
> > > deconvolution? This will dictate, to a large extent, the kind of memory
> > and
> > > CPU you need.
> > >
> > > An RTX 2080ti or Titan RTX will be competitively performant but much
> > > cheaper than a Quadro card, which offers no benefits for an image
> > analysis
> > > workflow unless you're running virtual machines. If your workflow is
> > > primarily single view deconvolution, multiple GPUs can also be a huge
> > boon
> > > in terms of processing throughput, especially if individual volumes can
> > fit
> > > in 11 GB of VRAM since you can fill a large workstation to bursting
> with
> > > 2080ti's for the price of 1 Quadro card. Even 2x Titan RTX can be had
> for
> > > less than the price of 1x Quadro 6000.
> > >
> > > Depending on the specific piece of software, many will not scale
> > > efficiently to >16 cores and a faster clocked CPU with fewer cores may
> be
> > > advantageous. If you can benchmark it on a high core count system, that
> > > should show whether your pipelines are able to keep that many cores fed
> > > before running into algorithmic, disk access or memory throughput
> > > bottlenecks.
> > >
> > > Best,
> > > Pavak
> > >
> > > On Tue, Jan 14, 2020, 1:27 PM Niyanta Kumar <
> > > [hidden email]> wrote:
> > >
> > > > *****
> > > > To join, leave or search the confocal microscopy listserv, go to:
> > > > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > > > Post images on http://www.imgur.com and include the link in your
> > > posting.
> > > > *****
> > > >
> > > > Hi all,
> > > > I plan to get a workstation that can handle confocal and lightsheet
> > data
> > > > analysis using Imaris and/or Arivis. Lightsheet data clearly has the
> > > higher
> > > > bar/needs in terms of specs. Below are the specs I am considering.
> Can
> > > you
> > > > please let me know if you have any recommendations?
> > > >
> > > > Lenovo Think Station P920
> > > > 3.7 GHz CPUs (Intel) 48 cores
> > > > Graphics: NVIDIA Quadro RTX 6000 or 8000
> > > > 4 X 3 TB of local SSD (multiple drives - one for the OS, one for
> > writing,
> > > > one for reading)
> > > > RAM: 500 GB
> > > > Monitor: dual 1920 x 1200
> > > > 300 TB NAS via 10 GB ethernet – file sharing
> > > > Mouse: 3 button wheel
> > > >
> > > > Monitor: thinking of a dual Lenovo Think Vision P32u-10 but it
> doesn’t
> > > > have a full array LED. Any suggestions? I hear the curved monitors
> can
> > > > cause issues with display when sharing screens over Webex etc.
> > > > Thanks,
> > > > Niyanta
> > > >
> > >
> >
>

--
Francesco S. Pasqualini
Visiting Professor University of Pavia
Associate Harvard University

tel: [phone] (IT)
tel: [phone] (USA)

[Niyanta Kumar]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Thanks Gary, Pavak, Vincent, and Francesco! I'm fairly new to the field so this is great. I was wondering if you'll could comment on your experience and related recommended PC specs needed when using Imaris or Arivis Vision 4D software in particular for lightsheet data analysis? I think Imaris only uses a single GPU. My typical data set is ~ 0.5 to 1 TB - labeled, cleared tumors/organs. A lot of the processing will involve fusion and analysis will be segmentation based. We are finalizing the purchase of a lightsheet instrument for inhouse use and currently generating data through a CRO (LifeCanvas) but will need to analyze data in house. I'm not working in a core setting in terms of throughput and my analysis station will handle a couple of data sets of the size mentioned over a month at a time. I think to your point I want to find a sweet spot between something to start with and what we can upgrade to over time if/as our needs increase. Any recommendations on monitors? I was thinking of a dual Lenovo Think Vision P32u-10. Does a full LED array help or is edge lit fine?

Best,
Niyanta

Niyanta Kumar, Ph.D.
Senior Scientist - Pharmacokinetics and ADME
Merck Research Laboratories, WP75B-2215
770 Sumneytown Pike, West Point, PA 19486
Phone: [phone]

-----Original Message-----
From: Confocal Microscopy List <[hidden email]> On Behalf Of Francesco Pasqualini
Sent: Wednesday, January 15, 2020 10:22 AM
To: [hidden email]
Subject: Re: Lightsheet imaging analysis Workstation Specs

EXTERNAL EMAIL – Use caution with any links or file attachments.

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Pavak, Vincent Gary, et al - thanks for the clarification. Best, Francesco


On Wed, Jan 15, 2020 at 4:13 PM PAVAK SHAH <[hidden email]> wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hi Francesco,
>
> I should clarify that it depends a lot on the software pipeline being used.
> The ones I am most familiar with for diSPIM data (MIPAV, Shroff lab
> software, and Stephan Preibisch's Multiview Reconstruction) don't
> inherently support parallel processing of a single dataset on multiple
> GPU's but multiple instances can be run separately on different
> datasets (or different segments of the same dataset). I have not kept
> up as closely with what folks use for fusing data from Z1, openSPIM,
> MuVi SPIM, or bespoke multiview systems.
>
> With enough memory, multiple fusions can also be performed on the same
> GPU, albeit with sub-linear gains in performance. In the case of MVR,
> my understanding is that GPU support may have become deprecated (at
> least I haven't succeeded in running the CUDA libraries it depends on
> in several
> years) and most users run it on CPU clusters now. I've been meaning to
> take a stab at recompiling the CUDA libraries to see if they'll run as
> is on 10.1, but haven't found the time to do so.
>
> Best,
> Pavak
>
> On Wed, Jan 15, 2020, 12:33 AM Francesco Pasqualini <
> [hidden email]> wrote:
>
> > *****
> > To join, leave or search the confocal microscopy listserv, go to:
> > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > Post images on http://www.imgur.com and include the link in your
> posting.
> > *****
> >
> > Hi Pavak et al,
> > This is very interesting and helpful.
> >
> > Do I understand you correctly that multiple GPU won't help with
> > multiview deconvolution?
> >
> > If there is a reference for deconvolution performances as a function
> > of various hardware options can anyone point it out?
> >
> > Thanks
> > Francesco
> >
> >
> >
> > On Wed, Jan 15, 2020, 12:57 AM PAVAK SHAH <[hidden email]> wrote:
> >
> > > *****
> > > To join, leave or search the confocal microscopy listserv, go to:
> > > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > > Post images on http://www.imgur.com and include the link in your
> > posting.
> > > *****
> > >
> > > Hi Niyanta,
> > >
> > > What lightsheet instrument and what is the size of your typical
> dataset?
> > Is
> > > your processing pipeline primarily for segmentation or multiview
> > > deconvolution? This will dictate, to a large extent, the kind of
> > > memory
> > and
> > > CPU you need.
> > >
> > > An RTX 2080ti or Titan RTX will be competitively performant but
> > > much cheaper than a Quadro card, which offers no benefits for an
> > > image
> > analysis
> > > workflow unless you're running virtual machines. If your workflow
> > > is primarily single view deconvolution, multiple GPUs can also be
> > > a huge
> > boon
> > > in terms of processing throughput, especially if individual
> > > volumes can
> > fit
> > > in 11 GB of VRAM since you can fill a large workstation to
> > > bursting
> with
> > > 2080ti's for the price of 1 Quadro card. Even 2x Titan RTX can be
> > > had
> for
> > > less than the price of 1x Quadro 6000.
> > >
> > > Depending on the specific piece of software, many will not scale
> > > efficiently to >16 cores and a faster clocked CPU with fewer cores
> > > may
> be
> > > advantageous. If you can benchmark it on a high core count system,
> > > that should show whether your pipelines are able to keep that many
> > > cores fed before running into algorithmic, disk access or memory
> > > throughput bottlenecks.
> > >
> > > Best,
> > > Pavak
> > >
> > > On Tue, Jan 14, 2020, 1:27 PM Niyanta Kumar <
> > > [hidden email]> wrote:
> > >
> > > > *****
> > > > To join, leave or search the confocal microscopy listserv, go to:
> > > > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > > > Post images on http://www.imgur.com and include the link in your
> > > posting.
> > > > *****
> > > >
> > > > Hi all,
> > > > I plan to get a workstation that can handle confocal and
> > > > lightsheet
> > data
> > > > analysis using Imaris and/or Arivis. Lightsheet data clearly has
> > > > the
> > > higher
> > > > bar/needs in terms of specs. Below are the specs I am considering.
> Can
> > > you
> > > > please let me know if you have any recommendations?
> > > >
> > > > Lenovo Think Station P920
> > > > 3.7 GHz CPUs (Intel) 48 cores
> > > > Graphics: NVIDIA Quadro RTX 6000 or 8000
> > > > 4 X 3 TB of local SSD (multiple drives - one for the OS, one for
> > writing,
> > > > one for reading)
> > > > RAM: 500 GB
> > > > Monitor: dual 1920 x 1200
> > > > 300 TB NAS via 10 GB ethernet – file sharing
> > > > Mouse: 3 button wheel
> > > >
> > > > Monitor: thinking of a dual Lenovo Think Vision P32u-10 but it
> doesn’t
> > > > have a full array LED. Any suggestions? I hear the curved
> > > > monitors
> can
> > > > cause issues with display when sharing screens over Webex etc.
> > > > Thanks,
> > > > Niyanta
> > > >
> > >
> >
>

--
Francesco S. Pasqualini
Visiting Professor University of Pavia
Associate Harvard University

tel: [phone] (IT)
tel: [phone] (USA)
Notice:  This e-mail message, together with any attachments, contains
information of Merck & Co., Inc. (2000 Galloping Hill Road, Kenilworth,
New Jersey, USA 07033), and/or its affiliates Direct contact information
for affiliates is available at
http://www.merck.com/contact/contacts.html) that may be confidential,
proprietary copyrighted and/or legally privileged. It is intended solely
for the use of the individual or entity named on this message. If you are
not the intended recipient, and have received this message in error,
please notify us immediately by reply e-mail and then delete it from
your system.

[Olaf Selchow]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hello everybody,

Happy to share a bit of my experience on this:

- Re the multiview deconvolution by Preibisch et al - with GPU enhancement: It works but my experience also is that it could be a bit easier to get the installation done. CUDA support is not depreciated though, to my knowledge.

- Typical data set size in what I have done in Light Sheet Microscopy in recent years is 500 GB to 10 TB both, in live imaging (long timer series with multiple views) and imaging of optically cleared specimen.

- Re the specs of a suitable machine, the following things I would consider (depends on how many people will use it, how many microscopes connected, installation location of the processing computer)

1. With Imaris and / or arivis vision4D to be run on the same machine as Fiji and maybe the manufacturers software (Zeiss ZEN, Leica LAS X, 3i Slidebook, etc) you want to run Windows, I guess.

2. If you want to be able to use it with multiple users in parallel you want/need Windows Server.

3. I would make sure that the microscopes are connected with direct 10 Gbit. Over an institutional network, even if 10 Gbit, you might be limited in bandwidth and so you can migrate the data only after acquisition from the acquisition PC to the Storage. This costs time, blocks the acquisition PC and essentially duplicates the required storage capacity you need. I always try to bring the microscopes with the processing & analysis computers in a 10 Gbit subnet that I can manage with a dedicated router/firewall/switch. That prevents interferences with  all the other traffic in your institute.

4. I would not use a NAS. Processing data by loading data into memory / software via network (and saving the results / temp data via the same line) can take ages. 10 Gbit is far too slow in my view. I’d use a prcessing machine with a strong RAID controler and a large RAID array directly attached to the processing unit. And I would save the data from the microscopes directly to this volume.

5. if you need super large RAM depends on how many people are supposed to work in parallel on this machine, if you want to use VMs, and what software you use (arivis needs much less RAM than others. Fiji benefits a lot from super large RAM, etc.). But if you choose the right motherboard and OS, you can always and easily upgrade RAM later.

6. CPU: generally, large multicore CPUs speed up things. But some software, even today, doesn’t make much use of parallelization. If you buy a very expensive dual-44-core CPU for thousands of $/€ you might end up with the software not using it. Actual check out the workflows. Some vendors might say „yes, our software uses all cores available“ , but in the end the processing function you are using most might still be running on a single or only 2 cores.

7. Monitor: I have worked with a number of 32 inch UHD 4K (3840 x 2160, 16:9) and 32 inch WQHD 3.6K (2560 x 1440, 16:9) monitors and never had a real problem. But thi smight depend on the GPU you use.

8. GPUs:
- if you want to use 3D renderings over remote access (e.g., RDP) sessions, I strongly recommend professional GPUs. I know, they are expensive. But the drivers on the Geforce or other gaming-grade GPUs can give you a hard time when working remotely. I have good experience with NVIDIA Quadro RTX boards (they are, in some GPU CUDA processing tasks, 2x faster than the previous P-seroes that is otherwise also perfectly fine.) For 3D viewing / rendering / analyzing data, make sure the VRAM on the GPU is 11 GB / 16 GB or larger.
- for some software and for VMs, it makes sense to think of the option of multiple GPUs. Maybe you just want to make sure you can fit a 2nd or 3rd later on. In SVI Huygens you can, for example, assign instances of DCV to certain GPUs - so multiple GPUs can speed up your work. You need to buy the respective licenses from SVI though.

9. Storage volume: make sure you have multiple (2 or more) fast volumes (RAIDs of HDD or SSDs) to have space for the software where it can save temp data - on an independent volume. Multiple simultaneous read/write processes can slow down even fast RAID configs. Also keep in mind that SSDs are more convenient and faster, but still more expensive and still have a higher failure rate. Mak sure you consider hard drives 7 ssds as consumables. In a RAID of 15 HDDs, it is perfectly normal that you have 1 HDD/year in average that fails and needs replacement. SSDs maybe even more often.

The Lenovo Think Station P920 is certainly a great hardware. You’ll still have to invest a bit of time and money to get it ready to work for your applications. Networking i, etc.

I would also point out that there is commercial options that provide you a turnkey solution with support that can scale / grow with your needs.
I have worked with ACQUIFER HIVEs a lot. Check with them or a similar provider if your budget allows a high end solution for 60k (+) $/€ and if you are looking for a solution provider that saves you from configuring network adapters yourself …
Note: I used to have consultancy projects with ACQUIFER and might have more in the future. So I am a bit biased, but mostly because I think they have great hardware and services and I worked wit them to position their products and improve them. I do not (!) have a direct financial interest in them selling a platform unit to you!

I hope this helps.
best regards,
Olaf

———
Dr. Olaf Selchow
--
Microscopy & BioImaging Consulting
Image Processing & Large Data Handling
--
[hidden email]

[Francesco Pasqualini]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Hi, quick follow up: Has anyone built a workstation for microscopy analysis
using AMD hardware? Their price/thread is way better than Intel's (both
customer- and server-grade CPUs) and the only downside seems to be that
they run a little hotter. Is there limited support in the various software
packages? Or is it just a matter of legacy infrastructure? Thanks, Francesco


On Thu, Jan 16, 2020 at 9:50 AM Olaf Selchow <[hidden email]> wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hello everybody,
>
> Happy to share a bit of my experience on this:
>
> - Re the multiview deconvolution by Preibisch et al - with GPU
> enhancement: It works but my experience also is that it could be a bit
> easier to get the installation done. CUDA support is not depreciated
> though, to my knowledge.
>
> - Typical data set size in what I have done in Light Sheet Microscopy in
> recent years is 500 GB to 10 TB both, in live imaging (long timer series
> with multiple views) and imaging of optically cleared specimen.
>
> - Re the specs of a suitable machine, the following things I would
> consider (depends on how many people will use it, how many microscopes
> connected, installation location of the processing computer)
>
> 1. With Imaris and / or arivis vision4D to be run on the same machine as
> Fiji and maybe the manufacturers software (Zeiss ZEN, Leica LAS X, 3i
> Slidebook, etc) you want to run Windows, I guess.
>
> 2. If you want to be able to use it with multiple users in parallel you
> want/need Windows Server.
>
> 3. I would make sure that the microscopes are connected with direct 10
> Gbit. Over an institutional network, even if 10 Gbit, you might be limited
> in bandwidth and so you can migrate the data only after acquisition from
> the acquisition PC to the Storage. This costs time, blocks the acquisition
> PC and essentially duplicates the required storage capacity you need. I
> always try to bring the microscopes with the processing & analysis
> computers in a 10 Gbit subnet that I can manage with a dedicated
> router/firewall/switch. That prevents interferences with  all the other
> traffic in your institute.
>
> 4. I would not use a NAS. Processing data by loading data into memory /
> software via network (and saving the results / temp data via the same line)
> can take ages. 10 Gbit is far too slow in my view. I’d use a prcessing
> machine with a strong RAID controler and a large RAID array directly
> attached to the processing unit. And I would save the data from the
> microscopes directly to this volume.
>
> 5. if you need super large RAM depends on how many people are supposed to
> work in parallel on this machine, if you want to use VMs, and what software
> you use (arivis needs much less RAM than others. Fiji benefits a lot from
> super large RAM, etc.). But if you choose the right motherboard and OS, you
> can always and easily upgrade RAM later.
>
> 6. CPU: generally, large multicore CPUs speed up things. But some
> software, even today, doesn’t make much use of parallelization. If you buy
> a very expensive dual-44-core CPU for thousands of $/€ you might end up
> with the software not using it. Actual check out the workflows. Some
> vendors might say „yes, our software uses all cores available“ , but in the
> end the processing function you are using most might still be running on a
> single or only 2 cores.
>
> 7. Monitor: I have worked with a number of 32 inch      UHD 4K (3840 x
> 2160, 16:9) and 32 inch WQHD 3.6K (2560 x 1440, 16:9) monitors and never
> had a real problem. But thi smight depend on the GPU you use.
>
> 8. GPUs:
> - if you want to use 3D renderings over remote access (e.g., RDP)
> sessions, I strongly recommend professional GPUs. I know, they are
> expensive. But the drivers on the Geforce or other gaming-grade GPUs can
> give you a hard time when working remotely. I have good experience with
> NVIDIA Quadro RTX boards (they are, in some GPU CUDA processing tasks, 2x
> faster than the previous P-seroes that is otherwise also perfectly fine.)
> For 3D viewing / rendering / analyzing data, make sure the VRAM on the GPU
> is 11 GB / 16 GB or larger.
> - for some software and for VMs, it makes sense to think of the option of
> multiple GPUs. Maybe you just want to make sure you can fit a 2nd or 3rd
> later on. In SVI Huygens you can, for example, assign instances of DCV to
> certain GPUs - so multiple GPUs can speed up your work. You need to buy the
> respective licenses from SVI though.
>
> 9. Storage volume: make sure you have multiple (2 or more) fast volumes
> (RAIDs of HDD or SSDs) to have space for the software where it can save
> temp data - on an independent volume. Multiple simultaneous read/write
> processes can slow down even fast RAID configs. Also keep in mind that SSDs
> are more convenient and faster, but still more expensive and still have a
> higher failure rate. Mak sure you consider hard drives 7 ssds as
> consumables. In a RAID of 15 HDDs, it is perfectly normal that you have 1
> HDD/year in average that fails and needs replacement. SSDs maybe even more
> often.
>
> The Lenovo Think Station P920 is certainly a great hardware. You’ll still
> have to invest a bit of time and money to get it ready to work for your
> applications. Networking i, etc.
>
> I would also point out that there is commercial options that provide you a
> turnkey solution with support that can scale / grow with your needs.
> I have worked with ACQUIFER HIVEs a lot. Check with them or a similar
> provider if your budget allows a high end solution for 60k (+) $/€ and if
> you are looking for a solution provider that saves you from configuring
> network adapters yourself …
> Note: I used to have consultancy projects with ACQUIFER and might have
> more in the future. So I am a bit biased, but mostly because I think they
> have great hardware and services and I worked wit them to position their
> products and improve them. I do not (!) have a direct financial interest in
> them selling a platform unit to you!
>
> I hope this helps.
> best regards,
> Olaf
>
> ———
> Dr. Olaf Selchow
> --
> Microscopy & BioImaging Consulting
> Image Processing & Large Data Handling
> --
> [hidden email]
>

--
Francesco S. Pasqualini
Visiting Professor University of Pavia
Associate Harvard University

tel: [phone] (IT)
tel: [phone] (USA)

[PAVAK SHAH]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

On paper AMD is now a clear winner in terms of performance, except in
highly vectorized workloads that can use AVX-2 / AVX-512 instructions on
Intel or software that was developed using Intel's MKL library for linear
algebra (MATLAB is a prominent example of this). As a consequence, many of
these applications run significantly slower on AMD hardware, or run less
stably. It's a sad state of affairs, but I would generally advise that you
stick with Intel platforms for now. Hopefully the future will bring better
software support for the "red team".

Best,
Pavak

On Mon, Jan 20, 2020, 2:49 AM Francesco Pasqualini <
[hidden email]> wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hi, quick follow up: Has anyone built a workstation for microscopy analysis
> using AMD hardware? Their price/thread is way better than Intel's (both
> customer- and server-grade CPUs) and the only downside seems to be that
> they run a little hotter. Is there limited support in the various software
> packages? Or is it just a matter of legacy infrastructure? Thanks,
> Francesco
>
>
> On Thu, Jan 16, 2020 at 9:50 AM Olaf Selchow <[hidden email]>
> wrote:
>
> > *****
> > To join, leave or search the confocal microscopy listserv, go to:
> > http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> > Post images on http://www.imgur.com and include the link in your
> posting.
> > *****
> >
> > Hello everybody,
> >
> > Happy to share a bit of my experience on this:
> >
> > - Re the multiview deconvolution by Preibisch et al - with GPU
> > enhancement: It works but my experience also is that it could be a bit
> > easier to get the installation done. CUDA support is not depreciated
> > though, to my knowledge.
> >
> > - Typical data set size in what I have done in Light Sheet Microscopy in
> > recent years is 500 GB to 10 TB both, in live imaging (long timer series
> > with multiple views) and imaging of optically cleared specimen.
> >
> > - Re the specs of a suitable machine, the following things I would
> > consider (depends on how many people will use it, how many microscopes
> > connected, installation location of the processing computer)
> >
> > 1. With Imaris and / or arivis vision4D to be run on the same machine as
> > Fiji and maybe the manufacturers software (Zeiss ZEN, Leica LAS X, 3i
> > Slidebook, etc) you want to run Windows, I guess.
> >
> > 2. If you want to be able to use it with multiple users in parallel you
> > want/need Windows Server.
> >
> > 3. I would make sure that the microscopes are connected with direct 10
> > Gbit. Over an institutional network, even if 10 Gbit, you might be
> limited
> > in bandwidth and so you can migrate the data only after acquisition from
> > the acquisition PC to the Storage. This costs time, blocks the
> acquisition
> > PC and essentially duplicates the required storage capacity you need. I
> > always try to bring the microscopes with the processing & analysis
> > computers in a 10 Gbit subnet that I can manage with a dedicated
> > router/firewall/switch. That prevents interferences with  all the other
> > traffic in your institute.
> >
> > 4. I would not use a NAS. Processing data by loading data into memory /
> > software via network (and saving the results / temp data via the same
> line)
> > can take ages. 10 Gbit is far too slow in my view. I’d use a prcessing
> > machine with a strong RAID controler and a large RAID array directly
> > attached to the processing unit. And I would save the data from the
> > microscopes directly to this volume.
> >
> > 5. if you need super large RAM depends on how many people are supposed to
> > work in parallel on this machine, if you want to use VMs, and what
> software
> > you use (arivis needs much less RAM than others. Fiji benefits a lot from
> > super large RAM, etc.). But if you choose the right motherboard and OS,
> you
> > can always and easily upgrade RAM later.
> >
> > 6. CPU: generally, large multicore CPUs speed up things. But some
> > software, even today, doesn’t make much use of parallelization. If you
> buy
> > a very expensive dual-44-core CPU for thousands of $/€ you might end up
> > with the software not using it. Actual check out the workflows. Some
> > vendors might say „yes, our software uses all cores available“ , but in
> the
> > end the processing function you are using most might still be running on
> a
> > single or only 2 cores.
> >
> > 7. Monitor: I have worked with a number of 32 inch      UHD 4K (3840 x
> > 2160, 16:9) and 32 inch WQHD 3.6K (2560 x 1440, 16:9) monitors and never
> > had a real problem. But thi smight depend on the GPU you use.
> >
> > 8. GPUs:
> > - if you want to use 3D renderings over remote access (e.g., RDP)
> > sessions, I strongly recommend professional GPUs. I know, they are
> > expensive. But the drivers on the Geforce or other gaming-grade GPUs can
> > give you a hard time when working remotely. I have good experience with
> > NVIDIA Quadro RTX boards (they are, in some GPU CUDA processing tasks, 2x
> > faster than the previous P-seroes that is otherwise also perfectly fine.)
> > For 3D viewing / rendering / analyzing data, make sure the VRAM on the
> GPU
> > is 11 GB / 16 GB or larger.
> > - for some software and for VMs, it makes sense to think of the option of
> > multiple GPUs. Maybe you just want to make sure you can fit a 2nd or 3rd
> > later on. In SVI Huygens you can, for example, assign instances of DCV to
> > certain GPUs - so multiple GPUs can speed up your work. You need to buy
> the
> > respective licenses from SVI though.
> >
> > 9. Storage volume: make sure you have multiple (2 or more) fast volumes
> > (RAIDs of HDD or SSDs) to have space for the software where it can save
> > temp data - on an independent volume. Multiple simultaneous read/write
> > processes can slow down even fast RAID configs. Also keep in mind that
> SSDs
> > are more convenient and faster, but still more expensive and still have a
> > higher failure rate. Mak sure you consider hard drives 7 ssds as
> > consumables. In a RAID of 15 HDDs, it is perfectly normal that you have 1
> > HDD/year in average that fails and needs replacement. SSDs maybe even
> more
> > often.
> >
> > The Lenovo Think Station P920 is certainly a great hardware. You’ll still
> > have to invest a bit of time and money to get it ready to work for your
> > applications. Networking i, etc.
> >
> > I would also point out that there is commercial options that provide you
> a
> > turnkey solution with support that can scale / grow with your needs.
> > I have worked with ACQUIFER HIVEs a lot. Check with them or a similar
> > provider if your budget allows a high end solution for 60k (+) $/€ and if
> > you are looking for a solution provider that saves you from configuring
> > network adapters yourself …
> > Note: I used to have consultancy projects with ACQUIFER and might have
> > more in the future. So I am a bit biased, but mostly because I think they
> > have great hardware and services and I worked wit them to position their
> > products and improve them. I do not (!) have a direct financial interest
> in
> > them selling a platform unit to you!
> >
> > I hope this helps.
> > best regards,
> > Olaf
> >
> > ———
> > Dr. Olaf Selchow
> > --
> > Microscopy & BioImaging Consulting
> > Image Processing & Large Data Handling
> > --
> > [hidden email]
> >
>
>
> --
> Francesco S. Pasqualini
> Visiting Professor University of Pavia
> Associate Harvard University
>
> tel: [phone] (IT)
> tel: [phone] (USA)
>

[George McNamara]

*****
To join, leave or search the confocal microscopy listserv, go to:
http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
Post images on http://www.imgur.com and include the link in your posting.
*****

Yes, my current office PC (mostly MetaMorph) is a nice AMD workstation
(~$1200, no including GPU or SSDs), 10Gbe Ethernet, 64 Gb ram, SATA RAID
array (and maybe ASUS Hyper M.2 card - holiday today, writing at home
without details or memory).

works great. Buy a high wattage power supply and 'appropriate' heat sink(s).

I also suggest: PCIe gen4 motherboard with lots of x16 slots, for
acquisition workstations (Intel does not currently support 'gen4', so
AMD CPU exclusive).

happy 2020,

George

p.s two lab users: nice AMD CPU PC desktops for $600 each. Both users
very happy with these, one does some image analysis [Metamorph, Fiji
ImageJ, review FV3000RS data], happy with it (re-used current monitors).


On 1/20/2020 5:48 AM, Francesco Pasqualini wrote:

> *****
> To join, leave or search the confocal microscopy listserv, go to:
> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
> Post images on http://www.imgur.com and include the link in your posting.
> *****
>
> Hi, quick follow up: Has anyone built a workstation for microscopy analysis
> using AMD hardware? Their price/thread is way better than Intel's (both
> customer- and server-grade CPUs) and the only downside seems to be that
> they run a little hotter. Is there limited support in the various software
> packages? Or is it just a matter of legacy infrastructure? Thanks, Francesco
>
>
> On Thu, Jan 16, 2020 at 9:50 AM Olaf Selchow <[hidden email]> wrote:
>
>> *****
>> To join, leave or search the confocal microscopy listserv, go to:
>> http://lists.umn.edu/cgi-bin/wa?A0=confocalmicroscopy
>> Post images on http://www.imgur.com and include the link in your posting.
>> *****
>>
>> Hello everybody,
>>
>> Happy to share a bit of my experience on this:
>>
>> - Re the multiview deconvolution by Preibisch et al - with GPU
>> enhancement: It works but my experience also is that it could be a bit
>> easier to get the installation done. CUDA support is not depreciated
>> though, to my knowledge.
>>
>> - Typical data set size in what I have done in Light Sheet Microscopy in
>> recent years is 500 GB to 10 TB both, in live imaging (long timer series
>> with multiple views) and imaging of optically cleared specimen.
>>
>> - Re the specs of a suitable machine, the following things I would
>> consider (depends on how many people will use it, how many microscopes
>> connected, installation location of the processing computer)
>>
>> 1. With Imaris and / or arivis vision4D to be run on the same machine as
>> Fiji and maybe the manufacturers software (Zeiss ZEN, Leica LAS X, 3i
>> Slidebook, etc) you want to run Windows, I guess.
>>
>> 2. If you want to be able to use it with multiple users in parallel you
>> want/need Windows Server.
>>
>> 3. I would make sure that the microscopes are connected with direct 10
>> Gbit. Over an institutional network, even if 10 Gbit, you might be limited
>> in bandwidth and so you can migrate the data only after acquisition from
>> the acquisition PC to the Storage. This costs time, blocks the acquisition
>> PC and essentially duplicates the required storage capacity you need. I
>> always try to bring the microscopes with the processing & analysis
>> computers in a 10 Gbit subnet that I can manage with a dedicated
>> router/firewall/switch. That prevents interferences with  all the other
>> traffic in your institute.
>>
>> 4. I would not use a NAS. Processing data by loading data into memory /
>> software via network (and saving the results / temp data via the same line)
>> can take ages. 10 Gbit is far too slow in my view. I’d use a prcessing
>> machine with a strong RAID controler and a large RAID array directly
>> attached to the processing unit. And I would save the data from the
>> microscopes directly to this volume.
>>
>> 5. if you need super large RAM depends on how many people are supposed to
>> work in parallel on this machine, if you want to use VMs, and what software
>> you use (arivis needs much less RAM than others. Fiji benefits a lot from
>> super large RAM, etc.). But if you choose the right motherboard and OS, you
>> can always and easily upgrade RAM later.
>>
>> 6. CPU: generally, large multicore CPUs speed up things. But some
>> software, even today, doesn’t make much use of parallelization. If you buy
>> a very expensive dual-44-core CPU for thousands of $/€ you might end up
>> with the software not using it. Actual check out the workflows. Some
>> vendors might say „yes, our software uses all cores available“ , but in the
>> end the processing function you are using most might still be running on a
>> single or only 2 cores.
>>
>> 7. Monitor: I have worked with a number of 32 inch      UHD 4K (3840 x
>> 2160, 16:9) and 32 inch WQHD 3.6K (2560 x 1440, 16:9) monitors and never
>> had a real problem. But thi smight depend on the GPU you use.
>>
>> 8. GPUs:
>> - if you want to use 3D renderings over remote access (e.g., RDP)
>> sessions, I strongly recommend professional GPUs. I know, they are
>> expensive. But the drivers on the Geforce or other gaming-grade GPUs can
>> give you a hard time when working remotely. I have good experience with
>> NVIDIA Quadro RTX boards (they are, in some GPU CUDA processing tasks, 2x
>> faster than the previous P-seroes that is otherwise also perfectly fine.)
>> For 3D viewing / rendering / analyzing data, make sure the VRAM on the GPU
>> is 11 GB / 16 GB or larger.
>> - for some software and for VMs, it makes sense to think of the option of
>> multiple GPUs. Maybe you just want to make sure you can fit a 2nd or 3rd
>> later on. In SVI Huygens you can, for example, assign instances of DCV to
>> certain GPUs - so multiple GPUs can speed up your work. You need to buy the
>> respective licenses from SVI though.
>>
>> 9. Storage volume: make sure you have multiple (2 or more) fast volumes
>> (RAIDs of HDD or SSDs) to have space for the software where it can save
>> temp data - on an independent volume. Multiple simultaneous read/write
>> processes can slow down even fast RAID configs. Also keep in mind that SSDs
>> are more convenient and faster, but still more expensive and still have a
>> higher failure rate. Mak sure you consider hard drives 7 ssds as
>> consumables. In a RAID of 15 HDDs, it is perfectly normal that you have 1
>> HDD/year in average that fails and needs replacement. SSDs maybe even more
>> often.
>>
>> The Lenovo Think Station P920 is certainly a great hardware. You’ll still
>> have to invest a bit of time and money to get it ready to work for your
>> applications. Networking i, etc.
>>
>> I would also point out that there is commercial options that provide you a
>> turnkey solution with support that can scale / grow with your needs.
>> I have worked with ACQUIFER HIVEs a lot. Check with them or a similar
>> provider if your budget allows a high end solution for 60k (+) $/€ and if
>> you are looking for a solution provider that saves you from configuring
>> network adapters yourself …
>> Note: I used to have consultancy projects with ACQUIFER and might have
>> more in the future. So I am a bit biased, but mostly because I think they
>> have great hardware and services and I worked wit them to position their
>> products and improve them. I do not (!) have a direct financial interest in
>> them selling a platform unit to you!
>>
>> I hope this helps.
>> best regards,
>> Olaf
>>
>> ———
>> Dr. Olaf Selchow
>> --
>> Microscopy & BioImaging Consulting
>> Image Processing & Large Data Handling
>> --
>> [hidden email]
>>
>