FW: IT questions
Locked
|
We have on all our equipment a small batch script that
automatically runs on start up and backs up data. It compares files and only
backs up new and it is one way to a 4Tb NAS drive (these are nice and cheap!).
The script runs on start up because when windows shuts down it seems to
disconnect the network connection prior to running shutdown scripts. We have
never taken the time to find a workaround. Jonathan Professor J.C.
Knowles BSc(hons), PhD, FIMMM, CEng, FRSC, CSci |
|
|
We are in the process of buying a web server for data storage for the microscope facilities. Till now we save data on the HD and this can stay on the system till data is analysed. Unfortunately, that is the ideal situation and often we find at the end of an imaging session that the HD is full. Several groups use now external HD for storage. However, to make it all better organized we have decided to get a web server. Our computer centre suggests:
Sun x4500 AMD Opteron (290) 2.80 GHz, 16 GB DDR1, 400MHz, 24 TB (500GB SATAs) with the budged we have. However, they also suggest that we buy 2 with one for backup. We were hoping to use OMERO for indexing of our data but at the moment that does not work. OMERO will convert the data to OME-TIFF, so you have double the amount of data. Other option would be to upload only to OMERO but it results in only OME-TIFF files which often cannot be read into our analysis software. I understand by just using the server as an extra HD the system does not know what data has been changed or moved and so what needs to be backed-up. As a result full back-ups have to be run. So two questions: 1) Has somebody experience with this Sun server? 2) Has somebody solved the problem with uploading files to the server and which files have to be backed-up? Thanks Kees Dr. K.R. Straatman Senior Experimental Officer School of Biological Sciences http://www.le.ac.uk/biochem/microscopy/home.html Postal address: Department of Biochemistry Henry Wellcome Building University of Leicester Lancaster Rd. Leicester LE1 9HN UK tel.: + 44 (0)116 229 7085/252 2263 fax: + 44 (0)116 229 7031 Times Higher Education University of the Year 2008-9 From: Confocal Microscopy List [mailto:[hidden email]] On Behalf Of [hidden email] Sent: 12 July 2009 10:38 To: [hidden email] Subject: FW: IT questions We have on all our equipment a small batch script that automatically runs on start up and backs up data. It compares files and only backs up new and it is one way to a 4Tb NAS drive (these are nice and cheap!). The script runs on start up because when windows shuts down it seems to disconnect the network connection prior to running shutdown scripts. We have never taken the time to find a workaround. Jonathan Professor J.C. Knowles BSc(hons), PhD, FIMMM, CEng, FRSC, CSci Professor of Biomaterials Science Head of Division of Biomaterials and Tissue Engineering, UCL Eastman Dental Institute, University College London, 256 Gray's Inn Road, London WC1X 8LD. UK. Tel +44 (0)207 915 1189 Fax +44 (0)207 915 1227 Mobile 07785 313615 Skype Me Editor, Journal of Biomaterials Applications Email [hidden email] Eastman website; http://www.eastman.ucl.ac.uk Personal website; http://www.homepages.ucl.ac.uk/~sfhvjck This email represents the views of the sender alone and must not be construed as representing the views of the Eastman Dental Institute. It may contain confidential information and may be protected by law as a legally privileged document and copyright work. Its content should not be disclosed and it should not be given or copied to anyone other than the person(s) named or referenced above. If you have received this email in error, please contact the sender. |
|
|
Hi,
We use the Biotrue CDMS data management software on our server. This allows simple upload and download and setting of permissions all in a browser based presentation. This means it is OS independent and also requires no instruction for end users on how to use it. I've tried the OMERO server and agree with you about the format issue. I understand why they are pushing the OME-TIFF format but I think user choice should be maintained and additional file standards supported on the server. The OMERO server software also seems to be more focused on server based image analysis than a customisable data storage system. It's important to remember though that it is only a beta. Regarding your questions - I don't have any specific experience about the server you mention but it should just work like any other. I would install linux to make things easy and definitely by two servers at the same time - backups need to start straight away. For the backups, although we use Biotrue for archiving on the server, we still back up the data on the hard drives in the same way you will need to do. It is simple to do this using the rsync command in linux which looks for differences between the backup and the original and copies over the data. I have used this with some scripts to generate daily snapshot backups, weekly backups and a monthly data archive which all run at night when the server is quiet. The clever trick is to use hard links which means that you store the whole data set only once and effectively list which of those files are contained in each snapshot. This allows the 11 "copies" I have on the backup server to only occupy the total space of 1 copy. There is more info here: http://www.mikerubel.org/computers/rsync_snapshots/ and if you want a copy of my scripts just get in touch. Hope that is of some help. Colin -- Dr Colin Rickman Centre for Integrative Physiology School of Biomedical Sciences University of Edinburgh Hugh Robson Building George Square Edinburgh EH8 9XD Tel: +44 131 6511512 Fax: +44 131 6503128 Straatman, Dr K.R. wrote: > We are in the process of buying a web server for data storage for the microscope facilities. Till now we save data on the HD and this can stay on the system till data is analysed. Unfortunately, that is the ideal situation and often we find at the end of an imaging session that the HD is full. Several groups use now external HD for storage. However, to make it all better organized we have decided to get a web server. Our computer centre suggests: > Sun x4500 AMD Opteron (290) 2.80 GHz, 16 GB DDR1, 400MHz, 24 TB (500GB SATAs) with the budged we have. However, they also suggest that we buy 2 with one for backup. > We were hoping to use OMERO for indexing of our data but at the moment that does not work. OMERO will convert the data to OME-TIFF, so you have double the amount of data. Other option would be to upload only to OMERO but it results in only OME-TIFF files which often cannot be read into our analysis software. I understand by just using the server as an extra HD the system does not know what data has been changed or moved and so what needs to be backed-up. As a result full back-ups have to be run. > > So two questions: > 1) Has somebody experience with this Sun server? > 2) Has somebody solved the problem with uploading files to the server and which files have to be backed-up? > > Thanks > > Kees > > > Dr. K.R. Straatman > Senior Experimental Officer > School of Biological Sciences > http://www.le.ac.uk/biochem/microscopy/home.html > > Postal address: > Department of Biochemistry > Henry Wellcome Building > University of Leicester > Lancaster Rd. > Leicester LE1 9HN > UK > tel.: + 44 (0)116 229 7085/252 2263 > fax: + 44 (0)116 229 7031 > > Times Higher Education University of the Year 2008-9 > > > > From: Confocal Microscopy List [mailto:[hidden email]] On Behalf Of [hidden email] > Sent: 12 July 2009 10:38 > To: [hidden email] > Subject: FW: IT questions > > > > We have on all our equipment a small batch script that automatically runs on start up and backs up data. It compares files and only backs up new and it is one way to a 4Tb NAS drive (these are nice and cheap!). The script runs on start up because when windows shuts down it seems to disconnect the network connection prior to running shutdown scripts. We have never taken the time to find a workaround. > > Jonathan > > > Professor J.C. Knowles BSc(hons), PhD, FIMMM, CEng, FRSC, CSci > Professor of Biomaterials Science > Head of Division of Biomaterials and Tissue Engineering, > UCL Eastman Dental Institute, > University College London, > 256 Gray's Inn Road, > London WC1X 8LD. UK. > Tel +44 (0)207 915 1189 > Fax +44 (0)207 915 1227 > Mobile 07785 313615 > Skype Me > Editor, Journal of Biomaterials Applications > Email [hidden email] > Eastman website; http://www.eastman.ucl.ac.uk > Personal website; http://www.homepages.ucl.ac.uk/~sfhvjck > This email represents the views of the sender alone and must not be construed as representing the views of the Eastman Dental Institute. It may contain confidential information and may be protected by law as a legally privileged document and copyright work. Its content should not be disclosed and it should not be given or copied to anyone other than the person(s) named or referenced above. If you have received this email in error, please contact the sender. > > -- Dr Colin Rickman Centre for Integrative Physiology School of Biomedical Sciences University of Edinburgh Hugh Robson Building George Square Edinburgh EH8 9XD Tel: +44 131 6511512 Fax: +44 131 6503128 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. |
«
Return to Confocal Microscopy List
|
1 view|%1 views
| Free forum by Nabble | Edit this page |