Opportunistic cross-platform distributed computing January 9, 2008Posted by gordonwatts in computers.
There is room for two types of distributed computing in particle physics. The first one is boring – large clusters of computers run by laboratories or large universities with 1000’s of CPUs. These are essentially very big batch platforms. There are lots of unsolved problems – for example, how do get the data in and out.
The second type is much smaller. Say a small set of machines you are using to help analyze your 4 TB of ntuple-data for your analysis. While the batch farms are bound to be run by compute professionals, these small clusters are bound to be run by, well, us.
This got me to wondering — in the university environment – why aren’t we using some cross platform tool for running our analysis? For example, at UW we have large collections of Windows machines freely available to us (well, not freely) – the undergraduate lab computers. We also have a small number of dedicated Linux machines. ROOT does hold us back — it is difficult to get the same code running on more than one platform.
I stumbled on this article while trying to track down some other information on distributed computing. It describes using Microsoft’s virtual machine (MS’s answer to Java). Which got me to thinking — why haven’t we seen anything like this in Java? Wikipedia has a few references to projects in Java – but nothing that has crossed over to physics. Is it ROOT that is holding us back? Or something else?
[Update: ROOT developer wrote in to ask if it was ROOT or C++ that was holding things back — yes, C++, which is of course what ROOT is based on. It should also be noted that we can only do a lot of what is possible now because of ROOT.]