It’s the UI that counts (technical) April 24, 2009
Posted by gordonwatts in CERNVM, computers.add a comment
In my last post I mentioned that I was using CERNVM. In order to use that I need a virtual machine program of some sort. I’m on windows (Windows 7 at the moment), and so the three main ones that are available to me (and free!) are Virtual PC, VMWare, and VirtualBox. I’ve talked about VirtualBox before. My opinion hasn’t changed about the thing.
But now I’ve done some speed measurements. For running and building software in the ATLAS environment, it seems like it is about 10% slower than Virtual PC (Microsoft’s edition). The networking seems to be about the same – at least as far as I can tell. So the question becomes: choose UI over functionality? For me the 10% hit isn’t big enough, so I’ll be sticking with it. But I thought it would be interesting to comment on this.
BTW, this was the 2.1.4 version of VB. The 2.2.0 version, which is the latest and sounds like it might be a bit more efficient, does not play with with Windows 7. Much as VMWare took over all my USB devices and wouldn’t give them back, VirtualBox 2.2 takes over my network connection and forces me to basically shut down and restart the networking stack in order to get them back. Hopefully there will be some bug fixes soon!
But something else came out of this. If you are planning on running virtual machines – run, do not walk, and buy/build a machine build on Intel’s Core i7 architecture (as I did). That home machine blew the doors off anything else I had around. x2 faster in many cases. When doing straight single-threaded calculations it was only 30%-40% faster than my other machines, so there must be some hardware improvements for virtualization in the hardware. Both Virtual PC and VirtualBox were able to take advantage of whatever improvements were there (err) out of the box.
Take What You Need April 22, 2009
Posted by gordonwatts in ATLAS, CERNVM, computers.add a comment
I don’t know where this idea came from, but it was brilliant – CERNVM. It solves two big problems, all at once. And in such a elegant way.
Here is the basic problem: you want to build and run ATLAS (or CMS, ALICE, LHCb, etc.) software on your local university machine or laptop. This is painful for two reasons. The first reason is that it is not likely you are running the proper kind of Linux. Scientific Linux was designed to run science and particle physics code, not run email or have a nice gui (i.e. it isn’t a Mac, or Windows, or ubuntu). There are ways around this, of course – start up a virtual machine and install scientific linux on it, etc. But then you hit the second problem: you have to install the ATLAS software. I’m not sure about the other experiments, but for ATLAS this is a 6 gig affair. And, each time a new version of the release comes out you have to install it all over again. In a virtual machine this can be painful (take hours of your time).
This is where CERNVM comes in. The first realization was that when you compile, build, and test-run, you need only about 10% of that 6 gigs of software. Of course, everyone needs a different 10%, but in general, it is only a small fraction of the release that is needed. So why download everything? The second realization was: automatically install each release or file only when it was needed – and automatically add each release. This second bit means that I, as an end user, never have to install another ATLAS release. Ever! How sweet is that!?!? I can just use it. As soon as a new release is out, and CERN publishes it, then I can access it!
CERNVM is a virtual machine. For folks that have been around CERN a while, they will know that CERNVM refers to the venerable IBM system that CERN used to have as one of its mainframes. According to Predrag Buncic, so I believe is the lead on the modern CERNVM project, that choice of names was on purpose. See Predrag’s talk slides from CHEP for some more details.
CERNVM accomplishes its magic with a FUSE file system. This brilliant open source project creates a virtual file system. Whenever you try to access a file in it, FUSE hands the request off to some user-written code. In CERNVM’s case, this code looks for the file up on a master server at CERN, and then downloads it locally. Once it has been cached locally, the file is accessible like any other file. So, CERN can publish a whole ATLAS release on their master servers, and then when I try to setup that release, CERNVM will automatically bring down exactly the files that I need to get my work done. Better yet, if I’ve already got the files locally, then I can hope on an airplane and everything will still work! Not to shabby!
The virtual machine part of this gets around the wrong OS type – it is based on Scientific Linux. Thus, the two main problems with running local build code are solved! Very nice, eh!?
Life as a Physicist 