jump to navigation

Take What You Need April 22, 2009

Posted by gordonwatts in ATLAS, CERNVM, computers.

cernvm I don’t know where this idea came from, but it was brilliant – CERNVM. It solves two big problems, all at once. And in such a elegant way.

Here is the basic problem: you want to build and run ATLAS (or CMS, ALICE, LHCb, etc.) software on your local university machine or laptop. This is painful for two reasons. The first reason is that it is not likely you are running the proper kind of Linux. Scientific Linux was designed to run science and particle physics code, not run email or have a nice gui (i.e. it isn’t a Mac, or Windows, or ubuntu). There are ways around this, of course – start up a virtual machine and install scientific linux on it, etc. But then you hit the second problem: you have to install the ATLAS software. I’m not sure about the other experiments, but for ATLAS this is a 6 gig affair. And, each time a new version of the release comes out you have to install it all over again. In a virtual machine this can be painful (take hours of your time).

This is where CERNVM comes in. The first realization was that when you compile, build, and test-run, you need only about 10% of that 6 gigs of software. Of course, everyone needs a different 10%, but in general, it is only a small fraction of the release that is needed. So why download everything? The second realization was: automatically install each release or file only when it was needed – and automatically add each release. This second bit means that I, as an end user, never have to install another ATLAS release. Ever! How sweet is that!?!? I can just use it. As soon as a new release is out, and CERN publishes it, then I can access it!

CERNVM is a virtual machine. For folks that have been around CERN a while, they will know that CERNVM refers to the venerable IBM system that CERN used to have as one of its mainframes. According to Predrag Buncic, so I believe is the lead on the modern CERNVM project, that choice of names was on purpose. See Predrag’s talk slides from CHEP for some more details.

CERNVM accomplishes its magic with a FUSE file system. This brilliant open source project creates a virtual file system. Whenever you try to access a file in it, FUSE hands the request off to some user-written code. In CERNVM’s case, this code looks for the file up on a master server at CERN, and then downloads it locally. Once it has been cached locally, the file is accessible like any other file. So, CERN can publish a whole ATLAS release on their master servers, and then when I try to setup that release, CERNVM will automatically bring down exactly the files that I need to get my work done. Better yet, if I’ve already got the files locally, then I can hope on an airplane and everything will still work! Not to shabby!

The virtual machine part of this gets around the wrong OS type – it is based on Scientific Linux. Thus, the two main problems with running local build code are solved! Very nice, eh!?



No comments yet — be the first.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: