If it does not exist that will force us to tear down and rebuild – in some totally unknown way – our model of physics. Our model that we’ve had for 40+ years now. Imagine that – 40 years and now that it finally meets data… poof! Gone. Or, we will find the Higgs, and we’ll have a mass. Knowing the mass will be in itself interesting, and finding the Higgs won’t change the fact that we still need something more than the Standard Model to complete our description of the universe. But now every single beyond-the-standard model theory will have to incorporate not only electrons, muons, quarks, W’s, Z’s, photons, gluons – at their measured masses, but a Higgs too with the appropriate masses we measure!

So, how do I know this is going to happen? Look at this plot that was released during the recent HCP conference (deepzoom version ) in Paris.

Ok, this takes a second to explain. First, when we look for the Higgs we do it as a function of its mass – the theory does not predict exactly how massive it will be. Second, the y-axis is the rate at which the Higgs is produced. When we look for it at a certain mass we make a statement “if the Higgs exists at mass 200 GeV/c2, then it must be happening at a rate less than 0.6 or we would have seen it.” I read the 0.6 off the plot by looking at the placement of the solid black line with the square points – the observed upper limit. The rate, the y-axis, is in funny units. Basically, the red line is the rate you’d expect if it was a standard model Higgs. The solid black line with the square points on it is the combined LHC exclusion line. Combined means ATLAS + CMS results. So, anywhere the solid black line dips below the red horizontal line means that we are fairly confident that the Standard Model Higgs doesn’t exist (BTW – even fairly confident has a very specific meaning here: we are 95% confident). The hatched areas are the areas where the Higgs has already been ruled out. Note the hatched areas at low mass (100 GeV or so) – those are from other experiments like LEP.

Now that is done. A fair question is where would we expect to find the Higgs. As it turns out, a Standard Model Higgs will mostly likely occur at low masses – exactly that region between 114 GeV/c2 and 140 GeV/c2. There isn’t a lot of room left for the Higgs to hide there!! These plots are with 2 fb-1 of data. Both experiments now have about 5 fb-1 of data recorded. And everyone wants to know exactly what they see. Heck, while in each experiment we basically know what we see, we desperately want to know what the other experiment sees. The first unveiling will occur at a joint seminar at 2pm on December 13th. I really hope it will be streamed on the web, as I’ll be up in Whistler for my winder ski vacation!

So what should you look for during that seminar (or in the talks that will be uploaded when the seminar is given)? The above plot will be a quick summary of what the status of the experiments. Each experiment will have an individual one. The key thing to look for is where the dashed line and the solid line deviate significantly. The solid line I’ve already explained – that says that for the HIggs of a particular mass if it is there, it must be at a rate less than what is shown. Now, the dashed line is what we expect – given everything was right – and the Higgs didn’t exist at that mass – that is how good we expect to be. So, for example, right around the 280 GeV/C2 level we expect to be able to see a rate of about 0.6, and that is almost exactly what we measure. Now look down around 120-130 GeV/c2. There you’ll notice that the observed line is well above the solid line. How much – well, it is just along the edge of the yellow band – which means 2 sigma. 2 sigma isn’t very much – so this plot has nothing to get very interested yet. But if one of the plots shown over the next year has a more significant excursion, and you see it in both experiments… then you have my permission to get a little excited. The real test will be if we can get to a 5 sigma excursion.

This seminar is the first step in this final chapter of the old realm of particle physics. We are about to start a new chapter. I, for one, can’t wait!

N.B. I’m totally glossing over the fact that if we do find something in the next year that looks like a Higgs, it will take us sometime to make sure it is a Standard Model Higgs, rather than some other type of Higgs! 2nd order effect, as they say. Also, in that last long paragraph, the sigma’s I’m talking about on the plot and the 5 sigma discovery aren’t the same – so I glossed over some real details there too (and this latter one is a detail I sometimes forget, much to my embarrassment at a meeting the other day!).

Update: Matt Strassler posted a great post detailing the ifs/ands/ors behind seeing or not seeing – basically a giant flow-chart. Check it out!

This is a picture of the Cockroft-Walton at Fermilab’s Tevatron. This is where it all starts.

It isn’t that much of an exaggeration to say that my career started here. You are looking through a wire cage at one half of the Cockroft-Walton – the generator creates a very very very large electric field that ionizes Hydrogen gas (two protons and two electrons) by ripping one of the protons off. The gas, now charged, can be accelerated by an electric field. This is how protons start in the Tevatron.

And that is how most of the experimental data that I used for my Ph.D. research , post-doc research, and tenure research started. Basically, my career from graduate student to tenure is based on data from the Tevatron. The Tevatron delivers its last beam this Friday, at 2pm Central time (the 30th).

I’ll miss working at Fermilab. I’ll miss working at DZERO (the most recent Fermilab experiment I’ve been on). I’ll also miss the character of the experiments – CDF and DZERO now seem like such small experiments. Only 500 authors. I feel like I know everyone. It is a community in a way that I’ve not felt at the LHC yet. And I’ll miss directly owning a bit of the experiment – something I joined the LHC too late to do. But most of all I’ll miss the people. True – many of them have made the transition to the LHC – but not all of them. For reasons of travel, or perhaps retirement, these people I’ll probably see a lot less over the next 10 years. And that is too bad.

I’ll remain connected with DZERO for some time to come. I’m helping out with doing some paper reviews and I’m helping out with data preservation – making sure the DZERO data can be accessed long after the experiment has ceased running.

Tevatron. It has been a fantastic run. You have made my career. And I’ve had a wonderful time with the science opportunities you’ve provided.

So long, and thanks for all the (anti-)protons.

The biggest thing I got back was that as the corrections become well known, they get automated – so there is no need for this two step process I outlined before – running on MC and data, deriving a correction, and then running a third time to do the actual work, taking the correction into account. Rather, the ROOT files are centrally produced and the correction is applied there by the group. So the individual doesn’t have to worry. Sweet! That definitely improves life! However, the problem remains (i.e. when you are trying to derive a new correction).

I made three attempts before finally finding an analysis framework that worked (well, four if you count the traditional approach of C++, python, bash, and duct tape!). As you can tell – what I wanted was something that would correctly glue several phases of the analysis together. The example from last time:

1. Correct the jet pT spectra in Monte Carlo (MC) to data
1. Run on the full dataset and get the jetPt spectra.
2. Do the same for MC
3. Divide the two to get the ratio/correction.
2. Run over the data and reweight my plot of jet variables by the above correction.

There are basically 4 steps in this: run on the data, run on the MC, divide the results, run on the data. Ding! This looks like workflow! My firs two attempts were based around this idea.

Workflow has a long tradition in particle physics. Many of our computing tasks require multiple steps and careful accounting every step of the way. We have lots of workflow systems that allow you to assemble a task from smaller tasks and keep careful track of everything that you do along the way. Indeed, all of our data processing and MC generation has been controlled by home-rolled workflow systems at ATLAS and DZERO. I would assume at every other experiment as well – it is the only way.

This approach appealed to me: I can build all the steps out of small tasks. One task that runs on data and one that runs on MC. And then add the “plot the jet pT” sub-task to each of those two, take the outputs, and then have a small generic tasks that would calculate a rate, and then another task that would weight the events and finally make the plots. Easy peasy!

So, first I tried Trident, something that came out of Microsoft Research. An open source system, it was designed to work with a number of scientists with large datasets that required frequent processing (NOAA related, I think). It had an attractive UW, and arbitrary data could be passed between the tasks, and the code interface for writing the tasks was pretty simple.

I managed to get some small things working with it – but there were two big things that caused it to fail. First, the way you pass around data was painful. I wanted to pass around a list of files to run on – and then from that I needed to pass around histograms. I wanted fine grained tasks that would manipulate histograms (dividing the plots) and the same time other tasks would be manipulating whole files (making the plots). Ugh! It was a lot of work just to do something simple! The second thing that killed it was that this particular tool – at the time – didn’t have sub-jobs. You couldn’t build a workflow, and then use it in other workflows. It was my fault that I missed that fact when I was choosing the tool.

So, I moved onto a second attempt. Since my biggest problem had been hooking everything up I decided to write my own. Instead of a GUI interface, I had an XML interface. And I did what is known as “coding-by-convention.” The idea is that I’d set a number of defaults into the design so that it “just worked” as long as the individual components obeyed the conventions. Since this was my own private framework there was no worry that this wouldn’t happen. The framework knew how to automatically combine similar histograms, for example, or if it was presented with multiple input datasets it knew how to combine those as well – something that would have required a another step in the Trident solution.

This solution went much better – I was able to do more than just do my demo – I tried moving beyond the reweighting example above and tried to do something more complex. And here is where, I think, I hit on the real reason that workflow doesn’t work for analysis (or at least for me): you are having to switch between various environments too often. The framework was written in XML. If I wanted a new task, then I had to write C++, or C# (depending). Then there was the code that ran the framework – I’d have to upgrade that periodically.

Really, all I wanted to do was make a stupid plot on two datasets, divide it, and then make a third plot using the first as a weight. Why did I need different languages and files to do that – why couldn’t I write that in a few lines??

Those of you who are active in this biz, of course, know the answer: two different environments. One set of code deals with looping over, possibly, terrabytes of data. That is the loop that makes the plot. Then you need some procedural code to do the histogram division. When that is done, you need another loop of code to do the final plots and reweighting. Take a step back. That is a lot of support code that I have to write! Loading up the MC and data files, running the loop over them, saving the resulting histogram. The number of lines I actually need to create the plot and put the data into the plot? Probably about 2 line or 3. The number of lines I need to actually run that job start to finished and make that plot? Closer to 150 or so, and in several files, some compiled and some interpreted. Too much ceremony for that one or two lines of code: 150 lines of boilerplate for 3 or so lines of the physics interesting code.

So, I needed something better. More on that next week.

BTW, the best visual analysis workflow I’ve seen (but not used) is something called VISPA. Had I known about it when I started the above project I would have gone to it first – it is cross platform, has batch manager, etc., integrated in, etc. (a fairly extensive list). Looking in retrospect it looks like it could support most of what I need to do. I say this only having done a quick scan of its documentation pages. I suspect I would have run into the same problem: having to move between different environments to code up something “simple”.

But I had a problem. By the time I stepped down in order to even show a plot in ATLAS you had to apply multiple corrections: the z distribution of the vertex was incorrect, the transverse momentum spectrum of the jets in the Monte Carlo didn’t match, etc. Each of these corrections had to first be derived, and then applied before someone would believe your plot.

To make your one really great plot then, lets look at what you have to do:

1. Run over the data to get the distributions of each thing you will be reweighting (jet pT, vertex z position, etc.).
2. Run over the Monte Carlo samples to get the same thing
3. Calculate the reweighting factors
4. Apply the reweighting factors
5. Make the plot you’d like to make.

If you are lucky then the various items you need to reweight are not correlated – so you can just run the one job on the Data and the one job on the Monte Carlo in steps one and two. Otherwise you’ll have to run multiple times. These jobs are either batch jobs that run on the GRID, or a local ROOT job you run on PROOF or something similar. The results of these jobs are typically small ROOT files.

In step three you have to author a small script that will extract the results from the two jobs in steps 1 and 2, and create the reweighting function. This is often no more difficult that dividing one histogram by another. One can do this at the start of the plotting job (the job you create for steps 4 and 5) or do ti at the command line and save the result in another ROOT file that serves as one of the inputs to the next step.

Steps 4 and 5 can normally be combined into one job. Take the results of step 3 and apply it as a weight to each event, and then plot whatever your variable of interest is, as a function of that weight. Save the result to another ROOT file and you are done!!

Whew!

I don’t know about you, but this looked scary to me. I had several big issues with this. First, the LHC has been running gang-busters. This means having to constantly re-run all these steps. I’d better not be doing it by hand, especially as things get more complex, because I’m going to forget a step, or accidentally reuse an old result. Next, I was going back to be teaching a pretty difficult course – which means I was going to be distracted. So whatever I did was going to have to be able to survive me not looking at it for a week and then coming back to it… and me still being able to understand what I did! Mostly, the way I normally approach something like the above was going to lead to a mess of scripts and programs, etc., all floating around.

It took me three tries to come up with something that seems to work. It has some difficulties, and isn’t perfect in a number of respects, but it feels a lot better than what I’ve had to do in the past. Next post I’ll talk about my two failed attempts (it will be a week, but I promise it will be there!). After that I’ll discuss my 2011 Christmas project which lead to what I’m using this year.

I’m curious – what do others do to solve this? Mess of scripts and programs? Some sort of work flow? Makefiles?? What?? What I’ve outlined above doesn’t seem scalable!

The break down is a bit under 4 million lines of C++ and almost 1.5 million lines of python – the two major programming languages used by ATLAS. Additionally, in those same C++ source files there are another about million blank lines and almost a million lines of comments. Python contains similar fractions.

There are 7 lines of LISP. Which was probably an accidental check-in. Once the build runs the # of lines of source code balloons almost a factor of 10 – but that is all generated code (and HTML documentation, actually) – so shouldn’t count in the official numbers.

This is imperfect because these are just the files that are built for the reconstruction program. This is the main program that takes the raw detector signals and coverts them into high level objects (electrons, muons, jets, etc.). There is another large body of code – the physics analysis code. That is the code that takes those high level objects and coverts them into actual interesting measurements – like a cross section, or a top quark mass, or a limit on your favorite SUSY model. That is not always in a source code repository, and is almost impossible to get an accounting of – but I would guess that it was about another x10 or so in size, based on experience in previous experiments.

So, umm… wow. That is big. But it isn’t quite as big as I thought! I mentioned in the last post talking about source control that I was worried about the size of the source and checking it out. However, Linux is apparently about 13.5 million lines of code, and uses one of these modern source control systems. So, I guess these things are up to the job…

So, lets tackle a ROOT problem.

ROOT is weird. It has made it very easy to do very simple things. For example, want to draw a previously made histogram? Just double click and you’re done. Want to see what the data in one of your TTree’s looks like? Just double click on the leaf and it pops up! But, the second you want to do something harder… well, it is much harder. I’d say it was as hard to do something advanced as it was to do something intermediate in ROOT.

Plotting is an example.

Stacking the Plots

I have four plots, and I want to plot them on top of each other so I can compare them. If I do exactly what I learned how to do when I learned to plot one thing, I end up with the following:

Note all the lines on black, thin, and on top of each other. No legend. And that “stats” box in the upper right contains data relevant only to the first plot. The title strip is also only for the first plot. Grey background. Lousy font. It should probably have error bars but that is for a later time.

h1->Draw();
h2->Draw("SAME");
h3->Draw("SAME");
h4->Draw("SAME");

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

So, everyone has to make plots like this. This should be “easy” to make it look good! I suspect with a simple solution 90% of the folks who use ROOT would be very happy!

So, someone must have thought of this, right? Turns out… yes. It is called THStack. Its interface is dirt simple:

THStack *s = new THStack();
s->Add(h1);
s->Add(h2);
s->Add(h3);
s->Add(h4);
s->Draw("nostack");

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }and we end up with the following:

THStack actually took care of a lot of stuff behind our backs.It matched up the axes, it made sure the max and min of the plot were correct, removed the stats box, and killed off the title. So this is a big win for us! Thanks to the ROOT team. But we are not done. I don’t know about you, but I can’t tell what is what on there!

Color

There are two options for telling the plots apart: color the lines or make them different patterns (dots, dashes, etc.). I am, fortunately, not color blind, and tend to choose color as my primary differentiator. ROOT defines a number of nice colors for you in the EColor enumeration… but you can’t really use it out of the box. Charitably, I would say the colors were designed to look good on the printed page – some of them are a disaster on a CRT, LCD, or beamer.

First, under no circumstances, under no situation, never. EVER. use the color kYellow. It is almost like using White on a White background. Just never do it. If you want a yellowish color, use kOrange as the color. At least, it looks yellow to me.

Second, try to avoid the default kGreen color. It is a flourecent green. On a white or grey background it tends to bleed into the surrounding colors or backgrounds. Instead, use a dark green color.

Do not use both kPink and kRed on the same plot – they are too close together. kCyan suffers the same problem as kGreen, so don’t use it. kSpring (yes, that is the name) is another color that is too bright a green to be useful – stay away if you can.

After playing around a bit I settled on these colors for my automatic color assignment: kBlack, kBlue, TColor::GetColroDark(kGreen), kRed, kViolet, kOrange, kMagenta. The TColor class has some nice palettes (right there in the docs, even). But it one thing it doesn’t have that it really should is what the constituents of EColor look like. These are the things that you are most likely to use.

Colors are tricky things. The thickness of the line can make a big difference, for example. The default 1 pixel line width isn’t enough in my opinion to really show off these colors (more on fixing that below).

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

After applying the colors I end up with a plot that looks like the following:

A Legend and Title

So the plot is starting to look ok… at least, I can tell the difference between the various things. But darned if I can tell what each one is! We need a legend. Now, ROOT comes with the TLegend object. So, we could do all the work of cycling through the histograms and putting up the proper titles, etc. However, it turns out there is a very nice short-cut provided by the ROOT folks: TPad::BuildLegend. So, just using the code:

c1->BuildLegend();

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }where c1 is the pointer to the current TCanvas (the one most often used when you are running from the command line). See below for its effect. The automatic legend has some problems – mainly that it doesn’t automatically detect the best placement when drawing for a stack of histograms (left, right, up or down). One can think of a simple algorithm that would get this right most of the time. But that is for another day.

Next, I’d like to have a decent title up there, similar to what was there previously. This is also easy – we just pass it in when we create the stack of histograms.

THStack *s = new THStack("histstack", "WeightSV0");

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

And we now have something that is at least scientifically serviceable:

One thing to note here – there are no x-axis labels. If you add an x-axis label to your plot the THStack doesn’t copy it over. I’d call that a bug, I suppose.

Background And Lines And Fonts

We are getting close to what I think the plot should look like out of the box. The final bit is basically pretty-printing. Note the very ugly white-on-grey around the lines in the Legend box. Or the font (it is pixelated, even when the plot is blown up). Or (to me, at least) the lines are too thin, etc. This plot wouldn’t even make it past first-base if you tried to submit it to a journal.

ROOT has a fairly nice system for dealing with this. All plots and other graphing functions tend to take their queues from a TStyle object. This defines the background, etc. The default set in ROOT is what you get above. HOWEVER… it looks like that is about to change with the new version of ROOT.

Now, a TStyle is funny. A style is applied when you draw the histograms… but it is also applied when it is created. So to really get it right you have to have the proper style applied both when you create and when you draw the histogram. In short: I have an awful time with TStyle! I’m left with the choice of either setting everything in code when I do the drawing, or applying a TStyle everywhere. I’ve gone with the latter. Here is my rootlogon.C file, which contains the TStyle definition. But even this isn’t perfect. After a bunch of work I basically gave up, I’m afraid, and I ended up with this (note the #@*@ title box still has that funny background):

Conclusion

So, if you’ve made it this far I’m impressed. As you can tell, getting ROOT to draw nice plots is not trivial. This should work out of the box (using the “SAME” option that I used in the first line we should get behavior that looks a lot like this last plot).

Finally, a word on object ownership. ROOT is written in C++, which means it is very easy to delete an object that is being referenced by some other bit of the system. As a result, code has to carefully keep track of who owns what and when. For example, if I don’t write out the Canvas that I’ve generated right away, sometimes my canvases somehow come out blank. This is because something has deleted the objects from under me (it was my program obviously, but I have no idea what did it). Reference counting would have been the right away to go, but ROOT was started too long ago. Perhaps it is time for someone to start again?

The code I used to make the above appears below. My actual code does more (for example, it will take the legend and automatically turn it into “lightJets”, “charmJets”, etc., instead of the full blown titles you see there. It is, obvously, not in C++, but the algorithm should be clear!

        public static ROOTNET.Interface.NTCanvas PlotStacked(this ROOTNET.Interface.NTH1F[] histos, string canvasName, string canvasTitle,
            bool logy = false,
            bool normalize = false,
            bool colorize = true)
        {
            if (histos == null || histos.Length == 0)
                return null;

            var hToPlot = histos;

            ///
            /// If we have to normalize first, we need to normalize first!
            /// 

            if (normalize)
            {
                hToPlot = (from h in hToPlot
                           let clone = h.Clone() as ROOTNET.Interface.NTH1F
                           select clone.Normalize()).ToArray();
            }

            ///
            /// Reset the colors on these guys
            /// 

            if (colorize)
            {
                var cloop = new ColorLoop();
                foreach (var h in hToPlot)
                {
                    h.LineColor = cloop.NextColor();
                }
            }

            ///
            /// Use the nice ROOT utility THStack to make the plot
            /// 

            var stack = new ROOTNET.NTHStack(canvasName + "StacK", canvasTitle);
            foreach (var h in hToPlot)
            {
                stack.Add(h);
            }

            ///
            /// Now do the plotting. Use the THStack to get all the axis stuff correct.
            /// If we are plotting a log plot, then make sure to set that first before
            /// calling it as it will use that information during its painting.
            /// 

            var result = new ROOTNET.NTCanvas(canvasName, canvasTitle);
            result.FillColor = ROOTNET.NTStyle.gStyle.FrameFillColor; // This is not a sticky setting!
            if (logy)
                result.Logy = 1;
            stack.Draw("nostack");

            ///
            /// And a legend!
            /// 

            result.BuildLegend();

            ///
            /// Return the canvas so it can be saved to the file (or whatever).
            /// 

            return result;
        }

        /// <summary>
        /// Normalize this histo and return it.
        /// </summary>
        /// <param name="histo"></param>
        /// <returns></returns>
        public static ROOTNET.Interface.NTH1F Normalize(this ROOTNET.Interface.NTH1F histo, double toArea = 1.0)
        {
            histo.Scale(toArea / histo.Integral());
            return histo;
        }

.csharpcode, .csharpcode pre
{
font-size: small;
color: black;
font-family: consolas, “Courier New”, courier, monospace;
background-color: #ffffff;
/*white-space: pre;*/
}
.csharpcode pre { margin: 0em; }
.csharpcode .rem { color: #008000; }
.csharpcode .kwrd { color: #0000ff; }
.csharpcode .str { color: #006080; }
.csharpcode .op { color: #0000c0; }
.csharpcode .preproc { color: #cc6633; }
.csharpcode .asp { background-color: #ffff00; }
.csharpcode .html { color: #800000; }
.csharpcode .attr { color: #ff0000; }
.csharpcode .alt
{
background-color: #f4f4f4;
width: 100%;
margin: 0em;
}
.csharpcode .lnum { color: #606060; }

Subversion, Mercurial, and Git are all source code version control systems. When an experiment says we have 10 million lines of code – all that code is kept in one of these systems. The systems are fantastic – they can track exactly who made what modifications to any file under their control. It is how we keep anarchy from breaking out as >1000 people develop the source code that makes ATLAS (or any other large experiment) go.

Yes, another geeky post. Skip over it if you can’t stand this stuff.

ATLAS has switched some time ago from a system called cvs to svn. The two systems are very much a like: centralized, top-down control. Old school. However, the internet happened. And, more to the point, the Cathedral and the Bazaar happened. New source control systems have sprung up. In particular, Mercurial and git. These systems are distributed. Rather than asking for permission to make modifications to the software, you just point your source control client at the main source and hit copy. Then you can start making modifications to your hearts content. When you are done you let the owner of the repository know and tell them where your repository is – and they then copy your changes back! The key here is that you had your own copy of the repository – so you could make multiple modifications w/out asking the owner. Heck, you could even send your modifications to your friends for testing before asking the owner to copy them back.

That is why it is called distributed source control. Heck, you can even make modifications to the source at 30,000 feet (when no wifi is available).

When I wrote that first blog post I’d never tried anything but the old school source controls. I’ve not spent the last 5 months using Mercurial – one of the new style systems. And I’m sold. Frankly, I have no idea how you’d convert the 10 million+ lines of code in ATLAS to something like this, but if there is a sensible way to convert to git or mercurial then I’m completely in favor. Just about everything is easier with these tools… I’ve never done branch development in SVN, for example. But in Mercurial I use it all the time… because it just works. And I’m constantly flipping my development directory from one branch to another because it takes seconds – not minutes. And despite all of this I’ve only once had to deal with merge conflicts. If you look at SVN the wrong way it will give you merge conflicts.

All this said, I have no idea how git or Mercurial would scale. Clearly it isn’t reasonable to copy the repository for 10+ million lines of code onto your portable to develop one small package. But if we could figure that out, and if it integrated well into the ATLAS production builds, well, that would be fantastic.

If you are starting a small stand alone project and you can choose your source control system, I’d definitely recommend trying one of these two modern tools.

Lets take a purely hypothetical situation. A person has joined a large scientific collaboration like CDF, DZERO, ATLAS, or CMS. As part of joining they agree to abide by a set of rules. For example, not discussing an analysis publically before it has been approved by the experiment.

I apologize in advance to those who are not part of this life, or who don’t care. This blog posting will be even less interesting than normal!

Here are the questions. I’m curious about the answers from both an ethics point of view and a political point of view. Or any other point of view you care to bring to bear. I’ve put my answers below. The setup below is hypothetical! And I have some personal issues with #7! #8 is the one I’ve gotten most push back on when talking with people.

1. You are a member of said collaboration and you anonymously post all or part of an internal document to a blog.
2. You are a member of said collaboration and you post non-anonymously to a blog.
3. The blog owner(s) are unaffiliated with any experiment. Are they obligated to take it down?
4. The blog owner is affiliated with the experiment (e.g. say someone posted an internal DZERO or ATLAS abstract to my blog). Are they obligated to take it down?
5. Is it ok for the experiment to ask the blogger to reveal the posters information? For example, the wordpress blogging platform, which I use, keeps internally a record, visible to me, of the posters IP address, which might be able to identify the poster. Is the answer any different if the blog owner is a member of the same experiment? How about a member of a competing/different experiment?
6. Does the blog owner have to respond with the information to the experiment?
7. What if the blog owner is a member of the same experiment? Do they have to respond then?
8. Does the experiment have to ask the blog owner for help?

Ok. So, here are my answers. These aren’t completely thought out, so feel free to call me out if I’m not being consistent. And these are my opinions below, no matter how strongly I state them.

1. This is clearly unethical. You are violating something that you agreed to in the first place, voluntarily. Further, by doing this anonymously you are basically trying to get away without being accountable – so you are taking no responsibility for your actions – which is also unethical. The PR result depends, obviously, on what is posted. If the topic is interesting enough to the mainstream, articles will end up on the mainstream news sites. If this damages the credibility of an actual result when it is released then real harm has been done. It is not likely that it will damage the credibility within the field, however.
2. For me this is more murky. You clearly have violated the agreement that you signed initially. But you have also made it clear who you were when you posted it – so you are taking responsibility and accepting the consequences for your actions. The first half you are not behaving ethically, but the second half you are. It seems the PR consequences are similar, except they will be much more personal because the press will be able to get in touch with you. A large faceless experiment, like DZERO or ATLAS, will have a much harder time countering this (people make better stories!).
3. Ethically, I don’t think you are obligated to take it down if you are not affiliated with any experiment. That was someone else’s agreement, and not one that you signed up for. I follow the thinking of various places that deal with whistleblowers. Now, the blog owner may have their own set of ethical guidelines for the blog, for example, “I will not traffic in rumors,” and then ethically they should not make an exception for a particular post. But that is strictly up to them – they could just as easily say that “this blog traffics in rumors!” The PR aspect of this really depends, if the blog is up front about what it is, then the PR won’t reflect on it as much as it will reflect on the rumor. If the blog does something that violates its own guidelines – like normally it ignores rumors except in this particular one because it is a big one – then part of the PR will be focused back on them. This is a wash, in my opinion.
4. If the blog was owned by a member of the same experiment then I do think they would be obligated to take it down. The blog owner, upon joining the experiment, agreed not to reveal secrets, and the blog is an extension of the person who made the agreement. From a PR perspective, this would put the blog owner in a fairly difficult position! First, most of us small-time blogs allow comments w/out waiting for approval, so it could be up for several hours before it gets taken down. Any of the RSS comment aggregators would easily have time to grab it before it disappeared. So, it would be out there for anyone with a bit of skill even if it had already been taken down. So the PR would, basically, be the same as the other case. But, if any press came to call the blog owner they would have to say “No Comment.” Ha!
5. So, it is fine for the experiment to ask the blog owner for any identifiable information about the poster. They are not violating any of their ethics. The PR response, however, can vary dramatically. After the experiment asks, the blogger could respond “Yes” or “No”. And then everyone moves on. But the blogger could also post a copy of the request and say something like “This 3000 person scientific organization is putting pressure on my to reveal my sources. This is a clear suppression of free speech, etc. etc.” What happens next is anybody’s guess and really depends on the blogger’s reputation, their popularity, who picks it up and runs with it, etc. So, anything from forgotten to a PR nightmare for the experiment. For a blogger that wants to prove that they will keep their rumor sources confidential – and thus get more rumors, this could be a big plus. Add this to the likelihood that there is no identifiable information, this makes me conclude it isn’t worth it. Now, if the blogger is a member of the experiment, or the blogger is well known to individuals on the experiment, a small conversation can happen over the phone or in person to see if the blogger might be willing to help out.
6. First, if the blogger is not a member of the experiment. In this case, I do not think there is any ethical reason for the blogger to respond. By the same token, I do not think the experiment can get bent-out-of-shape if the blogger declines to help. I don’t think there is any real PR aspect to this question (other than what was above). Something to keep in mind: depending on the severity of the leak, you may be ending or seriously affecting someone’s career (judge/jury/etc.) by giving up that technical information – which could be spoofed.
7. Now, if the blogger was on the same experiment, then things get more tricky. Ethically, you agreed to keep your experiment’s secrets, but you didn’t agree to tattle tail on a fellow collaboration member. I feel like I’m on thin ice here, so any comments yes or no to this would be helpful – especially because I could see myself in this position! While that may be the case, the experiment could bring a huge amount of peer pressure to bear on the blog author if they are a member. This effect should not be underestimated.
8. This may seem like an odd question. Think of it from this point of view. An internal document has just been leaked. You are one of 3000 people working hard on this experiment. Something that you’ve had no input into, and perhaps seriously disagree with, has been put out on the web. You are still bound by the agreement with the collaboration so you can’t counter why you think it is bad. You have to sand by, frustrated, as this document is discussed by everyone except the people it should be discussed by. Worse, what if this person who did the posting gets away with it!? There are no consequences to what they did? Worse, what if the collaboration changes the way it does internal reviews and physics in order to keep things more secret from even its own members to lessen the chances of another leak? Now the person doing the leak has seriously impacted your ability to work and nothing has happened to you. So, should the collaboration do all it can to track this leaker down? Whew. Yes. But what if tracking this person down causes more damage (like the free speech PR nightmare I mentioned above)? I have a lot of trouble answering this question. In isolation the answer to this is clearly yes. However, when the various possible outcomes are considered, it feels to me like it isn’t worth it.

One final thing. As far as I can see, it seems to me that no actual laws have been broken by any of the proposed actions. That is, you couldn’t sue in a court of law for any of the actions. There is no publically recognized contract, for example. Do people agree with that? Any key questions I missed that should be in the above list?

… means not mis-crediting a result

… means an obligation to society to not falsify results

… means not making false claims to gain exposure

… means respecting your fellow scientist and their results

… means not talking about things that aren’t public (or, say, that haven’t undergone an internal review)

… means playing by the rules you agreed to when you enter into a collaboration

It means being a scientist!

Integrity is more important that ever given how much the public eye is focused on us in particle physics.

Update: I should mention that this post was authored with Alison Lister.

She was using the Global Entry program. Having signed up and used it for my most recent flight… I’m a fan. It is fairly cheap – $100 bucks for 5 years. You do have to give up finger prints and picture to the US government – as far as I know that is the first set of finger prints any government agency has on record for me – so that was a little weird. As an example, on my last flight into IAD the plane doors were opened at 4:10 pm. At 4:22 pm I was in the X-Ray line. This included more than 5 minutes of walking since our plane was waaaay down the terminal. You use a kiosk instead of a person in the immigration area. I’d say it took the same amount of time as dealing with an officer who decided not to ask any question and if there were no lines – about 90 seconds or so. Extra bonus: no filling out those @*#&@ blue custom forms (there is an abbreviated version on the kiosk). And, when you go through customs, there is a separate line that lets you cut to the front (at least, in IAD). You just hand them a bit of paper that the immigration kiosk printed out and you are done.

I could imagine there are a number of circumstances that don’t make this worth it. If you always travel with kids under 14 you can’t use this (well, the kids can’t use this), if you always check baggage the time saved will be a small fraction of your total time, and I think there are only about 20 airports that support it (these are where your international ports-of-entry). Oh, and if you like watching people while standing in lines to relax after that long flight being cooped up… then this isn’t for you either.

My flight into IAD earlier this week was over an hour late. I had less than an hour to connect. A student of mine and I were both on the plane and both were on the connecting flight to Seattle. Neither of us had bags checked. The Seattle flight was in D29 in IAD (which means a long walk). I did a brisk walk and made it before boarding started. He had to sprint some of the way and made it after everyone had already boarded – but he still made it. BTW – I was also able to skip to the front of the X-Ray line which can be killer in IAD because I’d been upgraded on that last leg. That probably saved me an additional 10 minutes or so on this trip.

So… I’d recommend getting this if you flight internationally with any frequency. It definitely made that part of my trip quicker and, thus, more enjoyable!

As a side note… WHY don’t they design the airport so that if you don’t have to pickup your luggage you don’t have to go thought security again?

It happens once a year, of course: Cherry Blossom Season. You can find it all over – Japan is famous for it. But back at the University of Washington we have our own little grove of Yoshino Cherry trees on the Quad. For the two weeks or so the place becomes a bit of a tourist destination – it is packed with people. Some just sitting and reading, but most walking around and snapping pictures. I went a little crazy this year. If you love this stuff, you can find it all over the web. Here are links to some of the stuff I’ve taken:

• Pictures from a cloudy day on flickr.
• A large panorama view. This is probably the easiest one to get an understanding of what the square looks like.
• A giant 451 photo 3D reconstruction (a photosynth). I’m really looking forward to the technology (recently previewed) where you can walk around with a video camera and that is enough to build one of these!
• A desktop theme pack for Windows 7. If you like having your background image change every 30 minutes to a different view of cherry trees, well, this is for you!

Enough till next year!

The first I heard about this phenomena was some results presented by the PAMELA experiment. The results were very interesting – perhaps indicating dark matter. The scientists showed a plot at a conference to show where they were, but explicitly didn’t put the plot into any public web page or paper to indicate they weren’t done analyzing the results or understanding their systematic errors. A few days later a paper showed up on arXiv (which I cannot locate) using a picture taken during the conference while the plot was being shown. Of course, the obvious thing to do here is: not talk about results before they are ready. I and most other people in the field looked at that and thought that these guys were getting a crash course in how to release results. The rule is: you don’t show anything until you are ready. You keep it hidden. You don’t talk about it. You don’t even acknowledge the existence of an analysis unless you are actually releasing results you are ready for the world to get its hands on and play with it as it may.

I’m sure something like that has happened since, but I’ve not really noticed it. But a paper out on the archives on April 1 (yes) seems to have done it again. This is a paper on a Z’ set of models that might explain a number of the small discrepancies at the Tevatron. A number of the results they reference are released and endorsed by the collaborations. But there is one source that isn’t – it is a thesis: Measurement of WW+WZ Production Cross Section and Study of the Dijet Mass Spectrum in the l-nu + Jets Final State at CDF (really big download). So here are a group of theorists, basically, announcing a CDF result to the world. That makes a bit uncomfortable. What is worse, however, is how they reference it:

In particular, the CDF collaboration has very recently reported the observation of a 3.3 excess in their distribution of events with a leptonically decaying W+- and a pair of jets [12].

I’ve not seen any paper released by the CDF collaboration yet – so that above statement is definitely not true. I’ve heard rumors that the result will soon be released, but they are rumors. And I have no idea what the actual plot will look like once it has gone through the full CDF review process. And neither do the theorists.

Large experiments like CDF, D0, ATLAS, CMS, etc. all have strict rules on what you are allowed to show. If I’m working on a new result and it hasn’t been approved, I am not allowed to even show my work to others in my department except under a very constrained set of circumstances*. The point is to prevent this sort of paper from happening. But a thesis, which was the source here, is a different matter. All universities that I know of demand that a thesis be public (as they should). And frequently a thesis will show work that is in progress from the experiment’s point of view – so they are a great way to look and see what is going on inside the experiment. However, now with search engines one can do exactly the above with relative ease.

There are all sorts of potential for over-reaction here.

On the experiment’s side they may want to put restrictions on what can be written in a thesis. This would be punishing the student for someone else’s actions, which we can’t allow.

On the other hand, there has to be a code-of-standards that is followed by people writing papers based on experimental results. If you can’t find the plot on the experiment’s public results pages then you can’t claim that the collaboration backs it. People scouring the theses for results (as you can bet there will be more now) should get a better understanding of the quality level of those results: sometimes they are exactly the plots that will show up in a paper, other times they are an early version of the result.

Personally, I’d be quite happy if results found in theses would stimulate conversation and models – and those could be published or submitted to the archive – but then one would hold off making experimental comparisons until the results were public by the collaboration.

The internet is here – and this information is now available much more quickly than before. There is much less hiding-thru-obscurity than there has been in the past, so we all have to adjust.

* Exceptions are made for things like job interviews, students presenting at national conventions, etc.

Update: CDF has released the paper…

The company’s plan to digitize every book ever published and make them widely available was derailed on Tuesday when a federal judge in New York rejected a sweeping $125 million legal settlement the company had worked out with groups representing authors and publishers.

I am a huge fan of the basic idea. Every book online and digital and accessible from your computer. I’m already almost living the life professionally: all the journal articles I use are online. The physics preprint archive, arivx.org, started this model and as a result has spawned new types of conversation – papers that are never submitted to journals. Pretty much the only time I walk over to the library is to look at some textbook up there. The idea of doing the same thing to all the books – well I’m a huge fan.

However, I do not like the idea of one company being the gateway to something like that. Most of the world’s knowledge is written down in one form or another – it should not be locked away behind some wall that is controlled by one company.

I’d rather see a model where we expect, in the long term, that all books and copyrighted materials will eventually enter the public domain. At that point they should be easily accessible online. When you think of the problem like this it seems like there is an obvious answer: the Library of Congress.

Copyrighted books are a tougher nut to crack. There publishers and authors presumably will still want to make money off this. And making out-of-print books available will offer some income (though not much – there is usually a reason those books are out of print). In this case the Google plan isn’t too bad – but having watched journals price gouge because they can, I’m very leery of seeing this happen again here. I’d rather see an independent entity setup that will act as a clearing house. Perhaps they aren’t consumer facing – rather they sell access and charge for books to various companies that then make the material available to us end users. This model is similar to what is done in the music business. I purchase (or rent) my music through Zune – I don’t deal directly with any of the record labels. The only problem is this model doesn’t have competition to keep prices down (i.e. nothing stops this one entity from price gouging).

Lastly, I think having all this data available will open a number of opportunities for things we can think of now. But I think that we need to make sure the data is also available in a raw form so that people can innovate.

Print books are dying. Some forms will take longer than others – I would expect the coffee table picture book to take longer before it converts to all digital than a paper-back novel. But I’m pretty confident that the switch is well underway now. What we do with all the print books is a crucial question. I do think we should be spending money on moving these books into the digital age. Not only are they the sum of our knowledge, but they are also a record of our society.

There seems to be a concerted attack on science funding in the US at the federal level. The feds fund most research that is too long term for a company to fund – which is becoming more and more as the stock market forces companies to think more and more short term. A healthy research program in a country needs to contain a balance for the sake of the long-term health of the economy. And a healthy economy is the only way to make jobs. The large cuts that are reputed to befall the Office of Science, which funds most of the national labs, will force lab closures. Facilities where we do science – gone! 1000’s of people layed off. Heck, if you are trying to cut out 60 billion you can take a guess as to how many jobs that is worth. At $100,000 per person per year – so really nice jobs! – that is another .6 million added to the unemployment roles. Right. That’s going to turn out well!

Second is this constant discussion about teacher pay. I’ve seen comments on newspaper articles with statements like “we are just paying them to babysit our kids.” Seriously?? Maybe we should just eliminate the schools and have the kids all at home. No formalized education system. Now, that has never been done before! And so obviously it must be better! Oh… wait. I guess it has been done before. I think it was called the middle ages… Arrgh! Yes, our K-12 system needs some real work. But beating the crap out of teachers in newspapers is not the way to get good people into the classroom! And the idea that teachers are overpaid paid? Seriously? [I’m not trying to channel Grey’s Anatomy here] I find that hard to believe. Perhaps they are getting better retirement plans for what they are paid – but I suspect that is because when the unions couldn’t negotiate a pay raise – so they went for an increase in the pension. I wonder if you paid teachers a more fair wage, but kept their pension plans the same size, if the rate would be more in line with normal?

On a more local note, one of our state legislators was heard to say “Higher education is a luxury we no longer can afford.” I don’t even know where to start with that. Washington is like every other state, it has some rich people and some poor people. UW is a state school – the state provides subsidies for the in-state students to make it more affordable. A robust state and federal scholarship program back fill for people really in need. The idea is if you are good and you want to get a higher level education, the federal government, the state government, and the university will do its best to make sure that finances do not get in your way. This has been a bedrock of all higher education in the USA for many years now. Do we go back to a class based system? What are people thinking, really? I get they are trying to cut the budget, but think for a few minutes about the implications of what you are saying!

And to those who say education is radically more expensive than it has been in the past – at the UW that is definitely true that the cost an instate student pays has gone up a lot over the last 10-15 years. Definitely more than inflation(by a bit). But if you look at the amount of $$ the university pays to educate a single student that has remained almost constant. Wait. For. It… That is right! State support has dropped dramatically. So the university has to cut expenses and find other sources of income – i.e. raise tuition. Blaming the university for this is misplaced. Last year in the state of Washington after the state legislature cut the UW funding by 26% the university raised tuition by 14% over two years. Legislatures were known to stand up at town halls, etc., and express their displeasure at UW for doing that in hard economic times. I’m happy with them being displeased – I was displeased – but at least be honest and say that the state cut 26% of the university’s funding. It isn’t like that was a capricious raise!

Next is another is the push to increase the teaching load. I currently teach one class a quarter – so three a year (I get paid for only the 9 months that I’m teaching – I have to find my own funding for the rest of the year). That one class is about 3 hours in the class room in front of students. Pretty cushy, eh!? I taught graduate particle physics this year. This is my third year so I’d like to think that I know it by now (not) – but all told during the week it would take about 20 hours of my time. The first time I taught it – when I had to teach myself some field theory – it was taking more like 50 hours a week. When I teach the easier undergraduate courses I tend to have 100’s of students – so it also works out to be about 20 hours a week. Some weeks a lot less, some a lot more. So, it would seem I have at least enough time to take on another course! Except there is one big problem here – my job isn’t just to teach undergraduates. My job is to also teach graduate students, mentor post-docs, and do research. UW is the #1 public institution in the USA when it comes to bringing in $$ from grants. You add another class, then you will effectively change the nature of the University of Washington – make it a teaching institution rather than a research institution. The ramifications of something like that are huge – rankings, desirability, research & undergrads, etc. Do people to say things like this understand how all this is connected?

This last election brought in a lot of new people (at least at the federal level). I remember being elected to a few positions having to do with HEP. I had all sorts of ideas – but I discovered that when I arrived that all the decisions that had been made were all made for a reason! They weren’t arbitrary. You can’t go wrecking around like a bull in a china shop – you have to carefully consider what you are doing and the ramifications. I get the feeling many of these new folks just don’t care. Really just don’t care. Even worse, they don’t know history – which means they are doomed to repeat it. Many of the ideas on the table around America have been tried before – if not here, then other places. I would love them to take a careful look. There is plenty of room for new things to achieve some of the same goals – why not try them rather than closing your eyes and just letting the knife fall where it may? In physics we call this a “prescale” – we just randomly through out data because we have too much. Here we are randomly throwing out programs because we have too little. In both cases this is an implicit admission of defeat: we aren’t smart enough to make a strategic cut.

Ok. Enough. Thank goodness there is a counter balance in most cases to these drives to change things so radically. It won’t be pleasant, but the system is too large and what comes out of it too valuable to actually destroy it in a few short years, despite best efforts of some. Now that I’ve vented, back to working on my classes and my research!

Update: Fixed “under paid” –> “over paid”. Of all the typo’s!

Before I write more, I should point out that I very much have a dog in this race. Actually, perhaps a bit more than one dog. Funding for almost all my research activities comes via the National Science Foundation (NSF) – this is funded directly by congress. My ability to hire post-docs and graduate students, train them, do the physics – everything, is dependent on that stream of money. Also, two months of salary a year come from that stream. In short, almost everything except for the bulk of my pay. That comes from two sources: state of Washington and student’s tuition. A further chunk of money comes from the Department of Energy’s (DOE) Office of Science – they fund the national labs where I do my research, for example. In short, particle physics does not exist without government funding.

So when people start talking about large, across-the-board cuts in funding levels I get quite nervous. Many republicans in 2010 campaigned on cutting back the budget, hard:

“We’re broke, and decisive action is needed to help our economy get back to creating jobs and end the spending binge in Washington that threatens our children’s future,” Mr. Boehner said.

Up until recently they really haven’t said how they were going to do it – a typical political ploy. But now things are starting to show up: cut funding to 2008 levels, and then no increases to counter inflation. The latter amounts to a 2-3% cut per year. No so bad for one year but when you hit 3-4 it starts to add up. You’ll have to let go a student or perhaps down-size a post-doc to a student.

But what about all these other cuts? So… I’m a scientist and I want to know: Where’s the data!? Well, as any of you who aren’t expert in the ways of Washington… boy is it hard to figure out what they really want to do. I suppose this is to their advantage. I did find out some numbers. For example, here is the NSF’s budget page. 2008 funding level was $6.065 billion. In 2010 it was funded at a rate of $6.9 billion. So dropping from 2010 back to 2008 would be a 12% cut. So, if that was cut blindly (which it can’t – there are big projects and small ones and some might be cut or protected), that would translate into the loss of about one post-doc, perhaps a bit more. In a group our size we would definitely notice that!

But is that data right? While I was searching the web I stumbled on this page, from the Heritage foundation, which seems to claim reducing the NSF to 2008 levels will save $1.7 billion, about x2 more than it looks like above. Who is right? I know I tend to believe the NSF’s web page is more reliable. But, seriously, is it even possible for a citizen who doesn’t want to spend days or weeks to gather enough real data to make an independently informed decision?

Check out this recent article from the NYTimes about a recent proposal coming from  Congressman Jordan whose goal is to reduce federal spending by $2.5 trillion through fiscal year 2021 (am I the only one that finds the wording of that title misleading?). As a science/data guy the first thing I want to know is: where is he getting all that savings from? There are lists of programs that are eliminated, frozen, or otherwise reduced – but that document contains no numbers at all. And I can’t find any supporting documentation that he and his staff must have in order of have made that $2.5 trillion claim. So, in that document, which is 80 pages long, I’m left scanning for the words “national science foundation”, “science”, “energy”, etc. Really, there is very little mentioned. But I have a very hard time believing that those programs are untouched – as the article in the new york times points out, since things like Medicare, Social Security, etc., are left untouched (the lions share of the budget – especially in out years), and so all the cuts must come from other programs:

As a result, its effect on the entire array of government programs, among them education, domestic security, transportation, law enforcement and medical research, would be nothing short of drastic.

I agree with that statement. 2.25 trillion is a lot of cash! Can you find the drastic lines in that document? Well, perhaps you know more about Washington. I can’t. This gets to me because now if I have to get into an argument it is a very abstract one.

Pipedream: What I would love these folks to do is release a giant spreadsheet of the US gov’t spending that had 2008, 2009, 2010 levels, and then their proposed cuts, with an extra column for extra text. That is a lot of data, and would probably be hard to compile. But, boy, it would be nice!

Taking a test is not just a passive mechanism for assessing how much people know, according to new research. It actually helps people learn, and it works better than a number of other studying techniques.

I’m here to tell you: duh!

In fact, we’ve institutionalized this in our physics graduate schools. Most university physics departments have the mother-of-all tests. Here at UW we call it the Qualifying Exam. Others call it a prelim (short for preliminary). And there is a joke associated with this exam, usually said with some bitterness if you’ve not passed it yet, or some wistfulness if you long since have passed it:

You know more physics the day you take the qual than you ever do at any other time in your life.

The exam usually happens at the end of your first year in graduate school. The first year classes are hell. Up to that point in my life it was the hardest I’d ever worked at school. Then the summer hits, and you get a small rest. But it is impossible to rest staring down the barrel of that exam, often given at the end of the summer just before the second year of classes start. You have to pass this exam in order to go on to get your Ph.D. And for most of us, it is the last (formal) exam in our career that actually matters. So physiologically, it is a big hurdle as well.

How hard is it? My standard advice to students is that they should spend about one month studying, 8 hours a day. For most people, if they study effectively, that is enough to get by. Some need less and some need more. This is about what it took me. What is the test like? At UW ours is 2 hours per topic, closed book, and all it is is working out problems. No multiple choice here! It lasts two days.

So, how do you study? There is, I think, really only one way to get past this. For 30 days, 8 hours a day, work out problems. There are lots of old qualifier problems on websites. Our department provides students with copies of all the old exams. Even if you don’t know the solution, you force your self to try to work it out with out looking it up in a book – break your brain on it. Once you can solve those problems with out having to look at a text book, you know you are ready. Imagine trying to study by reading a text book, or by reviewing your first year homework problems. There is no way your brain will be able to work out a new problem after that unless you are a very unique individual.

Note how similar this is to the results shown in the article:

In the first experiment, the students were divided into four groups. One did nothing more than read the text for five minutes. Another studied the passage in four consecutive five-minute sessions.

A third group engaged in “concept mapping,” in which, with the passage in front of them, they arranged information from the passage into a kind of diagram, writing details and ideas in hand-drawn bubbles and linking the bubbles in an organized way.

The final group took a “retrieval practice” test. Without the passage in front of them, they wrote what they remembered in a free-form essay for 10 minutes. Then they reread the passage and took another retrieval practice test.

The last group did the best, as you might imagine from the theme of this post!

This is also how you know more physics than at any other time in your life. At no other time do you spend 30 days working out problems across such a broad spectrum of physics topics. If you study and try to work out a sufficiently broad spectrum of problems you can breeze through the exam (literally, I remember watching one guy taking it with me just nail the exam in about half the time of the rest of us).

Working out problems  – without any aids – is active learning. I suppose you could follow the article and say that forcing the brain to come up with the solution means it organizes the information in a better way… Actually, I have no idea what the brain does. But, so far this seems to be the best way to teach yourself. You are actively playing with the new concepts and topics. This is why homework is absolutely key to a good education. And this is why tests are good – if you study correctly. If you actively study for the test (vs. just reading the material) then you will learn the material better.

And we need to work better at designing tests that force students to study actively. For example, I feel we are slipping backwards sometimes. With the large budget cuts that universities are suffering one byproduct is the amount of money we have to hire TA’s to help grade our large undergraduate classes is dropping. That means we can’t ask as many open-ended exam questions – and have to increase the fraction of multiple choice. It is much harder to design a test that goes after problem solving in physics using multiple choice. This is too bad.

So, is this qualifier test hazing process? Or is there a reason to do it? Actually, that is a point of controversy. Maybe there is a way to force the studying component without the high-anxiety of the make-or-break exam. Certainly some (very good) institutions have eliminated the qual. Now, if we could figure out how to do that and still get the learning results we want…

Each Christmas break I tend to hack on some project that interests me – but is often not directly related to something that I’m working on. Usually by the end of the break the project is useful enough that I can start to get something out of it. I then steadily improve it over the next months as I figure out what I really wanted. Sometimes they never get used again after that initial hacking time (you know: fail often, and fail early). My deeptalk project came out of this, as did my ROOT.NET libraries. I’m not sure others have gotten a lot of use out of these projects, but I certainly have. The one I tackled this year has turned out to be a total disaster. Interesting, but still a disaster. This plot post is about the project I started a year ago.  This was a fun one. Check this out:

Each of those little rectangles represents a plot released last year by DZERO, CDF, ATLAS, or CMS (the Tevatron and LHC general purpose collider experiments) as a preliminary result. That huge spike is July – 3600 plots (click to enlarge the image) -  is everyone preparing for the ICHEP conference. In all the 4 experiments put out about 6000 preliminary plots last year.

I don’t know about you – but there is no way I can keep up with what the four experiments are doing – let alone the two I’m a member of! That is an awful lot of web pages to check – especially since the experiments, though modern, aren’t modern enough to be using something like an Atom/RSS feed! So my hack project was to write a massive web scraper and a Silverlight front-end to display it. The front-end is based on the Pivot project originally from MSR, which means you can really dig into the data.

For example, I can explode December by clicking on “December”:

and that brings up the two halves of December. Clicking in the same way on the second half of December I can see:

From that it looks like 4 notes were released – so we can organize things by notes that were released:

Note the two funny icons – those allow you to switch between a grid layout of the plots and a histogram layout. And after selecting that we see that it was actually 6 notes:

That left note is title “Z+Jets Inclusive Cross Section” – something I want to see more of, so I can select that to see all the plots at once for that note:

And say I want to look at one plot – I just click on it (or use my mouse scroll wheel) and I see:

I can actually zoom way into the plot if I wish using my mouse scroll wheel (or typical touch-screen gestures, or on the Mac the typical zoom gesture). Note the info-bar that shows up on the right hand side. That includes information about the plot (a caption, for example) as well as a link to the web page where it was pulled from. You can click on that link (see caveat below!) and bring up the web page. Even a link to a PDF note is there if the web scrapper could discover one.

Along the left hand side you’ll see a vertical bar (which I’ve rotated for display purposes here):

You can click on any of the years to get the plots from that year. Recent will give you the last 4 months of plots. Be default, this is where the viewer starts up – seems like a nice compromise between speed and breadth when you want to quickly check what has recently happened. The “FS” button (yeah, I’m not a user-interface guy) is short for “Full Screen”. I definitely recommend viewing this on a large monitor! “BK” and “FW” are like the back and forward buttons on your browser and enable you to undo a selection. The info bar on the left allows you do do some of this if you want too.

Want to play? Go to http://deeptalk.phys.washington.edu/ColliderPlots/… but first read the following. And feel free to leave suggestions! And let me know what you think about the idea behind this (and perhaps a better way to do this).

• Currently works only on Windows and a Mac. Linux will happen when Moonlight supports v4.0 of Silverlight. For Windows and the Mac you will have to have the Silverlight plug-in installed (if you are on Windows you almost certainly already have it).
• This thing needs a good network connection and a good CPU/GPU. There is some heavy graphics lifting that goes on (wait till you see the graphics animations – very cool). I can run it on my netbook, but it isn’t that great. And loading when my DSL line is not doing well can take upwards of a minute (when loading from a decent connection it takes about 10 seconds for the first load).
• You can’t open a link to a physics note or webpage unless you install this so it is running locally. This is a security feature (cross site scripting). The install is lightweight – just right click and select install (control-click on the Mac, if I remember correctly). And I’ve signed it with a certificate, so it won’t get messed up behind your back.
• The data is only as good as its source. Free-form web pages are a mess. I’ve done my best without investing an inordinate amount of time on the project. Keep that in mind when you find some data that makes no sense. Heck, this is open source, so feel free to contribute! Updating happens about once a day. If an experiment removes a plot from their web pages, then it will disappear from here as well at the next update.
• Only public web pages are scanned!!
• The biggest hole is the lack of published papers/plots. This is intentional because I would like to get them from arxiv. But the problem is that my scrapper isn’t intelligent enough when it hits a website – it grabs everything it needs all at once (don’t worry, the second time through it asks only for headers to see if anything has changed). As a result it is bound to set off arxiv’s robot sensor. And the thought of parsing TeX files for captions is just… not appealing. But this is the most obvious big hole that I would like to fix some point soon.
• This depends on public web pages. That means if an experiment changes its web pages or where they are located, all the plots will disappear from the display! I do my best to fix this as soon as I notice it. Fortunately, these are public facing web pages so this doesn’t happen very often!

Ok, now for some fun. Who has the most broken links on their public pages? CDF by a long shot. Who has the pages that are most machine readable? CMS and DZERO. But while they are that, the images have no captions (which makes searching the image database for text words less useful than it should be). ATLAS is a happy medium – their preliminary results are in a nice automatically produced grid that includes captions.

It isn’t difficult. The web already has a perfect data format for this – Atom (or RSS). Just imagine. Each source code repository you commit to would publish a feed of all of your changes (with a time stamp, of course!) in the Atom format. Heck, your computer could keep track of what files you edited and publish a list of those too (many cloud storage services already do do this). Make a plot in ROOT? Sure! A feed could be published. Ran a batch job? The command you used for submission could be polished.

Then you need something central that is polling those RSS feeds with some frequency, gathering the data, and archiving it. Oh, and perhaps even making it available for easy use.

Actually, there is a service that does this already. Facebook. Sure! Just tell it about every RSS feed and it will suck that data in. Some of you are probably reading this on Facebook – and this posting got there because I told Facebook about this blog’s Atom feed and it sucked the data in.

Of course, having a write-only repository of everything you did is a little less than useful. You need a powerful search engine to bring the data you are interested in back out. Especially because a lot of that data is just a random command which contains no obvious indication of what you were working on (i.e. no meta-data).

And finally, at least for me, I don’t really want something that is static. Rarely is there a project that I’m finished with and I can neatly wrap it up and move on. Heck, there are projects I put down and pick up again many months later. This ultimate logbook doesn’t really support that.

Perhaps it is best to split the functions. Call this a ultimate logbook a daily log instead, and then keep separate bits of paper where you do your thinking… Awww heck, right back to where we started!

BTW, if you think Facebook might be interesting as a solution here, remember several things. First, as far as I can tell, there is no way to search your comments or posts. Second, you might get ‘Zuckenberged’ – that is, the privacy settings might get changed and your logbook might become totally public.

• What goes into a log book: pictures, code, text, screenscrapes, files, plots, handwriting, paper
• What do you use: Evernote, old style (bound notebook), loose paper, wiki/twiki, yojimbo, google wave, email (as in email a plot to yourself), tiddywiki, blogging software, text file, DEVON Think Personal, Journler (now defunct).

One thing I didn’t ask about but all of you contributed anyway was how the logbook got used (there is no right way – the logbook has to work for you, of course):

• Gave up – nothing but an inbox
• Just keep track of thinking
• Exploded: link services to track papers, paper for jotting down notes, email, etc. – a bit of everything
• Every last thing goes into the logbook, including bathroom breaks.

No one mentioned using a kindle/nook to read their logbook, btw. For software that gets used most like a logbook it looks to me like Evernote wins.

For me the most surprising method was email. And by surprising, I  mean smacking myself on the forehead because I’d not already thought of it. Here is the idea: just email your log book entries – with files and attachments, etc., to your logbook email account. Then use the power of search to recover whatever you want. And since you can stick it on Gmail or Hotmail or Yahoo mail, you have almost no size restrictions – and it is available wherever you happen to have a internet connection. Further, since it is just email, it is trivial to write scripts to capture data and ship it off to the logbook.

Now, I’ll ramble a bit in way of conclusion…

Do you remember MIcrosoft’s failed phone, the Kin? It was basically a smart phone w/out the apps. But one of the cool things it did was called Kin Studio. The point was this – everything you did on the phone was uploaded to the cloud. All the text messages you sent or received, all the pictures you took, etc. Then on the web you could look back at any time at what you did and have a complete record. Now, that is a logbook.

Of course, there are some problems with this. Who wants to look at lots of messages that say “ok!” or “ttl” or similar? And the same problem would occur if we were able to develop the equivalent of the Kin studio for logbooks. It would be a disaster. Which I think gets to the crux of what many of you were wrestling with in the comments of those posts (and something I wrestle with all the time): what do you put in a logbook!? There is a part of me that would like to capture everything – the ultimate logbook. Given todays software and technology this wouldn’t be very hard to write!

In thinking about this I came up with a few observations of my own behavior over the last few years:

One way to look at this is: what do you look up in a logbook? I have to say – what I look up in my logbook has undergone some dramatic changes since I was a graduate student. Back then we didn’t have the web (really) or search engines. As a result writing down exactly what I needed to do to get some bit of code working was very important. Now it is almost certain I can find a code sample on the web in one or two searches. So that doesn’t need to go into the logbook anymore. Plots still go in – but 90% of them are wrong. You know – you make the plot, think you are done, move on to the next step and in the process discover a mistake – so you go back and have to remake everything. And put the updated version of the plot into your logbook. Soon it becomes a waste of time – so you just auto-generate a directory with all the plots. So it always has the latest-and-greatest version. Hopefully you remember to put some of those into your logbook when you are done… but often not (at least me).

What is the oldest logbook entry you’ve ever gone back to? For me it was the top discovery – but that was nostalgia, not because I needed some bit of data. I rarely go back more than a few months. And, frankly, in this day and age, if you do an analysis that is published in January, by July someone (perhaps you) have redone it with more data and a better technique in July. You need those January numbers to compare – but you get them from an analysis note, not from your logbook! In short, the analysis note has become the “official” logbook of the experiment.

I have to say that my logbook current serves two functions: meeting notes and thinking. Meeting minutes are often not recorded – so keep a record. Especially since I’m using an electronic notebook I can mark things with an “action” flag and go back later to find out exactly what I need to do as a result of that meeting. The second heaviest use for me is brainstorming. Normally one might scribble ideas on some loose paper, perhaps leave them around for a day or two, come back refine them, etc. I use my logbook for that rather than loose paper.

Now a days I definitely do not keep a log book in the traditional way. Certainly not in the way I was taught to use a logbook in my undergraduate physics classes! Here is a quote from an ex-student of mine (in the comments of one of the previous posts – and I can copy this because he already has a job!!):

I have a rather haphazard attitude toward these things–I have a logbook, but I use it to remember things and occasionally to sort out and prioritize my thoughts. So it’s fairly sparse, and it certainly would be of no help in a patent dispute! Often I keep my old working areas around on my computer, and I use them if I forget what I did in my previous work.

This is pretty typical of what I see in people around me in the field. Other commenters made reference to more careful use of logbooks. I wonder how much usage style varies by field (medicine, physics (particle vs. condensed matter, theory vs. experiment), engineering, industry vs. academic, etc.)?

Last month in San Francisco at the Web 2.0 Summit, where about 1,000 people heard such luminaries as Mark Zuckerberg of Facebook, Julius Genachowski, chairman of the Federal Communications Commission, and Eric E. Schmidt of Google talk about the digital future, the Wi-Fi slowed or stalled at times.

I like the way one of my students, Andy Haas, put it once. He was giving a talk at a DZERO workshop on the Level 3 computer farm and trying to make a point about the number and type of computers that were in the farm. He drew an analogy to the number of laptops that were open in the room. It can be a little spooky – almost everyone has one, and almost everyone has them open during conference talks. In Andy’s case there were about 100 people in the room. And when you are giving the talk you have to wonder: how many people are listening!?

There is another side-effect, however. It is rare that the hotel, or whatever, is ready for the large number of devices that we particle physicists bring to a meeting. In the old days it was a laptop per person and now add in a cell phone that also wants a internet connection. Apparently most conference organizers used to use to guess that it would be about 1 in 5 people would have a portable that needed a connection at any one time. Folks from particle physics, however, just blew that curve! The result was often lost wifi connections, many seconds to load a page, and an inability to download the conference agenda! As conference organizer we have long ago learned that is one of the most important things to get right – and one of the key things that will be used to judge the organization of your conference.

The article is interesting in another aspect as well (other that pointing out a problem we’ve been dealing with for more than 10 years now). WiFi is not really designed for this sort of use. Which leads to the question – what is next?