Scientific Integrity April 22, 2011
Posted by gordonwatts in physics, physics life, politics, press, science.17 comments
… means not telling only half the result
… means not mis-crediting a result
… means an obligation to society to not falsify results
… means not making false claims to gain exposure
… means respecting your fellow scientist and their results
… means not talking about things that aren’t public (or, say, that haven’t undergone an internal review)
… means playing by the rules you agreed to when you enter into a collaboration
It means being a scientist!
Integrity is more important that ever given how much the public eye is focused on us in particle physics.
Update: I should mention that this post was authored with Alison Lister.
We’re Broke… or not… where is the data!? January 26, 2011
Posted by gordonwatts in DOE, NSF, science, University of Washington, USA.6 comments
It is hard for me not to feel very depressed about the way government funding is going in Washington. Especially all the “cuts” that keep being mentioned. So I thought I’d spend an hour doing my best to understand what cuts are being talked about. Ha! Sheer fantasy!
Before I write more, I should point out that I very much have a dog in this race. Actually, perhaps a bit more than one dog. Funding for almost all my research activities comes via the National Science Foundation (NSF) – this is funded directly by congress. My ability to hire post-docs and graduate students, train them, do the physics – everything, is dependent on that stream of money. Also, two months of salary a year come from that stream. In short, almost everything except for the bulk of my pay. That comes from two sources: state of Washington and student’s tuition. A further chunk of money comes from the Department of Energy’s (DOE) Office of Science – they fund the national labs where I do my research, for example. In short, particle physics does not exist without government funding.
So when people start talking about large, across-the-board cuts in funding levels I get quite nervous. Many republicans in 2010 campaigned on cutting back the budget, hard:
“We’re broke, and decisive action is needed to help our economy get back to creating jobs and end the spending binge in Washington that threatens our children’s future,” Mr. Boehner said.
Up until recently they really haven’t said how they were going to do it – a typical political ploy. But now things are starting to show up: cut funding to 2008 levels, and then no increases to counter inflation. The latter amounts to a 2-3% cut per year. No so bad for one year but when you hit 3-4 it starts to add up. You’ll have to let go a student or perhaps down-size a post-doc to a student.
But what about all these other cuts? So… I’m a scientist and I want to know: Where’s the data!? Well, as any of you who aren’t expert in the ways of Washington… boy is it hard to figure out what they really want to do. I suppose this is to their advantage. I did find out some numbers. For example, here is the NSF’s budget page. 2008 funding level was $6.065 billion. In 2010 it was funded at a rate of $6.9 billion. So dropping from 2010 back to 2008 would be a 12% cut. So, if that was cut blindly (which it can’t – there are big projects and small ones and some might be cut or protected), that would translate into the loss of about one post-doc, perhaps a bit more. In a group our size we would definitely notice that!
But is that data right? While I was searching the web I stumbled on this page, from the Heritage foundation, which seems to claim reducing the NSF to 2008 levels will save $1.7 billion, about x2 more than it looks like above. Who is right? I know I tend to believe the NSF’s web page is more reliable. But, seriously, is it even possible for a citizen who doesn’t want to spend days or weeks to gather enough real data to make an independently informed decision?
Check out this recent article from the NYTimes about a recent proposal coming from Congressman Jordan whose goal is to reduce federal spending by $2.5 trillion through fiscal year 2021 (am I the only one that finds the wording of that title misleading?). As a science/data guy the first thing I want to know is: where is he getting all that savings from? There are lists of programs that are eliminated, frozen, or otherwise reduced – but that document contains no numbers at all. And I can’t find any supporting documentation that he and his staff must have in order of have made that $2.5 trillion claim. So, in that document, which is 80 pages long, I’m left scanning for the words “national science foundation”, “science”, “energy”, etc. Really, there is very little mentioned. But I have a very hard time believing that those programs are untouched – as the article in the new york times points out, since things like Medicare, Social Security, etc., are left untouched (the lions share of the budget – especially in out years), and so all the cuts must come from other programs:
As a result, its effect on the entire array of government programs, among them education, domestic security, transportation, law enforcement and medical research, would be nothing short of drastic.
I agree with that statement. 2.25 trillion is a lot of cash! Can you find the drastic lines in that document? Well, perhaps you know more about Washington. I can’t. This gets to me because now if I have to get into an argument it is a very abstract one.
Pipedream: What I would love these folks to do is release a giant spreadsheet of the US gov’t spending that had 2008, 2009, 2010 levels, and then their proposed cuts, with an extra column for extra text. That is a lot of data, and would probably be hard to compile. But, boy, it would be nice!
Going Against The Grain Is Hard December 21, 2009
Posted by gordonwatts in science.3 comments
I follow Cliff Mass’ Seattle weather blog. It is an excellent resource for explaining the various weirdness that is Seattle/NW weather (we have mountains and water all around us). He has a couple of themes that come up repeatedly in his blog – getting a new costal radar installation, improving math in the schools (why he started the blog, apparently). Recently, he had an interesting posting about the climategate emails. He starts off saying, as you might expect, that nothing that has happened calls into question the science behind climate change and evolution. But that isn’t want caught my eye. It was the following:
Are scientists human and sometimes doing things out based on human emotion or group think? Yes.
and then a bit later:
I am known as somewhat of a skeptic regarding global warming effects in the NW–although I do believe that greenhouse gases are a serious problem in the long-run. A group of us noted that the snowpack in the Cascades was NOT rapidly melting away, in contrast to some publications by some local climate scientists and publicized by Mayor Nickels.
…
We then wrote a paper on the subject (the main contributor being Mark Stoelinga) and submitted it to the Journal of Climate. I have published a lot of papers in my life (roughly 100) and I never had problems like we had with this paper. Very biased associate editor and some reviewers.
First, I don’t think I need to do much to convince you the first statement is correct. We, as scientists, like to think we are totally driven by the data – we just go wherever it leads us. I’d like to think this is mostly true. But, lets face it, science is hard. We are always operating at the edge of our understanding – that is the very point of the what we are dong. If the answers were easy, they would be boring (to us) and we would move on. In order to make any progress we have to make some assumptions and then test them. The tests are often not conclusive – they might say something like “well, there is a 35% chance that assumption is right” or something like that. Lets say that is the best assumption we have. Then we make progress based on it until perhaps a better microscope or similar comes online or we otherwise get a new data point that validates or contradicts the assumption.
Deciding what is a good assumption – especially when the data does not give a clear answer – is something of an art. A very human art. Take the current state of my field – it is all about the Standard Model. We know it is broken, but we don’t have data that provides a clear direction to fixing it. To make progress we have all decided it is the theory and we are madly testing every bit of it we can. So far the thing refuses to bend under any experiment we do. Sure, there are some internal self-consistency problems and the Standard Model does not contain anything that looks like dark matter – but neither of those things give us good guidance for a future model. We have been using the Standard Model for 35 some-odd years now. The stupid thing works.
Now, lets say you make a measurement that spot on confirms the Standard Model. It is a new measurement, or perhaps an old one, but to a much higher degree of precision. Both of these are interesting results and deserve publication. Since you matched the results of the Standard Model people will start out believing you. Internal review and peer review will look at your results carefully, certainly, but if you’ve done your job it won’t be hard to get the paper published.
Now, instead, lets say you have a new measurement that is substantially different from the Standard Model. Something you measured in the lab that really breaks it. This is the big one. Internally, you bet you look at it more carefully. It gets reviewed – lots of people pour over it. It takes much longer to get released. And when it comes for peer review everyone starts out from the “Really? They see something? I’m not sure – first time in 35 years? Are we sure they understand their errors?” As you might imagine, it starts from there and ends up taking a lot longer for a result like this to come out. Heck, if this result is wrong – well, you can imagine how embarrassing it will be for all involved (the people doing the analysis, the experiment, the lab, the journal, the peer reviewers, etc.).
As a result it is a natural by-product of the way we do science that a paper that goes against the grain will take longer and have more trouble than a paper that goes with the grain does.
That isn’t to excuse some of the difficulty it sounds like Cliff’s paper had – that sounds worse than the normal difficulty that one should have. And the visibly public stakes are much higher in the case of climate change, obviously – which makes me think that one would have even more difficulty – as Cliff and his collaborators did.
Keep in mind those brakes are there for a reason. If we have 35 years behind of us of one thing working and suddenly someone proposes a different mechanism or a result that makes no sense in this 35 year old framework, then the first thing you have to do is make sure that result makes sense – and it does get extra scrutiny.
BTW, you’ll note this means that results that are different from expected get a lot more scrutiny than results that match expectations. This doesn’t feel like – results should get the same scrutiny weather or not they are expected, right? Human nature, however, gets in the way. This is why Blind Analyses are performed (for example, the recent CDMS result that generated so much interest). The idea is you hide the final result from yourself until the last minute – and promise to report exactly what you saw when you unblind.
Dark Matter Discovered – Loosing Control Of Your Data October 26, 2009
Posted by gordonwatts in GLAST, physics, physics life, science.2 comments
Ok, so it is a sensationalist title. But it was triggered by archive submission with the following title: Possible Evidence For Dark Matter Annihilation In The Inner Milky Way From The Fermi Gamma Ray Space Telescope. Wow! That is quite a title!
First, a bit of background on this paper. This is authored by two theorists who analyzed publically released FermiLAT/GLAST data. Fermi is a NASA funded project and one of its stipulations is that all data it collects must be made publically available 6 months after it has been collected. The authors of the paper downloaded the data, used a simple background model, added in their dark matter theory, and did a fit. And pow:
The red points are the data from Fermi, the dash-dot line and the dotted line are backgrounds (galactic diffuse, and a single TeV source), and the dashed line is their model. Nice fit, eh? Yep – looking at this my first reaction is “Wow – is this right? This is big – how did Fermi miss this?” and then I run across the hall to find someone that actually knows this data well.
It turns out the basic problem with this analysis is that not all sources of background are included. This is the galactic center, and, as one would imagine, there are lots of sources there. Not just one TeV source modeled above. My impression from hallway conversations is that when you take into account all of these sources there is much less (if any) room left for the dark matter model. I don’t think that Fermi has published a paper on this yet, but I suspect they will try a some point soon.
Ok, so all’s well. Fermi will publish the paper and everyone will know the right way to do this non-trivial analysis. Except that things got away from them. Nature news has picked it up and wrote a short update. This is pretty widely read. Now Fermi has a PR problem on its hands – people are running around talking about their data and they’ve not really had a voice yet (the science coordinator for Fermi was interviewed for this bit, but her comments were relegated to the end of the post). Fermi is a big collaboration (yes, not the size of the LHC), even if their paper is close to publication it would probably be at least a month or more before the collaboration could agree on a response. So what to do?
There are a lot of issues surrounding making data public. To first order, it is the tax payers that are paying for these experiments, so the data should be public. On the other hand, you can already see that besides the work and infrastructure of making the data public (which costs real $$ – especially for a big experiment like Fermi or one of the LHC experiments), you have to respond to other folks that analyze your data – basically pointing out their mistakes and trying to help them along, even when they might be in competition with some of your internal analyses. In NASA’s case all the data has to be made public – it is written into every grant submission and NASA even provides money for it. This is not currently the case for particle physics. In many of these advanced experiments the data is quite complex – and someone that can’t depend on the large infrastructure of the experiment to help interpret it is bound to have some difficulties.
One only wishes that the authors had gotten in contact with some Fermi folks before submitting their note to the archive…
It’s not about the people July 24, 2009
Posted by gordonwatts in NASA, politics, science.4 comments
This is a copy of a guest post I made over at Gordon’s blog a few days ago (he is away and asked for some help).
So – who’s been following the Apollo 8th’s 40th anniversary? NASA has put together a pretty cool web site with lots of great resources. The Apollo mission was truly one of our crowning technological achievements. And at a time like this you can’t help but look forward at the same time you look forward. And you run right into the same questions as always: can America afford space exploration? What should the program look like in the near and long-term future?
Ignoring the moral factor for the country, the science case for exploration is pretty strong in my view. There is a lot we don’t know about our own earth, about the moon, and, especially, about near-by planets. Telescopes that are out of the earth’s atmosphere have unique advantages when it comes to deep space exploration. We have a lot yet to discover about the planet we currently live on and on the solar system, galaxy, and universe we inhabit. I’m always in favor of more knowledge. Can we afford it. The USA? Certainly! I can’t think of many crises in the USA that would warrant totally canceling the space program (or any other science program for that matter). No country can survive without a balance in how it spends money – on the present as well as the near and far futures. The global warming crisis has already taught us how important it is to know where we live and how we are affecting the environment around us (no matter which side of the debate you are on!).
So what should be space program look like? I think it should look more like this:
![rover_sundial[1]](https://gordonwatts.files.wordpress.com/2009/07/rover_sundial1.jpg "rover_sundial[1]")
and less like this:
![man-on-the-moon[1]](https://gordonwatts.files.wordpress.com/2009/07/manonthemoon1.jpg "man-on-the-moon[1]")
The reason is bang-for-the-buck. I think it is pretty hard to deny that sending people up into space is amazingly expensive right now. You have to get them up, and you have to get them back. You have to supply life support. Escape hatches. Lots of space you might fill with equipment is people and oxygen and etc. People are amazingly good at improvising. Something goes wrong – your buggy gets stuck on a rock or similar – people can adapt. That is where they really beat out a robot. We can get an idea of something that needs to be done, and quickly come up with a plan that balances all the risk factors, the obstacles, and still achieves the goal. Heck, if on the way to accomplishing the plan something interesting distracts us – well, that is useful too – perhaps we made one of those serendipitous discoveries; whatever: we can quickly evaluate the interesting thing and decide if it is just a rock reflecting light or some form of Martian currency. Finally, because we have to bring back the humans, bringing back that discovery has very little additional cost (other than biohazard containment!).
Robots have different strengths. Many of the things I’ve listed above robots aren’t so good at. On the other hand, they can stay on a planet for months or years. Need no life support – so getting them there is a lot cheaper. We are getting good enough at creating these exploration robots that we can given them simple tasks and they can take care of themselves. Since they are there for months or years at a time, it doesn’t matter if they aren’t extraordinarily efficient, we can just have them repeat the exercise several times until they get it right. And, perhaps because we don’t have to carry all the support equipment for humans out there, we can stuff an extra scientific instrument or two on the device and get that much more information out.
The thing is – the science case is overwhelmingly in favor of the robots right now. Check out the poster-child for this – the Mars Rover Missions. 5 years of exploration and science. Less than one billion bucks. What did putting a man on the moon cost? I can’t find firm numbers – but it isn’t uncommon to see numbers like 100 billion. Now, I don’t think going back to the moon would be that expensive – it usually is cheaper the second time around. Mars, which was Bush’s stated goal, would probably be that or close to it. For that kind of cost you could littler Mars with rovers and send a few to other planets.
Maybe these commercial endeavors getting people into space cheaply will change the cost/benefit equation. If successful they may well change my opinion of things like the space station – things in low earth orbit. If you could fly the parts up there cheaply and ferry people back and forth – then it might just be another expensive government lab where micro-gravity experiments could be done (in isolation too). But getting people further out to the Moon or Mars would still be very expensive – I doubt I’d be convinced.
Besides, the more experience we have with sending objects up the better prepared we will be when the aliens arrive. 😉
The Art Of Noise June 1, 2009
Posted by gordonwatts in science.4 comments
We like to say that science is deterministic. You do x,y, and z, and then a will happen. I 100% agree with this. Except when it comes to noise.
What is noise? This is part of the problem – it comes in so many forms. Noise is like a weed. We call a plant a weed when it doesn’t belong. Got grass in your garden? It is a weed. Got grass in your lawn? Definitely now a weed. Is your cell phone making buzz-buzz-buzz sounds on your computer speakers? Noise! Are you using the cell phone signal to talk? Not noise!
But electronic noise is more insidious than the weed analogy implies. If you have an iPhone you can do the following experiment. Sit down at a computer that has separate speakers – the key is that an exposed wire connects the speakers to the computer. Place the iPhone near the speakers*. Wait for a few minutes. Now you’ll hear the buzz-buzz-buzz. So you know the source – you put the iPhone near the computer and it makes that buzzing sound. Lets say you didn’t know that. You had the iPhone in your pocket, and you walked up to your computer – it would start buzzing. You’d notice that every time you were near you’d get the buzzing. You might conclude you were the source of noise (talk about a loud heartbeat!!). You’d be both right and wrong.
The beauty of science is that if you do a, b, and c, then x will happen. Electronic noise is very much about science – if you do x, y, and z then you’ll get noise. The problem is it is very very difficult to determine what x, y, and z are. Above x might be “walk near your computer”, and y might be “have iPhone in your pocket”. You might get x right away, and then give up on fixing the speaker. A computer is no good if you can’t be near it! If you figured out y you might be able to do something different (like leaving your iPhone across the room, or better shield your speakers).
This is why finding and diagnosing electronic noise is an art.
I was on shift for two nights last week. On Friday night we saw a strange noise pattern in D0’s calorimeter. The pattern of noise is called high noon. And it was in four crates of electronics – typical high noon noise is never seen in more than one crate at a time. Experts were here until 2am trying to figure out what the source was. No clue. Eventually they went home – have to investigate the next morning when everyone was awake. Impact on data seemed to be minimal, and the automatic noise detection algorithms seemed perfectly capable of detecting this type of noise.
There were other problems with the detector that night too. A muon phototube kept looking synchronization with the readout. Each time this happens an automatic system would rap its knuckles and bring it back into line. Finally, after it had lost sync about 20 times, we decided to hit it with a hammer: reset it. And lo… the noise in the calorimeter, which had been mocking us for the last 6 hours disappeared.
Keep in mind – the muon system is meters away from the calorimeter system. They share no common electronics – not even racks of electronics are shared by the two. But the muon system was acting like the cell phone, transmitting electronic noise and the calorimeter was acting like an antenna – the speaker wire – and picking up the noise. D0 has been running since 2001. As far as I know, no one had seen this particular failure mode before.
If anyone tells you science isn’t an art – that there is no room for creativity – they are full of it. There are a lot of unknowns in unsolved problems – and it takes some creativity to guess what those unknowns are. And that is also the point behind reproducibility – to make sure you found x, y, and z that produce effect a.
Long live art!
* BTW, the iPhone isn’t the only one that will do this, it is just the worst offender of any of the smart phones I’ve seen. And easy way to fix it is place the iPhone on some tin foil. I’m not sure if that is because the tin foil becomes an antenna and so the iPhone needs less power to communicate with the tower or because it provides some shielding. But it works. 🙂 See? Art!
2009. Ready or not January 2, 2009
Posted by gordonwatts in ATLAS, CERN, D0, Fermilab, LHC, politics, science.4 comments
We’ve made it through the first day of 2009. I have mixed feelings about this coming year.
- Federal Science Funding Levels. The economy is crashing down around our ears. Business responds quickly (layoffs :() – government is a bit slower. If things followed their natural course of action that would mean science funding, along with everything else, will take yet another hit. However, the incoming Obama administration seems to be committed to spending the USA’s way out of this recession, so in the end funding might not change very much. I am hopeful that hard sciences funding will remain at least stable.
- Federal Science Funding Directions. Climate change is what the Obama administration is focused on. There is a good chance that if you are researching something connected with climate change you may have access to increased funding opportunities. I would expect a funding profile similar to NIH’s funding during its years of increase. I would like to think that funding will spill over into the physical sciences – it should because there are connections between the physical sciences and clean air technologies. All of this is applied scientific research. I hope that the pure research funding gets an increase as well, as an investment in this countries future (particle physics is pure research, of course). I’m feeling neutral here.
- Federal Science. Obama’s science team is just a BLAST of fresh air when compared to the current administration’s. After all, his DOE nominee is a Nobel prize winning experimental physicist. Even if the science advisor isn’t elevated to a cabinet position (PDF), there will be someone in the room that knows a great deal about science, research, and how it is done. Even if there are cuts to science funding, I’m very hopeful there will be intelligent cuts rather that unscientifically motivated cuts. I’m very hopeful in this respect.
- State Universities. The economy in states is depressing. Some states, like my own (Washington) that rely on sales tax are being hit hard and very fast. State universities can’t escape that, obviously, and my university is no exception. Unfortunately, this usually translates to reduced raises, inability to counter offers from outside, reduced support for research, etc. In our own department I wouldn’t be surprised if some people left for other universities that, for whatever reason, were able to make good offers in this awful climate. There is, in fact, already evidence this is happening. The only consolation is most universities are in the same boat, and so most of them are having similar problems. I know less about private universities, but I do know the endowments of many of them are also having difficulty. I’m very downbeat about this: it will be a rough two years at least, I think.
- My Science. When it comes to the Tevatron and the LHC… Well, I see no reason that the Tevatron shouldn’t continue to break records in luminosity (they just broke one earlier this week). And the experiments will continue to be flooded with data. While it is possible for one experiment or the other to have a catastrophic failure, I doubt that will happen. And they should continue to produce papers and science at a furious rate. I also am looking forward to real LHC collision data this year. While I hope it will be at the full 14 TeV, I suspect it is more likely to be at 2 TeV, just a hair above the Tevatron’s luminosity. We’ll hopefully know what the machine scientists think about that sometime in February. I’m really hopeful about this.
- New Years Resolutions. Well, I made only one. That way I have a hope of keeping it: make bread more often. 🙂 I think there is a chance that I will keep this one. Especially now that I’ve said it publically. 🙂
Of course, this should also be a fun year, as noted by the Beacon News:
Frustrated with their failed attempt to destroy the world in 2008, the scientists at Fermilab and their counterparts at Switzerland’s CERN physics lab resolve to perfect their new device, the Large Planet-Sucking Black-Hole-o-Tron.
Here is to another great year of data collection and science at the Tevatron and first collision data at the LHC!
Dance your Science December 8, 2008
Posted by gordonwatts in life, science.1 comment so far
Apparently, the Association for the Advancement of Science put together a contest for Ph.D. research: “Dance your Ph.D.”:
The contest is open to anyone who has (or is pursuing) a Ph.D. in any scientific field, such as physics, chemistry, biology, psychology, anthropology, or in science-related fields such as mathematics, engineering, linguistics, bioethics, the history of science, etc. regardless of whether you’ve remained in academia.
1. Make a video of your own Ph.D. dance.
2. Post the video on YouTube. (Some tips.) In the video’s information box, include your Ph.D. title and any relevant information (scientific or artistic) that you’d like to share.
All the entries and the four winners can be found at this page. There was only one entry in physics – astrophysics – oh well.
Something Smells Funny Out There November 25, 2008
Posted by gordonwatts in science.1 comment so far
Overbye has a good article in the Science Times today. It discusses all the recent results from satellite experiments that, given proper interpretation, may indicate further direct “particle” evidence for dark matter. That would be huge!
The reason I mention the article, however, is Overbye puts in a bunch relating to the competition between the various theorists who are trying to explain the dark matter signals. For example,
That theory was called “a delightful castle in the sky” by Dr. Gordon Kane, who said he was glad it kept Dr. Arkani-Hamed and his colleagues busy and diverted them from competing with him.
And there are a bunch more comments similar to that. Worth the read for the science as well as some of the story about how science is done. It ends with a great quote by ex-UW postdoc, who is now a professor at NYU, Neil Weiner:
In a year or two, we’ll either not be talking about this idea at all, or it will be all we’re talking about.
I love his optimism. If we actually know something definitive in a year or two that would be great! Sadly, in that time scale I don’t think the LHC will have very much to contribute (perhaps in two??)!
The Atom Smashers November 24, 2008
Posted by gordonwatts in Fermilab, science, USA.10 comments
The Atom Smashers (http://www.pbs.org/independentlens/atomsmashers/) will show on PBS on Tuesday night. It looks like it focuses on Fermilab and the particle physics research occurring there. I like their tag line:
After funding cut backs, Fermilab—a premier U.S. government research laboratory focusing on particle physics—is struggling to survive. Physics, politics and international competition collide as scientists race to find one of the most elusive sub-atomic particles ever theorized: the Higgs boson.
Elsewhere on the site the film makers claim they don’t try to answer questions – but rather to get you to "think":
We hope this film will raise the awareness of America’s strange relationship with science. We don’t attempt to answer questions in our film, but rather to raise them. Is this research worth doing? Should we care about it? Should the U.S. participate in it or let it get done elsewhere? Also, we hope to help demystify science and scientists. We’d love it if a viewer came away thinking, “You know, those scientists are not really that different from me."
That last line being one of the main points of this blog!! Leave a comment if you get a chance to see it – I’d like to know what you think!
This show is part of PBS’ Independent Lens project. I have no idea if it will be available online. I hope so as I don’t have a TV receiver (their videos online are all very short, so I might be out of luck)!
P.S. Sorry about the links (and lack of them) – the computer I’m on doesn’t have my normal blogging software and so is a pain-in-the-butt to use.
Life as a Physicist 