The Ultimate Logbook January 8, 2011
Posted by gordonwatts in logbooks, physics life.trackback
I couldn’t leave this alone. I mentioned the ultimate logbook in my last posting. This is the logbook that would record everything you did and archive it.
It isn’t difficult. The web already has a perfect data format for this – Atom (or RSS). Just imagine. Each source code repository you commit to would publish a feed of all of your changes (with a time stamp, of course!) in the Atom format. Heck, your computer could keep track of what files you edited and publish a list of those too (many cloud storage services already do do this). Make a plot in ROOT? Sure! A feed could be published. Ran a batch job? The command you used for submission could be polished.
Then you need something central that is polling those RSS feeds with some frequency, gathering the data, and archiving it. Oh, and perhaps even making it available for easy use.
Actually, there is a service that does this already. Facebook. Sure! Just tell it about every RSS feed and it will suck that data in. Some of you are probably reading this on Facebook – and this posting got there because I told Facebook about this blog’s Atom feed and it sucked the data in.
Of course, having a write-only repository of everything you did is a little less than useful. You need a powerful search engine to bring the data you are interested in back out. Especially because a lot of that data is just a random command which contains no obvious indication of what you were working on (i.e. no meta-data).
And finally, at least for me, I don’t really want something that is static. Rarely is there a project that I’m finished with and I can neatly wrap it up and move on. Heck, there are projects I put down and pick up again many months later. This ultimate logbook doesn’t really support that.
Perhaps it is best to split the functions. Call this a ultimate logbook a daily log instead, and then keep separate bits of paper where you do your thinking… Awww heck, right back to where we started!
BTW, if you think Facebook might be interesting as a solution here, remember several things. First, as far as I can tell, there is no way to search your comments or posts. Second, you might get ‘Zuckenberged’ – that is, the privacy settings might get changed and your logbook might become totally public.
Life as a Physicist 
What I hate about Facebook (and also other web-based solutions for log-book) is that I don’t own the data. I don’t care that Mr.Facebook can read it. But he can remove, modify it,… I want a solution where I control the data. There could be a synchronised copy somewhere on the server, that’s fine. But I should own the master copy. That’s why I, for example, publish my photographs on Picasa (the master is on my disk, the copy is on the Picasa site) and put only links on FB.
Unfortunately, more and more applications (especially for smartphones) are like that – even when it’s completely useless, the data are on some server somewhere.
This sunday morning I was glad google owns me. Not only do they index all my email, keep track of all my google queries (and which results I eventually clicked on) but also these days they index all my instant messenger conversations.
Poking around some old code I realised I had to find a “hack” from months ago. I knew I had discussed the problem and solution with someone (what happens if you cast a uint to int, store it in a ttree as int, but want to recover the original uint in python code) so all I had was “months ago, convert python unsigned integer”, low and behold I found it within seconds! Try doing that with a paper logbook!