Life as a Physicist

Presentation September 20, 2009

Ok. Really. This is my last post of Video for a while. Ever since I started the DeepTalk project I’ve started to be much more aware of how conference data is put out on the web. So it has become a bit of a soap-box for me. But this is the last one for a while, I promise.

During my last several posts on this there have been a bunch of comments on how other conferences have presented their video online. I thought I’d give you my opinion.

• Pycon 2009 – the annual Python conference. At first I was hopeful about this – the web page is quite nice and you’ll notice right at the top there is a nice iCal link so you can download the schedule. However, the schedule is just that – a schedule. You can’t get access to the links to the talks or video from there. Associated with the web page is a RSS feed too – which is excellent – I could now use my pod-cast software (any software should be able to read it) and I could download the audio of the whole event. Sweet. However, there is no way to connect the slides and the video or audio together via a program (as far as I can tell). The video looks like it is all archived on blip.tv. The beauty of this system is that it makes files availible in lots of formats (see this talk, click on the “files and links” to see). AND there is a small little RSS link at the bottom – so I can get all the talks down as video to my podcast software (the default seems to be the MP4 format, which satisfies most of my requirements as a good video format). So this conference has made its schedule available in a standard format (iCal), made all of its videos available in a standard format (blip.tv). I’d like to see some integration between the two so that one could find the slides, abstract, and video together, using a program.
• Strings ‘07 Conference – a conference on strings. The conference website is basically a series of static web pages – including the schedule (I’ve extracted that page – but you can get to it by looking at the home page –> Scientific Program –> Speakers&Titles). There are links to the slides and Video. The video is in MP4 format (fantastic!). None of this is discoverable, unfortunately, by a program – you would have to scrape the web page in order to find it. Chimpanzee, who has left a lot of comments on these video posting, has done some work with this conference, putting it in iTunes as a show. Unfortunately, unless you have iTunes installed, this is not very useful as it brings you to an Apple page that asks you to download and install ITunes. However, Chimpanzee did put this on blip.tv as a several shows (one show per day – I think from the point of view of subscribing I’d have preferred a single show for the whole conference). Also, the nice RSS feeds to blip.tv are well hidden. So, well done with mp4 and PDF files up there. The blip.tv solution is quite nice, again. The static web page that links them together isn’t so good – it isn’t very discoverable, unfortunately.
• Lepton-Photon 2009 – The agenda is posted in the standard agenda software in use in HEP, Indico, which makes it easily exportable. Each talk has a link to the PDF as well as a Video link. Unfortunately, the Video leads to a RealMedia file – which my open source tools cannot play. So the video format doesn’t pass muster.

I am pleasantly surprised by blip.tv. It looks like a very nice service. I have no idea what their business model is. The good news is that people won’t watch most talks from a physics conference very much – so they will require very little bandwidth.

No conference gets it quite right (IMHO), but they all come close. From my point-of-view, combining Indico with blip.tv seems like a fairly ideal solution given current technology constraints.

Two quick notes. First, there has been a hope that perhaps HTML5 would standardize a single video format – and we could all just depend on all browsers running it without having to install plugins like the security-ridden Flash or RealMedia. This is not to be, however. There is an excellent blog series for those of you who want to know what is happening to HTML5 that I stumbled on. This posting makes it clear that a preferred video format no longer exists in the standard (for details, see the change log for the standard).

Second, I keep holding up Indico as a nice way to post meeting agendas. But perhaps there is a standard for this sort of thing? A microformat or perhaps something form the Semantic Web? Then Indico (and everyone else) could produce that for various tools to parse. I only took a brief search, but didn’t find anything.

Time shifting Video: Recording September 6, 2009

In my first post on video there were a few comments on the effort required to record the video in the room. The basic question from Chip was the following:

The question I have is the on-site effort and expense. Take the PyCon setup: any clue what synch software they used? Because of the zooming, they had a person with a camera. Maybe I’ve not noticed, but having the slides small and the person large is an interesting idea. With the slides separately available in full-resolution, one could use the on-screen slide images as just a key to tell you when to actually click on the full size ones. Usually, it’s the other way with the person being very small and the slides larger. In fact, pedagogically, having the viewer then have to manipulate something during the talk would keep them in the game, so to speak.

Ok, there are several questions. First point: I want to be able to view this stuff on my MP3 player – so “keeping someone in the game” is not what I have in mind for that sort of viewing.

Now, the more important thing: cost of recording. There was a reply to this from Tim:

Why don’t you just record the video from the camera and the input to the projector? This would seem like an easy way to get synchronized slides.

For some dumb reason that hadn’t occurred to me – get a VGA splitting and hook its input up to your computer. The Lepton-Photon folks seem to have basically done that:

Judging from the quality of the slides (which is worse here because this was a low resolution image), I’d guess they had a dedicated camera recording the slides rather than actually looking at the computer output. A second stream focused on the presenter and they can use common post-processing tools to combine the two streams as they have above. In fact, the above is from a real-time stream. I don’t know what tool they used, but I can think of a few open-source ones that wouldn’t have too much difficulty as long as you had a decent CPU behind you. On caveat here: in a production environment I have no idea how hard it is to capture two streams and keep them in sync. If they are on two computers they you need software to make sure they start at the same time. Or if there is a glitch and you loose one, etc.

Chip also asks the key question:

what did it cost?

I’m not sure what the biggest expense for these things is – but it is usually culprit is the person doing the work – so I’ll go with that. To record a conference I assume you need to setup the video, run the system while it is recording, and then post-process the video to make it available on the web. The post processing could be fairly time consuming: you have to find where each talk ends and the next one begins, cut the talks, render the final video, etc.

Thinking about this, it seems like one could invest a little money up front and perhaps drop the price quite a bit. First, making software to record the two streams and keeping track of the sync can’t be too hard to write. On the windows platform I’ve seen plenty of samples using video and doing real-time interpretation. Basically, at the end of the day you would want two files with synchronization information: one with video focused on the slides, and the other on the person (with a decent audio pickup!)

If one wants to stream the conference live – that is harder. I don’t know enough about streaming technology to know how it would fit in above without impacting the timing – which is fairly important for the next step.

A human could probably recognize almost the complete structure of the conference from the slide stream alone. I suspect we could write a computer program to do something similar. Especially if we also handed the computer program the PDF’s of all the talks. Image comparison is probably decent enough that it could match almost every slide to the slide stream. As a result you’d get a complete set of the timings for the conference – when the title slide when up, when the last slide was reached, when the next talk started. Heck, even when every single slide was transitioned. You could then use these timings to automatically split the video into talk-by-talk video files. Or generate a timing file with detailed information (I’d love slide-by-slide timing for my deeptalk project). During this step you could also combine the two streams, much as is done in the above live stream I recorded. You could even discard the slide stream and put high quality images from the PDF in its place.

I doubt this would be perfect, but I bet it would get you 90% of the way there. It would have trouble at the edges – before the conference started, for example. Or if someone gives a talk with no slides or slides that are very different from the ones it is given to parse. But, heck, that is to be fixed in Version 2.0. I do not know if 90% is good enough for a project like this.

Seems like a perfect small inter-disciplinary project between CS and physics (with a small grant for one year of work). I wonder how far fetched this is?

Time shifting a Conference: Video Formats August 31, 2009

I got a few interesting comments when I wrote about discoverability of conference video the other day. I’ve been away for a week so instead I thought I’d write a whole new post. But I wanted to change the topic a little bit – motivated by two things: Lepton/Photon and a comment by chimpanzee. And sorry if this gets a little technical… I’m on a rant here!

First Lepton/Photon. The conference is over now. Check out the agenda:

This got me very excited! Look at that – a link to the video! Yes! Finally! This is what I’d been hoping for in the last post – making it easy to find video for talks! Woo hoo!

Click on it and… they require the RealPlayer to be installed. Bummer. The RealVideo format is a proprietary video format. You have to have RealPlayer installed in order to use it. There are some open source implementations of RealVideo out there, but I think they do an older version of the RealVideo format (for example, VLC claims to know what to do with the RealVideo streams, but falls over before it plays anything). For most people that may not be a big deal – just install RealVideo. I personally have a problem with the RV software. But for the purposes of this post my problem is I can’t download the video stream and pack it up into my mp3 player (I have a Zune). For $40 bucks, I might be able to do it for an iPod (not clear from their website).

If anyone knows a way to play those above files without having to use the RealPlayer software, I’d love to know!

Second, there was the comment by chimpanzee. I recommend reading the fully thing – I’m going to cherry pick for this post and stick with video formats:

For those who love “competition” when it comes to codecs – the competitive war just heated up. Google, just bought On2/VP8 and apparently is going to Open Source VP8.

I know nothing about VP8 other than what is on on2’s web site. It is currently a proprietary video codec. And chimpanzee says in his comment it seems reasonable that Google would open source it. For any of you that have downloaded video files from the internet (Bad! Bad!) you already know there are a plethora of video formats out there and one often needs to install lots of different codecs to get them all to play. VP8 will not solve this, at least not in the near term (<5 years). But this got me thinking – we can solve this now, can’t we?

So, I have a modest proposal. Physics conferences should archive the video of their conferences in a format that plays natively on the n most used operating systems out-of-the-box (where n is > …):

• Linux – this is funny. In HEP we mostly use Scientific Linux. This is not optimized for watching video. So choose the most popular distro – Ubuntu I think? I used to have a distro of that running on my laptop but had to delete it for space reasons, so I couldn’t test it…
• Mac – A recent version of OSX running on Intel Mac’s.
• WIndows – this is tricky. XP is the most popular version out there, however the OS is quite old and plays almost nothing modern out of the box. Plug-ins are available (including my favorite – wow I hate the new sourceforge) that will allow it to play almost anything. So that isn’t good. Vista, I think, is in the same boat – it doesn’t have much in the way extra codecs. W7, however, supports most formats I’ve seen out there (I couldn’t find the docs on microsoft site, but I did find this which matches my experience with the release candidate). So, I think we are almost forced to pick Windows 7 for the Windows branches.
• iPhone/mobile – this I’m not too worried about. Usually if the host system can play it (iTunes, Zune, etc.) then it can be transcoded and placed on the moble device.
• Others?

Given all this, it strikes me that MP4 is the only video format that comfortably fits into this. There are plenty of open source tools – heck, tools in general – that allow you to manipulate it to your hearts content. Play it on your mobile player, your TV, heck, most modern burner software can burn it to a DVD if you want. Further, if you are like me, and want to manipulate the video for whatever reasons, well, you can because there are so many tools.

So, that is my modest proposal for archiving.

Streaming is more complex. I don’t know as much about streaming. I’d be inclined to vote for MP4, but I’m not sure how well it works in a streaming protocol.

I have to sneak one vacation picture in… The French countryside is pretty amazing… This is near the salt-flats out side of Rochelle.