EVO SUCKS July 14, 2008
Posted by gordonwatts in university.trackback
This post should be titled “Congratulations to John!” and all I should be saying in it is “John passed his General exam! This is the second of three exams you have to pass in order to get your Ph.D. at UW. Now it is on to the Higgs hunt at D0!” or something similar.
Instead I got so angry at his General exam that I was writing 4 letter words all across my copy of his slides.
No fault of his. It was EVO‘s fault. EVO is a multi-point video conferencing system. That is, a bunch of people can join and we can all send and receive each other’s video. There were going to be three or four of us watching John’s general exam remotely, and this was the option we settled on. It is, after all, the video conferencing system that is being pushed the hardest for ATLAS.
What a train wreck. What a @*&#&@ disaster. First of all, it kept cutting out. After some undetermined period of time the data stream would drop, video would freeze, and then go away. And the sound would drop. It wouldn’t auto-reconnect. And I couldn’t just end the meeting – I had to restart the whole EVO app. And each time you restart this java-based app, it seems to go and redownload everything from the net. What the heck? Right when you need it to be fast!
Then there was the 20 minutes or so when I was trying to get John to talk a little faster and he couldn’t hear me. Actually, no one at that end could hear me. Finally, using Skype’s instant messaging I was able to get the operator at the other end to reset the connection. Then it worked again.
And during the closed session part of the exam the operator had to leave the room. Not 5 minutes after he left the connection at UW dropped. I had to call up various phones in the department until I found someone that would run up to the 5th floor and notify the guy that he needed to get back in the room (thanks Ivy!). By that time it was just me – so we just used Skype, which worked without flaw for the test of the talk.
EVO has a nice looking snazzy interface. It looks like the authors spent a bunch of time on the interface. I wish they had invested that kind of time in the actual infrastructure and making the conference work1 I could care less about the interface. And ATLAS really wants to make this the official version of video conferencing? EVO doesn’t even have built in echo-canceling – something pretty much everyone else does.
My normal video conferencing solution is ESNET‘s ad-hoc system. You usually have to pay for a client (a dedicated room machine runs a bit under $2K). Linux has a free client, and Polycom sells a $99 Windows client. The Linux client is noticeably lower quality, but works well enough. The others are fantastic. And it “just works”. However, according to ALTAS policy using it is not “open” — because the dial-up number is in the USA, not at CERN. So that is out for a future solution, apparently. Skype is out too — as soon as you do a call with more than 2 people you loose the ability to video conference. What else is out there that is cross platform?
I HATE EVO. What a disaster. I was doing John’s General this way because I was trying to save the $7K or so it would have cost myself and another committee member to fly back in the height of summer vacation travel. But not I’m not so sure $7K wouldn’t have been worth it for that 2.5 hour meeting. What a mess.
N.B. Yes, this is my second b**ch post in a row. Don’t know what is getting into me.
Life as a Physicist 
you think this desire of the HEP community to invent everything themselves (from scratch) is ever going to change? I struggle to believe that these days there arent solutions out there that you could at the very least extend.
I’ve not done much research for this — just enough to know that my favorite tool Skype can’t be used out-of-the-box. But I agree, there are others, and there must be something suitable. You want sort of a weak-social networking app connected with a full blow video conferencing client.
Dude,
Seems you had a bad experience and then drank too much coffee before posting. No problem. Send me a log of all the problems and I’ll forward them to EVO Support.
By the way, ATLAS is not pushing any one technology. I hope it did not appear that way in my ATLAS Week talk. We are just using what is available. EVO, when working, addresses the long list of requirements of HEP (multi-platform, no direct communication cost, no expensive hardware requirements, phone bridges, etc.) If Skype could do this reliably, that would be great. If you know of better tools, we could recommend that CERN IT try them out.
ESnet is a possible alternative, provided everyone in the collaboration can sign up and use it and provided we all get H.323 compatible systems (easy to do these days). Still, I doubt the DoE will want to support the whole world, again, as they have with EVO.
Concerning the download, that is Java Web Start. It is EVIL. The developers agree and will get rid of it soon (I have been told). A small applet must be downloaded every time. If there is a new version of the software available, it must be downloaded, as well. This worked during development, but needs to evolve to a Skype-like situation (“There is a new version available, shall I download it now or later?”).
Finally, i am surprised you like the EVO interface. I find it too complex. I’d like most of the buttons and menus hidden in the background. I like simple, like Google and Skype. I also do not like having both a web interface and a tool interface. I just want the latter (like Skype).
This year, the CERN IT guys are surveying the field for potential tools, including those with expensive licenses. If you have any proposals for alternatives, let them (me) know. Next year, we will be paying for the service, one way or another, so let’s choose the right one.
See ya,
Steve (hn-atlas-collaborativeTools@cern.ch)
Ummm, no. There wasn’t too much coffee involved. We went through 2 days of testing and setup before the exam. We did our best to be conservative with the setup. Even had the $200 buck CERN recommend echo canceling mic and speakers. I was so pissed off because the technology failed so badly.
The EVO interface is a bit dissonant – that is true, but it isn’t bad. I’ve used worse. It certianly says “designed by a programmer” – but I’m ok with that because I am a programmer. I am willing to forgive quite a bit if it works… but when it doesn’t work the snazzy GUI stuff just pissed me off because it seems like more time was spent working on the interface rather than on the core functionality.
The Skype interface is nice – simple and it works. EVO could have a simple interface if it just worked as well. 🙂 Having things to adjust (like bit rate) is helpful when trying to understand what is causing your sound to skip. In Skype this is done automatically.
I think ad-hoc is great. That is by far the most reliable system I’ve used (more than Skype). And supports multi-point conferencing. Skype does not do that, as far as I know. Or you have to pay – no idea how much that is or if it is even possible. And it is cross platform.
Other tools I have no idea about — I’ve not done much of a survey and, frankly, I just want a tool that works. I’m willing to invest some time and $$ in making it work – but I don’t want to pave this particular road.
[…] is a reaction to post EVO Sucks in another […]
> And each time you restart this java-based app, it seems to go and redownload everything from the net. What the heck? Right when you need it to be fast!
If you start WebStart application in the “offline” mode, it will not check for a new version and it will not re-install anything. I’ve written some instruction about it here:
Thanks! That is a nice work around until the developers fix that (which, according to Steve, they are planning on doing).
I totally agree with the reliability issue. Everything else goes out the door. I guess I have just had better experience.
One strange thing in the system that has screwed things up for me in the past are the Pandas. I frankly did not and still do not want to know about these things (which ought to always be in the background). However, once I had to learn to make sure the standard CERN room was picking up a standard CERN Panda, in order to avoid firewall issues. Once that was dealt with, all worked well.
I do highly suggest you send feedback (even a cut and paste of this blog) to and/or to . These guys do help and they might be able to figure out what was going on. For now, my opinion is that we need to make these things work, simultaneous to seeing if there are other options. Maybe a few bug-fixes (like the recent DB server update) are all that is needed. It won’t help John, but maybe the next…
O.K. enough for me. Thanks for letting me share your pain.
[…] July 22, 2008 Posted by gordonwatts in Uncategorized. trackback Well, I held EVO up the other day as something that wasn’t doing well. And Skype as something that was done right. Of course, today, using the new version of Skype, […]
We have had poor experience with EVO as well (people being cut out, huge amount of noise on the line, etc.). Here are some suggestions for the EVO developers:
1) We _only_ use EVO for voice, and not video. A lot of people do this, as a lot of time the video is basically superfluous when everyone already has the slides etc. I have a sense that a lot of bandwidth might be being reserved for video even when it isn’t being used. There should be an option for a voice-only meeting (or it should sense when there’s no video) and use the extra bandwidth for improving/optimizing the voice connection. Does EVO do this internally? If not, it _definitely_ needs to.
2) Often there is a problem with one (unknown) person’s connection (noise on their line, etc.). The organizer of the meeting should be able to go around the participants and mute / remove each one for 5 seconds, one by one, as a test, when problems like this crop up. This is an absolutely necessary addition.
Excellent questions. I’ll forward them to the EVO developers tomorrow. If I get a response I’ll either reference it here in the comments or cut/paste if I can’t do that.
And one more — EVO should have either a blog, webpage, or Hypernews where people can (semi-publicly) post complaints and questions, so that people can see how others are solving (or not solving) problems with it.
HI
CMS and ATLAS have been spending money on EVO for quite a while. I agree with Steve, the interface is too busy. I agree with Gordon, it’s too unreliable and frustrating. This is one of those tools that has to work 100% of the time because when it fails, it fails in a very public and personal way: when you’re trying to communicate. It costs you in a very big way that cannot be patched up later. The meeting is gone.
The HEPAP subpanel recommended that the agencies create a group to study Commercial products for video conferencing. No action on that. They too have been swept off their feet by VRVS/EVO. The whole world does video conferencing, but HEP decided it needed to invent its own. It’s really a shame: hard to support at the level required to make a real professional product, and has too much invested already to give it up.
Meanwhile, ESnet just works. Every time. From every personal computer platform and from essentially every meeting-room-sized setup. So, naturally it can’t be our preferred system!
Well, it’s our preferred solution. Even if there is some department function that, *because of HEP research reasons* (ie, away from home), I’m not able to attend, then I declare that an acceptable use of ESnet and I use it. Probably breaking about 12 federal laws?
So — if you are a member of ATLAS there is a hypernews group devoted to this stuff: https://hypernews.cern.ch/HyperNews/Atlas/get/collaborativeTools.html
But you need a password to get in.
I wrote to the developers about this, and this is what I got back (thanks!):
Hello Gordon,
firstly thank you to let us know about your bad experience. This is
the only way for us to help the users and improve the system.
I understand that the purpose of your blog is not compare
videoconference systems but let me give you some additional
information. EVO developer team is made of 3 FTE (Full Time
Equivalent). The Skype team is at least hundreds developers. EVO is a
multi video, multi user service provided at a world wide range.
Moreover EVO allows the user to connect different videoconference
standards (like H.323, SIP, AccessGrid and telephone) what Skype
doesn’t. It would be so easier to have a close system like Skype but
we are building a system to fit the need of our users (the HEP
community).
Around 600 unique users are participating every day to an average 150
meetings. A meeting could handle 100 users at the same time.
Now lets come back to your problem:
We know how frustrating could be such conference when something screw
the communication.
Could you tell me if #1 was connected on wifi or not? Do you remember
which Panda (EVO server) where were connected #1 and #3.
It is difficult to know what happened on the network afterwords.
Our main recommendations are:
1. push the HELP button (of Koala) and type a small description as
soon as you encounter a problem. It will attach the logs to your email
and we will be able to connect to your meeting and understand/fix the
issue.
2. if there is a really important meeting or conference you can ask
us in advance to monitor it (especially for the first time use).
3. if you can’t get a proper connection via a computer and internet
please use our telephone gateway (could be use from any standard
telephone and cellphone).
There are several possible reason to get audio drops:
1. network cut (if it is local we can’t fix it but if there’s a better
path to another panda Koala will automatically reconnect to a better
one).
2. too tiny bandwidth (automatically Koala reduce the video frame rate
of the transmission and switch to a more compressed audio coded). Some
of the solution could be done manually.
3. the CPU of the machine is overloaded. In such case close all the
unnecessary applications and stop to transmit video and display the
videos.
What I suggest is to make some test with us in the coming days to
check that machine/network that was used is correctly set and if we
can reproduce the encountered problems.
Regards,
Gregory Denis.
Dear Gordon,
I am sorry to hear that you had some issues during your last EVO session and I understand very well how much it could be frustrating.
Also, looking at the all comments, especially on the blog, I sorry to say that I found them completely unfair for the underfunded team who have been working hard to develop and provide a reliable EVO service for the HEP community (with their unique requirements). For a large parts, problems encountered by users are due to their local environment/set-up (network, PC/Mac, Microphone, H.323 device, ..) and not EVO. The EVO team members are always available to help and troubleshot the issue even if it is not EVO related and/or do some test sessions. It takes sometimes few minutes to troubleshot the issue but they have at least to be informed.
As Gregory mentioned, we run currently around 150 international meetings a day involving more than 550 sites (with even meeting involving more then 100 sites that will be impossible to run over and ad-hoc ESnet bridge for example). Under the current resource, it has been a challenge to maintain the service and develop/enhance the necessary parts continuously requested by the users community.
I have been working over the last 15 years on networking and collaborative tools together with my ESnet colleagues and similar teams from others National Research and Academic Networks, and I can tell you I don’t know anyone that will provide this level of service and development with the current level of manpower and resources. It is certainly one of the last differentiator between the HEP community and others disciplines.
As always, we always appreciate constructive remarks and critics.
Regards,
— Philippe Galvez
EVO Project Manager
I’ve had mixed experiences with EVO – it’s in my experience somewhat stabler than VRVS was, and its clients seem to be better at dealing with my hardware. And, thank heavens, you can book ad-hoc rooms. On the other hand I can’t say many good things about the whole Panda deal (the autoselection usually seems to avoid the closest choice – the best one for Chicago shouldn’t be at CERN!), and the A/V quality is so-so. There’s also a long-standing incompatibility between IPv6 and the SIP client which is kind of silly.
I know nothing of ESnet; I come from a world which didn’t use it, and I’ll speculate that was because we weren’t a DOE lab…
I am not very impressed with Phillipe’s response. There have been several comments that are in the constructive category and I see no response on his part regarding them. Fortunately I know that the other members of the EVO team are very helpful, and I am hopeful they will work toward making their product more reliable.
[…] usually don’t put stuff up here about things like Skype (unless in some other context) – but they just came out with a new version both for the Mac and Windows. Get it. The claim is […]
Hi,
I just happened to read this blog (my second time in few weeks) and just want to reply to “Anonymous”.
Sorry for the misunderstanding, I didn’t mean that all the post were not constructive but some certainly yes. As we all agree, it doesn’t help to say EVO doesn’t work, etc.. when we really don’t know the cause of problem. This remind me the time (still now), where for each issue we complained about the network even when we got a blue screen on Windows.
Coming back to some remarks:
o The blog is a good idea. I am not sure if we have to maintain it ourselves since it probably have to be monitored, etc.. and have limited time. Most of the users come directly to us for remarks and suggestions. For the blog, the aim will be mainly to share information between users.
o There are a lot of EVO meeting with audio only. The participant just doesn’t have to send video (which is not mandatory). Sometimes, they just use the share desktop to broadcast the desktop together with the audio. There is also a option in koala that when set on will reduce the video stream received/sent up to have only audio in case of high CPU or packet loss.
o The mute/unmute, video/audio, kick-off users, etc.. functions is already there since a while (you can even mute phone connections). By default the person who book the meeting has these privileges and can also extend this privileges to others. All is well described in the documentation..
Hope this help,
— Philippe
Hey, I was wondering does EVO use Skype? Does it use your network resources as a ‘Node’ Like Skype does?
Please Advise
Thanks
-Mike
No, it does not. EVO is strictly point-to-points, not p2p as is Skype. You can also disable that feature on Skype – there is some supernode setting that is possible. They had to do this in order to prevent some schools and other bussinesses from banning it.
Thanks Gordon!
CERN is moving to Vidyo. EVO will be phased out. Thank you CERN!