I’ve posted about Summit before, so not much new, but my talk with Rafael from the NYSE (mostly Rafael’s talk), drew a standing room only crowd (correction: no standing room at all!), which was nice to see again. Red Hat Summit was pretty much the usual; food was pretty terrible (or perhaps San Diego’s Summit was just delicious) but still enjoyed hanging out with friends very much. I also hosted a Campground session afterwards with some of the Red Hat Systems Management guys. Boston was extremely nice, and even had it’s own mini-heatwave to make us North Carolina folks feel like home. Found a little free time to check out the New England Aquarium, which has a fantastic penguin exhibit. It’s not as fancy as Shedd in Chicago (no Beluga whales or fancy theater show), but it’s really good.

Thursday I spoke at the configuration management summit at USENIX (held at the Sheraton, where Summit was 2 years ago) — along with Mark Burgess from cfengine, Aaron Peterson from chef/opscode, and Narayan Desai from bcfg2 and Argonne National Labs. We did regular talks and also the panel thing, which was a bit like I imagine testifying before Congress must be like. No, Mr. Senator, I do not use OS/2! During the afternoon Barcamp sections I also presented on some other systems management tool projects I’ve worked on in the past, which I hope was useful to folks as they fill in the other gaps of their systems management tool suites. It was hot getting back to South Boston — no AC in the T stations — but the underground electric bus tunnels were pretty awesome. Moral of the story though — if you are holding a conference, don’t have it at the same time as vendor conference and an O’Reilly conference devoted to the same subject. 60 or so people came, but there could have easily been a lot more.

Incidentally I’m going to be leaving Puppet Labs after this week, for a new position that I’m very excited about (more details to be announced later). As this signals my exit from open source software and systems management software, I intend to do something heretical that I’ve been unable to do for some time. Maybe I’ll buy an XBox or install Windows 7. Maybe I’ll put a hat on a penguin. Anyway, I’m looking forward to it a ton.

I’m just about done uploading pictures from SouthEast Linux Fest 2010. As Ian Weller mentioned, we had a pretty good random photo walk in the middle of it, and I think that’s a fine conference meme to continue in the future.

But seriously. Damn. What a huge conference for being in Spartanburg, SC. No disrepect, home of the mighty Wofford Terriers, but wow, we did not expect this from you. At all. The South East Linux Fest crew — David, Jeremy, scores of volunteers — well done. You shame Raleigh, Durham, and RTP. You bring envy to Atlanta. You bring folks travelling from near and far. SELF Folks, you not only put my previous attempt at Evil Robot Conference to a level of shame where it looks like kickball to the scale of the world cup, but were also incredible hosts to all of us. Thank you immensely.

I gave a talk about Puppet and why datacenter automation is important, with a few nods to previous projects I’ve worked on (cobbler, Func) thrown in — pretty well received I think, about 60 people were there and there were a lot of good questions. There were a lot of other good talks though, I’m most excited about Ian’s datanommer as it inherits a lot of legacy (euphemism intended) from some past projects of my own. I was also quite happy to see all the new folks who turned out for the FAD– quite a good difference from the usual FudCON where we typically already know most of everyone. I think this underscored, very very well, how much of a democratic and friends-helping-friends kind of community Fedora (and Linux) can be. Major points for reaching out and keeping the barrier to contribution down. Ubuntu can have some very polished initiatives, but they need to work at being less centralized, that is for certain. I hope they learn that lesson as they grow.

It was also nice to talk with lots of folks I had met at previous conferences, plus the cool folks at OpenNMS, ZenOSS, sqlite, and XenSource. I will certainty be back next year. You should to. Especially if it’s right next to the Krispy Kreme again.

Cinnamon Apple Filled, I am glad you are back.

The scenerario is this — lots of apps produce error messages. When the user can’t understand the error message or warning, that’s a problem. This could be, for instance, an error message that says “Your veeblefritzer is demodulated”. What does that mean? Or you may have an error message you have a bug report on, but you don’t know that it’s occurring as much as it actually is. All of these errors in an app can increase user frustration to the point where they’ll (maybe) stop using your application — and initial impressions are important. Users will almost always google for information about these problems.

There’s a pretty good concept I latched onto called the “30 minute rule”. It means after 30 minutes of fighting with anything, your users are busy, and will probably give up. Often that means they need to achieve complete success in that 30 minutes, or at least see there’s some light at the end of the tunnel. So minimizing those annoyances is pretty darn key for any project.

So here’s the trick — if you filter through Google Analytics search results (I really should write a script to automate this), removing the genuine search terms, you can get a pretty good idea about what messages from your application are the most confusing. You can see how often they are searched for, over time, for arbitrary intervals. It helps especially if you have the Analytics tool installed in your bug tracker. It would be even better if it were installed on a mailing list archive, though I think in most cases that’s not going to happen.

Another thing some apps try to do is submit tracebacks to a server — which also is a good idea, but I’d really like to see a good OSS hosted service for OSS apps (i.e. free) for that kind of purpose. i.e. not something kernel or firefox specific, but that I can insert into any application and get error counts for each log line. Though that doesn’t neccessarily solve the case of confusing application messages which aren’t tracebacks. You could also write something like an OSS Splunk that submitted logs to a central server, though it would be very difficult to ensure the logs were not containing proprietary information — opt-in would be pretty low.

I think there’s lots of statistical noise here and it’s probably not perfect, but everything you can do helps.

The Google trick is pretty useful, but not necessarily to everyone … what do people think about open & generic traceback and error service? (Perhaps if it was configurable on a per-application opt-in basis and had very tight controls on who could read the output?)

Maybe call it “ErrorHub”.

Also, for those who haven’t got enough of hearing me talk, yesterday I was on This Week In Cloud Computing with Jared Goralnick, CEO of Away Find. It has a bit of a different format than the usual podcast, and was a lot of fun. Click the link if you’d like to watch/listen.

Later this month, if you can’t wait, I’m giving a talk in the Cloud Track of WWW 2010 in Raleigh.

The docs site is based off of the Rails Guides and is generated from Markdown, which is now my new favorite way to generate documentation. A bonus is that Wikis like Redmine already support Markdown, so there is almost no conversion to move things between them.

Whether or not linking the full context of the book was actually legal, I should say that without even reading the text, I have two better links for you.

here and here.

If I post to my blog, my blog posts to my twitter that I’ve updated by blog, and it shows up on my blog that I’ve twittered about all of that (self reference warning!). It’s all rather crazy, and I imagine I use less of these things than many people (my PS3 is not connected to the tweetosphere, for instance). Currently if I post a blog entry, someone may comment on it on facebook, and someone may “like” a tweet on buzz. Thus commenters are isolated, often having no perception of the isolation. Metadata is also lost in translation and there’s often no way to know the original source of the content.

The analogy of blood types also seems to apply. Facebook is A+, a universal receptor, but is not a very good donor. Flickr is more of a universal donor, but can’t accept input very well. Twitter does both well, but not at a very high level of data. The idea of interacting with eleventy billion different websites and not being able to visualize this graph is conceptually interesting and patent worthy. And I’m a Computer Science guy who specializes in the meta problem of software that manages other software. Imagine the problem if you’re into some other humanities subject, like, say, Underwater Basketweaving or Necromancy. Or, say, if you double majored in both. Problems.

I’m also wondering how close we are to getting a few possible social network feedback loops set up, where one network updates another network saying it just updated itself on the same network (ad nauseum), and possible photocopier degradation in that signal due to metadata loss that makes it hard to detect the events were uncorrelated. It seems like it’s probably possible today, and probably not all sites are insulated against this.

I am also waiting for the day when Twitter can communicate with Buzz over Farmville protocol.

Clearly, the answer has to be a really big standards committee. I’m sure of it. Or perhaps a Philosophy Thesis.

A trip to San Francisco to sit on our Puppet Training class gave me an opportunity to visit a place I had long known about on Flickr… the San Francisco Airport BART Station. If you’re into traffic cone photography, you’re already in on it, I’m sure.

Puppet training was remarkably informative … not only watching Teyo teach about exported resources, storeconfigs, and custom functions/facts (and seeing Bruce plow through creation of new Puppet types in Ruby), but I also had the chance to talk to numerous happy users about future things we can do with Puppet. Amusingly, I ran into quite a few Cobbler users too, which definitely helps as an icebreaker. It was very nice meeting and talking with everyone.

Airplane landed a little late so I didn’t get to see the Golden Gate, but maybe next time. A day later and I could have seen the Maverick surfing competition. (John McCain should announce. It would be hilarious.)

So, I needed something to scan and past OSS things I’ve been involved with were logical first targets. To index the history on Cobbler into a concise video, I ran the following:

gource -s 0.03 --auto-skip-seconds 0.1 --file-idle-time 500 --max-files 500 --multi-sampling -1280x720 --stop-at-end   --output-ppm-stream - | ffmpeg -y -b 3000K -r 24 -f image2pipe -vcodec ppm -i - -vcodec mpeg4 gource.mp4

Want to run this yourself? You will likely have to build gource from source. I’ll warn you that building from source involves installing a ton of deps, though all are in Ubuntu 9.10, and once ./configure finally passes it does build fast. Fedora was downlevel with respect to ftgl, and compiling ftgl from source was difficult, hence the Ubuntu usage. The parameters I use above result in a large video (75MB) but are intended for YouTube HD.

The result is below. Note that I didn’t keep my source control commit attribution for the first couple of years on the project (lesson learned in how to use git!), I used to do development on devel and switched to master (this video shows master), and koan was grafted into the cobbler tree late in the game. Early on, I committed from two different user IDs. As a result, the video is not perfect — things tend to “pop” into view as releases happen. You’ll see the first outside attribution happen about 1/2 way through, though of course this was happening much much earlier. Still, the acceleration at the end, I think, means we achieved something pretty decent. Not all projects do.

Perhaps this is a start of a good meme. Get your code up on YouTube. Show us the life of your code and who you collaborate with.

Watch

I can see gource being immediately useful for a one major purpose. When evaluating OSS software for use in business, you always need to know if the community is solid and self sustaining. This allows you to watch a short video and find out. Coupled with looking through the mailing list archives, that’s a pretty good check. It can also help identify interesting patterns of large scale refactoring, new development, or stagnation.

Gource may also a great way to explain open source to people who don’t immediately understand how collaboration can work, and how contributors come and go.

That is what I call good TV. It is also rather trippy to look at. Please turn up the Floyd.

If we had a free supercomputer and infinite development time, my ultimate dream visualization would be a 20×20 foot wall section of these graphs, showing multiple projects side by side, with developers flying between projects. Who flies between projects? Is that common? What are the clustering patterns of these projects that share contributors? Where are the hubs and spokes? Are the hubs bigger projects than the spokes? (Can we get that in 3D?). There is something to be learned here, even if we don’t know what that is.

Install Gource. Let’s see your project video.