michaeldehaan.net

It’s Not Just About Llamas

Joining Reductive Labs

with 9 comments

As this thread indicates, I am very glad to announce I’ve joined Reductive Labs as their new Product Manager.

As most folks on Fedora Planet know, killer OSS communities and taking over the computer world with intelligent software are some of my favorite things. Puppet excels at both of these, and Reductive Labs is full of amazing people. I simply cannot think of any better place to be right now. The present state of things with Puppet is great, and there are lots of places it can go.

It is also great to be seeing many of the same nics on the lists and IRC again, and I plan to be running into a lot more of you at conferences as well. If anyone wants to talk shop about their Puppet usage or management challenges, I’ll always be up for it. I’m mpdehaan on #puppet and michael@reductivelabs.com over email.

Written by mpdehaan

February 1, 2010 at 11:41 pm

Posted in linux, puppet

x.org configuration for Dell 24″ Ultrasharp + Ubuntu 9.10

leave a comment »

In case this proves useful, the following works for me:

Section "ServerFlags"
        Option          "DontZap"               "false"
EndSection

Section "Screen"
Identifier "Default Screen"
Monitor "DELL U2410"
DefaultDepth 24
SubSection "Display"
Depth 24
Virtual 3200 2000
Modes "1900x1200" "1600x1200" "1280x1024" "1152x864" "1024x768" "800x600" "720x400" "640x480"
EndSubSection
EndSection

Section "Module"
	Load	"glx"
EndSection

Section "Device"
	Identifier	"Default Device"
	Driver	"fglrx"
EndSection

It seems like the proprietary driver (and not using the built-in Intel card) is necessary on my laptop. However, go-go-gadget Display Port.

Written by mpdehaan

January 28, 2010 at 10:11 pm

Posted in Uncategorized

Shine On You Crazy Tunnel

leave a comment »

Last time through O’Hare, I failed to find the shiny blinky concourse that seems to be the airport photo meme of the decade (at least on Flickr). Found it this time. It is in the tunnel between concourse B&C.

Written by mpdehaan

January 24, 2010 at 7:00 pm

Posted in Uncategorized

The Frogger

leave a comment »

Ground Kontrol has an excellent collection including the vector graphics Star Wars (wow, first time seeing one of these, fantastic), Asteroids, Missile Command, Frogger, Turbo Ms. Pacman, Galaga, Discs of Tron (my personal favorite), and some “newer” stuff I remember feeding many quarters into when I was growing up — Sunset Riders, X-Men, etc. Missed some of my racing/driving favorites like Pole Position, Bump N Jump, Stun Runner, Spy Hunter, Out Run, etc. They did have a most excellent selection of pinball tables, more than I’ve seen in one place, even “back in the day”. Pinball tables eat quarters FAST. Did I mention they have beer (and special holders for them?).

Written by mpdehaan

January 24, 2010 at 6:45 pm

Posted in Uncategorized

The New Yardbirds?

leave a comment »

Awesomeness via Make

Written by mpdehaan

January 19, 2010 at 8:09 pm

Posted in Uncategorized

Another Gource Video: Func

leave a comment »

As a follow up to my previous post, here’s a video from Func’s source control history, from conception until today. It differs a fair amount from Cobbler, mainly because commit attribution was preserved early on (due to using git am and related tools).

Written by mpdehaan

January 15, 2010 at 3:37 am

Posted in Uncategorized

Source Code Visualization with Gource.

with 8 comments

Gource is an amazing program for visualizing commit history in a git-based code project. What I like about it is that it can also show what areas of the project are active in an easy to understand way, to show whether there is community around a whole project or just aspects of it. What looks like a shiny useless visualization is, in fact, pretty useful stuff. I’ll get to that in a bit.

So, I needed something to scan and past OSS things I’ve been involved with were logical first targets. To index the history on Cobbler into a concise video, I ran the following:

gource -s 0.03 --auto-skip-seconds 0.1 --file-idle-time 500 --max-files 500 --multi-sampling -1280x720 --stop-at-end   --output-ppm-stream - | ffmpeg -y -b 3000K -r 24 -f image2pipe -vcodec ppm -i - -vcodec mpeg4 gource.mp4

Want to run this yourself? You will likely have to build gource from source. I’ll warn you that building from source involves installing a ton of deps, though all are in Ubuntu 9.10, and once ./configure finally passes it does build fast. Fedora was downlevel with respect to ftgl, and compiling ftgl from source was difficult, hence the Ubuntu usage. The parameters I use above result in a large video (75MB) but are intended for YouTube HD.

The result is below. Note that I didn’t keep my source control commit attribution for the first couple of years on the project (lesson learned in how to use git!), I used to do development on devel and switched to master (this video shows master), and koan was grafted into the cobbler tree late in the game. Early on, I committed from two different user IDs. As a result, the video is not perfect — things tend to “pop” into view as releases happen. You’ll see the first outside attribution happen about 1/2 way through, though of course this was happening much much earlier. Still, the acceleration at the end, I think, means we achieved something pretty decent. Not all projects do.

Perhaps this is a start of a good meme. Get your code up on YouTube. Show us the life of your code and who you collaborate with.

Watch

I can see gource being immediately useful for a one major purpose. When evaluating OSS software for use in business, you always need to know if the community is solid and self sustaining. This allows you to watch a short video and find out. Coupled with looking through the mailing list archives, that’s a pretty good check. It can also help identify interesting patterns of large scale refactoring, new development, or stagnation.

Gource may also a great way to explain open source to people who don’t immediately understand how collaboration can work, and how contributors come and go.

That is what I call good TV. It is also rather trippy to look at. Please turn up the Floyd.

If we had a free supercomputer and infinite development time, my ultimate dream visualization would be a 20×20 foot wall section of these graphs, showing multiple projects side by side, with developers flying between projects. Who flies between projects? Is that common? What are the clustering patterns of these projects that share contributors? Where are the hubs and spokes? Are the hubs bigger projects than the spokes? (Can we get that in 3D?). There is something to be learned here, even if we don’t know what that is.

Install Gource. Let’s see your project video.

Written by mpdehaan

January 15, 2010 at 2:05 am

Posted in linux

VirtualBox & Wireless

with 2 comments

I’ve arrived at a fairly nice home virtual machine developer setup, that no longer cares whether I am docked or undocked. VirtualBox can set up a bridge around wlan0 by using the GUI — no manual setup, and it can also bridge wlan0. AFAIK, Virt Manager can still do neither of these things, though perhaps if you edit the libvirt XML and dnsmasq configurations you could get close. Further, the newly created bridges do not cause problems with NetworkManager or Firefox (as I’ve had happen in Fedora numerous times).

What I have:

  • Virtual machines are set up to be bridged on wlan0
  • Virtual machines are configured statically, i.e. 192.168.1.150+N (outside the DHCP range of the router), gateway=router_ip, dns=host_ip
  • Host runs dnsmasq with each machine in /etc/hosts

(For purposes of full disclosure, this is on Ubuntu 9.10)

I would like to be using virt-manager and supporting my former Fedora cohorts instead, but this setup is way too portable and awesome. I’m also running on a Dual 3GHz system with 8GB of RAM, and … for some reason, VirtualBox seems faster (even with /dev/kvm present) and the graphics are also much better.

Anyway, I’m pleasantly surprised by this being as manageable as it is for a bridged setup.
I’d like it to be even easier, but I realize virtual machines outside of NAT are largely a server use case, so much more than that would probably be overkill — and server folk can handle it. I still can’t help but wonder what would happen if Virt-Manager was as usable as Virtual Box, and Virtual Box had Virt-Manager’s dnsmasq (with dynamic DNS so hostnames just worked everywhere) and also added a UI to configure it.

An aside: /sbin/dhclient-script clobbering /etc/resolv.conf? Not cool. Not cool.

(Really I’d like universally unique IPs all of the time, regardless of what networks I’m on. And a magic IP that is always the IP of the host the guest is running on. And SkyNet…)

Written by mpdehaan

January 10, 2010 at 3:18 am

Posted in Uncategorized

Parsing CacheGrind from Ruby

leave a comment »

Here’s a little script to show how many times files are accessed during a function call (or integration test) series, for use with tools like XDebug. In my particular application, I had a large codebase and didn’t know what files were touched (or not) during a relatively complex call chain.

require 'optparse'
require 'ostruct'

options = OpenStruct.new()
options.compact = false
OptionParser.new do |opts|
   opts.banner = "Usage: cache_blaser.rb [options]"
   opts.on("-c", "--compact", "Report on directories, not files") do |c|
      options.compact = true
   end
end.parse!

called = {}

ARGV.each do |filename|
   open(filename) do |handle|
      handle.each_line do |line|
          if line=~/^fl=(.*)/
             key = $1
             key = File.dirname(key) if key.include?("/") and options.compact
             called[key] = called.has_key?(key) ? called[key]+1 : 1
          end
      end
   end
end

keys = called.keys.sort do |a,b|
    (called[b] == called[a]) ? b <=> a : called[b] <=> called[a]
end

keys.each do |file|
    begin
        printf("%06d | %s\n", called[file], file)
    rescue Errno::EPIPE
    end
end

Usage: (top 50 most accessed files)

ruby cache_blaster.rb /tmp/cachegrind* | head -n 50

The output is number of times something in each file was referenced. This could easily be adapted to also include function calls, or list top function calls for each file.

080976 | php:internal
029876 | /path/to/file/a.php
009454 | /path/to/file/b.php
005875 | /path/to/file/c.php
004552 | /path/to/file/d.php
002433 | /path/to/file/e.php
001522 | /path/to/file/f.php
(etc)

Obviously all this profiling data is already scanned by tools like KCacheGrind, though who really likes GUI tools for more complex data mining and for generating custom reports?

Written by mpdehaan

January 8, 2010 at 9:52 pm

Posted in Uncategorized

Flattening Hashes In Python

with 5 comments

I was looking for a simple way to diff two complex hashes. In order to make the output nicely readable for humans, I’d first like to flatten the hashes. For example:

test = {
         "a" : [ "dog", "cat", "chicken" ],
         "b" : {
             "c" : 0,
             "d" : [ "red", "yellow", "blue" ],
         },
         "e" : "shiny"
   }

Becomes:

{
    'a': ['dog', 'cat', 'chicken'],
    'b.c': 0,
    'b.d': ['red', 'yellow', 'blue'],
    'e': 'shiny',
}

Here is a very long Perl module that does this. Here’s my cut:

  def _flatten_ds(self, ds, result=None, memo=""):
        if result is None:
            result = {}
        assert type(ds) == type({})
        for (k,v) in ds.iteritems():
                if memo == "":
                    new_memo = k
                else:
                    new_memo = "%s.%s" % (memo,k)
                if type(v) == type({}):
                    self._flatten_ds(v, result=result, memo=new_memo)
                else:
                    result[new_memo] = v
        return result

Almost 300 lines shorter than the Perl module :)

Written by mpdehaan

January 5, 2010 at 4:02 pm

Posted in linux