Flattening Hashes In Python

Posted: January 5, 2010 in linux

I was looking for a simple way to diff two complex hashes. In order to make the output nicely readable for humans, I’d first like to flatten the hashes. For example:

test = { 
         "a" : [ "dog", "cat", "chicken" ],
         "b" : {
             "c" : 0,
             "d" : [ "red", "yellow", "blue" ],
         },
         "e" : "shiny"
   }

Becomes:

{   
    'a': ['dog', 'cat', 'chicken'], 
    'b.c': 0, 
    'b.d': ['red', 'yellow', 'blue'],
    'e': 'shiny', 
}

Here is a very long Perl module that does this. Here’s my cut:

  def _flatten_ds(self, ds, result=None, memo=""):
        if result is None:
            result = {}
        assert type(ds) == type({})
        for (k,v) in ds.iteritems():
                if memo == "":
                    new_memo = k
                else:
                    new_memo = "%s.%s" % (memo,k)
                if type(v) == type({}):
                    self._flatten_ds(v, result=result, memo=new_memo)
                else:
                    result[new_memo] = v
        return result

Almost 300 lines shorter than the Perl module :)

Advertisement
Comments
  1. Klaus says:

    Hi,

    the module may be a nice and general implementation, but your python implementation can be implemented 1:1 in perl, to be honest…

    Just wanted to know:

    sub flatten {
    my ($ds, $result, $memo) = @_;

    my %empty = ();
    $result = \%empty if (!defined $result) ;

    my @elt;
    while (@elt = each %{$ds}) {
    if ($memo eq “”) {
    $new_memo = $elt[0];
    } else {
    $new_memo = “$memo.$elt[0]“;
    }
    if (ref $elt[1] eq “HASH”) {
    $result = flatten($elt[1], $result, $memo=$new_memo);
    } else {
    $result->{$new_memo} = $elt[1];
    }
    }
    $result;
    }

    Klaus

    • mpdehaan says:

      Of course. Mainly I wanted to share the shorter algorithm, and feed it to Google.

      I’m sure it can be done in less code or more efficiently as well, which I haven’t really tried to do.

      • Anonymouse says:

        I was trying to view a verbose a hash, and I accidentally wrote that in a one-liner:

        ohai | ruby -ryaml -e “puts YAML::load(STDIN.read).each_pair{|k,v| puts k; v.respond_to?(:keys) ? v.keys.each {|z| puts ‘ ‘ + z.to_s + ‘ ‘ + v[z].inspect } : puts(‘ ‘ + v.to_s) }” | less -S

      • mpdehaan says:

        With semicolons and nested ternaries, it’s not a one liner, it’s just bad code formatting :)

  2. rich says:

    Perl and Python are very similar languages in what you can express. The next level languages are things like Lisp. Above that, take a look at OCaml + camlp4, for a unique combination of powerful language and macros.

  3. I’m not really trying to say it was powerful or it wasn’t :) Nor was this about Perl at all. It’s just that I didn’t see an example of this on Google when I was looking for a Python version, so I’m sharing mine.

    Besides, all the cool kids are using Erlang and Intercal.

  4. Klaus says:

    Hehe, we still should define a version for a Turing Machine, just for reference … Then the discussion, which language is the be best would be moot :-) I wouldn’t want to start a religious war about it (not even about Emacs vs. vi, even though emacs is best :-) )

    BTW, the recursion is nice. I like recursion…

    Klaus

Leave a Reply

Please log in using one of these methods to post your comment:

Gravatar
WordPress.com Logo

Please log in to WordPress.com to post a comment to your blog.

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Connecting to %s