Flattening Hashes In Python

Posted: January 5, 2010 by mpdehaan in linux

I was looking for a simple way to diff two complex hashes. In order to make the output nicely readable for humans, I’d first like to flatten the hashes. For example:

test = {
         "a" : [ "dog", "cat", "chicken" ],
         "b" : {
             "c" : 0,
             "d" : [ "red", "yellow", "blue" ],
         },
         "e" : "shiny"
   }

Becomes:

{
    'a': ['dog', 'cat', 'chicken'],
    'b.c': 0,
    'b.d': ['red', 'yellow', 'blue'],
    'e': 'shiny',
}

Here is a very long Perl module that does this. Here’s my cut:

  def _flatten_ds(self, ds, result=None, memo=""):
        if result is None:
            result = {}
        assert type(ds) == type({})
        for (k,v) in ds.iteritems():
                if memo == "":
                    new_memo = k
                else:
                    new_memo = "%s.%s" % (memo,k)
                if type(v) == type({}):
                    self._flatten_ds(v, result=result, memo=new_memo)
                else:
                    result[new_memo] = v
        return result

Almost 300 lines shorter than the Perl module :)


Comments
  1. Klaus says:

    Hi,

    the module may be a nice and general implementation, but your python implementation can be implemented 1:1 in perl, to be honest…

    Just wanted to know:

    sub flatten {
    my ($ds, $result, $memo) = @_;

    my %empty = ();
    $result = \%empty if (!defined $result) ;

    my @elt;
    while (@elt = each %{$ds}) {
    if ($memo eq “”) {
    $new_memo = $elt[0];
    } else {
    $new_memo = “$memo.$elt[0]“;
    }
    if (ref $elt[1] eq “HASH”) {
    $result = flatten($elt[1], $result, $memo=$new_memo);
    } else {
    $result->{$new_memo} = $elt[1];
    }
    }
    $result;
    }

    Klaus

    • mpdehaan says:

      Of course. Mainly I wanted to share the shorter algorithm, and feed it to Google.

      I’m sure it can be done in less code or more efficiently as well, which I haven’t really tried to do.

  2. rich says:

    Perl and Python are very similar languages in what you can express. The next level languages are things like Lisp. Above that, take a look at OCaml + camlp4, for a unique combination of powerful language and macros.

  3. I’m not really trying to say it was powerful or it wasn’t :) Nor was this about Perl at all. It’s just that I didn’t see an example of this on Google when I was looking for a Python version, so I’m sharing mine.

    Besides, all the cool kids are using Erlang and Intercal.

  4. Klaus says:

    Hehe, we still should define a version for a Turing Machine, just for reference … Then the discussion, which language is the be best would be moot :-) I wouldn’t want to start a religious war about it (not even about Emacs vs. vi, even though emacs is best :-) )

    BTW, the recursion is nice. I like recursion…

    Klaus

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <pre> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>