Text Mode Interface for Microsoft Word Prototype
________________________________________
< it looks like you are writing a letter >
----------------------------------------
\ ^__^
\ (oo)\_______
(__)\ )\/\
||----w |
|| ||
Minimal Python JSON Serialization/Deserialization
Cobbler leveraged JSON for saving object representations to disk (though not for wire transport, where it used XMLRPC, but that is another story). The object representation system in 2.0, which I called “FIELDS” was used to define the members of each object was a little more heavy weight than I liked. This made it difficult to add new “nouns” into Cobbler, and made modelling system interfaces (which I didn’t want quite to be their own objects), rather hard. Why? The serialization and de-serialization to and from objects was not as recursive as it could be. We also had a top level orchestrator object called “config” that did not do much, that I always wanted to get rid of.
I have.
Here is a rough example of doing all of that more automagically.
The base class is /somewhat/ gnarly because of the automagic, but I don’t think the object classes could be any simpler. Notice the type checking that is also built in. Bonus! So you can work in JSON objects all day, but your code gets to work in /objects/, not error prone nested hashes.
Extra validation can be added by writing setter functions, which are, by default, optional .. you get free type checking without the setters.
Streaming HTTP Uploads With Python’s Poster
There’s not much to say beyond the documentation examples, but I was looking for a way to do streaming uploads from Python, and poster delivers nicely on the client side.
Writing a server side streaming downloader in mod_python is infinitely easier and looks something like this:
def handle_upload(self):
self.form_data = util.FieldStorage(self.request)
for key in self.form_data.keys():
value = self.form_data[key]
if type(value) == types.InstanceType:
if isinstance(value, util.StringField):
pass
elif isinstance(value, util.Field):
filename = value.filename
content_type = value.type
base = os.path.basename(value.filename).strip()
fd = open("/tmp/apache/%s\n" % base, "wb")
while True:
data = value.file.read(1024)
fd.write(data)
if data == "":
fd.close()
break
return apache.OK
Security and error handling (as well as writing the rest of the mod_python handler, however minimal that might be) are left as an exercise to the reader.
Theoretical Attack Vector?
WRT This, I think there are potentially greater problems. Fedora has standards of packaging specification review, but not so much review of package content or updates (beyond, perhaps diffs to the specfile, which go to a mailing list in CVS — the tarballs are not subject to close inspection). While it is bad enough that this allows a user access to install a well-intentioned package with an unpatched vulnerability when it is not otherwise installed (requiring first some form of local access, or another exploit or bridge sufficient to talk to dbus?), what is not mentioned is it is fairly easy for a person to masquerade as a good guy, package a popular technology, and replace it with an disguised evil payload six months later. RPM scripts run as root, and are just as dangerous as the software they install.
I don’t know exactly /how/ you fix that attack vector, it is a social one. So if Fedora is thinking about reviewing security policies, also consider we don’t /really/ know everyone, and we can’t know. I would be more worried about that than the PackageKit one. With the PackageKit one, I’d just mostly be really annoyed that someone I gave an account to could fill up my hard drive space and I wouldn’t know exactly what they installed so I could clean up after.
My point is that even though the Package is signed, that doesn’t mean they are (neccessarily) safe. More reason for not allowing extra people to install packages.
I’m only bringing this up because I get tired of fighting off Ubuntu people, and such security features as PackageKit being wide open do not make it easy. With all the attention on SELinux, etc, I see no reason Fedora could not be both usuable and also a security bastion with similar reputations as OpenBSD. In the worst case, perhaps just ask the user what they want in the installer and default to some of the more secure options — or default to the most secure options and provide a setting in a configuration panel somewhere to open it up.
It’s Not What You Know
Repeating favorite quote: “A Mentat needs data”. Dune.
The most valuable aspects for computer-science guy is not just what he knows how to do in terms of code. It is a combination of (A) what you know about everything, (B) what you can infer and absorb, and (C) the logical transformations and intuitive leaps you can make on that data to arrive at new conclusions. Familiarity with certain code or tools? Irrelevant — all of these things can be learned. It is logic that is important. That is hopefully why you have a job. This is why you think you can come up with better plans than many of the business guys you know — you probably can (though you might not be good about talking to customers). But wait? That’s not about code, right? Right. It is about logic and knowing how to apply that logic.
In other words, ideally, you are (whether you like or not) becoming a human computer by having too much proximity to the real thing. While you went to school to learn how to program, what is happened is that the computer has learned to program you, and as a result you really learned something completely different. The code you work on is only a practice ground where this process is accelerated. I think this is why some software companies can become intensely political — smart folks start wanting to program their own reality. Anyway, what is happening is the computer is teaching you to think differently — and you are also more sensitive to noticing ways that things perform sub-optimally — logical problems MUST be fixed! (And this leads to total failure when the logical engines of different smart folks conflict… they think both sides are illogical! BOOM! What really needs to happen is a reset to first principles and a re-evaluation of all logic; but group-think does not usually arrive at this! Decree is actually better in these instances than attempting consensus — though you’ll only be able to lead those that agree with you, so choose wisely).
Think of this the next time you feel incredibly frustrated because your parents don’t understand “Correlation Does Not Imply Causation”, or why the US House and Senate don’t understand “Nash Equilibrium” — this pain you feel is there because the computer is programming you. It is not because you learned these concepts and they didn’t … it is because these concepts are now burned into your brain at a level beyond mere recall and understanding. You didn’t used to be that way. Other folks would just be content with chaos or error — because they wouldn’t /see/ the chaos or error.
I think the world would be really interesting if more Computer Science folks focused a bit less on code and focused on applying logic directly into politics, business, and other models — where they seem (to me) to be less applied. After all, code is just a mechanical transformation on an arbitrary construct. Politics, Business, other things, are just similar transformations on different storage mediums. Businesses? Do value your C.S. folks as more than just coding machines. They want to think and improve arbitrary problem-spaces. There is no other outlet. It is an occupational hazard.
Logic. It’s what’s for Dinner.
(Also: don’t let the machine eat your soul … that would also be bad)
Ack & Xargs For Global Replacement
For future reference:
ack "term_one" * -l | xargs -n1 --replace sed -i 's/term_one/term_two/' {}
RHEL 4/5, IIRC don’t require “-n1″ and “–replace”.
iPhone Car Adapters: Found a Really Good One
My previous Monster audio adapter thingy (TM) was great for a line-in on the iPod, but did not charge the iPhone, and also displayed a warning about incompatiblity and suggested that you switch the phone to airline mode. The different pin-out, or whatever was the cause here, and needing another adapter is kind of dumb.
Anyway, while you can pay $45 for an overpriced Apple AV adapter and then buy a Y splitter to merge RCA to TRS stereo (and the Apple Store has no alternatives for AUX ports), I really like the Kensington LiquidAUX that I grabbed off Amazon. One of the best features is that it keeps the phone from going into “auto lock after 2 minutes mode” so you can easily switch between songs while in the car (I plan to get a mount that pairs with my existing ProClip, I think, since I already have a mount for the GPS it could share), and also nicely pauses the iPhone when the ignition cuts off…. so it doesn’t keep playing while you put it in your pocket.
For about $25, it is pretty cool.
In other news, having a public IMAP and CalDAV server at work now means I can sync multiple mail accounts again, even though I have the gmail accounts configured with Exchange sync. More places should do this.
Vim Setup For Working With Large Codebases
Vim is awesome for working with small projects where you know where everything is. What if you don’t? Here’s my current setup, as a form of backup. Hopefully this is useful to someone else as well.
ctags setup:
#!/bin/sh
cd basedev
ctags -R --totals=yes --tag-relative=yes \
--extra=+f \
--PHP-kinds=+cf \
--regex-PHP='/abstract class ([^ ]*)/\1/c/' \
--regex-PHP='/interface ([^ ]*)/\1/c/' \
--regex-PHP='/(public |static |abstract |protected |private )+\s*function\s+([^ (]*)/\2/f/'
.vimrc:
nmap <C-f> :NERDTreeToggle<CR> nmap <C-b> :BufExplorer<CR> nmap <C-t> :TlistToggle<CR>
Installed VIM plugins:
So to open a file in the source tree, I can just:
cd ~/code/basedev :tag SomeClassName
And then the rest of my workflow is:
Control+f = Toggle the file explorer tree Control+b = Pick an open file to edit Control+t = List functions in the file (Control+w to change windows) Control+] = Jump to tag under cursor :Ack Foo = Look for files that contain Foo and potentially open them
All of this works without dealing with vim split windows or tabs, which are, IMHO, both very annoying.
Simple, and works. And, best of all, it is not Eclipse.
Suggestions on your other favorite vim plugins are very welcome. While Ack is nice on the command line, I think there are probably better file search plugins for vim that people prefer?. I am not using any “project” style plugins at the moment. My workflow for remembering a workspace is, so far, writing down the names of classes I’ve found along the way that might be worth remembering, and opening them again with “:tag Foo” as needed. I’ll edit this page over time to reflect my current setup.
Game of Life on A Rubik’s Cube, and Other Weirdness
From a random discussion at work (spawned from my de-stickered already-been-solved Rubik’s Cube), I think someone should build something like this:
Imagine Conway’s Game of Life (GOL) being played on all six surfaces of a connected Rubik’s Cube (RK) covered with LEDs, such that formations could travel across the edges of a face and onto the other edges. Each GOL cubeface is, say, arbitrary MxM, for instance 99×99 — however each twistable cube portion is only 33×33. As a result, if the microcosm traversing the surface of the cube were to stagnate, it could be radically altered with a twist of the cube and drastically alter the game state. This could be a quite interesting piece of art.
An alternative simulation requires contextually aware e-paper, which is much more technologically complicated, because currently such things don’t know where every “pixel” of the paper resides in 3D space. Anyway, suppose there is a GOL being played on a MxN sheet of paper (perhaps both sides), that is totally flexible. Imagine if it could be folded to connect various portions of the page, such as to temporarily form a tube, or folded in various shapes. Further, imagine the following with multiple sheets of contextually aware e-paper, so if you picked up two GOLs and set them together, a creature from one would have a chance to travel to an adjacent page.
We all agreed that GOL in itself was not super-interesting, though I think the idea of cellular automata (or possibly even some form of e-pets) that could move between various surfaces could be pretty neat. A related concept could involve wearing an e-shirt and leaning up against an e-wall with your e-shirt. Just like a “wet paint” scenario, some of the advertisements living on the wall could crawl onto your shirt until you shook it off. Similary, assume a GOL type automata living on your shirt interacting with a GOL automata living on the wall, or having a clean “white” shirt that gets a GOL automata on it by standing too close to the wall. Careful what walls you lean on, you could upset the equilibrium of a wall and kill it… and it might take someone else leaning on the wall to bring it back.
Vim Rsync Hooks — Remotely Copy Certain Files Upon Save
Challenge: Have local files in a developer setup for quick searching of a large tree of code, but mirror them to a remote test server any time they are edited, since your local development platform is not your test platform.
Solution: vim post hooks! Note that this is a work in progress.
/usr/bin/vim_mirror.py:
#!/usr/bin/python
import sys
import os
import subprocess
REMOTE_ROOT="user@host:/path/to/somedir"
if __name__ == "__main__":
if len(sys.argv) > 1:
tokens = sys.argv[1].split("/")
# as implemented this script will ONLY mirror all files rooted in "/basedev"
# to a given remote root... this will need to be tweaked for different
# developer setups in a future version of this script. Ideally it will take a list
if tokens[0] == '' and tokens[1] == "basedev":
local_file = sys.argv[1]
remote_part = "/".join(tokens[2:])
cmd = "rsync %s %s/%s" % (local_file, REMOTE_ROOT, remote_part)
print cmd
subprocess.call(cmd, shell=True)
.vimrc:
au BufWritePost * :!/usr/bin/vim_mirror.py %:p
