Russ, you probably remember that Aki did a linuxfb version of drawterm for Tiny Horrible Xen; I'd bet the changes could be adapted to Plan 9 Port without too much trouble, really the Linux framebuffer is pretty simple.
If you install Plan 9 Ports, check out /usr/local/plan9/src/cmd/devdraw. Russ can correct me if I'm wrong, but I think that's the only place the graphical programs interface with X. You'll need to find the places where devdraw is using X, and replace them with framebuffer code.