"such as speed of switching quickly between browser and vm terminal, the fact you need a fairly good computer to run even quite a modest server with a gui and if you don't use a gui, you're limited to 256 colours"
This implies you are running a terminal emulator in X on the VM and doing your coding there?
I use VMs for development all the time, and a headless VM with SSH access is perfect. It's almost identical to doing development locally.
the options as I see it are either do everything over ssh to a headless vm as you suggest which definitely has its advantages or working inside a full-screened vm/ booted environment. Perhaps not the best justification, but I prefer the latter because I can include my gvim setup and use the mouse for resizing columns, browsing code and using NERDTree. I miss that when running vim over ssh.
You can edit and browse on your host VM. Just adjust the VM's network settings and use a shared host-guest folder for the code (what exactly this ends up being varies based on the virtualization).
You really can make it a near exact match for a local development environment. Bridge your ports to your expected ones on the host machine, share the code folders, and SSH in if you need to punt a dev server when you make changes (or set your Editor/IDE up to do that for you). Browser can still point at local:8000 or whatever you expect.
Vim should be able to do everything that GVim can do, if you add a few lines to your configs. I'm not sure what specifically though.
I bring my entire development setup with me when using a server, and preparing it is as simple as cloning Homeshick (or Homesick if you have Ruby), cloning your dotfiles repository, and installing any applications that you want to use.
This implies you are running a terminal emulator in X on the VM and doing your coding there?
I use VMs for development all the time, and a headless VM with SSH access is perfect. It's almost identical to doing development locally.