Been a while since I did this, a lot of things have happened.
Managed to fix up per page verified exec. I was horribly abusing the UVM subsystem by setting flags I shouldn't be touching. Hence the horrible and weird crashings, I was claiming back pages that no longer belonged to me. Ugly. I found this with some help from Chuq. Fixing it was a bit more involved, I needed to unlock the pages to avoid a deadlock when I did a VOP_GETATTR but needed the pages back so I could check their fingerprints, which mean I needed to do a getpages to get the pages back but the function I was in was being called by getpages... round and round in circles. The only reason I needed VOP_GETATTR was to get the device and inode numbers so I could look up the entry in the veriexec hash tables. Chuq suggested I could get around the circular references by setting up a hash table with the vnode pointer as the index that pointed to the details I needed. This neatly solved the issue but presented it's own problems, firstly I needed to associate the vnode to the device and file id's. I extended the getnewvnode() to add in two new arguments so the device and file id could be passed in, this way an association between the new vnode and the veriexec entry can be made in the vnode hash table. Doing this required touching a lot of file system code to get the right numbers passed in. The worst was NFS where the device and file id are not available when the new vnode is allocated, in this case I defer the vnode->veriexec association until the VOP_GETATTR is done, this should be safe since any file that is executed or read needs to get the permissions first so the association will be set up before it is needed. The other end of this is, of course, when the vnode is no longer required and gets recycled for a new use the old association needs to be broken. Chuq suggested a hook that gets run when the vnode gets cleaned (more on hooks later), at the moment I have hard coded a clean up into the vnode recycle code which works but is not the best. After doing all this and having a few false starts I finally have per page veriexec where it should be, fs independent and functional. I have sent the diffs off to Elad so that he can look at them and also so I am not the only one with them (mmm paranoia).
The follow on project was to finally do the generic hook stuff so that I didn't generate yet another set of hook functions in the kernel just for the vnode association clean up. I posted a proposal to tech-kern and received back a suggestion that we use the FreeBSD eventhandler code. I was happy with that idea, I have imported this into a private tree and modified things to suit NetBSD better. I have this all working now and have migrated some of the hooks over to using the code. I will update my private tree to the latest NetBSD and make sure nothing is broken and then put up some diffs on tech-kern and see what happens. Once the generic hooks are in I will go back and modify the per page veriexec stuff to use them to break the vnode->veriexec association.
