There is something vaguely frustrating about finding and fixing a bug in some software thinking that you have found your problem and things will start working right only to find that the problem is still there - you found a bug but not the one you were searching for.
I had this today, I have been off and on trying to track down why aspell seg faults when NetBSD curses is used, the problem has the hallmarks of memory being overwritten. I built a version of libcurses with dmalloc and it was telling me that a boundary was being overrun. After a bit of digging I found that in __init_get_wch there was a memset used to clear a buffer but the size argument was way too big causing memset to stomp past the end of the array. I fixed this and dmalloc no longer complained when I ran my simple test code but aspell still seg faults and the stack backtrace from the core file still looks as mangled as it did before my fix. So, yes, I definitely fixed a bug - just not the one I was aiming to fix.
Friday, June 29, 2012
Wednesday, June 27, 2012
Been distracted fixing a scrolling bug in the menu library (libmenu). When I did the code I only really tested the code in "row major" mode where the menu items are laid out left to right in the desired number of columns, the other mode "column major" for want of a better term lays the menu items out going down the columns - in both cases the number of rows in a menu is determined by the number of items to display. You are supposed to be able to navigate around the menu items by using up/down/left/right commands (there are a lot of other commands too...). To make the navigation easier I pre-calculated the neighbours when the menu is being created and stored them in the item structure. Lots of edge cases, literally, doubly so because you can either have "cyclic" navigation which will wrap around the edges of the menu or "non-cyclic" where navigation stops at the edge of the menu. I had to totally rewrite the neighbour calculation and tidy up the menu drawing. Now it works as it should in both modes.
I have also played around with getting rescanning of the usb bus working. I am not sure if I am happy with what I have, you can rescan an attached device and it will be detached and reattached, you can rescan an entire bus but I am not sure if that does anything. More testing is required.
I have also played around with getting rescanning of the usb bus working. I am not sure if I am happy with what I have, you can rescan an attached device and it will be detached and reattached, you can rescan an entire bus but I am not sure if that does anything. More testing is required.
Thursday, June 14, 2012
The fix is in....
I have committed the changes to umodem (plus usb.h) that allow the driver to attach the serial port on the leostick, I expect this will work for other arduinos too - they should "just work" in NetBSD-current now. The problem of the drivers detaching and not reattaching when the arduino gets reprogrammed needs to be worked on still.
Tuesday, June 12, 2012
Arduino success
I have managed to upload an example sketch to the leostick, it took a bit of sleuthing to get all the bits right.
step 1: I used the boot configuration editor (boot -c) to disable the uhidev driver so that it would not claim the leostick, this meant that the ugen driver claimed it instead which is what I wanted. I already had a program that used the ugen ioctl USB_GET_CONFIG_DESC to get the configuration description of the device. It turned out the leostick had three interfaces, the first is a CDC ACM (usb serial) the second looks to be CDC Data but didn't make much sense and the third is the HID that uhidev was latching on to. Given that the NetBSD umodem driver supports the CDC ACM interface type it was a matter of working out why umodem was not attaching. It turns out that the umodem match was checking that the interface protocol attribute was set to a particular number, on the leostick the protocol was 0.
step 2: fix the umodem driver. I just modified the attach so that it would attach if the protocol was the AT protocol or if protocol is 0. Rebuilt the kernel, installed, rebooted and now when I plug the leostick in the umodem driver claims the serial port. I could connect to the serial port and see output from the leostick.
step 3: I built the blink sketch from the examples in the arduino IDE. I had built a native version of avrdude and put it in the right place in the arduino tree. I tried using the upload using programmer menu item but this errored out. I replaced avrdude with a script that logged the command line parameters and tried to manually run avrdude with the logged parameters but had the same errors. After a bit of fumbling around I found that some of the avrdude parameters weren't right for my environment plus I could only upload a sketch when the bootloader was active (the first 7 seconds after reset). So, I finally managed to upload a sketch by firstly pressing reset and then running the command:
/usr/pkg/bin/avrdude -C/home/user/blymn/arduino-1.0.1/hardware/tools/avrdude.conf -patmega32u4 -carduino -P/dev/ttyU1 -Uflash:w:/tmp/build1833870341762611968.tmp/Blink.cpp.hex:i -v
I was still stumped for a while because it seemed like I couldn't upload a sketch without doing a reset but adding the option to force the baud rate to 1200 would let me upload again without resetting:
/usr/pkg/bin/avrdude -C/home/user/blymn/arduino-1.0.1/hardware/tools/avrdude.conf -patmega32u4 -carduino -P/dev/ttyU1 -Uflash:w:/tmp/build1833870341762611968.tmp/Blink.cpp.hex:i -v -b 1200
The only problem now is that the NetBSD usb drivers detach from the leostick and don't reattach when a sketch is uploaded when a sketch is already running so I still need to reset the leostick to get the devices back... inconvenient.
step 1: I used the boot configuration editor (boot -c) to disable the uhidev driver so that it would not claim the leostick, this meant that the ugen driver claimed it instead which is what I wanted. I already had a program that used the ugen ioctl USB_GET_CONFIG_DESC to get the configuration description of the device. It turned out the leostick had three interfaces, the first is a CDC ACM (usb serial) the second looks to be CDC Data but didn't make much sense and the third is the HID that uhidev was latching on to. Given that the NetBSD umodem driver supports the CDC ACM interface type it was a matter of working out why umodem was not attaching. It turns out that the umodem match was checking that the interface protocol attribute was set to a particular number, on the leostick the protocol was 0.
step 2: fix the umodem driver. I just modified the attach so that it would attach if the protocol was the AT protocol or if protocol is 0. Rebuilt the kernel, installed, rebooted and now when I plug the leostick in the umodem driver claims the serial port. I could connect to the serial port and see output from the leostick.
step 3: I built the blink sketch from the examples in the arduino IDE. I had built a native version of avrdude and put it in the right place in the arduino tree. I tried using the upload using programmer menu item but this errored out. I replaced avrdude with a script that logged the command line parameters and tried to manually run avrdude with the logged parameters but had the same errors. After a bit of fumbling around I found that some of the avrdude parameters weren't right for my environment plus I could only upload a sketch when the bootloader was active (the first 7 seconds after reset). So, I finally managed to upload a sketch by firstly pressing reset and then running the command:
/usr/pkg/bin/avrdude -C/home/user/blymn/arduino-1.0.1/hardware/tools/avrdude.conf -patmega32u4 -carduino -P/dev/ttyU1 -Uflash:w:/tmp/build1833870341762611968.tmp/Blink.cpp.hex:i -v
I was still stumped for a while because it seemed like I couldn't upload a sketch without doing a reset but adding the option to force the baud rate to 1200 would let me upload again without resetting:
/usr/pkg/bin/avrdude -C/home/user/blymn/arduino-1.0.1/hardware/tools/avrdude.conf -patmega32u4 -carduino -P/dev/ttyU1 -Uflash:w:/tmp/build1833870341762611968.tmp/Blink.cpp.hex:i -v -b 1200
The only problem now is that the NetBSD usb drivers detach from the leostick and don't reattach when a sketch is uploaded when a sketch is already running so I still need to reset the leostick to get the devices back... inconvenient.
Sunday, June 10, 2012
Arduino on NetBSD
I have been toying with using an arduino board to create a low power data logger. Not surprisingly there is no pre-packaged arduino IDE for NetBSD but there is for Linux and NetBSD can run Linux binaries. I downloaded the 32bit arduino-1.0.1 software and installed that. When I tried to run the arduino app it told me I was missing java. My previous attempts at getting java and NetBSD to play nicely have not turned out well but I thought I may as well give it another go and downloaded java for 32bit Linux. I installed java and set my PATH to point at it and then ran arduino again. Much to my surprise the IDE actually runs and seems to mostly work apart from it steadfastly refusing to see any serial ports. I was able to build an example sketch but I cannot upload it to a board because of the serial port problem. The IDE gives you the option to upload using using a programmer in which they just call avrdude, this fails because libusb is missing. A quick look in NetBSD pkgsrc shows that avrdude is actually there so I can build a native version of this tool. In fact, now that I look it seems that all the "avr" tools required to crossbuild binaries for the arduino boards are available in pkgsrc. So I could build native tools and use a makefile to produce and upload the binaries. I think I will stick with just replacing avrdude for the moment.
As for a target... I have a leostick which was given away at the 2012 LinuxConf.au, I have tried plugging this into my laptop and it is attached as a keyboard and mouse which is not very helpful, no serial port is attached. Apparently this works on some later versions of linux, I guess due to changes in the CDC ACM driver, in NetBSD this seems to be handled by umodem but I haven't yet managed to get it to attach with this driver. We shall see how it goes.
As for a target... I have a leostick which was given away at the 2012 LinuxConf.au, I have tried plugging this into my laptop and it is attached as a keyboard and mouse which is not very helpful, no serial port is attached. Apparently this works on some later versions of linux, I guess due to changes in the CDC ACM driver, in NetBSD this seems to be handled by umodem but I haven't yet managed to get it to attach with this driver. We shall see how it goes.
Sunday, June 03, 2012
Oops
OK - This place was ignored for quite some time, lets see if we can do something regular from now on.
What has happened the last 4 and a bit years? Well, verifiedexec per page signatures is still not in the tree. I merged the bottom end of the uvm routines but there was push-back on how I had done this from others, citing the long function argument list as being not a good thing. I have the actual code working, not only uvm but also the per-page verifiedexec stuff. I wrote a simple test that consisted of a binary that was stored on a NFS share - the binary just calls a couple of functions located in different memory pages (I inserted a bunch of unused code to get the functions on different pages). There are two versions of the binary the "good" version which has a matching veriexec signature and is the one initially run and the "evil" version that has a modified second page. The test consisted of starting the "good" binary and checking both function calls are the good ones, then on the NFS server overwrite the "good" binary with the "evil" one, once this is done the resident executable pages were flushed using msync (this is just a short cut you could force resource starvation). With an unmodified veriexec, the "evil" function would be executed, with the per-page modification the binary was terminated as soon as the pager attempted to bring in the modified page. This is good but I really need to fix up the UVM modifications to make them less convoluted.
Part of the reason veriexec hasn't progressed much in the intervening time is that I started working on automatically testing libcurses. This is quite a complex thing to do because curses expects to talk to a terminal. I developed a test framework that takes a simple list of commands and runs them against a test program and then verifies the output matches what is expected. This framework works with the NetBSD ATF (Automated Test Framework) and has been committed to the tree along with a small set of tests. More tests are on the way, there are an awful lot to write and it is a slow process but definitely worth it as a number of previously unreported bugs have been found and fixed.
I talked about both veriexec and the curses testing at BSDCan 2012, the actual talks are here for veriexec and here for curses testing there are papers and the slides from the presentations available at the aforementioned links.
What has happened the last 4 and a bit years? Well, verifiedexec per page signatures is still not in the tree. I merged the bottom end of the uvm routines but there was push-back on how I had done this from others, citing the long function argument list as being not a good thing. I have the actual code working, not only uvm but also the per-page verifiedexec stuff. I wrote a simple test that consisted of a binary that was stored on a NFS share - the binary just calls a couple of functions located in different memory pages (I inserted a bunch of unused code to get the functions on different pages). There are two versions of the binary the "good" version which has a matching veriexec signature and is the one initially run and the "evil" version that has a modified second page. The test consisted of starting the "good" binary and checking both function calls are the good ones, then on the NFS server overwrite the "good" binary with the "evil" one, once this is done the resident executable pages were flushed using msync (this is just a short cut you could force resource starvation). With an unmodified veriexec, the "evil" function would be executed, with the per-page modification the binary was terminated as soon as the pager attempted to bring in the modified page. This is good but I really need to fix up the UVM modifications to make them less convoluted.
Part of the reason veriexec hasn't progressed much in the intervening time is that I started working on automatically testing libcurses. This is quite a complex thing to do because curses expects to talk to a terminal. I developed a test framework that takes a simple list of commands and runs them against a test program and then verifies the output matches what is expected. This framework works with the NetBSD ATF (Automated Test Framework) and has been committed to the tree along with a small set of tests. More tests are on the way, there are an awful lot to write and it is a slow process but definitely worth it as a number of previously unreported bugs have been found and fixed.
I talked about both veriexec and the curses testing at BSDCan 2012, the actual talks are here for veriexec and here for curses testing there are papers and the slides from the presentations available at the aforementioned links.
Subscribe to:
Posts (Atom)