Niall’s Weblog

I wrote the other day about porting software to Linux. I talked about some of the differences between the platforms in terms of the implementation of some well-known C library functions, and hinted at some other incompatibilities. Well, now I’m going to write about some of the other issues I encountered.

There are a number of security-oriented libc functions originating in OpenBSD which are very useful but have had varying degrees of uptake from other OSes. The classic examples are the well-known strlcat and strlcpy functions. As is discussed in this paper, these functions offer a less error-prone interface (specifically in the realm of truncation detection) compared to the strncat/strncpy ANSI C functions. Although they’ve been integrated into to at least Solaris, FreeBSD and NetBSD libc’s, for some bizarre reason they are not included in GNU libc. An additional OpenBSD function I make use of is strtonum. The issue here is that for my software to build on platforms other than OpenBSD, I must include copies of these functions and build them on platforms which don’t already have them. This leads quickly to the need for a way of dynamically detecting at build time whether these functions are available on the host system. Traditionally, the solution to this situation has been to use GNU Autoconf. However, I’ve never been a big fan of autoconf, mostly since its so complicated and tricky to get working with your project. Many people have been dissatisfied with autoconf though, and so alternatives now exist.

Much as I like the simplicity and ubiquity of make, I didn’t fancy implementing all the function call dependency checking in it. Furthermore, I am only familiar with the BSD flavour of make, and didn’t have much inclination to port my makefile over to other platforms. So I decided that I would try out one of the contemporary build systems. I came across something called SCons. I read the documentation and liked it because it seemed to make the tricky things pretty easy. Additionally, its written in Python - which I quite like - and seems to be pretty portable. At least, there are packages for OpenBSD, Linux and Windows.

I actually ended up with two different build systems. A BSDmakefile which will build my application under OpenBSD with no dependencies beyond the base install, and an SConstruct file for all the other platforms - requiring the presence of SCons. I’ve tested the SConstruct build under both Ubuntu and Centos Linux and it works great. I hope to add support for Solaris and Mac OS X in the near future - as soon as I get access to these operating systems.

I have been making very good progress with my BitTorrent implementation recently. However, I have felt the need to make it compile out of the box under systems other than OpenBSD. The first system I have ported to is Linux, specifically Ubuntu 7.10. In doing the port, I found some interesting differences between OpenBSD and Linux.

mmap() differences

Unworkable (not officially released yet, but blog readers can have a sneak peek) uses the mmap() system call rather heavily. For each piece in a torrent, I create at least one corresponding mmap’d segment - more if the piece spans multiple files. OpenBSD’s mmap() accepts arbitrary offsets, which is very straight forward. Linux, however, requires that the offset be aligned to page size. That is, offset must be a multiple of the page size.

On a POSIX operating system, the page size can be queried easily using the sysconf libc function, passing the _SC_PAGESIZE value. Once I have the page size, I find the nearest multiple to the offset we really want, and mmap that. Then I do some pointer arithmetic to get to the ‘real’ offset from there, and do all my reads and writes with a ’synthetic’ pointer into the mmap as the base address. The alignment requirement makes things a little more complicated, but its not too tough to handle.

dirname() / basename()

I use dirname() to ensure that directories exist before I try to write files underneath them - something analogous to mkdir -p. Under OpenBSD, it is safe to call dirname() directly on your path string, and use the returned value. On Linux however, dirname() and basename() modify the string you give it, meaning that you can’t re-use it. This caused me a bit of confusion until I realised what was going on. Under Linux, its necessary to create a copy of your string before passing it to dirname(), so that your original is not mangled.

snprintf()

The snprintf() function is very useful for conveniently building strings in C. I use it especially in Unworkable’s HTTP/1.0 client implementation, to build the query string. Under OpenBSD, its fine to snprintf a variable into itself, e.g.

snprintf(x, XLEN, "%s = %d", x, y);

Under Linux, this doesn’t work - in fact, glibc’s snprintf() fails completely silently in this case, from what I could tell. It was necessary for me to snprintf() into a temporary buffer, and copy that around, rather than just using the snrpintf() on its own, like you can under OpenBSD.

Well, thats it for now. I’m going to write more about the porting process, in terms of adding missing functions and dealing with Make incompatibilities, next.