WARNING: (added 2013-11-16) I have not been involved in running any sort of mail server for several years now. In particular, I don't follow what is going on the the qmail world. This patch never came with any sort of warranty, but that is triply true now. Even if a serious bug or security vulnerability came to my attention, I would not be in a position to fix it. If you do need to use this patch, please read the code and form your own understanding of how it works.
There will probably never be any support for dotlocking in the official release of qmail, for the reason that it is not sufficiently reliable. Still, dotlocking is popular in a world filled with NFS file systems (a problem in its own right), because ordinary file locking tends to create all sorts of problems with hung stat and lock daemons and stuff. So I have made a patch to allow qmail to use dotlocking on mailbox files. Of course, the long range solution is to use maildir instead, and then all this will be moot.
A word on the implementation of dotlocking: Basically, dotlocking works as follows. In order to gain the lock on a file FILE, a program first generates a unique filename, typically of the form FILE.lock.something-unique. Then that file is linked to FILE.lock. If this fails, it is presumed that somebody else holds the lock. If it succeeds, the unique file name is removed, the program performs whatever actions it wanted to perform on FILE, and then it releases the lock by deleting FILE.lock.
The good news is that this works on all kinds of UNIX systems, even over NFS almost no matter how broken the implementation. Otherwise, file locking over NFS is a tricky business at best.
The bad news is that dotlocking is prone to leaving FILE.lock lying around, in case the machine or program crashed at the wrong time. This is sufficiently much of a problem that the algorithm needs to be changed. The policy at our site, and what this patch implements, is the following: No program is allowed to hold a lock for more than 60 seconds, and any lock older than that may be stolen. With this patch, qmail-local or maildir2mbox will detect an approaching 60 second deadline, clean up, and commit suicide. Any message(s) not added to the mbox will be left in the queue to be delivered later (qmail-local) or in the maildir (maildir2mbox).
The bad, bad news is that, particularly on a heavily loaded machine, the cleanup and suicide may not complete in time, leading to a corrupted mailbox.
Finally, a piece of good news (and good advice): If you use maildirs instead of mboxes, none of these problems apply. By all means, do so if you can. This patch is for those who cannot, and who have implemented a dotlocking policy at their site.