Here’s how to get hexdump to give you a suitable output for embedding in a C file:

hexdump -e "8/1 \"0x%02x, \" \"\n\"" <file>

(This post is for my benefit as much as anyone else.)

I’ve been working on some kernel code recently. The default log buffer size is only 4 KB so it doesn’t take much before you lose messages (not that I recommend you use logging for debugging purposes, but there are some times when it’s necessary). Anyway to increase it, you need to set the msgbuf boot argument. For example, in Terminal type this:

sudo nvram boot-args=msgbuf=65536

I came across this code recently:

level = kIOStorageAccessNone;
while ( ( object = objects->getNextObject( ) ) )
{
    if ( object != client )
    {
        level += _openClients->getObject( ( OSSymbol * ) object );
    }
}

Basically, it’s going through a list of clients that it services and works out what the aggregate access level is for all the clients. Here are the possible access levels:

enum
{
    kIOStorageAccessNone          = 0x00,
    kIOStorageAccessReader        = 0x01,
    kIOStorageAccessReaderWriter  = 0x03,
    kIOStorageAccessSharedLock    = 0x04,
    kIOStorageAccessExclusiveLock = 0x08
};

Now, from looking at this code, it looks wrong because of the use of the +=. Surely |= is more appropriate. It’s obvious to me now, but it took me some time before I found the bit where they’d overridden the += operator:

inline void operator+=( IOStorageAccess access )
{
    _access = ( ( _access - 1 ) >> 1 ) & 7;
    _access = gIOMediaAccessTable[ ( ( access - 1 ) >> 1 ) & 7 ][ _access ];
    _access = ( ( _access & 7 ) << 1 ) + 1;
}

Anyway, this, in my opinion, is a classic example of an abuse of C++ operator overloading. The rule when writing code should be to make it as readable and as obvious as possible first before you start trying to be clever. If the code had been something like this:

    clientLevel = _openClients->getObject( ( OSSymbol * ) object )->unsigned32BitValue( );
    level = aggregateLevel( level, clientLevel );

it wouldn’t have cost me any time as it would have been clear what was going on.

It’s for similar reasons that I always put brackets round expressions that combine && and || clauses: not everybody knows the order of precedence of these two and it’s easy to forget; adding brackets means you don’t need to remember.

For example, I would always do:

    if ((a && b) || (c && d))

rather than

    if (a && b || c && d)

I know it’s tempting to use that clever bit of the language that you just learnt about but in doing so you might make it harder for the person that follows you. The general rule should be: unless it is a common idiom, make it clear what’s going on and if a better known alternative exists, use that instead.

Disk Arbitration is what is used on the Mac to handle automatic mounting of volumes and various other disk related things. When you insert a disk, the kernel will instantiate drivers for it and notify the Disk Arbitration daemon. It will then ask filesystem plugins if they recognise the volume and if they do it will usually proceed to mount them.

Using Disk Arbitration, you can send requests to eject disks, mount and unmount volumes, rename volumes and you can claim a disk for exclusive use. For each of those operations you can also approve or reject requests that other applications make. It’s not actually enforced: you could still force a volume to be unmounted and claiming a disk for exclusive use doesn’t stop anyone else from using it (like advisory file locks). You can see some of these notifications flying around by using disktool with the undocumented -y flag. You can get more debug by hacking the com.apple.diskarbitration.plist file in /System/Library/LaunchDaemons and adding the -d flag to the argument list. You’ll then get /var/log/diskarbitrationd.log.

diskarbitrationd is the main daemon that’s responsible for all this stuff. The source code for it and related things can be found on the Open Source part of Apple’s site. To use Disk Arbitration from within your application, you need to link to the Disk Arbitration framework. It was a private framework in Panther (10.3) but was made public in Tiger (10.4). Things like Spotlight use it to know when to stop indexing a volume when you want to unmount a volume and you should use it too if you’re writing a similar application.

There are a couple of files that it looks at to tell it what to do. /etc/fstab is one of them. The shipped version (in Leopard) has some examples in the comments of /etc/fstab and the man page looks pretty good. By making the appropriate change to this file, you can, amongst other things, stop a volume from being automatically mounted. The other file it looks at is /var/db/volinfo.database. This text file stores the setting of the “Ignore ownership of this volume” flag that you see in Finder.

Disk Arbitration will also give you information about all the disks and volumes on the system. Here’s the kind of information it might have about your startup volume:

{
    DAAppearanceTime = 244372661.099552;
    DABusName = PMP;
    DABusPath = "IODeviceTree:/PCI0@0/SATA@1F,2/PRT2@2/PMP@0";
    DADeviceInternal = 1;
    DADeviceModel = "WDC WD1600JS-40NGB2                     ";
    DADevicePath = "IOService:/AppleACPIPlatformExpert/PCI0@0/AppleACPIPCI/SATA@1F,2/AppleAHCI/PRT2@2/IOAHCIDevice@0/AppleAHCIDiskDriver/IOAHCIBlockStorageDevice";
    DADeviceProtocol = SATA;
    DADeviceRevision = "10.02E04";
    DADeviceUnit = 0;
    DAMediaBSDMajor = 14;
    DAMediaBSDMinor = 3;
    DAMediaBSDName = disk0s3;
    DAMediaBSDUnit = 0;
    DAMediaBlockSize = 512;
    DAMediaContent = "48465300-0000-11AA-AA11-00306543ECAC";
    DAMediaEjectable = 0;
    DAMediaIcon =     {
        CFBundleIdentifier = "com.apple.iokit.IOStorageFamily";
        IOBundleResourceFile = "Internal.icns";
    };
    DAMediaKind = IOMedia;
    DAMediaLeaf = 1;
    DAMediaName = "Macintosh HD";
    DAMediaPath = "IODeviceTree:/PCI0@0/SATA@1F,2/PRT2@2/PMP@0/@0:3";
    DAMediaRemovable = 0;
    DAMediaSize = 138155196416;
    DAMediaUUID = <NSCFType: 0xd32f150>;
    DAMediaWhole = 0;
    DAMediaWritable = 1;
    DAVolumeKind = hfs;
    DAVolumeMountable = 1;
    DAVolumeName = "Macintosh HD";
    DAVolumeNetwork = 0;
    DAVolumePath = file://localhost/;
    DAVolumeUUID = <NSCFType: 0xd3316b0>;
}

That’s obviously for a specific partition on a disk. There’s another entry that represents the whole disk.

Something to note regarding disk/volume UUIDs: GUID partition schemes have a UUID that is used for the partition but in addition to that, there is a UUID that is stored as part of the file system (in the case of HFS+, it’s not actually a UUID but eight bytes that get turned into a UUID), so if you do end up using them, make sure you know which one you’re dealing with. The UUID reported by Disk Arbitration happens to be the file system UUID but the UUID property of IOMedia objects is from the partition table.

One thing that isn’t ideal is there’s no actual notification indicating when the system wants to eject a disk. You can use the eject approval notification, but then there’s no way to find out if the eject was ultimately approved. This would be a problem for, say, an indexing application, where it might want to release all resources for the disk (so that the eject will actually succeed) when an eject is requested. Spotlight probably uses the eject approval callback, but the only way it would be able to restart indexing on a volume that failed to eject, is by checking again after a certain amount of time (I don’t know if it does that).

Another problem I found recently is that bad clients can cause it to eat memory and this in turn can cause kernel memory leaks (of IOMedia objects). For example, on the currently shipping version of Leopard, the WindowServer process doesn’t appear to be properly processing Disk Arbitration notifications (probably because it uses a funny run loop) which is ultimately causing IOMedia objects to hang around in the kernel. It’s not a big deal but it does need fixing.

Oh, and one other thing, the system does not like it if Disk Arbitration crashes. Even though launchd will restart it, you still need to reboot as Finder and other processes don’t connect to the new version.

Lock free data structures are interesting I think. It’s usually much simpler to keep things simple and use regular locks but if you have a need for speed, lock free data structures can give that to you.

I was curious to know what difference it would make for a simple FIFO queue so I knocked up a mostly lock free based implementation and a conventional lock based version. I chose to make things slightly harder by choosing a fixed size queue and having it block when the queue was full.

The lock free version was nearly thirty times faster on the machine I tested it on. Disclaimer: I did this pretty quickly and I wouldn’t be surprised if there are bugs (I haven’t tested it on a multiprocessor weakly ordered architecture machine).

Here’s the code.

Now that the iPhone NDA has been lifted, all whining is now focused on App Store rejections. I’m sure you’ve all just read Gruber’s post. I’d just like to throw another theory in: what if Apple is rejecting these applications because they offer features that they are planning to add to their applications. If they allow these third party applications, then a) the fanfare when they announce their equivalent feature isn’t as good and b) they’re going to piss people off who’ve developed the third party application and also those that bought the application. By rejecting these applications early, they’re doing the developers a favour: better rejected now than find their application obsolete in a few months time. Obviously, it would be better if you could get rejected even earlier as Gruber and others have suggested.

I’ve just done the iTunes 8 update along with the QuickTime and other updates listed and found that OS X had a problem starting up afterwards. Anyway, I’ve tracked down the problem to it being a corrupted /etc/authorization file. Restoring this file fixes the problem. I’m not 100% sure it wasn’t something else I was doing that caused this but thought I’d post in case someone else encounters the problem.

Update: I thought I might explain how I figured out it was the /etc/authorization file was corrupt since you could use the same techniques to debug other start up issues.

The first thing I did was to boot up from another drive and run Disk Utility: no problems. Then I had a look at the system.log file in /var/log/. It showed a bunch of errors but the first one indicated that securityd had crashed. The crash log (stored in /Library/Logs/CrashReporter/) showed a crash in CFDictionaryContainsKey which got me thinking it was a problem with a property list file somewhere. To figure out which property list file, I booted the system in single user mode (Command-S) and typed:

    mount -uw /
    fs_usage >/var/log/fs-usage &

and then:

    exit

to continue booting. After rebooting into a working system, I was able to examine the /var/log/fs-usage file. I looked for the last thing that securityd did before it crashed and that was to read /etc/authorization and sure enough the file was garbage when I looked at it.

NSTableView allows you to have variable row heights in Leopard but here’s some code (MIT license) that will work back to 10.3. It works around a few awkward issues in Tiger and earlier that I suspect are fixed in Leopard.

I recently went through the process of making our applications signed which is something that’s new to Leopard.

I just wanted to mention a few things that I came across and aren’t necessarily in Apple’s documentation.

Signing Frameworks

When you sign frameworks, you have to sign a specific version. So, let’s say your framework is called CSMail, you’d sign CSMail.framework/Versions/A. If you try and just sign the top level folder it will silently fail, as will CSMail.framework/Versions/Current (see “Symbolic Links” below).

Symbolic Links

Any symbolic links will be silently ignored and this extends to the path you give to the codesign command line utility. I think there’s actually a problem with symbolic links: you can add them to a Resources folder and it won’t invalidate the signature (whereas you cannot add normal files). I’ve reported this to Apple (rdar://problem/6050445).

Resources

Everything in the Resources is signed according to resource rules. You can see what the default rules are by signing something and having a look at the end of the generated CodeResources file. Here’s a bit from the default rules:

<key>^Resources/.*\.lproj/</key>
<dict>
    <key>optional</key>
    <true/>
    <key>weight</key>
    <real>1000</real>
</dict>

It should be clear that this means that all the .lproj folders in the Resources folder are optional which allows you to strip out unneeded localised files.

You can use your own rules if you need to; the resource rules file that you pass in to codesign is the same as the CodeResources file but with just the rules key. We needed to omit a particular file that might get installed in the Resources folder by an old version of our application when upgrading to a newer version. Using custom rules isn’t officially supported by Apple but I cannot imagine it will be broken for a while (as it would affect many Apple applications). Nevertheless, there’s no point using them unless you need to.

Helper Tools

Since helper tools should be separately signed, they shouldn’t be placed in the Resources folder, they should go in the MacOS folder if they’re simple tools or the Contents folder if they’re bundles.

When you sign helper tools (that aren’t bundles), you’re supposed to add an Info.plist to the executable although I’m not sure why you have to do it since you can specify an identifier as a parameter to codesign. Apple’s documentation shows a way to add an Info.plist file using the linker but I wrote a script that creates an object file from an Info.plist file. The advantage of doing this is that you can configure Xcode to run the script and it will properly track the dependencies whereas if you just add it as a parameter to the link command, modifying the Info.plist file will not cause things to be rebuilt. The script has the added benefit of expanding macros in the Info.plist file which we use to substitute things like the current project version.

The other issue we have is that we use the same helper tools for different variants of our main application. For example, we have iDefrag, iDefrag Lite, the demo version and a specific version for one of our customers. Each of those variants has a different bundle ID and we want to use that same bundle ID for the helper tools. Rather than have four different targets for each helper tool (we have 4 different helper tools so we’d end up with 16 different targets), I wrote a simple tool that updates the bundle ID within an embedded Info.plist file (source is here). One thing to watch out for is that it cannot increase the size of the embedded Info.plist file; you must make sure that the bundle ID only decreases in length. Also, it only works for 32 bit architectures since we have no need for 64 bit architectures at the moment.

I hope this helps someone.

Joel Spolsky recently posted an article about disabling menu items and this met with negative responses from John Gruber and Daniel Jalkut.

I think Joel has an interesting point. Whilst I agree with John and Daniel that it’s bad if you can’t tell if a menu item is disabled, there have definitely been cases where it’s not been clear to me why a menu item is disabled and some kind of indication would have been helpful.

A related issue came up with one of our products, iDefrag. In earlier versions we had a “Go” button on the toolbar that was disabled when it wasn’t possible to start defragmentation and the usual reason for that was that it was because you can’t defragment the disk you booted from. We got no end of support about this so we changed the button so that you could still click it but the button was displayed in red (rather than green) to indicate that it wouldn’t work. If you click the button, a message is displayed informing you why you can’t defragment and what to do about it.

Taking this approach for menus would mean that you would be able to click a menu item, but it would still be displayed differently. I’m not sure whether that’s a good idea or whether a different approach would be better, such as using tooltips to indicate what the reason is.

For what it’s worth, I think John’s comment:

This is why Spolsky is a Windows developer, not a Mac developer.

is offensive and incorrect. Just thinking about this issue makes it exactly the kind of attention to detail that all developers should aspire to.