Mac OS X Server AFP high CPU load problem

This discussion thread over on Apple’s forums documents a large number of system administrators who have their AFP (Apple File Protocol) creating a high CPU load and thus making the server unusable for their users.

Suggested workarounds which have anecdotally provided varied success (including none 🙁 ) include:

  • Turning off Spotlight on client machines
  • Turning off Spotlight indexing on the shared volumes
  • Executing

    defaults write /Library/Preferences/ -dict-add afp_wan_threshold -int 1000
    defaults write /Library/Preferences/ -dict-add afp_wan_quantum -int 131702

    on all 10.5.x client Macs.

  • Installing Security Update 2009-001 which states

    Description: A race condition in AFP Server may lead to an infinite loop. Enumerating files on an AFP server may lead to a denial of service. This update addresses the issue through improved file enumeration logic. This issue only affects systems running Mac OS X v10.5.6.

The problem was originally posted with respect to Mac OS X 10.5.4 on 29th of August 2008, and still seems unresolved for many people. And people wonder why it’s hard to argue that Mac OS X Server belongs in the data centre.

Leopard’s Apache2 doesn’t rewrite

Have finally moved my postfix mail and blog servers to Leopard. Unfortunately none of the WordPress permalinks would work, despite having tried everything I could think of to enable URL rewriting, and testing with a fresh test blog install. Seems others have similar problems here.
and here

Fortunately fink can do an Apache install, with a mod_rewrite that actually works.

fink install apache2-common
daemonic install apache2
(I think I must have done a fink install apache2-mpm-prefork somewhere along the line too, but fink’s not listing it as installed)
After this I noticed lots of error messages in the System log about launchd trying to start Apache unsuccessfully:

Aug  6 00:00:05 botrytis-local org.apache.httpd[90412]: (13)Permission denied: make_sock: could not bind to address [::]:80
Aug  6 00:00:05 botrytis-local org.apache.httpd[90412]: (13)Permission denied: make_sock: could not bind to address
Aug  6 00:00:05 botrytis-local org.apache.httpd[90412]: no listening sockets available, shutting down
Aug  6 00:00:05 botrytis-local org.apache.httpd[90412]: Unable to open logs
Aug  6 00:00:05 botrytis-local[244] (org.apache.httpd[90412]): Exited with exit code: 1
Aug  6 00:00:05 botrytis-local[244] (org.apache.httpd): Throttling respawn: Will start in 10 seconds

So I thought I’d disable the built-in System apache:

cd /System/Library/LaunchDaemons
sudo launchctl unload -w org.apache.httpd.plist

and then removed /System/Library/LaunchDaemons/org.apache.httpd.plist to a safe position in case I ever need it. Of course this is messing with a System file, which you shouldn’t do, but I figure I can probably live with that.

Meanwhile, blog permalinks now seem to be working properly. Yay!

Now I just need to work out why I keep getting this in the system log
fseventsd[67]: callback_client: ERROR: d2f_callback_rpc() => (ipc/send) timed out (268435460) for pid 263

Lingering Leopard Lepidoptera

Back in Mac OS X 10.5.0, Princeton University’s Office of IT identified some problems with Mac OS X. Sadly some of them still seem to be around in 10.5.4, namely the Location menu in the Network Preference pane that has deleted locations re-appear, or that just seems to randomly mangle itself if you make any addition or deletions.

Apple Design Award Winners 2008

Incorporating some iPhone categories this year (because hey, they’re cooler than those boring old Macs), we have

Enable Apple Remote Desktop like features in Leopard’s Screen Connect

A very cool article over at Macworld shows how to enabled hidden features in Leopard’s built in Screen Sharing to give Apple-Remote-Desktop-like features such as 

  • Drop colour resolution to improve performance on low bandwidth connections
  • Switch between controlling and observing the remote Mac
  • Locking out the remote Mac’s keyboard and mouse, or hiding on-screen activity
  • Taking screenshots

Time capsule – take one a day

Picked up a 500Gb Time Capsule. Got home and was somewhat in a hurry. Power cable was a bit hard to get pushed in properly, although there was no visible evidence of why this would have been the case in either the plug or the socket.
Ignored the “Install this first” CDs, as usual, and fired up Airport Admin utility under Leopard on the wired LAN and it found the existing Airport Express. Did a “Save As…” to save the config of the existing Airport Express (including passwords). Then unplugged the Airport Express and plugged in the Time Capsule, attached to the network cable that used to be plugged into the airport express. Interestingly the assistant offered me the chance to use the Time Capsule to replace an existing wireless access point, but it wanted me to find it (and I’d just unplugged it :). So I plugged it back in to power and the LAN, but the Assistant couldn’t seem to find it. I gave up at this point, hit the Manual button and just imported the settings from the Airport Express (seemed like a safe bet) into the Time Capsule’s settings. This seemed to work. Changed security to WPA Personal and restarted the Time Capsule.
So, now turn on Time Machine on the Leopard box and get it started. That was at 18:30, and it’s now done 47.3 of 69.44Gb (1,396,130 items from my 80Gb drive) after some 5.75hrs (over Gigabit).
Meanwhile, every other machine had to have its Airport connection changed to WPA Personal, and even though I’m pretty sure I typed the password in several times, it took a while to register and stick (at least I hope it’s stuck now!).
Are network speeds faster? Hard to tell. I haven’t maxed out the speed as currently we need backward compatibility with 802.11g until the next round of upgrades. It offers 802.11n on 2.4 or 5GHz, and 802.11a compatibility as well as 802.11n/b/g.
Other noteworthy things are that the Airport Admin utility displays a set of warnings about the current Time Capsule configuration (ie no DNS, multiple DHCP, etc.).
The Time Capsule also offers to sync with a timeserver, and also flash its light if there’s a software update available. This is much less useful as it’s going to be stuck in a separate room where I’m unlikely to see its flashing light.
There’s also support to “Advertise configuration globally” via Bonjour, or so it seems to make it available globally over the internet. I don’t enable this.
I’d heard rumours of it running hot. Sure, it’s warm, but not really any warmer than my ADSL router, and it’s been doing a lot more work for the past 5 hours. Plus it’s got an internal power supply, so it’ll be warmer from that alone.
The other thing is now there’s an extra shared volume appearing in the Finder. Logging into it reveals a shared disk onto which I can put stuff. Cool. Checking in the Airport Admin shows it’s a guest read-writable volume, which is probably not good by default. Now locked down 🙂
So far, it seems to be behaving pretty much as expected. I could do setup in a hurry without too much pain. It claims to be backing up at a reasonable speed.
Now I just need to test recovery 🙂