EricBlog

End of Entourage

Goodbye, Entourage. This week I had my second major personal e-mail outage of the year. This time it was a silently corrupted identity in Entourage 2008. The symptom was that I stopped receiving mail. After several hours of diagnostics, I determined that Entourage could sync almost any folder with our Exchange server except the Inbox folder. I rebuilt my Office 2008 database (successfully, unlike last time), but the problem persisted.

I used one of my two free Office 2008 tech support calls with Microsoft to determine that the root cause of this was a corrupted identity, which can not be repaired. This despite the fact that the database tool could not find any corruptions. Their answer was to build a new identity, then export my 12000 e-mail messages from the old identity to the new one.

I asked about whether this could happen again, and the answer I got was "it depends. Some people have this happen more frequently than others.". As a manager with close to 40 people on my team and a variety of cross-functional responsibilities, my single most important work tool (besides my voice) is e-mail. I simply cannot afford the reliability problems I've had with Entourage 2008 in the last four months.

Therefore, I decided to export my old mail to Mail.app instead, and try it an iCal for a while. I ran into some problems with Entourage creating .mbox files that Mail gave up on reading partway through. The answer to that was to download Thunderbird 2.0 and use it to read the Entourage .mbox files, then let Mail.app import the Thunderbird .mbox files instead. This works, and now I have my entire set of mail stored in Mail.app.

We'll see if I can get used to Mail's oddities relatively quickly. I will miss how Entourage remembered the last ~20 folders I'd moved messages to, and made moving more messages to one of the same folders easier than it is in Mail. I won't miss that in Entourage dragging a message from the inbox to another folder copied it rather than moving it. I wont' miss Entourage's inability to create real HTML links in e-mail messages (where the text displayed is something other than the URL of the link).

Wish me luck.

Annoying documentation niglets

Well, I've been working with a partial Roller 4.0 install for the last several weeks while I tried to figure out why the planet aggregator would not work. I didn't get much help from the dl, since my problem was "bizarre". I should have known that this was a synonym for "you missed something so basic we can't recognize the problem".

It turns out that I had the config entry to enable the planet aggregator in the wrong properties file. Despite the effort to make Roller 4.0 simpler to install, there are still issues. In particular, the user documentation specifies only one properties file (roller-custom.properties). There is a bug documented that the planet aggregator actually requires its own properties file with some duplicated entries (e.g. the database connection configuration entries). I assumed that the setting that indicated that the planet aggregator was enabled belonged in this file; it turns out it needs to be in the roller-custom.properties file.

Now that I am past this little niglet, hopefully the rest of the roller 4.0 experience will be good.

Leopard's Time Machine corrupts Entourage 2008 database backups

When I installed Leopard, I decided that since Apple had embedded an incremental backup solution into the OS, that this would make it more likely that I could keep my backups current. I'm not really good at remembering to take the time to back up my notebook normally, and this seemed like it would be a great solution.

In evaluating this, I read several articles that claimed that using Time Machine with the Office 2008 identity database was suboptimal because it would take a lot of backup space, since the large database file would be completely backed up each hour. This didn't bother me much, because my mail database is so critical to how I work that I would trade a lot of backup disk capacity for a guarantee of losing no more than an hour of e-mail updates due to a hard disk failure or other issue.

Then it happened. I had sync problems after upgrading to 10.5.2 (see my other post on this topic), and in the process of trying to resolve them, I accidentally destroyed all of my calendar data. I figured this wouldn't be a problem; I could just restore the identity folder from my Time Machine backup and I'd be OK. After doing the restore, Entourage would not start because the database file was corrupt. I tried to rebuild the database using the Microsoft database utility. After running for ~18 hours, it failed with an unrecoverable error.

That's when I started to look at the database file in each of the last several Time Machine backups. There were four consecutive hourly backups between 10am and 2pm in which the database file was the same size and had the same last modified timestamp. Since I'd sent and filed numerous messages during that time, it seemed to indicate a problem. I tried restoring each of these backups, and all were corrupt. I then went further backward to the first backup that had a different last modified timestamp on the database file. It too was corrupt. Again there were several hourly backups with this same last modified timestamp, although there should have been modified data between each backup.

I then gave up and called Microsoft. While I was on the phone with them, I found this KB article that states "If Time Machine starts an automatic backup while Entourage 2008 is modifying the user database, the backup process may damage the backup copy of the user database.". The Microsoft support rep I talked to on the phone was more blunt; she said that it was unlikely that any of my time machine backups of the database would be whole & usable, and none would be recoverable via the database utility due to the nature of the corruption.

After close to two days using Outlook via Parallels while I tried to figure out a way to get a relatively recent backup restored, I gave up and went back to the last SuperDuper backup I had before I started using Time Machine. I lost months of e-mail because of this problem, and I'm not at all happy. It seems highly unlikely to me that Apple and Microsoft would not have found this during any reasonable testing. If they had, there should at least have been documentation of the problem, and more likely they should have flagged the Office identify file to be skipped by Time Machine, with a clear notice to this effect in both the Leopard installer and the Office 2008 installer.

Leopard 10.5.2 Sync Conflicts between Entourage and iCal

I have Entourage configured to sync my Exchange calendar to iCal, so that I can use iTunes to transfer it to my iPhone. It took a long time to get this set up initially, and during this process I ended up using Syncrospector under Tiger to clear out the Sync Services truth database to finally stop getting duplicate entries and/or unresovable conflicts. After doing so, however, it was very usable.

I want the sync to be one-way for this calendar, so that there is no possibility of sync conflicts damaging my Exchange calendar. 90+% of my appointments are work-related. I keep the rest in a separate iCal calendar, and therefore I can see both on my iPhone normally. There is a setting for this in Entourage sync services that allows you to specify that the sync is one-way, and the Entourage data will always overwrite the iCal data.

Then I upgraded to 10.5.2. Right after the upgrade, I started getting notifications every couple of hours that there were > 200 sync conflicts between iCal and Entourage. If you think about how the setting works, this should be impossible on its face. I've configured the sync to overwrite iCal's version of the calendar; since I did so, how could there be a conflict? In any case, no option I selected in the conflict resolution dialog would clear these exceptions.

I decided to re-do what I'd done originally to get this working smoothly. I turned off syncing from Entourage. I deleted the Entourage calendar within iCal. I deleted the synchronization-related preferences files in /Library for both Entourage and iCal. I stopped the Microsoft database daemon process. I used Syncrospector to empty the Sync Services truth database. I then started Entourage and re-enabled sync services. I then restarted iCal.

The Entourage calendar had been recreated in iCal, but it was empty. I looked at the Truth database in Syncrospector and noticed that it had > 4000 entries from Entourage, so the Entourage sync was working. I looked at the sync state information for iCal in Syncrospector, and noticed that the "fast" sync status for iCal showed errors. I tried it again, and it failed again. I changed the sync type to "Refresh Sync", and that failed too. I then made my fatal mistake, and changed the sync type to "Push the Truth". I understood this to mean that it would push the existing truth data to iCal. Instead, it pushed the "Truth" that the Entourage calendar should be empty to the Sync Services truth database. Worse that that, Sync Services then overrode my Entourage preferences and pushed the Sync Services truth to my Entourage calendar and deleted all of the entries in it. Which then Entourage kindly synced to the Exchange server and deleted all of my calendars there.

There are multiple bugs/bug-like features here. First that you can configure a one-way sync and end up with unresolvable sync conficts. Second that you can use a developer utility like Syncrospector to override a one-way sync definition. Third that the wording in the Syncrospector utility is open to such dangerous misinterpretation.

I thought this problem would simply be annoying; I'd just restore the Entourage database from the most recent Time Machine backup and move on. Little did I know that I wasn't done finding Leopard/Entourage bugs and bug-like features. In fact, the worst was yet to come. See my upcoming post how Time Machine corrupts backups of the Office 2008 identity database.

Finally have Parallels working again

After an extensive amount of work, I finally have Parallels working again the way I want it to. One shared hard disk image for both my work and personal accounts on my mac, so I can use the same Windows install for both. The more recent builds of Parallels have change things a bit; they both now store the pvs files in the user accounts /Library folder; earlier builds of parallels could share a pvs file in /Users/Shared. Now I have the hard disk image in /Users/Shared, and separate .pvs files (virtual machine definitions) in each user account.

I also struggled with updating a copy of Windows from May 07 (my last good backup before the Windows config got hosed). Windows Update via the web browser just ran my CPU utilization up to > 100% forever without appearing to make progress. I restarted Windows, and set the windows update automatic timer to noon (so it would run while the machine was up). This worked, when I shut it down in the afternoon, it had 19(!) updates to install before shutdown completed. Hopefully that was all of them...

It appears that Parallels finally fixed shared networking to work properly. When I use it at work, I have a wireless network connection that requires a VPN to access the corporate network, and a wired Ethernet connection that does not. Earlier parallels builds insisted on using the wireless connection, and therefore being unable to resolve names on the network. This current build seems to correctly figure out that I have two connetions, and that it should try to resolve names on both before giving up.

Now that I'm done with Windows Update, it does seem that the VM is less of a hog; I don't notice as much that it is up and running in the background.

Trying Parallels build 5580

So I posted a couple of weeks back about how my Windows VM under Parallels had become damaged; I was no longer able to run Windows Update, and I kept getting a popup on each boot about "Error loading Tcp Mib library". I restored the disk image from a backup, and thought I was good to go. Earlier this week, I started up parallels to use IE again, and got an error message that the virtual disk was corrupted and could not be booted. Argh.

I wasn't sure how long this had been the case, so I checked my Time Machine backups to see how many different timestamps I had since I'd restored the original version. Only one, and it was corrupt too. Double argh. So today I downloaded the beta build for Leopard (5580), and again brought back the old Windows image from my April/May backup. Hopefully this one will last for a while; I really don't enjoy the extended process of parallels tools installs followed by multiple windows updates. That ain't exactly speedy.

I really, really wish Apple would build a full-featured Exchange replacement so that I could get rid of Parallels entirely. I would live with Microsoft's remote desktop client for IE when necessary if that was all I needed from Windows.

Installed my cell phone signal booster

My only reluctance in purchasing the iPhone was the relatively poor coverage that ATT has in my home and near where I work. I love having an iPhone, but hate that in my kitchen and most anywhere else on the first floor of my house I get really bad coverage. So I decided to try a cell phone signal booster. After looking around on the Internet, I settled on the zBoost YX510-PCS-CEL. It seemed to get mostly decent reviews, and promised to work for both the iPhone and my wife's Verizon phone.

I was hoping to avoid having to install the external antenna on the roof, and after crawling around in my attic with my iPhone set in field service mode, I was able to figure out that on one wall of the attic I got excellent (5 bars) coverage. I put the external antenna on that wall, and ran the cable through the ceiling of my linen closet (along with the power connector). Can't say it is the prettiest or most professional installation, but it is invisible from inside the house.

Once installed, I did see why some of the reviewers complained. Right next to the booster, I get five bars (instead of 2). This drops off pretty quickly; I only get ~2 bars downstairs. However, the call quality (as measured by talking to my wife when I called her on the land line phone) is distinctly improved (verified by unplugging the extender and repeating the same experiment).

Interestingly, the field test mode doesn't report that much of a boost (hovers around ~91 db for the best two towers without the booster, and ~70 db with the booster). Apparently, that is enough to make a quality difference on the call.

Why would you run Windows any way other than in a VM?

I just finished restoring my Windows virtual disk from a backup in April. At some point in the last couple of months, it got seriously hosed. At startup, I'd always get an "Error loading Tcp Mib" error, and I could no longer run Windows Update.

I googled around for a while about the error, and didn't find anything that helped solve the problem. Microsoft support is free if you can't run Windows Update, so I opened an incident. 48 hours later I got a e-mail from an MS support rep that said the following (a direct quote):

Dear Eric,

Just wanted to say hi and check if there is anything that I can do for you on this particular Service Request. If so, please do not hesitate to let me know and I will be happy to help.

I'm standing by for your reply.

Best Regards,

(name withheld)
Microsoft Windows Update Support Professional

This is not an especially confidence-inducing e-mail, since I'd described the problem (including the error number) when I filed the incident. It was also interesting that it had been well over the 24 hours promised for the incident. Since I'd had a two-week go-round with Microsoft Entourage support that initially cost me $250 before they refunded the money (because I had to solve the problem myself), I decided that the best bet was to drop back to a backup of the VM from before the problem occurred.

This reminded me again why virtualization is so useful. Restoring a VM (a single OS file) is so much simpler and easier than a full restore, especially if you've been storing your documents outside the VM's hard drive. Since Parallels allows me to use documents in my host OS (Mac OS X), I didn't lose any data when I restored to a backup that was 6 months old.

OK, now Roller is getting way cool

I've been using Roller since 1.0, and I'm impressed with the latest production release (3.1). The new themes (see www.rollerthemes.com) are great.

I haven't started using the topic cloud stuff yet, but I'm looking forward to doing so. Now if only there was an editor plugin that worked correctly in Safari...

Photo Nirvana

Now that I've nearly achieved music nirvana, next on the list is photo nirvana. Specifically, I want a well-designed and easy to maintain way to manage my pictures.[Read More]

Trying out Planet Roller's feed aggregation

So I've been investigating more what you can do in blogspace, and what you can do in particular with Apache Roller (the blog system on my site). I just enabled the Planet RSS aggregator, and am experimenting with the results.

[Read More]

Flexible Spending Accounts

I just finished filing medical and child care flexible spending reimbursements. Have you ever stopped to think about what an incredible waste of time, energy, and money this type of program is?

There is a huge administrative structure that requires work by corporations, the government, and each individual citizen that costs all of them time, money, and energy. The government has to define the rules and set up monitoring to ensure they're followed. Each corporation has to set up and administer a plan. Each citizen has to bet some of their money that they'll incur at least a specific amount of allowable expenses. Then each citizen has to record and file paperwork to document the expenses they did incur. Someone has to administer the paperwork and funding transfers for all of this.

And what is the purpose? To reduce the tax impact on individuals. This has to be the least efficient way possible to avoid giving the federal government some of your money. Imagine how much more efficient it would be to just reduce the tax rate for people earning less than X by some small percentage!

Perhaps worst of all, this type of program is likely to miss helping those most in need; those who work for companies that do not provide flexible spending accounts, or who don't have the time or knowledge to participate.

The most significant beneficiaries of this type of program are the lawyers and accountants who have a new line of business in setting up and maintaining these plans.

Next time you hear someone propose a stupid idea like this, make sure to let your Congressman know that they need to find a better way to offset the costs of child care and health care on working Americans.

Really non-intuitive way to add network shares to Leopard Finder sidebar

So I do like the sidebar on the Leopard finder, and I really like the implementation of autofs that makes it super simple to map NFS shares. I have an NFS server at home, and tons of NFS servers at work. The main remaining frustration I had was the inability to browse NFS shares via the finder. I was only able to browse shares by using the Goto function and typing in a directory; it was impossible to navigate from the root filesystem into the share. I'd read that it was possible to add shared folders to the sidebar, and so I tried dragging a folder into the Places section of the finder sidebar, to no avail.

Lo and behold in the comments for someone's Leopard comments, someone mentioned that you can add mounted shares to the Places section by dragging the folder over there while holding down the apple & option key together. This makes a significant usability leap forward for me, and I just can't believe that the technique for doing this is so obscure. I'm starting to look forward to going back to work now so that I can add the shared folders I use there to the finder settings on my other account.

I honestly don't know why the UI is build so that you cannot drag a folder onto the word PLACES and have it get added to the places dialog. This seems like a significant usability hole that could be filled relatively easily. If there is an OS-X UI convention that prevents this, I'd love to hear about it.

Music Nirvana (nearly) Achieved

A few years ago, I started to see the potential of the combination of the Internet, digital music, and mobile technology. I could feel that it should be possible to collect all of my music onto a computer, have it fully indexed & searchable, and available pervasively throughout my house. I was imagining the ultimate expression of this, which would be a computer in audio form factor that would catalog your CD collection automatically, make audio controls available via the web, and therefore place total control of your collection at your fingertips.

I'm starting to benefit from the realization of this vision through the excellence of implementation of Apple's iTunes, iPhone, and some of its supporting infrastructure. As I write this, I'm listening to my most recent playlist both from my notebook and my living room speakers (song is being streamed to an AirPort Express). I have Signal installed, so I can carry my iPhone with me and walk away from my notebook, and still have the control to skip songs, change volume, change playlists, search for other songs to play, or anything I'd like.

Most of my collection is on my iPhone itself, so if I wanted to just use earphone/earbuds, I could have nearly the same experience (but somewhat less comfortably).

I've had pieces of this working for a few years. I bought the AirPort Express to stream music to over a year ago, and stopped using it because it dropped out too often. Now that I have the AirPort Extreme with Gigibit Ethernet, I haven't had a single drop.

I spent a few hours yesterday searching for album art online, so that I could have a nearly complete collection for display via CoverFlow in either iTunes or on the iPhone. There are a variety of classical CDs that I still don't have covers for, but I can scan them if it really bugs me (I did save all of the covers...)

The only remaining fly in the ointment at this point is the lack of a reasonable audio-style form factor box from Apple that I could use to replace the Linux box in my house, so I could run the music completely off my server and still access iTunes DRM-protected tracks. The pieces to stream non-DRM tracks are clearly available for Linux, but I don't want to spend time on a solution that isn't complete.