While making the changes to Mori’s code for 1.7, I encountered some oddities in test results, and it turned out there was a bug which I had introduced in an earlier release. While it doesn’t appear to endanger data in Mori notebooks, it might not return all the results you expect in a search, or in entry summaries.

In addition, it has what I hope are a couple of performance improvements, continued improvements to Italian localization, and a work-around for Leopard’s insistence to treat non-Apple Spotlight metadata files as third-class citizens.

Normally when Spotlight discovers a file has been created or changed, it will ask the responsible program to figure out what’s inside, and feed it back to Spotlight. But one of the drawbacks to Spotlight’s design is it lacks the ability to define containers, or documents which contain logically distinct elements such as the chapters of a book, pictures in a photo album, or entries from a Mori notebook; and which can nest other containers as well. Treating a document as a single entity, Spotlight will open a document at the beginning (or maybe the place where the cursor was the last time it was open), even if what you’re looking for is somewhere near the end.

Because it doesn’t understand that a file can have distinct elements, the development teams for other Apple software (e.g., iPhoto, Safari, Stickies, etc.) came up with a scheme to trick Spotlight by creating new files with the data for those elements. So that’s how Jesse coded Mori’s behavior: duplicate the data for that logically distinct element in its own file. A separate copy of each element’s data in its own file. One extra file per element. That means the space taken up by your data is easily half again more than if Apple just added a container definition for Spotlight metadata (once for the notebook, another for the entry metadata file, and the third copy in Spotlight’s database).

But that isn’t all. While we’d like to keep all those extra files inside a notebook bundle (a directory which Finder treats as a file), because Spotlight treats a document as a single element it won’t look for those files inside the bundle. So Mori creates those files in the metadata cache folder (in your Library/Caches/Metadata folder), along with the metadata files from some of Apple’s programs. If you open the metadata folder and look at these files, you’ll see they have numbers to help Mori figure out which entry contains that data. But when you do a search using the Spotlight menu, and when you select menu item ‘Show All’ and the results are displayed in the Finder, you won’t see the numbers; you’ll see the titles for the entries they represent.

Leopard however, isn’t so democratic; which is why users where complaining about the entries when Leopard was released. First off, it ignores any non-Apple metadata files in the cache folder unless you set your Spotlight preferences to use those files. Secondly, it will ignore the title info embedded in the entry metadata file and just display the file’s actual title, meaning the number. How’s that for Apple undermining the work of third-party developers?

So the workaround I came up with is to add the entry’s title (or Untitled, if it has none) at the beginning of the filename, so you at least have an idea which entry matches your search terms.

(Thanks for wasting about a whole month total of my development time on that alone, Apple. I feel the love.)

I am, of course, more than happy to eat crow should I be proven to be completely mistaken or speaking from out-dated information. It’s easily worth it in order to improve the user experience.

Regardless of the rationale for the design decisions, enjoy, and thank you for being part of the community and continuing to support Mori!

While trying to solve a user’s problem with an mGTD script, I came across a subtle issue that demonstrates some issues that arise when violating a programming philosophy, tackling bugs in other people’s code, and general uncertainty whenever coding in AppleScript.

Working with AppleScript is generally considered iffy, because a lot seems ambiguous and so much is dependent on how the dialect is interpreted and how scriptable apps handle some of the application events which scripting is dependent on. I’ve written scripts before, some I’m pretty awed by (that it works, actually, but also what it does), but I’m still hesitant to tackle some scripting issues. In addition, being a GTD greenhorn, and an mGTD noob made trying to respond to this issue authoritatively very questionable.

Thankfully, BMEGuy, mGTD’s author and all-around community nice guy, tackled the question with a quick solution. But the updated script was still problematic, and so I felt I really needed to participate in coming up with a solution.

Again, being an mGTD noob and all, it took me at least half an hour to figure out how the plugin worked, and the script on top of that. Then, after I was able to get the script to run, it worked for me. Hmm.

But that’s because I was testing with an entry with a date due of today. Once I switched it to later in the week, the entry was still showing up for today. Isn’t that odd? It seemed I had inadvertently left in the date line from the original script. When I removed it, I witnessed the same problem.

It turns out there’s a bug in MOX 10.4.11’s iCal 2.0.5 (I’m guessing it’s present in earlier versions as well) where it doesn’t properly update the calendar display for new events made by the script. You won’t see it in the monthly view. However, you might notice a little oddness in the weekly view.

You can see the event if you add ’show theEvent’ after the script makes a new display alarm for the event (between the 2nd and 3rd ‘end tell’ up from the bottom). This will display it’s properties in the info drawer, but you won’t see the event anywhere on the calendar (in either week or month view) until iCal is restarted.

Running the script in monthly view doesn’t show any artifact in the calendar, but the data is shown in the info drawer.

You could also run the script in the weekly view and then switch to the monthly view, in which case you get this:

So now that the question of the event’s presence in the calendar was settled in my mind, I had to figure out why my faulty script displayed the event, but not the proper one; and how to coax iCal to display it.

Being unfamiliar with mGTD still, I tried to figure out the difference between the attribute name “dateDue” and due date. due date is one of the standard properties for entries in a Mori document. attribute name “dateDue” is a user column added in the example mGTD notebook. You can view them all the user columns by selecting the menu item Edit > Edit Notebook Columns…

Okay, good so far, but why would one cause iCal to display properly and not the other? After moving the due date line about for a while, I checked Script Editor’s Event Log, and saw

The event reply for the due date had a missing value! Mori wasn’t returning a value for the due date property because it wasn’t set (and wouldn’t be in the example notebook). Now I had to find a way to use one of those missing values to make theEvent visible without setting it to the wrong date. And the problem with that is most of the properties used in Mori’s entries aren’t appropriate for an iCal event.

I eventually thought about re-ordering the messages to iCal instead of being so fixated on a change in the messages to Mori or playing with the properties being set in creating the event. What I came up with was a plan to use the messed up missing value date as before to make the event visible first, and then set the date correctly. The code turned out like this:

tell application "Mori"
	tell current entry
		set theDate to (get attribute name "dateDue")
		set faultyDate to due date
		set theName to name
		set theNote to note
	end tell
end tell
tell application "iCal"
	tell calendar "Scramble" -- the user should specify the name of the target calendar here
		set theEvent to make new event at end with properties {description:theNote, summary:theName, start date:faultyDate, allday event:true}
		tell theEvent
			make new display alarm at end with properties {trigger date:theDate}
		end tell
		-- show theEvent
		set theEvent's start date to theDate
	end tell
end tell

And to my surprise, it worked! So as I began gathering the materials together for my reply to the issue, I noticed something in the event’s info drawer that had escaped my attention before:

iCal, that’s just crazy talk! But at least it would explain why it would display traces of an event, if anything at all; and why it wasn’t noticeable earlier: iCal would correct the event data when reading it in when it started (”iCal database, that’s just crazy talk!”). But somebody forgot to add a sanity check when creating a new event from the properties passed to it by our script. (This is an example of why the Once and Only Once principle should be heeded. If there’s only one place where events are synthesized from pre-recorded values, whether those values are from a stored file, a script or the UI, then all those code paths will benefit from any sanity checks added to event creation.)

Knowing this, here’s another means of working around this bug, by sending iCal info that won’t confuse it:

tell application "Mori"
	tell current entry
		set theDate to (get attribute name "dateDue")
		-- set faultyDate to due date
		set theName to name
		set theNote to note
	end tell
end tell
tell application "iCal"
	tell calendar "Scramble" -- the user should specify the name of the target calendar here
		set theEvent to make new event at end with properties {description:theNote, summary:theName, start date:theDate, end date:(theDate + 1), allday event:true}
		tell theEvent
			make new display alarm at end with properties {trigger date:theDate}
		end tell
		-- show theEvent
		-- set theEvent's start date to theDate
	end tell
end tell

Thinking about these two solutions it’s clear that picking the latter one, with well-formed properties, is the safest choice to make. Here’s additional proof: the first solution, the one which plays with the start date to make the event appear, will indeed make the event appear. But if there’s less than 24 hours until the event begins, it will appear on the wrong date and still require iCal to be restarted to appear in the proper location!

It just goes to show you, while you might be able to get away with just the barest minimum, and someone else might normally clean up after you, it’s best if you did the job correctly from the start in case your safety net disappears from under you.

Because of the difficulties fixing the toolbar bugs and getting Leopard compatibility complete (or reasonably so), Mori has quickly approached version 1.6.10 (not yet, only 1.6.7 has been released so far, but bear with me). This has some odd psychological barrier attached to it, as we seem to recognize it as a significant occasion, a hurdle we do not wish to cross.

I’ve been collecting fixes into a single release, attempting to conserve version numbers. There’s currently improved “Check and Repair Notebook”, more cautious handling of user preferences, improved Italian localization, a fix on the Drag and Drop stall, clearing of compiler warnings, some refactorings, and more unit tests. A couple more fixes I’d like to incorporate into this one: fixed word count (whether it’s in English or Greek), correct Smart Folder behavior and making wildcards optional in search terms.

But anything.10 is an artificial milestone, rather than a significant one. And MOX has already passed it and even gone on to 10.4.11, so what’s troublesome about it now? And with the new versioning class I added to Mori back in 1.6.4 or so, it should be able to handle even version 1.6.99 if necessary!

So I’ll be trying to post new updates more frequently. I’m not certain how frequently it’ll be, but I’d like to get to the point where there was a nightly build, like the Safari team provides. That’s too frequent for most users, of course, but then you’d be able to skip a few interim releases until something you need is included. The other benefit it would provide is allow me to move all the apps forward a little bit at a time, rather than doing continuous development on one app for weeks at a stretch before rotating development to the others.

It looks like the showstoppers which were part of 1.6.4, 1.6.5 (and even 1.6.3 counting the toolbar) have been dealt with. The latest build even seems to function normally in Leopard.

Thus, now that Leopard 10.5.1 has been released, and my Leopard install was for testing purposes anyway and not as my primary development environment, I’m going to install it and test the 1.6.6 build against it as well to make sure it still works. That is, after a stopover in Tigerland just to make sure that nothing in Leopard loused up my main volume.

The problems in Leopard appear to result, not from changes in the architecture of the Cocoa classes, but in how stringent the standards for values passed to them are, and how they deal with values which are unacceptable (invalid parameters and exception handling). Sure, code should only pass correct data all the time, but sometimes our expectations are off. Sometimes we get incorrect data ourselves (Garbage In Garbage Out), and sometimes, bad things happen in the real world in which computers operate.

Tiger used to be somewhat more non-chalant about this issues. It would ignore all but the most egregious problems, unless the programmer or the user asked otherwise. Leopard seems to be more restrictive and demanding of Mac developers and the code they write. Again, not bad by any means, except for the unexpected and the change in behavior for things that used to work before.

So, with more testing and better debugging (so why can’t I get “debug” suffix to work?!), we’ll hopefully see this occur less often in Mori’s code.

Now…back to that checklist.

Ever since I decided to treat my blog more like Twitter, and just write micro-events rather than an entire epistle, writing either has come to a virtual standstill. (Except of course for the firestorm that has been Mori v1.6.4, v.1.6.5, and v.1.6.6 which is now undergoing 3rd party testing and I’m still trying to squash that “freezes while writing in Leopard” bug.)

Option-clicking the ‘Run’ icon in Xcode3 causes Mori to execute, then gdb starts up and attaches to Mori’s process, then Mori quits. Huh?

As if that weren’t enough, everytime gdb starts up, it spews out a lot of warnings about object files it can’t find. Like so,

warning: Could not find object file “/BinaryCache/Libsystem/Libsystem-111~176/Root/usr/local/lib/system/libc_debug.a(errno.o)” - no debug information available for “/SourceCache/Libc/Libc-498/sys/errno.c”.

warning: Could not find object file “/usr/local/lib/system/libcommonCrypto_debug.a(md2_dgst.o)” - no debug information available for “/SourceCache/CommonCrypto/CommonCrypto-32207/Source/Digest/md2_dgst.c”.

warning: Could not find object file “/usr/local/lib/system/libcommonCrypto_debug.a(md4_dgst.o)” - no debug information available for “/SourceCache/CommonCrypto/CommonCrypto-32207/Source/Digest/md4_dgst.c”.

warning: Could not find object file “/usr/local/lib/system/libcommonCrypto_debug.a(md5_dgst.o)” - no debug information available for “/SourceCache/CommonCrypto/CommonCrypto-32207/Source/Digest/md5_dgst.c”.

warning: Could not find object file “/usr/local/lib/system/libinfo_debug.a(gethnamaddr.o)” - no debug information available for “gethnamaddr.c”.

warning: Could not find object file “/var/tmp/Libm/Libm-287.1~6/Libm.build/Libm_debug.a.build/Objects-normal/ppc/scalb.o” - no debug information available for “/SourceCache/Libm/Libm-287.1/Source/PowerPC/scalb.c”.

What kind of railroad are we running here?

Anyway, at this point I’m planning on changing the file format after v1.7 ships. It’s just making it too tough to do some fixes. Mori still won’t require Leopard for some time yet, but I have to make my job, and putting out updates, somewhat simpler.