An Urgent Mori 1.6.10 Release To Correct Bugs, and Workaround Spotlight Flaws

While making the changes to Mori’s code for 1.7, I encountered some oddities in test results, and it turned out there was a bug which I had introduced in an earlier release. While it doesn’t appear to endanger data in Mori notebooks, it might not return all the results you expect in a search, or in entry summaries.

In addition, it has what I hope are a couple of performance improvements, continued improvements to Italian localization, and a work-around for Leopard’s insistence to treat non-Apple Spotlight metadata files as third-class citizens.

Normally when Spotlight discovers a file has been created or changed, it will ask the responsible program to figure out what’s inside, and feed it back to Spotlight. But one of the drawbacks to Spotlight’s design is it lacks the ability to define containers, or documents which contain logically distinct elements such as the chapters of a book, pictures in a photo album, or entries from a Mori notebook; and which can nest other containers as well. Treating a document as a single entity, Spotlight will open a document at the beginning (or maybe the place where the cursor was the last time it was open), even if what you’re looking for is somewhere near the end.

Because it doesn’t understand that a file can have distinct elements, the development teams for other Apple software (e.g., iPhoto, Safari, Stickies, etc.) came up with a scheme to trick Spotlight by creating new files with the data for those elements. So that’s how Jesse coded Mori’s behavior: duplicate the data for that logically distinct element in its own file. A separate copy of each element’s data in its own file. One extra file per element. That means the space taken up by your data is easily half again more than if Apple just added a container definition for Spotlight metadata (once for the notebook, another for the entry metadata file, and the third copy in Spotlight’s database).

But that isn’t all. While we’d like to keep all those extra files inside a notebook bundle (a directory which Finder treats as a file), because Spotlight treats a document as a single element it won’t look for those files inside the bundle. So Mori creates those files in the metadata cache folder (in your Library/Caches/Metadata folder), along with the metadata files from some of Apple’s programs. If you open the metadata folder and look at these files, you’ll see they have numbers to help Mori figure out which entry contains that data. But when you do a search using the Spotlight menu, and when you select menu item ‘Show All’ and the results are displayed in the Finder, you won’t see the numbers; you’ll see the titles for the entries they represent.

Leopard however, isn’t so democratic; which is why users where complaining about the entries when Leopard was released. First off, it ignores any non-Apple metadata files in the cache folder unless you set your Spotlight preferences to use those files. Secondly, it will ignore the title info embedded in the entry metadata file and just display the file’s actual title, meaning the number. How’s that for Apple undermining the work of third-party developers?

So the workaround I came up with is to add the entry’s title (or Untitled, if it has none) at the beginning of the filename, so you at least have an idea which entry matches your search terms.

(Thanks for wasting about a whole month total of my development time on that alone, Apple. I feel the love.)

I am, of course, more than happy to eat crow should I be proven to be completely mistaken or speaking from out-dated information. It’s easily worth it in order to improve the user experience.

Regardless of the rationale for the design decisions, enjoy, and thank you for being part of the community and continuing to support Mori!

How to Install the iPhone 2.2.1 SDK on a PowerPC-based Mac

Update: I forgot to mention that I got help from the great folks at NSCoder Nights in San Francisco when mucking about the installer and Xcode files. Particularly helpful were Bruce Spath and Dan Grover. Thanks again, guys!

Here’s how you can install the iPhone SDK for 2.2.1 on a Mac running at least MOX 10.5.5:

1. Ensure you have at least 6 gigs of disk space available. If you have tried to install the iPhone SDK on the target volume before, it may state an upgrade will be performed instead of an install. Sadly, the only solution I currently have for this situation is to uninstall Xcode using /Library/Developer/3.1/uninstall-devtools.
2. Download the SDK disk image.
3. Mount the image by double-clicking it.
4. Copy the mounted volume to a hard drive.
5. Navigate to iPhone SDK.mpkg/Contents/iPhoneSDK.dist in the copied folder and replace line 340 which should be
start_selected = "isIntel() && hasRightOS() && agreedToSLA()"

with
start_selected = "true"

6. Run the installer, selecting either the default location /Developer or another directory name if you’re looking to preserve your current Xcode installation.
7. After a successful installation, navigate from the installation directory (default of /Developer) to /Platforms/iPhoneSimulator.platform/Developer/Library/Xcode/Specifications/iPhone Simulator Architectures.xcspec, and make the following two changes.

First, replace

{ Type = Architecture;
Identifier = Standard;
Name = "Standard (iPhone Simulator: i386)";
Description = "32-bit iPhone Simulator architectures";
ListInEnum = YES;
SortNumber = 1;
RealArchitectures = ( i386 );
ArchitectureSetting = "ARCHS_STANDARD_32_BIT";
},

with

{
Type = Architecture;
Identifier = Standard;
Name = "Standard (iPhone Simulator: i386)";
Description = "32-bit iPhone Simulator architectures";
ListInEnum = YES;
SortNumber = 1;
RealArchitectures = (
i386,
);
ArchitectureSetting = "ARCHS_OLD_STANDARD_32_BIT";
},
{
Type = Architecture;
Identifier = Standard;
Name = "Standard (iPhone Simulator: ppc)";
Description = "32-bit iPhone Simulator architectures";
ListInEnum = YES;
SortNumber = 1;
RealArchitectures = (
ppc,
);
ArchitectureSetting = "ARCHS_STANDARD_32_BIT";
},

then, replace

{ Type = Architecture;
Identifier = i386;
Name = "Intel";
Description = "32-bit Intel";
PerArchBuildSettingName = "Intel";
ByteOrder = little;
ListInEnum = NO;
SortNumber = 105;
},

with

{
Type = Architecture;
Identifier = i386;
Name = Intel;
Description = "32-bit Intel";
"PerArchBuildSettingName" = Intel;
ByteOrder = little;
ListInEnum = NO;
SortNumber = 105;
},
{
Type = Architecture;
Identifier = ppc;
Name = "Minimal (32-bit PowerPC only)";
Description = "32-bit PowerPC ";
"PerArchBuildSettingName" = PowerPC;
ByteOrder = big;
ListInEnum = No;
SortNumber = 201;
},
{
Type = Architecture;
Identifier = ppc7400;
Name = "PowerPC G4";
Description = "32-bit PowerPC for G4 processor";
ByteOrder = big;
ListInEnum = NO;
SortNumber = 202;
},
{
Type = Architecture;
Identifier = ppc970;
Name = "PowerPC G5 32-bit";
Description = "32-bit PowerPC for G5 processor";
ByteOrder = big;
ListInEnum = NO;
SortNumber = 203;
},

Now go ahead and start Xcode and when you select the “File > New Project…” menu item, you should see a darling iPhone category for projects. Also, run the iPhone simulator in /(Xcode install path)/Platforms/iPhoneSimulator.platform/Developer/Applications/iPhone\\ Simulator.app, it’s really mind-blowing to run it on your desktop, especially one Apple tells you isn’t able to run their iPhone SDK.

By the way, if you don’t feel like going through these steps yourself I’ve put together iPhoneSDKonqueror, an app which will handle these steps for you in a mostly automated manner. You should buy it. It’s only USD$5 and if you appreciate the ability to use your PowerPC Mac to write apps for the iPhone, it’s the right gesture to make to me.

Mori Update: More Bugfixes and More Frequent Updates

Because of the difficulties fixing the toolbar bugs and getting Leopard compatibility complete (or reasonably so), Mori has quickly approached version 1.6.10 (not yet, only 1.6.7 has been released so far, but bear with me). This has some odd psychological barrier attached to it, as we seem to recognize it as a significant occasion, a hurdle we do not wish to cross.

I’ve been collecting fixes into a single release, attempting to conserve version numbers. There’s currently improved “Check and Repair Notebook”, more cautious handling of user preferences, improved Italian localization, a fix on the Drag and Drop stall, clearing of compiler warnings, some refactorings, and more unit tests. A couple more fixes I’d like to incorporate into this one: fixed word count (whether it’s in English or Greek), correct Smart Folder behavior and making wildcards optional in search terms.

But anything.10 is an artificial milestone, rather than a significant one. And MOX has already passed it and even gone on to 10.4.11, so what’s troublesome about it now? And with the new versioning class I added to Mori back in 1.6.4 or so, it should be able to handle even version 1.6.99 if necessary!

So I’ll be trying to post new updates more frequently. I’m not certain how frequently it’ll be, but I’d like to get to the point where there was a nightly build, like the Safari team provides. That’s too frequent for most users, of course, but then you’d be able to skip a few interim releases until something you need is included. The other benefit it would provide is allow me to move all the apps forward a little bit at a time, rather than doing continuous development on one app for weeks at a stretch before rotating development to the others.

A (Hopefully Brief) Update on the Delay of Mori 1.6.8

As I was reviewing the code changes I had made, I slowly realized that the Mac’s user preferences system had a minor philosophical flaw which makes for dangerous situations in the program.

One of the goals of this version is improved recovery from faulty stored values relating to the preferences and UI, so you can continue with only minor inconvenience and perhaps a need to again set preferences to your liking. Not adequately performing this resulted in some of the missing toolbar and empty window problems that were experienced before. (For a more exciting example of cascading faults, read about the 2003 Northeast blackout.)

So after some delays in getting the unit tests for this new component to execute properly, I’m now adding that component into the program. Once I have the Cocoa preferences system replaced, I’ll be releasing 1.6.8. Then I’ll work on the possible fix for the SpotLight ‘odd name’ bug for a quick-turnaround (later today) 1.6.9.

If that particular bug can’t be quickly fixed, I’ll work on some other long-standing issues in Mori for an update that will be released after a few days.

Incidentally, Tobias had a valid concern regarding the update process: “Where does testing feature in this? In olden times an update was a build with only known problems since a few people played with it for a week before it was awarded a version number. WebKit offers a choice between bleeding edge nightly r1234 and release 3.0.”

I plan to continue to rely on the Beta Test Group for testing updates, with a release candidate at least one version ahead of what is shipping. However, at the time of this writing there are 320 open bug reports for Mori, even after having pre-release testing candidates as part of the process. That just isn’t right.

I’m primarily depending on the unit tests to prevent shipping defects to you. They serve to expose bugs in existing and new code; and by adding tests for the types of bugs identified in the bug reports, the tests will continue becoming more valuable to the release process.

The purpose is to continuously speed up the process by improving its results. It shouldn’t be a shock when a program works right off the bat. The delays are (partly) associated with having so many steps of repeated inspections on the same system. It multiplies the time it takes to get a new release out the door. The less I tolerate it, the quicker it gets done correctly.

Happy New Year and a New Mori Update

In case you’re running one of the older releases of Mori, one with a bug in the update mechanism, this is just a brief note to let you know Mori v1.6.8 has now been released!

The fixes include:

  • Rewrote user settings handling subsystem (to prevent wayward settings from affecting other operations).
  • Improved notebook check and repair functionality to fix orphaned entries and entry data records.
  • Fixed stall when dropping items into Mori’s source or entries lists.
  • Fixed (or at least improved) Italian localization.
  • Fixed faulty version update notification.

The code I had hoped to incorporate into the update, but which didn’t make it (because it was already three weeks or so overdue) included fixes for Smart Folder Rules, non-wildcard searches, and word-count bugs, which are among those to be included in v1.6.9. Also to be included are user-settable font settings in the source and entries views, and the fix for the font prefs for notes.

Work on v1.6.9 has already begun, and my work in v1.6.8 improved the ability to perform testing in new releases. The Beta Test Gang will find the first test version on the site Jan 2nd.

Mori v1.6.9 is Being Packaged for Release

This has taken much longer than I hoped for, and it lacks the features I wanted to add in there, but due to another empty-window bug I’m releasing Mori 1.6.9 today as a stopgap measure.

I had hoped for a quick release to get the fix out to those who are upgrading now (particularly from older, Hog Bay Software versions), but I had upgraded my Xcode installation mid-stream, which is generally a bad idea if you can’t test the effects a tool migration will have on your projects. And Mori is quite unique in the way it’s constructed, being almost entirely plugins, and plugins which run plugins.

It took a week to resolve unit testing problems, some of which were due to Mori’s unique project files, some of which were due to changes in the layout of Xcode’s tools directories and one due to a bug in the unit tests for the ‘Check and Repair Notebooks’ functionality (which I wasn’t going to risk going out with a bug whose cause I hadn’t determined).

Anyway, testing problems aside, I’m packaging the latest for release now. You’ll be able to read the release info shortly.

Double Vision in Xcode doesn’t Double Your Fun

The Mac developer scene is loaded with a lot of helpful and supportive developers out there, sharing ideas, code and encouragement with one another. I’ve benefitted from the development community, as have the Mori and Clockwork code. I say this not only as one who’s currently putting new code into them, but seeing the comments Jesse left in the code before me!

So I’d like to drop an idea into your lap you might find useful for the right type of application. As there are different types of software one can develop, e.g., operating systems, utilities, paint programs and the like, they have particular usage characteristics that they encourage. Some, like digital notebooks and desktop timers for example *cough*, tend to remain in operation while the user is logged into his account. Usually, this isn’t a problem for the user. Unless, of course, the user is also someone further developing said software. Then several questions have to be answered, such as, “How do I run unit tests on builds while still running a stable version of my app?” and “How do I try out the latest build without affecting the files which are currently open in my app?”

On operating systems based on more primitive process management, it isn’t troublesome because each built application is fairly independent of others, and often multiple invocations of a single app can be run concurrently. It wasn’t problematic on the Mac either back on MacOS 9 and earlier, and even in the early MOX releases. But now with Launch Services, running test versions of apps you run on an all-day basis can be problematic. It’s best to give that test version some different signature so its defaults and file changes don’t corrupt your normal environment.

Note that this tip doesn’t prevent two versions from munging the same files; even two unrelated apps can mess up a data file if they’re updating it at the same time. What this tip does is provide an alternate identity, if you will, for test builds so MOX doesn’t try to hand it the same data set it gives the release builds of your app. You mustn’t try to open files in the test build that are currently opened by a release build.

Here’s how I do it:

1. In the Info.plist file, use the C pre-processor’s conditional compilation directive to give your app its normal metadata when a release build is made, but a special set of metadata for all other build configurations:

#if defined(Release)
<key>CFBundleIdentifier</key>
<string>com.apokalypsesoftware.Mori</string>
#else
<key>CFBundleIdentifier</key>
<string>com.apokalypsesoftware.Mori-debug</string>
#endif

Update: I forgot to mention that Xcode 3 and later versions now use specialized editors for the Info.plist files. To override this editor selection, bring up the file’s Info window (by selecting the file and pressing cmd-I) or the Inspector window and change the File Typesetting from “text.plist.whatever” to “text”. Once you add the conditional to Info.plist you can change File Type back and access Info.plist in its raw form from its target’s Properties tab. That way, you can view it in either mode within Xcode. However, making any changes in Xcode’s plist editor will wipe out any conditional directive in the file, so use it only for reviewing current settings, not for actual editing!

2. In the Target Info window (brought up via the ‘Project > Edit Active Target “Mori”‘ menu item), select the Build tab, specify “All Configurations” for the Configuration pop-up menu, and enter “plist” in the search toolbar widget to bring up the relevant setting items.

3. For the “Info.plist Other Preprocessor Flags” setting, enter -traditional as the value. While not needed to support the test version twin, it will help to prevent any URLs included in the Info.plist from being swallowed up by the pre-processor and possibly causing you sleepless nights because neither MOX nor the Xcode build system have the slightest idea of what the proper format for URLs are, but will behave erratically when it isn’t just so. So it’s best to make it a non-issue from the start. (No charge for that and other tips related to usage of the pre-processor you can find in Technical Note TN2175. Incidentally, TN2175 says to use “-traditional”, but I use “-CC”. Use whatever works for the version you’re on.)

4. For the Info.plist Preprocessor Definitions setting, enter $(CONFIGURATION). Xcode will change the display to read <multiple values>, but that’s okay because the value will vary according to the current build configuration. It will define a value that matches the build configuration that the pre-processor will test against in the Info.plist. (Try switching the setting in the Configuration pop-up menu of the Build tab to watch the value change to match the current setting.) This is what will direct the pre-processor to output the correct value for CFBundleIdentifier.

5. Check the Preprocess Info.plist File checkbox so Xcode invokes the pre-processor on the Info.plist before copying it to the application.

When built, you’ll be able to run the app concurrently with the release version, and without causing conflicts with data files in use by it. Of course, you won’t have your preferences set up, but that’s only a problem until you save preferences for the test version. (Remember, if you try to get around that problem by copying the release version’s defaults file that it also holds the recently opened files. Purge, close or what-have-you as appropriate!)

One of the extra things I attempted to do in this technique was to provide an alternate name for the private builds or test versions so they would be distinctly named for testers. We used to use naming conventions such as “Mori 1.7β4″ when distributing such versions back in the pre-MOX days, but including the metadata keys CFBundleExecutableCFBundleName, and CFBundleDisplayName don’t have any noticeable effect. Any tips in this regard would be greatly welcomed.

I had also attempted to get the pre-processor to generate the bundle identifier with “Mori-$(CONFIGURATION)” that fit whatever build configuration was in effect automatically, which would’ve avoided the use of the #if defined directive. Unfortunately, because the macro expansion would generate Mori-Release that method proved ineffective.

A corresponding idea is to use an alternate icon (via the CFBundleIconFile specifier) to represent the test version of your app. It helps you and your testers realize when odd behavior is due to running the wrong build just by looking at the dock or application-switcher. It’s also fun to double-click a document only to realize the test app is opening vital data meant only for the stable version. Hilarity is sure to ensue. But I’m a professional, don’t you kids try this at home!

If you don’t have the construction guy handy, you can resort to the technique I used before I discovered Info.plist processing: MOX’s own stamping of missing dock files makes for a handy visual guide to your test app. To try it yourself, add the debug version from within your build/Debug folder to your dock items. That’s right, the debug version (or whatever you use for test builds). Now delete it and empty the trash. (You will be building new ones, won’t you?) Now click on the app icon you just added to the dock. It adds a big fat ‘?’ to your lovely icon, doesn’t it? (At least it should if you deleted the correct icon.) Now rebuild your app and run it from within Xcode.

Now bring up the application switcher (cmd-tab). What do you see? A big fat ol’ ‘?’ emblazoned on the debug version of your app! Oh, you don’t? You see the normal icon? Yeah, you’re on Leopard, aren’t you? This trick doesn’t work as well on Leopard because MOX validates the file state more often than Tiger did. Such as, starting the test build from the Dock instead of only within Xcode. On Leopard, clicking on the dock icon when there’s a built debug version of your app in the debug folder will restore its unstamped icon. But anytime the question mark goes away, just delete the debug version, empty the trash, and click on the dock icon again.

Have fun, and keep ‘em flying!

bitShifter Joins Apokalypse, Revamped/Pro Mori 1.7 Release Imminent, Cocoalogue: Our Smalltalk-based Scripting System, and a Cancer-Fighting Bundle

As you can see, another bevy of announcements all bunched up in one. Did you feel like it was time to unwind? Nope. Me neither!

First off, an announcement that I put off while other things had to be wrapped up: Apokalypse Software Corp. has acquired the rights and the code to bitShifter, the file encryption utility initially developed and marketed by MemSculpt and then ForgEdit. As with prior acquisitions, the current licensees’ purchases will be honored, updates will be made, etc. I’m still trying to understand what some of these licenses were (perpetual what, now?), but they won’t be problematic as the key is in gaining more licensees rather than bleeding the current userbase dry with the nickel-and-dime tactics I despise so much.

However, the name and logo will need to be changed, as the original developer cannot relinquish them…or something. (bitShifter is vague and ambiguous or too techie sounding, anyway.) It is being revamped, and will be re-launched shortly as an Apokalypse product. The price will likely remain $99 USD if I can figure out what type of license that bought. (Perpetual what, again?)

Now that Apokalypse has encryption technology, I’m putting Mori 1.7 out with encryption ASAP. Unfortunately, this means some of the really cool features I originally planned to incorporate in the release will be delayed until 1.8, including enumerated entries, and continuous text. What’s still in? LinkBack, customized labels, font settings for source & entry views, better keyboard navigation, outlining improvements, self-downloading updates, and now encryption.

I’ve also gone over the features of quite a few of Mori’s competitors, and realize how undervalued Mori is. So I’ll be creating a version with fewer capabilities at the current price to stay at the lower end, and the price of the full-featured version will be raised to $99.95 USD. On top of that, a Pro version will be released at $199.95 USD. Mori licensees will automatically be bumped up to the Pro license when the update is released; so as I promised you before, you won’t have to pay extra for the 1.7 upgrades. In fact, as I’m still planning to put out a 1.8 release, current licensees won’t have to pay for any 1.X upgrades.

Next on the list is the big project I had been working on when I purchased Mori and Clockwork from Jesse. It’s a programming system based on the Smalltalk programming language and it’s called Cocoalogue. What’s so special about it? It’s an interpretive system, with programs written in a shebang-prefixed text file like most scripting languages available on UNIX-like platforms. The Smalltalk-based syntax is virtually identical to Smalltalk-80 with extensions for declaring classes, methods and data types (with strong- and static-typing). It supports dynamic run-time features including blocks, automatic garbage-collection and data translation. And it has, as the name Cocoalogue would indicate, a bridge to Ma’s Cocoa frameworks. I’ll go into greater detail on these features and Cocoalogu’s current limitations in my next post. This product hasn’t been released yet, and will be priced at $129 when it is.

But I’m making the announcement now for a very good reason. You’ll be able to get a licensed copy of Cocoalogue today before it’s made available anywhere else, including the Apokalypse website, through the PMC Software Build Your Own Bundle program. Seth Dillingham, another indie Mac developer, has put together a special bundle where you can purchase Cocoalogue, Mori, Clockwork or a family pack at discounted prices. You could even get a discount on over 120 other fabulous programs from Mac indie developers as well! Not that you want to.

So go on to the bundle site, remembering that it’s the only way to get in on Cocoalogue now, and for the substantial savings you’ll get on Mori Pro 1.7 by getting a Mori license today.

And don’t forget: these sales are going to fund cancer research and treatment, so please don’t be stingy on what are already terrific deals. A lot of folks are counting on you!