An Urgent Mori 1.6.10 Release To Correct Bugs, and Workaround Spotlight Flaws

While making the changes to Mori’s code for 1.7, I encountered some oddities in test results, and it turned out there was a bug which I had introduced in an earlier release. While it doesn’t appear to endanger data in Mori notebooks, it might not return all the results you expect in a search, or in entry summaries.

In addition, it has what I hope are a couple of performance improvements, continued improvements to Italian localization, and a work-around for Leopard’s insistence to treat non-Apple Spotlight metadata files as third-class citizens.

Normally when Spotlight discovers a file has been created or changed, it will ask the responsible program to figure out what’s inside, and feed it back to Spotlight. But one of the drawbacks to Spotlight’s design is it lacks the ability to define containers, or documents which contain logically distinct elements such as the chapters of a book, pictures in a photo album, or entries from a Mori notebook; and which can nest other containers as well. Treating a document as a single entity, Spotlight will open a document at the beginning (or maybe the place where the cursor was the last time it was open), even if what you’re looking for is somewhere near the end.

Because it doesn’t understand that a file can have distinct elements, the development teams for other Apple software (e.g., iPhoto, Safari, Stickies, etc.) came up with a scheme to trick Spotlight by creating new files with the data for those elements. So that’s how Jesse coded Mori’s behavior: duplicate the data for that logically distinct element in its own file. A separate copy of each element’s data in its own file. One extra file per element. That means the space taken up by your data is easily half again more than if Apple just added a container definition for Spotlight metadata (once for the notebook, another for the entry metadata file, and the third copy in Spotlight’s database).

But that isn’t all. While we’d like to keep all those extra files inside a notebook bundle (a directory which Finder treats as a file), because Spotlight treats a document as a single element it won’t look for those files inside the bundle. So Mori creates those files in the metadata cache folder (in your Library/Caches/Metadata folder), along with the metadata files from some of Apple’s programs. If you open the metadata folder and look at these files, you’ll see they have numbers to help Mori figure out which entry contains that data. But when you do a search using the Spotlight menu, and when you select menu item ‘Show All’ and the results are displayed in the Finder, you won’t see the numbers; you’ll see the titles for the entries they represent.

Leopard however, isn’t so democratic; which is why users where complaining about the entries when Leopard was released. First off, it ignores any non-Apple metadata files in the cache folder unless you set your Spotlight preferences to use those files. Secondly, it will ignore the title info embedded in the entry metadata file and just display the file’s actual title, meaning the number. How’s that for Apple undermining the work of third-party developers?

So the workaround I came up with is to add the entry’s title (or Untitled, if it has none) at the beginning of the filename, so you at least have an idea which entry matches your search terms.

(Thanks for wasting about a whole month total of my development time on that alone, Apple. I feel the love.)

I am, of course, more than happy to eat crow should I be proven to be completely mistaken or speaking from out-dated information. It’s easily worth it in order to improve the user experience.

Regardless of the rationale for the design decisions, enjoy, and thank you for being part of the community and continuing to support Mori!

The 21st century has barely started, and I’m already wishing it would end

Even after releasing two updates to the same app in as many days, there are still unresolved, critical issues that have to be corrected. I’m attempting to get the toolbar items and empty windows bugs fixed in Tiger, and solve some of the more blatant of the Leopard issues.

Speaking of which, I hate Leopard. It froze while I was away from my machine. Don’t just put lipstick on a pig and tell me it’s a healthier beast. Get a healthier beast.

Stacks? Pretty. (At least the grid layout. The arc toss is a hurler.) I can’t navigate through nested folders with it though. Less functional. Would I rather have prettier than less functional? Definitely not in this case.

So I rebooted into Tiger and answered some emails. I’m going to see if Thunderbird (yes, I don’t use Mail.app for real email) will allow me to link to its database in the Library (~/Library/Thunderbird) or whether it’s yet another example of me wanting to be so unconventional that I can’t get tools to do what I want.

Then I’ll head back into Leopard and add some more unit tests and debugging and hopefully get a stable update into users’ hands by morning. I’m hoping the debug suffix works better for me in Leopard than Tiger’s did.

Late Night Cruisin’

Ever since I decided to treat my blog more like Twitter, and just write micro-events rather than an entire epistle, writing either has come to a virtual standstill. (Except of course for the firestorm that has been Mori v1.6.4, v.1.6.5, and v.1.6.6 which is now undergoing 3rd party testing and I’m still trying to squash that “freezes while writing in Leopard” bug.)

Option-clicking the ‘Run’ icon in Xcode3 causes Mori to execute, then gdb starts up and attaches to Mori’s process, then Mori quits. Huh?

As if that weren’t enough, everytime gdb starts up, it spews out a lot of warnings about object files it can’t find. Like so,

warning: Could not find object file “/BinaryCache/Libsystem/Libsystem-111~176/Root/usr/local/lib/system/libc_debug.a(errno.o)” – no debug information available for “/SourceCache/Libc/Libc-498/sys/errno.c”.

warning: Could not find object file “/usr/local/lib/system/libcommonCrypto_debug.a(md2_dgst.o)” – no debug information available for “/SourceCache/CommonCrypto/CommonCrypto-32207/Source/Digest/md2_dgst.c”.

warning: Could not find object file “/usr/local/lib/system/libcommonCrypto_debug.a(md4_dgst.o)” – no debug information available for “/SourceCache/CommonCrypto/CommonCrypto-32207/Source/Digest/md4_dgst.c”.

warning: Could not find object file “/usr/local/lib/system/libcommonCrypto_debug.a(md5_dgst.o)” – no debug information available for “/SourceCache/CommonCrypto/CommonCrypto-32207/Source/Digest/md5_dgst.c”.

warning: Could not find object file “/usr/local/lib/system/libinfo_debug.a(gethnamaddr.o)” – no debug information available for “gethnamaddr.c”.

warning: Could not find object file “/var/tmp/Libm/Libm-287.1~6/Libm.build/Libm_debug.a.build/Objects-normal/ppc/scalb.o” – no debug information available for “/SourceCache/Libm/Libm-287.1/Source/PowerPC/scalb.c”.

What kind of railroad are we running here?

Anyway, at this point I’m planning on changing the file format after v1.7 ships. It’s just making it too tough to do some fixes. Mori still won’t require Leopard for some time yet, but I have to make my job, and putting out updates, somewhat simpler.

Mori v1.6.6 Clearing for Takeoff

It looks like the showstoppers which were part of 1.6.4, 1.6.5 (and even 1.6.3 counting the toolbar) have been dealt with. The latest build even seems to function normally in Leopard.

Thus, now that Leopard 10.5.1 has been released, and my Leopard install was for testing purposes anyway and not as my primary development environment, I’m going to install it and test the 1.6.6 build against it as well to make sure it still works. That is, after a stopover in Tigerland just to make sure that nothing in Leopard loused up my main volume.

The problems in Leopard appear to result, not from changes in the architecture of the Cocoa classes, but in how stringent the standards for values passed to them are, and how they deal with values which are unacceptable (invalid parameters and exception handling). Sure, code should only pass correct data all the time, but sometimes our expectations are off. Sometimes we get incorrect data ourselves (Garbage In Garbage Out), and sometimes, bad things happen in the real world in which computers operate.

Tiger used to be somewhat more non-chalant about this issues. It would ignore all but the most egregious problems, unless the programmer or the user asked otherwise. Leopard seems to be more restrictive and demanding of Mac developers and the code they write. Again, not bad by any means, except for the unexpected and the change in behavior for things that used to work before.

So, with more testing and better debugging (so why can’t I get “debug” suffix to work?!), we’ll hopefully see this occur less often in Mori’s code.

Now…back to that checklist.

In Mori 1.6.6, Windows are Still Coming Up Empty…Sometimes

Unfortunately, one of the fixes I thought I made didn’t carry over from one of the work directories (four at last count) I used to develop for the various MOX versions.

Or I was completely delirious at the time and never made the fix at all.

Whatever the cause, 1.6.6 still shows empty windows as a result of the toolbar problem. You can see when Mori gets confused by checking your console logs. You’ll see an entry like

2007-11-20 11:58:08.592 Mori[8503] *** -[NSCFArray objectAtIndex:]: index (0) beyond bounds (0)

Here’s the cause, and one of three possible paths you can take to get around this while you wait for the, hopefully finally fixed at last, v1.6.7 update: In order to improve your user experience (are you feeling it?), I wanted to move your old toolbar preferences settings to the new internal naming system. So it performs a check to see if the two settings are different, erasing the current one and copying over the new one if they are. However, if your previous toolbar had more items than the current one does, it will come up short during the compare and thus cause a Cocoa exception.

Mea culpa. Mea fixa isa cominga uppa.

    In the meantime, here are three ways to get your notebooks back all shiny and fresh.

  1. Add a bunch of extra items to your toolbar. Probably at the very beginning where they’re easier to get to and remove.
  2. Rename your preferences file (com.apokalypsesoftware.Mori.plist in your Preferences folder) until such time as Mori 1.6.7 is released and things have settled down. Or, if it doesn’t particularly bother you to personalize your toolbar again you could just delete that file.
  3. Just open all those important notebooks. Close them. Re-open those notebooks. All your data now appears because Mori has updated the toolbar info. (Or you can just do this for those notebooks you need for the moment. The fix will be released shortly…once I check a few things here first!)
  4. My apologies again to those who were alarmed at opening an empty notebook.


Mori v1.6.6 is Taxiing to the Runway

16112007

I just shipped the latest Release Candidate (Mori 1.6.6alpha89) to the beta testers, and I feel pretty certain it’s fine.

I’m about to do some final checks in Leopard (10.5.1) and may even install 10.5 again to double-check there while I await word from the testers.

I’m going to add a special dialog thanking the beta testers for their bravery and helpful comments and pointers. Not only have they helped to nail the big “empty windows” and Leopard freezing/crashing bugs, but their insights and explanations have revealed some things that I wasn’t aware of in Mori’s functionality! It should take maybe two or three days of coding and testing to add to Mori.

But it’ll have to wait as I know a lot of folks have been patiently enduring the problems in hope of a quick and stable release. So let me just thank them here for their fantastic service to me and Mori’s user community:

Bob Embry
Michael Koch
Rolf Schmolling
Joe Wicentowski

For their patient vigil and endurance, I am grateful.

Probable Release of v1.6.7 Today

Since it appears that the users still experiencing the “empty window” or missing notes problem are running Leopard and have some strange mixture of preferences (particularly “Enable check spelling as you type in new windows”), I should be able to get some update out today to Tiger users at least.

I received word a short while ago from users experiencing this specific symptom, and byakkie, who thankfully took the time to experiment with all the preferences to help pin down the most likely causes, regarding the results of running an experimental build to deal with the big remaining issues. With those results, and knowing it’s Leopard-specific (those more stringent parameter requirements!) I’ll reboot into Leopard in a moment and get down to exorcising that gremlin. If it’s like the other Leopard problems encountered it should be relatively easy to find the problem, and hopefully not-too-difficult to engineer a solution.

While I had awaited word, I worked on streamlining and pruning certain aspects of the build process and decided to tackle the recurring error message on my console:

*** -[NSKeyedUnarchiver decodeObjectForKey:]: cannot decode object of class (WebView)

which is always cause for concern for developers when things go wrong that you didn’t even realize were going on! “WebView?! Since when is there a WebView in an outliner?”

So I tackled it and discovered it is part of Blocks’ software update service, which checks for new versions of Mori and displays the release notes when found. Once I linked the WebKit framework into the Blocks software update plug-in, the release notes window popped up to announce a new version! Then it had a redirect issue I had to fix (because www.apokalypsesoftware.com redirects to apokalypsesoftware.com). Then I discovered the links in the release notes weren’t pointing to the correct location. And so on.

This all leads me to one conclusion: you folks are hard-core Mori users! Do you Fred Flintstone your way to work? Do you eat stale bread? Do you really love Mori’s features so much that you’ve been willing to put up with the broken stuff for so long? (And just what Mori features are you particularly fond of? I’d like to intensify that experience for you!) It’s just a more pleasant experience when the software does things for you like it should. Wow!

Well, while I am half-joking, I am truly impressed and grateful that you’ve been willing to endure some inconvenient obstacles just for the productivity that you gain with Mori. Thank you so much! I plan to continue fixing these annoyances as features are added, so that you find there’s more that can be done by Mori and less roadblocks it puts in your way.

Mori: v1.6.7 Post-Mortem and Upcoming Changes

Well, it looks like the latest toolbar fix has finally stuck. There were actually a few, very subtle interrelated items, and some procedural issues that cascaded into others. Code was shifted around hither and yon, resulting in elimination of two of a main class’ instance variables and simplification of logic in several methods.

Speaking of which, I’d really love to try out Xcode3’s new refactoring tools, but that would mean

  1. spending more time in Leopard, which freezes on me,
  2. and spending more time in Xcode3, whose text editing I already loathe.

I will need to investigate how well refactoring works in TextMate (which I already own and use from time to time when bumping against some other Xcode2 limitation.)

The pressure cooker development level of the past couple of weeks really put the development process here through the wringer. Some things held up. Some things fell apart. So more ‘behind the scenes’ development is going to take place here to minimize the interference caused by the mismatch between what needs to be done and what the tools require me to do next.

    Some observations of the release’s development:

  1. The primary, agile-based, processes were abandoned in the panic over the continued toolbar problems and Leopard incompatibilities.
    There was no established process for handling emergencies, so the chaotic edit-compile-test behavior resurfaced. This is not to be repeated.Corrective Measures: A plan of action for dealing with emergencies must be established. Exercises and drills prepared and performed so the proper outcome occurs next time.
  2. Users seemed to be unaware, or unconvinced, of my plans for Leopard.
    Although I had made statements regarding the planned support for Leopard, it was only in response to questioning by users. I had failed to provide information ahead of time and prominently.Corrective Measures: While users refer to the forum when there are questions or they have some issue to resolve, they refer to the blog constantly. It’s best to have already disclosed plans so users have time to assimilate needed info and prepare accordingly. Blog. Blog. Blog. Forum posting: link to blog. Then blog.
  3. Unit testing got skipped.
    It was unavoidable given the current process because tests didn’t cover the critical issue being developed: the user interface (i.e., toolbar items and empty windows).Corrective Measures: Mock objects are inadequate to sufficiently test the UI. For all my blustering that the UI is testable, it’s clearly time I seek or develop the necessary tools and put them to use.
  4. Subversion is a win, but it’s an ugly win.
    Being able to restore files, or the project, of a particular beta or release build, or of a particular date and time, is great. Being able to make wholesale changes to the project, then abandoning them, or keeping them along a separate branch to continue experiments at a later date without juggling project directories here and there, is great. Being able to merge or contrast multiple working project directories from separate environments (Tiger, Leopard, laptop, and the v1.6.3 release) quickly and easily, is great.Nibs saved as a smattering of files in the repository: not so great. Confusing subversion with its own metadata when copying/adding directories: not so great. Performing an add/delete when moving or copying directories and files instead of adding “move” or “copy” semantics to the system: not so great. Poor integration mechanism for multiple offline revisions: not so great.

    SCM is great. I’ve used CVS for years. I’ve even set up another project or two using Subversion before I bought Mori and Clockwork from Jesse. But I don’t see where it’s an obviously better package than CVS, but perhaps that’s because I’ve learned to work around CVS’ shortcomings. (Reviewing the feature list at http://subversion.tigris.org, the only practical feature I see that means anything to someone actually using Subversion is command-level manipulation of directories, although the handling of renames is poor. Hrmm. And svn+ssh access to the repository, with Xcode integration even!)

    Corrective Measures: For the time being, the messiness of the metadata in copies and moves will have to remain unsettled. As long as the structure ends up correct, and it doesn’t take long to get it to that state, it remains tolerable in the face of bigger issues. A procedure must be established to handle the circumstance where developers are offline and must “commit” in some manner which is transferrable to the repository when they are back online.

  5. Debugging is severely inadequate for professional-grade development.
    Can you tell Apple’s team has a severe case of he-man, mouse-haters club insecurities? (Either that, or they’re operating in crunch mode so much, they don’t have time to develop the appropriate tools. Nah!) The debugging facilities they provide for developers operate primarily through the use of environment variables, console output and functions and output which require gdb to access.ZeroLink didn’t work well and was abandoned; Fix and Continue is still problematic; and the vaunted debug libraries (to help you catch errors in the parameters passed to the AppKit and Foundation libraries) hasn’t worked right since 10.4.3.

    My own debugging facilities are rudimentary and lacking in depth. More of the failures in the field should be communicated back to me (remember Safari’s ‘bug’ button?). The crash reporting mechanism should work when the crash occurs, not the next time Mori runs. Also, exceptions are logged to the console, but otherwise go unnoticed by Mori and the user.

    Corrective Measures: Extend the runtime monitoring and browsing tools. Rewrite the crash reporting mechanism to activate when the application terminates. Add an exception handler which sends reports back. Develop proper high-level debugging tools for Objective-C, the UI, CoreData and Bindings. Coding takes place at a higher level. Debugging should be at least at that same level. -fobjc-gc exists for a reason. Take advantage of it!

  6. The build process is inconsistent, confused and unstable.
    The various plugins’ build settings aren’t consistent. Extraneous resources (images, sounds, etc.) are stored in the nibs. At times the build directory must be purged for a functioning executable to be built.Corrective Measures: A Build Settings Table has already been prepared and Mori and its plugins have had their settings documented. The settings for the Blocks plugins has yet to be documented. The effect of some settings has to be determined, which can then be propagated to the targets as appropriate. The project file should be purged of duplicated actions, unnecessary references, etc.
  7. Debugging Mori when it’s used for normal work results in too much human activity thrashing.
    Because the debug and test versions use the same preferences/file settings, the release version used for normal work had to be exited to avert data corruption in the notebooks.Corrective Measures: This issue was resolved with the Oneill branch. However, that uses a separate target to achieve its distinction. Some means of specifying a special bundle id for the debug and test builds must be developed to accomplish a similar effect, perhaps through preprocessing the Info.plist.
  8. Checklists are great
    Being able to state that processes are being followed, builds are complete, and updates were released correctly, is great.Corrective Measures: Checklists are great, but scripted procedures are better. Automate as much of every process as possible.
  9. To-Do lists, Getting Things Done (GTD), or whatever Time Management activity that is put into practice definitely helps.
    I’ve got the audio book. Mori’s got the plugin. I’ve got Taskpaper, too. I’ve got a Handspring Visor Edge and its Palm Desktop software. Something, anything, that helps track tasks that need to be done so nothing falls between the cracks is a plus. They are effective at keeping things moving forward, but I’m not efficient at it and a lot of discipline and effort must be used to keep moving things forward.
    Corrective Measure: I used TaskPaper to handle the tracking during this hectic period. I’ve got a blog entry in preparation, but basically, I found it a great way to get into the GTD system due to its simple interface. I must continue developing my understanding of this system to manage my activities, and see about getting the various software/devices integrated better.

Next Batch of Changes

I’ve already begun work on v1.6.8: Improved checking and repair functionality for notebooks. Correct Italian localization, thanks to Mario Pettenghi. The code for the Blocks framework will be tweaked so it compiles without triggering warnings (e.g., unused parameters, missing prototype, etc.). Additional unit testing for the UI, and refactoring of UI code.

These improvements to the code will set up the continuation of progress for v1.7, which should then be ready during the holidays. At least that’s the current goal.

A Week of Fist Shaking at BOFH and Open Source Developers

The past week was one of the worst in recent computer experiences I’ve had, surpassing the Leopard install. In fact, the last time my experience has been this bad was when I last fiddled with website software. I was going to abandon the whole server transfer/upgrade plan due to the issues the upgrades were causing, but I remembered the transfer itself should be fairly trouble free. I’ll only upgrade a minor version; not a whole version, which would require adding a lot of code to custom modules Jesse wrote. Should be.

I can appreciate the time and effort invested by those who work on open source projects, but developers who complain that users aren’t migrating to their latest efforts quickly enough are both arrogant and naive. Customers prefer to be able to continue using the data they’ve stored in the current versions to losing them when they migrate to the newer ones. I’m pretty sure it’s the same for OSS users. Much as I love writing code, I have enough to write already thank you very much. And I don’t appreciate having to spelunk through your code to figure out what those data structures you’ve added are supposed to do (”ooh, shiny, new!”) just because you’re too lazy to write the docs (”the code are the docs!”) and impatient to play with your new ideas.

Although the site I inherited from Jesse used a CMS system distinct and incompatible from the one I had selected when I started my own, I didn’t want to switch it around or try doing new things with it because I felt the continuity was better than taking the time to rebuild a site from scratch. But now I can appreciate Jesse’s decision to do such a thing. If I have to spend the downtime upgrading parts that already work well just to get the features needed for other modules, I might as well fix it to my own needs and ignore the chaos that OSS developers are creating in the system. Now the whole NIH question is thrown out the window. It’s no longer a matter of, “Oh, I can do a cooler system.” Now it’s a question of the developers themselves causing users to abandon their system. A question of distrust and self-preservation.

Hopefully, any remaining misconfigurations in the website will be rectified before a second person notices it.

If you sent an email in the past 24 hours (of 2007-12-05) and you haven’t gotten a response already I apologize, but you’ll need to resend it. My site host doesn’t transfer files, emails, settings, or the like between accounts; and well, it has probably been crushed by the lumbering floes. In fact, if you’re waiting for a response from me on anymatter, please jog my memory with another email.

Anyway, at least the site now has some breathing room, and I can continue improving the products.

Mori, PIMs, Pricing and the Business of Software (Was Re: Mac PIMs in General (was NightHawk)

A few days ago, there was a thread on the Macintosh PIMs group that descended into a diatribe against the current state of PIM software and the cost of software. In response, I wrote what turned into a very long, poorly-conceived, and most likely ill-advised response to some of the opinions voiced. Very few quotes are enclosed, as it’s mainly a response, not a rebuttal.

Please forgive what is sure to be a foolish action on my part, but nothing concerning the current state of affairs will improve by actively avoiding public discussion of the issues. None of my comments are an attack on the people whose comments I responded to, particularly db whom I responded to especially. Consider his remarks a proxy for a lot of the “it costs too much” complaints I see on sites like VersionTracker, MacUpdate, iusethis, and elsewhere on the Internet. And it’s with the intent to publically respond to those complaints that I choose to do so here, rather than the Macintosh PIM group’s mailing list. My apologies to the Mac PIMs group, and the rest of the netizens (although there are worse things one should unsee). With that in mind, here is Apokalypse’ contribution to the conversation. Your comments are welcome.

Ted Goranson wrote [Context added so his position is somewhat clearer. His remarks are included as Mori is mentioned. — AG]
> the value added. Users who know nothing about development somehow
> expect the same, unreasonably low pricing scheme.
>
> These people, for example are why we don’t have Incontrol and
> Infodepot, why we lost MORE. Why Mori was all but lost.

db wrote:
> I’m going to try again (to change this topic ;-)
>
> Ted/Edward,
>
> I appreciate the understanding and constructive disagreement, however
> large or small, though actually I don’t think we really disagree about
> anything substantial, other than pigs flying. I have proof, from the
> state of Maine no less, which is the original home state of Mori and
> still the home of Mori’s original Developer, HogBay Software:
> http://sendbread.com/
>
> Mori got smart and moved to Florida, well before winter set in. I have
> no idea where the pigs went.

db wrote:

> And you expect Mori to get
> it right with a part-time developer.

Since there’s been some mention made of Mori of late, I believe it’s in the community’s best interest for me to stir things up a tad bit more in the hope that by sharing my experience as Mori’s owner/developer, a better understanding of the current state of the Mac software market might help to improve the situation for us as users and businesses.

And I know, db, that you like to constantly talk about Mori’s “original” developer, in a wistful (or rueful) tone, but the truth is Jesse wasn’t developing Mori when I purchased it from him, had lost all interest in continuing development months before, and never wants to touch Mori code again. This isn’t an attempt to be cruel, but to make the point with finality. If anyone has any bugs to report or feature requests they want to make, they’ll have to tell me. I’m the only one who’s responsible for the condition it’s in now, and the only one who’ll be able to make the necessary changes. I’m the one taking Mori into the future. Or to put it into less delicate terms, Mori may once have been Jesse’s little girl, but I’m making her a woman.

The Low-Baller Wins Myth

> There are some folks who will defend M$ pricing, or the price of an
> unlocked iPhone in Germany. That’s OK, but not me.

> If you recall, I am asking for a more modest personal edition price,
> to help the product succeed by increasing market penetration (and
> frankly, user goodwill) in order to make an integrated PIM available
> to more users. I’ll bet that your Mac and it’s apps save you $1000s
> more than they cost if you use them effectively in a work environment.
> But if Apple sold them at a price that more closely approached the
> savings gained by it’s users, users would have revolted and there
> would be no Macs. The Apple IIe saved me a ton of time in college but
> that doesn’t mean I would have been willing or able to pay two or
> three times as much for that option. My strategy is to charge as
> little as I reasonably need to and I’ll keep busy. If some fool wants
> to pay someone more thinking they must be getting a something better.
> Fine. I don’t like working for fools anyway.

> I think their is a marketing possibility that more at lower pricing
> will offset less at higher. Not everyone is a fool.

The problem with your strategy is it fails to account for shortfalls in sales volumes, changes in the market, and other unforeseen events; both corporate and personal. For a lower price to make a difference it has to be substantially lower, and for volume to make up the loss in margin it has to be several times higher. Selling 25% more of a half-off product won’t cut it. And when your target market is the price-conscious, economic conditions which impact their budget also impacts your sales. Wal-mart successfully, though unintentially, demonstrated that for us these past two years.

People spend 50-300% more for an Apple iPod than for anyone else’s portable media player. Not just 20-30%, but up to 300%! That’s two to three times more for a device to play songs or display pictures. The multiple is even higher when you consider the features lacking in an iPod which are available in other players. So you think iPod buyers are fools. I think your opinion that price is the main consideration for consumers is faulty. Or is there some special excuse we give Apple for pricing above the market, instead of “a more modest personal edition price”? (The iPod Shuffle furthers my argument, not yours, as the Shuffle is priced against Apple’s own products, not the rest of the market: it is has even less features.)

When you made that crack about me developing Mori part-time, I was insulted. But then I realized that, in all honesty, I am only developing Mori part-time. I’m also handling web-site duties part-time, which include controlling spam, updating the site software, touching up the databases, writing blog entries, performing backups, and preparing traffic reports. I’m also fielding customer support, whether it’s email, IMs, the fora, or bug/feature tracking. I’m also handling marketing, which includes contacting blog writers, contacting writers and editors in the media, preparing market plans, product literature, artwork, etc. I’m also writing user docs, screencast scripts, tutorials, and the like, in several languages. I might be doing all this, but my development work on Mori itself is, as you say, strictly part-time.

Most software developers in the Mac market are small shops. MicroISVs. Indies. Whatever the term, sometimes it’s a shop of two or three people doing product development. By far, though, the bulk of the product developers today are one-man shops. Somebody working solo. And that solo developer typically doesn’t make enough to support himself on his products’ sales. It usually doesn’t matter because these indies are usually students or employees of another company. Their product is just something they whipped up for their own needs or interests, and they decided to offer it for sale to make a few bucks (just like the Apple story we’re all so fond of).

However, I’m neither a college student nor employed anywhere else. I’ve even turned away contracting offers due to the backlog of development tasks. So I have to question what your beliefs are when I read your crack about my work on Mori as nothing more than part-time in the same paragraph where you complain that Mac software is overpriced, and how small shops can’t afford market research!

A Cowboy [Coder] Isn’t A Landowner

> I think many (not all) of the little one-man shops fail because they
> lack the willingness or ability to see or use the advantages of
> cooperation with others. They sometime simply want to be in charge,
> their own boss, and see cooperation, of course, as giving up control.
> That’s the way it is when you work with others. Unless they have such
> a big hit that allows them to hire others, they’d be far better off
> cooperating with others. Look how many GTD and info management apps we
> have from very small shops. Few have a chance at decent success
> working alone, and especially when competing with the larger shops
> which simply engender more consumer confidence because of their size,
> never mind having more resources to begin with.

Now you’re finally saying things I can almost completely agree with: most indies want to strike out on their own so they can be their own boss. The problem is most lack the chops to do it. Being intelligent in one field, many assume they know what it takes to be successful running a venture entirely on their own. Being socially awkward, many are too untrusting of others to risk venturing with them. Indeed, many realize the likelihood is that any new venture will go belly up in less than five years. Though most entrepreneurs fear personality clashes with potential partners, the most common cause of failure is insufficient resources.

What incentive does one developer have to cooperate with another? to give his source code and a promise to share profits with a competitor? I’ve repeatedly attempted to persuade other developers to work with me, but loved as I am for my winning personality and disarming smile, I’ve been unable to convince them to abandon their products and support mine instead! Crazy, no? Perhaps they need some sort of compensation for their investment in their product and their customers; some security or other evidence of the legitimacy of the deal, and its probability of success. Would having a bankroll improve the likelihood of him joining me so that we “have a chance at decent success”? How do I accumulate this bankroll with your strategy “to charge as little as I reasonably need to and I’ll keep busy”? While being a low-price leader may be a marketing strategy, volume isn’t proof of commercial success. Long-term commercial success is dependent on your money being busier than you.

There are a couple of your comments that undermine your entire “more modest personal edition price, to help the product succeed by increasing market penetration” fallacy. One is, “Unless they have such a big hit…” Unless? So you admit the chance of that happening is slim. And if the chances of having a big hit and the volume that it creates are slim, then prices will have to remain high to stay in business. Also, if you think having a low price will guarantee a big hit, you are guaranteed that a competitor will come along and undermine your sole competitive advantage with an even lower price.

The other comment debunking your assertion is, “…larger shops which simply engender more consumer confidence because of their size, never mind having more resources to begin with.” Without the margins and volume to build up your resources, how do you expect to engender more confidence in consumers?

If you don’t make enough profit with the early adopters of your program, you’ll never last long enough to develop the additional features, user resources, documentation, etc. to be purchased by the mainstream. In addition, your organization will likely implode due to an inability to adequately provide service for your customers.

You probably have a larger selection of PIMs to choose from now than ever before, so why aren’t you satisfied? It’s because they aren’t as powerful or feature-rich as the old ones were. You’d say they lack the quality, or aren’t of the same caliber as the old apps. I’m pointing out that, twenty years later, the apps are substantially less expensive as well; and they don’t get better because the developers can’t afford to invest more development time and money in them!

Road Closed Due to Growth

Do you know what it takes to add that power and those features you long for to the software on the market? It doesn’t take listening to the customer, because customers have been talking about their needs for years. It doesn’t take writing better docs, which people don’t like to read anyway. It doesn’t take promotional discounts or educational versions, which have a limited lifespan in effective marketing.

It takes engineers. Software engineers and time. Time to think about how the current product was architected. Time to think about what features need to be added. Time to think about what features can be added given the current state of the product. Time to design the code to add those features to the product. Time to code the features into the product. Time to test the code. Time to fix the code. Time to redo the steps again and again until it’s ready to be released. Or worse, until they run out of time.

Do you think engineers are given special treatment for all this wonderful code they’re adding? Do real estate developers or hotels put a roof over their head because they’re improving products? Are hospital visits, medical treatment, or even health insurance without a price because we’re indispensable? Do engineers get any food or caffeine of any quantity without charge because of their role in society? Or should they be required to sacrifice their own needs and wants for transportation, entertainment, family, etc. to fulfill some “higher calling”?

Someone who enters the PIM market as a business isn’t looking to scrape enough money to buy himself a shiny new MBP for Xmas. There’s more than just the cost to purchase equipment. Or pay electric bills. Or Internet access. Or to purchase technical books and journals. A business can’t just make enough to cover the salaries of its employees, its legal fees, its taxes, etc. It has to cover the cost to invest in growth: of its products, its corporate infrastructure, and its owners.

Someone has to foot the bill for all these things while time is being spent adding those improvements you want so much, whether it’s a VC, an angel investor, a spouse, family, friends, whatever; and that someone is going to want a return on their investment. You get a lot of turnover in this industry because these would-be entrepreneurs discover the return on their investment just isn’t satisfactory.

A competent software engineer can make at least $75K/year, even as a fresh graduate. For a 2080 hour year, your product has to bring in $36.06 per hour to cover his salary. His salary alone. If you don’t want him to work on development part-time, like I do, you have to pay others to do the marketing, technical writing, artwork, administering servers and websites, the business-administration-type stuff, etc. So, say you as the business owner make a worst-case salary of $80K, and your developer makes $75K per year. So after other operating expenses of $65K, and a 20% profit for the year, a business should bring in $264K. That is for strictly online sales without a marketing program. It excludes marketing expenses such as advertising, sales commissions, packaging, product literature, trade show exhibitions, etc.

That’s $129.513/hr in sales for a company to be comfortably profitable. (Oh, did we forget to deduct the processing fees deducted from sales by the payment processing firm? You can go out of business if you forget these details!) If we don’t make that, there isn’t a point pretending we run a business. And if you don’t have a business standing behind the software you use to manage your information, quit pretending to be surprised when it’s no longer under development, being supported, or that the features you enjoyed on the old packages will ever return in anything new.

Companies are bought and sold. So are product lines. Products with a sufficient revenue stream continue in the market, regardless of their origin. Products that aren’t worth the trouble die; regardless of how loved they were and how missed they’ll be. So if you’re not prepared to spend the money necessary to obtain a solid, powerful package now, then let time take its course. The part-time developer you’re supporting will either get the features added in there eventually, or drop the product for something more rewarding in his life.

Those who cannot learn from PIM history are doomed to re-key their data

Let me explain why there isn’t a strong third-party PIM in the Mac market. Future product development is based on past product development. Whether it’s the profits from past products, or using the codebase of previously engineered products (e.g., Cocoa and Carbon), one product is built upon others. And whenever someone gets the bright idea to write another to-do list or contact manager or agenda application, whether it’s to learn how to develop for the Mac, or because they have more time than money, they have to develop the basic functionality first. Then, they think, “This is so helpful for me, I bet other people can use it too,” so they make it available to others as freeware or shareware, presumably to others who perceive a similar lack in existing apps or possessing a similar lack of funds.

Then, his app begins to find users. Slowly, of course, because it’s new and the majority of people will let others be the early-adopters. But his app will find some users because it sufficiently meets their feature/price requirements. But! its feature set is shallow because he started from scratch. And! there are a few bugs here and there that need to be fixed. And! it doesn’t sync with Apple’s bundled PIMs. And! it lacks support for their phone or pda. He doesn’t have time to add innovative features because he’s too busy trying to catch up. Now it stops being just a hobby and starts to be real work. Then he thinks, I’ve got to get something more out of it. If he feels he can do it, he’ll start charging (more) for it.

Now there are people who don’t mind spending hours in front of a TV set, playing video games, or downloading and reading stuff off the Internet. It’s something to occupy their time. A way to unwind. Maybe someone likes to tinker with cars, spending months to strip down a junked Mustang and rebuild it into a street monster. It’s a nice way for him to while away the days. Perhaps when he’s finished he’ll just cruise the strip in it. Maybe he’ll street race. Maybe he’ll sell it off and use some of the money to buy another clunker and start over again.

Regardless of the number of times he rebuilds cars for fun, his mindset changes when he begins to treat it as more than just a hobby. His goals will be different. The decisions he makes will take on a whole new importance, and even the tools and processes he uses will have changed.

It’s the same way with software development: you can afford to waste time and money on it when it’s just for kicks. But when it competes with the rest of your life, when a child becomes ill, your spouse loses a job, or your kids are in college, those things that occupied your time are re-evaluated; and you decide whether or not it needs to be treated more seriously, and how committed you are to its success.

And businesses that were profitable in this market re-evaluate their returns due to the competition. They consider whether branching out to other product lines or other platforms will be more rewarding. Developers, large and small, let sales coast without active development. Eventually, their products are outdated, or the platform is (like Mac OS 9 or Tiger). Then some developer decides there’s no to-do list or contact manager or agenda app that matches his feature/price requirements, so he writes one from scratch…

And that is why the PIM market, indeed, the Mac software market in general, is so poor today, and only a few categories have clear market leaders.

As far as PIM goes, it’s obviously an inadequate term for the types of products that fit in that category. There are calendars, to-do lists, project planners, address books, outliners, and on and on, but Mori is specifically a digital notebook app. And while users and developers can add agenda, contact management, GTD, file management or even wordprocessing and spreadsheet functionality through the use of scripts and plug-ins, it doesn’t come with the features typical of those applications. In fact, a lot of the design work I’ve been doing over the past couple of months has been to reduce the excess behavior in Mori and recast its feature set with an eye towards note taking and organizing superiority. Any additional behaviors will have to be the result of plugins. This means the plugin API will be more sophisticated though.

The point is, I’m not going to add features to Mori to handle all PIM needs. And I’m not going to cater to the notetaking needs of every Mac owner out there. Apokalypse will be focusing on the professionals who understand the value of their time, and demand software that delivers productivity gains that make them look good. SOHO users. It isn’t that I don’t appreciate everyone else, but I can’t afford to help everyone else, and the quality of the products will suffer badly. This is one reason why I’ve continued Jesse’s practice of sending prospects to other products, you can’t be all things to all people.

So, as a user of PIM products, figure out what your needs are. If you’re just looking to keep a list of your friends and family members’ contact info, track class activities or have the occasional sticky, the iApps bundled with the Mac should be enough. If your needs are more sophisticated and your time is a resource you use to produce money, find the software that will maximize your productive use of time in managing your tasks, contacts, info, etc. and purchase it. Even at minimum wage rates, you should recoup your investment within the first week!

If you want engineers and small businesses in general, and me in particular, to improve the quality of your life, you’re going to have to improve the quality of mine. How can we maintain a continuing relationship otherwise?