While making the changes to Mori’s code for 1.7, I encountered some oddities in test results, and it turned out there was a bug which I had introduced in an earlier release. While it doesn’t appear to endanger data in Mori notebooks, it might not return all the results you expect in a search, or in entry summaries.

In addition, it has what I hope are a couple of performance improvements, continued improvements to Italian localization, and a work-around for Leopard’s insistence to treat non-Apple Spotlight metadata files as third-class citizens.

Normally when Spotlight discovers a file has been created or changed, it will ask the responsible program to figure out what’s inside, and feed it back to Spotlight. But one of the drawbacks to Spotlight’s design is it lacks the ability to define containers, or documents which contain logically distinct elements such as the chapters of a book, pictures in a photo album, or entries from a Mori notebook; and which can nest other containers as well. Treating a document as a single entity, Spotlight will open a document at the beginning (or maybe the place where the cursor was the last time it was open), even if what you’re looking for is somewhere near the end.

Because it doesn’t understand that a file can have distinct elements, the development teams for other Apple software (e.g., iPhoto, Safari, Stickies, etc.) came up with a scheme to trick Spotlight by creating new files with the data for those elements. So that’s how Jesse coded Mori’s behavior: duplicate the data for that logically distinct element in its own file. A separate copy of each element’s data in its own file. One extra file per element. That means the space taken up by your data is easily half again more than if Apple just added a container definition for Spotlight metadata (once for the notebook, another for the entry metadata file, and the third copy in Spotlight’s database).

But that isn’t all. While we’d like to keep all those extra files inside a notebook bundle (a directory which Finder treats as a file), because Spotlight treats a document as a single element it won’t look for those files inside the bundle. So Mori creates those files in the metadata cache folder (in your Library/Caches/Metadata folder), along with the metadata files from some of Apple’s programs. If you open the metadata folder and look at these files, you’ll see they have numbers to help Mori figure out which entry contains that data. But when you do a search using the Spotlight menu, and when you select menu item ‘Show All’ and the results are displayed in the Finder, you won’t see the numbers; you’ll see the titles for the entries they represent.

Leopard however, isn’t so democratic; which is why users where complaining about the entries when Leopard was released. First off, it ignores any non-Apple metadata files in the cache folder unless you set your Spotlight preferences to use those files. Secondly, it will ignore the title info embedded in the entry metadata file and just display the file’s actual title, meaning the number. How’s that for Apple undermining the work of third-party developers?

So the workaround I came up with is to add the entry’s title (or Untitled, if it has none) at the beginning of the filename, so you at least have an idea which entry matches your search terms.

(Thanks for wasting about a whole month total of my development time on that alone, Apple. I feel the love.)

I am, of course, more than happy to eat crow should I be proven to be completely mistaken or speaking from out-dated information. It’s easily worth it in order to improve the user experience.

Regardless of the rationale for the design decisions, enjoy, and thank you for being part of the community and continuing to support Mori!

When I purchased Mori, one of the first things I did was generate some documents about its codebase. For this, the main tool I used was Xcode’s Class Model tool to generate charts of the various classes involved. I spent several days laying out the classes on the charts, then printing and then folding and taping the pages together. (This is a process which I am replacing with specific related classes that occupy only one or two pages, so I can keep them in a notebook; or sending a PDF to Kinko’s the next time I need to print the whole chart.) They helped me get an understanding of how code was laid out, and their relationships.

After a while, the tape that held the charts up on the wall would lose their adhesiveness and down they’d come…again and again, eventually falling into disuse. Having gotten a digital camera, I spent a day reassembling the crumpled remains of the charts about a week or so ago, and snapped a few shots which I present here.

There are 33 classes, and 25 protocols defined for the document back-end plugin, and quite a few supplementary category methods extending Cocoa’s foundation classes.

The user interface plugin has 85 classes, 18 protocols, and its own quantity of supplementary category methods.

At least, that’s what Mori’s classes looked like when the shots were taken. Today it looks a tad different.

Oneill, the branch for Mori 1.7 is under active development again and will bring substantial changes to the UI and back-end architecture. You shouldn’t notice any hiccups in the file format, but you should see better functionality and performance.

At least that’s what the testing support should help me do. Thankfully, I make heavy use of the Subversion SCM system. Just in case.

It doesn’t matter how much we complain about time, we’re not being realistic or honest about our use of it. Everyone gets the same amount every day; that is, if we’re getting the whole day. But asking for more time to do things has to strike someone as having too many things to do, and being grown up enough to abandon or postpone those things for which there isn’t time.

There are plenty of Time Management systems out there. Gettings Things Done (GTD) is a recent one with a lot of buzz and adopters. There are plenty of GTD software packages for those who prefer computer-based systems to paper, including two versions for Mori: Jeff Fisher’s mGTD plugin and Jim Harrison’s MoriGTD scripted system.

Now, I’ve had the audiobook for GTD for years but I’ve never listened to it. (I did at least listen to Merlin Mann’s David Allen interviews when I was on the road.) And now that I’m developing (and using) Mori I’ve even installed the mGTD plugin, but that was more for testing purposes than actually knowing what GTD was about (or how to use it).

So when Jesse introduced his own GTD app, TaskPaper, as a simpler way to getting things done with “paper-like simplicity” I was of course concerned that it would cannibalize Mori sales. But TaskPaper only manages tasks, not notes like Mori does. So unless your notes are one-liners used only to manage your activities, you’ll still need to grab a copy of Mori.

But the real reason not to be concerned is TaskPaper is an awesome app in its own right. Jesse prodded me a couple times to try it, and even provided me with a license. So the next time I needed to jot down a task reminder, instead of using Mori, I opened up TaskPaper and gave it a try.

Now I have to admit, being both a GTD noob and a programmer, TaskPaper’s interface threw me at first. Let’s face it, any device with less than five buttons on it leaves me scratching my head. But I do know how to use an outliner. So with that interface in mind I set out to discover just how it was earning such wonderful reviews.

But TP isn’t exactly an outliner. It just uses an outline-like interface, and I emphasize that point. You’ll be productive very quickly, but I also got my list complicated very quickly by adding sub-sub-headings to my project. (But you have to expect that from a GTD noob and a programmer.) The beauty is that by using a very simple text editor/outliner interface, TP doesn’t complicate the ability to change your organization of tasks and projects. Nor does it complicate your ability to add extra info about the task (its metadata). Avoiding special columns and a form interface, TP instead uses the following special text characters to signify meaning: “:” for projects, “-” for tasks, “[tab]” to nest tasks within other tasks (which are themselves nested within a project), any non-reserved character for notes, and “@” for the interesting context.

Contexts are GTD’s way of giving you a way of lumping similar activities together. They aren’t necessarily part of the same project, but just things which are done at the same place (e.g., @home, @mall), the same type of activity (@computer, @phone), and so on. TP refers to them as tags, and uses the normal GTD syntax (I guess) to refer to them.

TP’s interface is very sparse, which means less clutter and distractions. It’s nonetheless powerful, letting you filter to a specific project or context. The point is to spend more time doing your activities than actually organizing what they are. And TaskPaper handles that beautifully (and elegantly). I now understand what the Hog Bay Notebook users have been clamoring for!

I also have a better understanding of how GTD works, even more so after I sat through the mGTD screencast put up by Jeff, which in turn helped me make better use of TP and GTD.

There are definitely some features I’d like to see added, though:

The UI signifies task completion by changing the text style to strike-through, but the only way to mark the task as completed textually is by adding the context @done or the keyboard shortcut cmd-D. (You can also mark it completed by clicking on the open circle in the left margin or the menu item “Project > Mark as Done”, but that requires lifting your hands from the keyboard to use the mouse. I would prefer some other delimiter such as “x”, or a checkmark to signify completion, just for continuity.

I’d like to see true tags supported, perhaps using Chris Messina’s “#” tag delimiter.

So while there are some features I’d like to see added for more sophisticated capabilities, for GTD noobs like myself (still) and those who just want to work and not manage yet another system, TaskPaper should be the first stop to GTD (and priority) mastery!

In time, I might become profficient enough at GTD to be able to use mGTD or MoriGTD. But for now I just need to start the “filtering the inbasket thing” to help things along. That, and redefine my understanding of the word “superficial”.

For an entrepreneur, particularly a one-man shop like a microISV, the problem of product development is one of what to produce and the feature mix it should have.

The problem is that there are different cultural values that could be represented by a product. Not just the obvious geographical, ethnic, age and gender cultural values, but values which are more subtle, and reflect an individual’s vocation and skill set more than any genetic or environmental demographic factor. These are reflected in the ways an engineer, a marketer and the customer think about a product’s design.

The Engineer
The engineer is a problem-solver. His main interest is in using his tools to build solutions.

He thinks in terms of his solution to a problem versus the previous method. Ergonomic and aesthetic qualities are not considerations for the engineer, features are. Features. Options. Variability. He’ll look to add other components to his solution so it solves a wider range of problems and is thus more useful (volume, speed, duration, size, angle, etc.). Adapting his solution for other uses though, will throw the engineer into bewilderment. “Why would you want that?”

The engineer who makes use of a variety of tools to solve his problems will build solutions to others’ problems in the same way: a plethora of tools in a variety of qualities.

The Marketer
The marketer is a persuader and influencer. His main interest is in getting his message adopted by the customer.

Because the marketing department is one of the few which are customer facing, they generally have a sense of what issues their current customers face, what they need, what price point they’ll tolerate, etc.

Due to his preoccupation with perceptions and feelings (prestige, security, intelligence etc.), the marketer’s input on product designs focuses mainly on the physical attributes (e.g., color choices, materials, textures) that will reflect qualities that bestow intangible or perceived value to the user.

The Customer
The customer is a doer and a responder: performing his routines, following his processes. He may be looking for a solution to some problem he’s currently facing (e.g., new car, faster time-to-market, rising prices). Or, as is most often the case, he may not realize there’s a problem to which someone will happily sell him the solution.

While he might rationalize his decisions with some logical arguments (takes less space, improves productivity, etc.), it is for the emotional value (trendiness, security, etc.) of owning the product for which he actually chooses.

Putting it into Perspective
This difference in perspective is vital for the microISV to know and understand in developing his products, as the tool which was developed to solve his problem is often not what the general buying public will want. Engineering a solution is one thing. Creating the want for the solution is what business is about.

In the Mac market, this conflict in viewpoints is easily demonstrated in the topic of “Delicious Generation” applications.

Many of the indie developers/microISVs developing for the Mac have criticized the arrival of apps which they have denounced as all flash and no substance. Apps such as Disco, whose designers put more emphasis on the visual quality of the product (to the extent it rendered smoke effects when burning a CD or DVD) rather than the functionality it provided, are seen as providing very little value to the consumer. These developers have a dislike for the hype surrounding these types of products, seeing them as distractions from real needs which can be solved by more functions, options, and the like.

An interesting counter-reaction was given by John Gruber in his presentation Consistency vs. Uniformity in UI Design at the C4[0] Mac developer conference, however. His description of Disco? “That’s f—— gorgeous!”

The lesson for entrepreneurs, indie developers, and anyone overseeing development of a new product: while engineers may carry the bulk of the responsibility for developing the technology that goes into your product, until machines start buying products, or your target market isn’t other engineers, you must rein in their enthusiasm to over-engineer. Think about how you’ll market the product first. Understand what the customer is motivated to want to buy. Bring them into the discussion as early as possible. Then you can engineer a solution within that envelope.

Describe. Develop. Deliver.

As I was putting the finishing touches on this entry (links, pics, and such), I came across this fabulous example.

Software Development Isn’t an Art IV

For precept must be upon precept, precept upon precept; line upon line; here a little, and there a little. — Isaiah 28:10

The identification and use of design patterns has moved our understanding and the current state of our profession forward. But without an understanding of the critical factors for developing and successfully completing quality software projects, stagnation at the current state is all that can occur. And stagnation is all that’s occurring due to our unwillingness to exert more rigor in the design of software.

When Walt made storytelling the linchpin of his animated films, the other studios were unable to compete on a long-term basis regardless of the gimmicks used. They hired away his animators. They hired away his composer. They hired some of his storymen. But they didn’t use his process of refinement and experimentation to develop and improve the stories they told, and it reflected in the quality of their films.

Developing software without a written, documented and tested plan is like making a movie without developing the characters and the plot. Or, to use a metaphor you might find easier to relate to, like building a house without a blueprint. You’re just eyeballing it and you’ll have to continue making adjustments to your code as you integrate it into the rest of the system. And hope it works out for the best.

But you know it won’t. You know you”ll have to spend more time trying to get components to work together properly. You know you’ll have to redo production code and unit tests repeatedly. You know customers will change requirements that will adversely affect the system’s integrity downstream. Does it seem like a good idea to add extra work to your plate because you didn’t take the time to execute a proper design in the first place?

The design isn’t in your code. “If” statements aren’t design constructs. Neither are functions. They’re execution constructs meant for computers to execute.

Design languages help you understand and analyze designs. They help you switch contexts much quicker than code ever could. So when you design, use the language of design; don’t use code. Code is for computers to follow, not people.

Software Development Isn’t An Art IV

Let’s face it. The software industry just doesn’t eat its own dog food.

Debuggers? What are they for? Why doesn’t the system point out when memory is clobberred or a pointer is NULL?

Hardware is cheap. Software development costs are much higher. Why is that? Hardware developers have no trouble using simulators in the development process. The cost for building first-runs is quantifiable, as is the errors and rework associated with them. Hardware engineers and their managers can see the cost of defects in manufactured products is too high not to use additional tools, such as mock-ups and prototypes, to aid designers. So why don’t software projects also use simulations and other tools to improve the quality of the process? Poor management. Proud and arrogant developers.

Programming today is done much the way animating was during the twenties, before Walt Disney pushed for process improvements in his studio. After a director selected a storyline, the animators would have to produce a certain amount of feet of film, and come up with the gags and bits of story to fill it. Then the work produced by the various animators would be spliced together and, hopefully, they had a reel where each animator’s contributions formed a somewhat coherent plot.

Visual gags were a requirement because the story and character development were not considered important. So the use of gags and other gimmicks became necessary to sell a cartoon to film distributors.

Code shouldn’t be designed as it’s written. If you believe this contradicts the agile philosophy, in particular XP, you’re probably opposed to TDD as well which is a heavier process method than just straight writing code. Writing the actual code to be used in a production environment should be one of the last steps in the development process. A lot of effort must go into reviewing and validating the requirements, the implementation languages and the development toolchain, the architecture and interfaces, the components’ roles and interactions, the execution platform, etc. before the production code itself can be written.

If you do not have the behavior of the code firmly in mind before you begin writing it, you cannot reasonably expect it to behave well with others, and you will be doing a lot of rewrites. And debugging. A lot of it.

Software Development Isn’t an Art III

One pitfall that programmers typically fall into is the need to just dive right into the code. (And production code at that.) They’re in such a hurry to do the job they enjoy the most that they fail to consider how to do it best. That means design. That means analyze the problem before you try to code the solution. That means test what you’ve come up with to validate it before you attempt to codify it.

Pulling examples from the animation profession again, here is a list of the test processes and artifacts used to validate the direction the production’s elements are going (or get it off the ground). There are story treatments, scripts (AKA screenplay), storyboards, and animatics (leica reels) which go through constant revision. There are concept drawings and paintings, character sketches, model sheets, maquettes, set designs, and color tests to develop the look and feel of the film. There are musical scores, bar sheets (AKA dope sheets), and exposure sheets (X-sheets) to work out the timing of the vocals and music with the action. There are layouts to determine camera angles and how motion will be staged.

Even after all this design and timing work has been done, an animator will thumbnail how his drawings will progress before actually beginning (or if he needs to, rethink) his drawing. As far as the animated drawings are concerned, there are roughs which are pencil test and studied before the inbetweens which are then cleaned up before the final ink & paint, and none of these drawings are even seen in the final production. In CGI they use wireframes to review the animation and camera movements before a scene’s rendered frames are generated.

As software developers, we have quite a few tools available to us to speed up the development process. There is the ol’ flowchart and pseudocode. Or we can go with a more in-depth, and sophisticated, Nassi-Schneiderman, Warnier-Orr, Jackson Structured Programming, or UML diagramming notation.

There are Story Cards, CRC Cards and my personal, more permanent, whiteboard substitute.

We can write Use Cases, Unit Tests and Functional Tests (also known as acceptance tests).

There are even Spike Solutions and Prototypes to test ideas, not just UIs, before committing too much effort and cost to the wrong direction.

How many of these or other non-production materials do you use to design your software and provide documentation for new members to your team (think agile communication that scales)? The importance of these tools in improving the quality of your code cannot be understated. Better designs make for better code.

If you read the comments for the link to bar sheets above, you’ll see that this issue of pre-planning strikes at the heart of the quality of the final product. Particularly this comment (which I’m reproducing here because Safari doesn’t scroll directly to it):

Stephen Worth said…

The most sloppily drawn, poorly timed television programs have the highest budgets. It doesn’t cost any more to draw well and plan out your timing. In fact, careful timing can save money…

It eliminates the need to “trim” shows to “make them play”, cutting down on the amount of wasted animation.

If action is timed to a beat, even if it’s a bit slow or a bit fast, it still feels “right”. When you time with no rhythm, you have to do multiple pencil tests and revise your work to parallel park it into looking “natural”. [Emphasis mine. -- AG]

Animation is the one medium where every single aspect of what is on the screen can be carefully planned. [O RLY? -- AG] If a director chooses to ignore some aspect and let it go to chance, that just means he’s a lazy director. It has nothing to do with money.

We have quite a few tools available to us as software developers. We just need to exercise the discipline to use them if we’re going to raise the quality of our work and be recognized as a true engineering profession.

Truth be told, I’m not big on software development. I became a programmer by hanging out at Radio Shack and trying to figure out what made the beasties run, but it wasn’t the career I had intended to enter. I grew up doing cartoon animation, and it was an expensive hobby in those days.

But this posting isn’t about my career choices. This is about software development and the myth, often voiced by programmers, that what they do is an art.

Now, there are exceptional programmers. And there are exceptional programmers who are artists. There are men who, with an understanding of the programming language and the architecture of the hardware on which it runs, can craft simple and elegant solutions in less than one tenth of the execution space and time that ordinary programmers do. But the programming profession is far from an art form.

Programming isn’t an art form not because it’s derived from mathematics or is executed on emotionless, sterile computing machinery; but because not enough is understood about what makes an effective program and how to build one. The vagaries of the practice don’t prove it’s art. It proves it isn’t.

To demonstrate this difference in more concrete terms I’ll illustrate with an overview of the development of an animated film, which I hope you would consider to be an art.

Initially, a story treatment is presented giving an overview of the film’s premise, the characters involved and the plot. If interested, the studio will pick up a treatment and secure a script. If the script is approved then various studies are made dealing with specific aspects of the story, characters, lighting, colors, sets, props, pacing, etc. Voice talent is selected for the characters. As the story is developed, storyboards are made illustrating the progression of a scene. An animatic, or Leica reel, is made combining dialogue, storyboard sketches and sound effects to provide the team with a sense of the film so they can detect weak or unworkable issues. As action is animated, the drawings are filmed as pencil tests to offer a quality review. As pencil tests are shot and approved and final dialogue, sound effects and scores are recorded they’re incorporated into the animatic so the team can gauge the film’s progress. The animators’ drawings are inked and painted, formerly by hand. Backgrounds and special effects are drawn and painted and merged with the colored animation and filmed. Eventually the animatic is transformed into the final picture and can be packaged and distributed. Of course, artwork and performances for marketing materials, trailers, and publicity are also necessary.

Production proceeds in a manner that is logical from a production standpoint, not one parallel to the film’s time. Sequences are animated out of linear order. Voice actors in a scene are usually not recorded in the same session. One animator draws the lead character in one sequence of the film while another draws the same character in a later sequence, and additional characters in those sequences are drawn by yet other animators. And normally, different sequences are in different phases of production at any given point. Scenes may have to return to an earlier phase several times before approval, but the film will still be completed properly.

This development cycle typically lasts three to four years for a feature-length animated film. How is it that a schedule for something as vague and variable as a filmed story can be well-defined enough to meet a release date? Because the problem is well-understood. Because the metrics (e.g., duration of a scene, characters per scene, feet of film an animator can output per week, etc.) are known. Because decisions and directions are constantly tested and screened every step of the way. Of course, though steps have been taken to minimize the risk of script changes, re-animating or re-filming sequences, etc., sometimes substantial and costly changes must be made as the film nears completion. In animation, as in software development, it’s very expensive to fix poor design decisions when they’ve been executed.

Every artist participating in the development process knows that though his contributions are a tiny portion of the final product, the rest of the team are dependent on his output for its success. Consider the outcome of the artist’s mindset, and how lacking it is in software development today: a development team of three to four hundred individuals capable of completing a three-year project on schedule and within budget repeatedly!

No, software development isn’t even close to being an art.

Technorati Tags:
software-development, programming, animation, art