Talk Like A Duck

I was playing around with Pixelmator today, and made a new sketch of Ruby the DuckDog whose photo graces the masthead here.

When the old dog say her picture, she told me that she wouldn’t mind sharing it, so I decided to put it on a few items which the two or three fans of this site might enjoy.

You can find them at the Talk Like A Duck Store.

Consider this post the grand opening. The small profit from any purchases will help keep Ruby in DuckDog food, and support the activites here at TalkLikeADuck HQ.

Martin Fowler, just wrote a typically insightful article about the number of development teams he has encountered, which, with the goal of being agile, adopt scrum for project management, but run into problems when they fail to adopt approaches to actually executing the project which enable successful iterative development.

I highly recommend that anyone interested in truly being 'agile' digest this article.

As an aside, as a side-effect of reading this article, I once again found myself looking at the original wiki, Ward Cunningham's WikiWikiWeb, which was pointed to by several links in Martin's article. Every time I find myself browsing around on Ward's wiki, I find at least three useful ideas.

Joe Van Dyk, a reader, posted a question in response to my recent article about my work towards a new gem providing iCalendar support for Ruby

How do people represent recurring events in a database? I’ve never been able to come up with a good way.

My particular problem is that each event that reoccurs needs to be able to be worked with individually (i.e. comments added to it), deleted, moved, etc.

Rather than answering directly in the comments section, I'll give my thoughts on this as a new article

My approach to this would start with recognizing the difference between an event, and an occurrence of that event. Each event would be stored as a single entity on the database, no matter how many occurrences that event had. Each event entity would have a start date-time which would be the start time of the first occurrence, and a last occurence end date-time which would be the end of the last occurrence if the event had a defined last-occurrence, or null if the recurrence set was open ended.

A non-recurring event would be handled as a degenerate case of a recurring event, with a single occurrence.

The idea is to allow for a database query which returns the set of (recurring) events which have any occurrences between two, points in time. The use-case here is displaying a calendar page, where I need to find all of the events for a given, day, week, month, etc.<\p>

Once I had those events, I'd ask each one for a list of occurrences within the target range, which for some events and some ranges might be empty.

Individual Occurrences

As Joe points out there are time when you need to talk about individual occurrences of an event. RFC 2445 talks about this. There are basic mechanisms in iCalendar to support use-cases like editing a series of recurring events. So iCalendar, defines three attributes which combine to connote a particular event, set of occurrences, or a particular occurrence

UID

A globally unique identifier which identifies a particular event with all of it's occurrences. The set of occurrences may change if the event is edited, with different edits being identified by the ...

Sequence Number

The revision sequence number of a calendar component, in this case a VEVENT, with a given UID

Recurrence-ID

Which represents either a particular occurrence of an event identified by a UID and sequence number, or subsequence either at the beginning or end of the list of occurrences for an event. The value of the recurrence id is a string containing either:

1. A string representation of the start time and date for the occurrence or
2. A string representation of the date of the occurrence in the case of an anniversary (all-day) event or
3. Either of the first two, preceded by "THISANDFUTURE:", indicating all occurrences starting with the one identified by the date or date-time or
4. Either of the first two, preceded by "THISANDPRIOR:", indicating all occurrences starting with the first through the one identified by the date or date-time or

I'd put any attributes associated with an individual occurrence into a separate entity belonging to the event, which would have many of these, and identified with a recurence-id.

Just a word of caution, here. This article reflects a mixture of things I have actually done in the past, with some thought experimentation. I haven't actually had to deal with attributes for occurences, but what I've described 'feels right' to me as a first approximation.

I'll be speaking about dynamic languages at RubyRX which will be held from February 19-21, at the Mariott in Research Triangle Park, NC. If you are attending, say hi, I'd love to meet you.

Instead of blogging much of late, apart from the odd twitter tweet, and playing with alpha versions of Maglev as they come out, I’ve been spending most of my time working on a new implementation of RFC 2445 “Internet Calendaring and Scheduling Core Object Specification (iCalendar)”, in Ruby.

My previous employer wanted to implement iCalendar recurring events to allow their customers to import and export events from their favorite calendar application, be that MS Outlook, the Mac iCal app, Google calendar, or whatever. RFC 2445 is the “linqua franca” for calendar applications. Unfortunately there wasn’t enough in the budget to do this given the technical challenges which I’ll describe shortly. Since I’ve had a lot of free time on my hands since late November when the budget for anything seemed to sublimate along with the company because of the financial crisis, I’ve been chipping away at it anyway.

Although there are several extant ruby implementations of iCalendar, the most popular being the icalendar and vPim gems, which do a reasonable job parsing and generating icalendar files, none of them provide an implementation of the harder aspects of iCalendar: time zone representation and recurring events.

You have to understand that RFC 2445 is the demon spawn of IBM (via Lotus) and Microsoft, and like many such standards, appears to be a camel with humps full of the quirks of the proprietary products from the author’s companies, and when those companies are IBM and Microsoft, and it’s the late 1990s, the standards don’t always conform to todays practices.

Does Anybody Know What Time Zone It Is?

While Ruby has libraries which deal with time zones (TZInfo), and recurring events (Runt, and TExp) in both cases they aren’t usable out of the box to support iCalendar.

TZInfo, like most of the computing world, uses the Zoneinfo (a.k.a. Olsen) database of time zone information, which allows referencing time zones by standard names. RFC 2445 requires each iCalendar file or datastream to contain a definition of each time zone referenced within the file. The time zone definitions name the time zone, give an initial offset from utc, and either a list of transition times between standard and daylight saving time, or recurrence rules defining those times.

So one of my goals is to allow Ruby applications to import iCalendar files containing time zone definitions and make use of those definitions with the contents of the imported calendar, and to allow calendars produced by Ruby applications to properly produce valid iCalendar time zone definition components.

Right now, I’ve done most of the groundwork for the second goal. I can generate a time zone definition (technically a VTIMEZONE component) from a ZoneInfo time zone name and a starting and ending time, the component is tagged with an extended attribute so that if I import a file produced by this library I can use TZInfo to do the time zone mappings for the calendar.

I put the second goal for time zone components off for the time being while I’ve been working on the other major problem area, if for no other reason that iCalendar time zone components make use of some of the same machinery as recurring events.

Recurring Events, or Deja Vu, All Over Again

Runt and TExp are both based on Martin Fowler’s paper on a pattern for implementing recurring events. Martin’s approach is quite elegant. Unfortunately, it’s rather inadequate to implement iCalendar’s notion of recurring events.

The primary component of a recurring event definition in iCalendar is a ‘recur’ value, which describes how to enumerate a set of occurrences given a starting time. The actual definition is quite complex and hard to understand. A ‘recur’ val has a basic frequency, like MONTHLY, or WEEKLY, or MINUTELY, and an interval. The frequency and interval can be used to express a recurrence pattern like, “every 3 days”. Then there are optional parts like, BYMINUTE, BYYEAR, BYWEEKDAY, which if they represent a period of time the same as or longer than the frequency act as filters, for example “FREQ=MONTHLY;BYMONTH=1,2” represents an event which happens every January and February at the given time. If a BYxxx part represents a shorter period than the frequency, it adds additional occurences, so “FREQ=DAILY;BYHOUR=11,14” represents an event which recurs every day at both 11:mm a.m. and 2:mm p.m. where the mm is taken from the separately provided start time.

A second way to represent a list of occurrences is explicitly with a list of date or date-time values.

That’s all complicated enough, but a given event can have multiple ‘recur’ values and lists, each of which can either add (RRULE, RDATE) or prevent (EXRULE, EXDATE) occurrences from being included in the final event.

While I’m pretty sure that a lot of this capability never gets used in the real world, it’s hard to determine what subsets the various calendar applications actually use. The complexity of iCalendar has led to efforts like and Calsifywhich is chartered by the Internet Engineering Task Force with simplfying RFC 2445 ””, along with the related RFC 2446 “iCalendar Transport-Independent Interoperability Protoco (iTIP)”, RFC 2447 ” iCalendar Message-Based Interoperability Protocol (iMIP)”, and RFC 3283 “Guide to Internet Calendaring” and CalConnect a consortium working on usage standards to aid interoperability of applications the existing versions of those standards

In the spirit of Postel’s Law I’ve decided to take on accepting everything that RFC 2445 allows.

Where Things Stand

First I considered doing this as an extension to either the iCalendar or vPim gems, each has it’s advantages and disadvantages. After considerable thought I decided to start afresh, and build a behavior-driven implementation as much as possible. I’ve found myself taking a slight detour from strict test-first design in the case of the implementation of enumerating ‘recur’ values. Since there the specification is so convoluted, it was more natural to try to ponder the overall problem, and write ‘black-box’ specs directly from the RFC. It took me several days of pondering all of the examples in RFC 2445, and a few ‘shower thoughts’ before I got the general approach to enumerating occurrences. In either case, I’ve been using RSpec as my specification/behavior driving/acceptance testing tool, with great pleasure.

So far, I’ve got specs and code for:

1. Parsing a calendar envelope
2. Parsing an event
3. Producing a VTIMEZONE calendar component for a given time range from a TZInfo::Timezone
4. Parsing ‘recur’ values
5. Attribute reading and writing for ‘recur’ values
6. Enumerating occurrences for a ‘recur’ value given a starting time

Having just completed the last of these and passing all of the example use cases given in section 4.8.5.4 of RFC 2445, I feel like I’m over a big hump, with a few, hopefully smaller, humps to go like the merging of RRULEs, EXRULEs, RDATEs and EXDATEs; and giving imported VTIMEZONE components the ability to actually do time conversions. Other that that I think it’s just a series of several smaller tasks. One of the reasons for posting this is to give myself some incentive to keep at it by going public!

It’s not quite ready to foist on the world just yet, but I hope to do that soon. My inclination is to release this as open source. I must say that since I’ve been without paying work for several months now, and this has been a self funded project, I’d like to figure out how to generate some revenue from what has been a considerable amount of work. I’d welcome any ideas to that end.