Death by Email

January 16, 2005 12:43 AM

I had nothing much else to do this afternoon, so I installed the latest Mac OS X Tiger seed on my home iMac. Mentioning this probably violates some NDA or other, but I figure so long as I don't divulge anything that Steve Jobs hasn't already demonstrated a thousand times, I should be safe.

OS X's Mail client has been spotlight-ised, of course, with smart folders and impressively fast searching through my 300Mb or so of archived mail1.

One smart mailbox that is included by default is a collection of all your unread messages across all your mail folders. Turning on Mail for the first time, I was greeted with the news that I had 55,000 unread messages.

Needless to say, in the time it would take me to even sort through 55,000 messages and intelligently throw away the ones I wasn't going to read, I'd have accumulated 50,000 more. So I created another smart mailbox of all unread mail that was more than a week old, and with a short, guilty mouse-click, marked them all as having been read.

Then, after making a cup of tea, I went through the rest. Mail is now happily reporting that not a single message of my entire mail archive is unread. I suspect that's untrue for at least 95% of the messages.

The problem is mailing-lists. I'm genuinely interested in the goings-on of the ruby-lang mailing-list, but at over a hundred messages a day, it's simply impossible to keep up with it. I'm genuinely interested in firewall-wizards, but even though it only sees 10-20 messages on a busy day, I'd probably rather read 20 more mails from the Ruby list.

And so on. Before you know it, you've got 55,000 unread messages.

Mailing-lists suck. So many projects rely on mailing-lists, even to the point of requiring you to subscribe to a list before you can communicate a bug, submit a patch or even provide feedback. And every list is mental overhead that you just don't need if you're planning on spending any time not reading email.

And don't tell me how RSS is going to save me from email. I have 550 unread articles waiting for me in NetNewsWire.

The best cure for this particular malaise that I've encountered was kernel-traffic, a weekly newsletter summarising the significant goings-on of the linux-kernel mailing-list. Unfortunately, kernel-traffic is an exception that I've never seen translated effectively to another list: it requires a rare level of commitment from an individual editor, and an even rarer talent for summarisation.

One of the more convincing arguments for wikis has been that they offer projects an escape from email Hell. Conversations on a wiki almost inevitably lend themselves to summarisation as the site's (usually self-appointed) WikiGnomes point people to previous discussions, and act as a distributed force to clean up threads into pages. These post-discussion pages then become a natural part of the project's documentation.

While I was writing this post, 8 new messages arrived in my ruby-lang-new folder. I didn't have time to read them.

1 My work mailboxes are bigger, possibly by an order of magnitude, but testing against them would involve upgrading my Powerbook to Tiger. I don't really want to put a pre-release operating system on the machine I do all my coding on.

14 Comments

It appears, that you're not aware of http://news.gmane.org/, which is a basically a web interface and an nntp newsserver, which provides very convinient access to thousands (?) of mailling lists of open source projects (your particular interest might be missing, but still). You can even post to a mailling list through a newsreader without the need to subscribe to the particular mailling list. This is very usefull, when you only have the occasional question without the need to be constantly subscribed. Higly recommended.

email is broken. No doubt about it. I have 4,000 unread messages from one list alone and I'm never going to get to and anyway - more than half of them are either 'unsubscribe me' or 'yeah, you're right...'

I like the idea of the smart folder in Mail.app.

I think most e-mail apps have terrible UIs for handling large volumes. The three-pane system seems fine for low volumes, but most people have a huge Inbox where things just drop to the bottom (or top, depending on preference). There's also the issue that the most popular non-webmail e-mail client (Outlook) has search functionality that blows chunks (to use the vernacular :). Mail.app's find-as-you-type is the way things should work (though I would like to have an "Advanced" option which allows more criteria (like limiting to time frames)).

It's also that most people have exteremely bad habits with regards to how they use e-mail. This is parlty an issue of how most e-mail programs create messages and replies (and also because simply use their Inbox as a contact folder -- finding a person's e-mail address and hitting "reply" to start a new thread).

Top posting [1] [2] is used a lot by most people (since it's the default in Outlook and Mail.app) and it makes many things more difficult. There's also the point that most people don't trim their messages so there's this huge amount of cruft at the bottom of messages (plus HTML).

Currently I think Usenet is the best model for large volume discussions. You can have hundreds of conversations (threads) around a general topic (Usenet group) and deal with the volume fairly well. With a decent reader (not Outlook) you can follow large threads fairly easily. Add filtering and killfiles for the "silly" people and you've got ways to ignore noise.

Most news readers also have good threading support. There are conversations that have over a hundred posts (some even with different subject lines) and it's very easy to follow the flow of the conversation and who is talking / responding to whom. This is all but impossible with most e-mail clients (though Gmail is decent in it's threading support [3]). This long conversation of course is only one in many dozen others -- and they don't interfere with one another because of good threading [4]. GMAME (mentioned in the first comment) is very good, and is very good at this. Mailman [5] is also good at threading.

People in Usenet tend to be a bit more acquainted with the "proper" ways of posting messages so posts are trimmed, replies are usually "inline" or bottom posted. With e-mail, most people are simply thrown in and are not "indoctrinated" so you get short messages, with IM-style spelling, lack of capitalization, and many ellipses ("...") containing over a dozen periods.

In conclusion to this rant and rambling monologue, e-mail is different things at different times. Sometimes it's a well-formed essay, sometimes a quick Post-It(tm) note, and at times the equivalent of a quick word over the cubicle wall. It continues to be the "killer app" of the Internet (regardless of the deluge of spam), and thus is used as one of the primary ways to communicate (even it it's the wrong medium) because it is available to everyone just about everywhere (thanks to webmail). Other methods of communicating may be appropriate at times, but they're not always available so e-mail is used as the stop gap measure.

Hence the flood of messages.

P.S. I also don't like the way many people just drop links in e-mail messages. I prefer 'footnote style' way of having links. It's easier to read the text if there isn't this huge URI in the paragraph. The links are pointed to via numbers and can be at the bottom of the message like here, or at the end of the paragraph like in Debian Weekly News [6].


[1] http://www.dickalba.demon.co.uk/usenet/guide/faq_topp.html
[2] http://www.google.com/search?q=top+posting
[3] http://www.google.com/search?q=Gmail+threading
[4] http://www.jwz.org/doc/threading.html
[5] http://www.gnu.org/software/mailman/
[6] http://lwn.net/Articles/117337/

Gmail is pretty good for mailing lists... Searching all the mail you have received is pretty good. If it allowed distinguishing between read-mail and un-read-mail for searches, that would be even better.

I've just spent a good portion of a lunch lecturing jiramike on the joys of imap, where you can happily have your work email in multiple locations, which makes testing clients a breeze and far far less of a commitment.

Now just make sure you nag enough internally to make it happen!

I've been nagging quietly for about a year now. IMAP at Atlassian seems to be like the 'good china'. It exists, but it only seems to be able to be used on special occasions. :)

On the other hand, I consider having computer at home on which I can't receive work-related email to be an overall positive.

I third the mention of gmane, though in some ways it just ends up being another different place where you have hundreds to thousands of unread items. Since you (Charles) are a Mac user, you can use MT-Newswatcher, which has one of the better interfaces geared for high volume (i.e., that lets you ignore the vast majority of the incoming items, and they eventually just go away).

My bugaboo is that email UIs still give us only ONE BIT to mark a message as "read" or "unread". I want time- and subject-based filters and at least a dozen or more categories of unread-ness.

[As I wrote here: http://zia.pycs.net/2003/3/7/
I use a word I made up 7 or 8 years ago, "nitam", to mean any email message, USENET article, or weblog entry (etymology: net-item, or acronym for "News ITem, Article, or Message"; pronunciation: like "night-um").]

The use of that word emphasizes my attitude -- that I, as the reader of a nitam, should be able to decide which kind of UI to use for that category of nitam; the UI style should not be dictated by which of these channels happened to contain the nitam.

I find your predicament amusing, mainly because of a mental image I had when I was reading your post.

You seem to be subscribing to these mailing lists with the best of intentions, that some day you may find the time to read them. It reminded me of someone who buys exercise equipment off infomercials with the best intentions of using it, but then feels guilty every time they open the closet or look under the bed and see all the unused equipment.

I don't actually understand why you subscribe when you can read most of the mailing lists on through USENET gateways or on the web. It seems like a bit of a waste of disk space and bandwidth (and closet space).

Web interfaces to mailing lists, for the most part, suck. I guess I could gate my mailing-lists to a private instance of Confluence, but since it's an archive, not a mail client, it doesn't have (and doesn't plan on having) any way to track what I have, or haven't read.

Whenever I have to find anything GMane, I end up wanting to tear my hair out. I'd rather have the mail sitting around on my machine so I can search and manipulate it locally.

Usenet interfaces are much the same as email. I was a Usenet junkie for several years, and quit for the same reason I can't follow email - the volume of what I wanted to read overwhelmed my ability to consume it. The only difference is that most usenet servers expire messages fast enough to silently remove anything you're unlikely to get around to reading (At the cost of not being able to track back through long threads you entered late without resorting to Google Groups)

Predictably, I have to mention ZOË [1] which does a pretty good job at handling significant amount of email. And even knows what mailing lists are [2] :)

[1] http://zoe.nu/
[2] http://zoe.nu/itstories/story.php?data=stories&num=29&sec=2

[shameless plug]
Every month I summerize the interesting posting in the performance discussion groups in our monthly newsletter. And don't worry, I can't help but editorialize so it's not a stright review ;)

you can subscript at http://www.javaperformancetuning.com/newsletter.shtml if you're interested in yet another summary.

Re: "... an exception that I’ve never seen translated effectively to another list":

The Python development list has had such editors for some time, though they apparently stopped a couple of months ago. See

http://www.python.org/dev/summary/

my last contribution to Usenet is from around 1996 I think, so gave up on that one long ago. The only use I find in the total overkill of millions of postings over there is doing a search on Google Groups every now and then. So Usenet can still come in handy when I'm looking for someone who struggled with the same problem as me on that moment.
I unsubscribed from my last mailinglists about two years ago. It was during the time I was discovering weblogs as being more interesting to read than mailing lists. They were an eye opener as to offering information on (techrelated) topics to my interest. Much more interactive. But, I also found myself trying to keep up with more and more websites.
Then this RSS and feed thing came into my view. Now, I can't imagine surfing without my newsreader. Guess that's just an ongoing development, using the tools available at any certain moment in time. The only thing is that sometimes I wonder how people can still be interested in Usenet anyway, when there's so much more easy to use tools available (take Wiki's e.g., but also modern forums with a decent login and a much better tracking system. Etc., etc.

the best solution is a tropical island sans internet .. peace ;-)

Previously: XML is Not the Answer

Next: Language of the Year