The following is a rudimentary set of instructions for setting up, on a Linux server, a Darcs repository which uses the most basic possible method for collaboration, in which a single administrator manually applies patches¹ sent him/her via email (automatically, via the darcs send command issued by the contributor).² Some notes on producing those patches and managing the repository are also included. Further information, including info on how to retract changes, is available in the Darcs manual.

cd proj_folder
darcs init
darcs add -r *

darcs record -a

darcs record -am "Project imported."

cd /path/to/project
darcs apply /path/to/patch

darcs push username@website.com:/path/to/repos/proj_name

darcs get http://www.website.com/repos/proj_name

darcs whatsnew

darcs record -a

darcs send -a

darcs send -a --output=/tmp/changes.patch

darcs add myscript.py

darcs pull http://www.website.com/repos/proj_name

darcs tag 1.0

darcs get --tag "1.0" http://www.website.com/repos/proj_name

Over the past year or two I’ve become increasingly convinced that one of the keys to personal productivity is to spend as little time as possible connected to the Internet. For some–for example, those who need to be constantly available via an instant messenging client for collaboration with remotely located coworkers–that may not be practical. Others may be so disciplined that they don’t suffer from the temptation to succumb to the manifold distractions presented by Web browing and emailing. But for those of us who do not fall into one or both of these categories, offline living–connecting to the Internet only for say, an hour a day, or two hours a day two or three days a week, depending on your circumstances–can be a major productivity booster.

In a guest post for the Getting Things Done-oriented personal productivity blog 43 Folders¹, Paul Ford describes some of the symptoms of Internet-induced productivity loss:

My work requires me to patiently work through things and come up with fresh ideas. And I can honestly say that since broadband Internet came to my home a year and a half ago my stock of new, fresh, fun ideas has grown very thin. It’s just too much. My mind can’t wander, because, with anything that interests me, I can look it up on Wikipedia to gain some context. Before I know it I’ve got thirty tabs open at once in Firefox. Then new email comes in.

Ford admits to being a "weak-willed" and prone to "disasters, like reading Slashdot threads, or meandering through Google News, or ‘researching’ something on Wikipedia and following a chain of fifty links."

These descriptions resonated with me pretty deeply, and my guess is that for most people they have a more than familiar ring.

Recognizing that the Internet is an indispensible tool for virtually anyone living in a modern society, "knowledge worker" or otherwise, how does one practically go about leading an offline life?

In my case, it means no Internet access at home. I have a Wifi-enabled notebook computer which I use to connect to a wireless network at the university where I study, or at a nearby coffee shop. I usually connect about three times a week for around 2 hours at a time, browsing the Web for news, researching projects for my graduate study, sending email, administering Web site projects, etc. (Admittedly, my current life as a graduate student with minimal formal work responsibilities makes offline living less challenging for me than it might otherwise be, but I hope to be able to apply the principles of offline living whatever the circumstances I might find myself in in the future.)

Now, the key thing to recognize here is that I do not restrict time at the computer to these online sessions. I spend many more hours with my computer offline reading downloaded articles, programming, reading and composing emails, etc. The benefit of being offline is that it short-circuits such time-wasting propensities as the link-clicking excursions Ford describes and the temptation to check your email every 15 minutes. If I find that I need access to an item I failed to download for offline browsing, I make a note to look for it the next time I’m online.

The following is a list of a few tools that facilitate offline living.

Firefox Web browser with Scrapbook Firefox extension

This is the number one necessity for anyone who aspires to successful offline living. Scrapbook is a tool that allows you to create and maintain a library of downloaded articles (or entire sites, for that matter) for offline browsing in a very agile manner. The ability to categorize and search that library is a secondary benefit.

Desktop email client (E.g., Thunderbird or Microsoft Outlook (Express))

You can’t aspire to offline living if you only do email using Hotmail or Yahoo! Web mail. You’ll need to be able to read and compose emails offline, and send them when you connect again.

RSS feed reader

Subscribe to the RSS feeds of your favorite sites. Some will provide the full text of the article, and you’ll be able to read them offline through the reader itself. For those that don’t, you may want to use the reader’s article flagging feature to remind yourself to download the article the next time you’re online.

Note taking/task management program

You’ll need a program that allows you to create a list of reminders for tasks (whether an article to read or some other online activity) that can only be accomplished online. Which program you use will depend on your taste. Scrapbook has a notes feature you may find useful. I currently use gJots for Linux, a free-form hierarchical notetaking tool.

Darcs revision control system (for programmers)

More mainstream revision control systems like CVS or its successor Subversion require frequent online access to the central repository. Darcs (and similarly distributed systems like Arch) is different. From the Darcs Wiki: "You can commit changes even if you’re on the road with no access to the server. That’s because your own working directory is a repository in its own right. When you get home, you do a ‘darcs push’ to commit to the public server."

An objection to all this might be that the very concept of "offline" could very well be obsolete some five or ten years into the future when the airwaves are bathed with ubiquitous free Wifi. (Indeed an undisciplined person aspiring to apply the principles of offline living at home might discover that his or her neighbor has an open Wifi connection.) Despite the perceived benefits of such a state of affairs, it would be unfortunate for a number of reasons. At the very least, offline living would become more challenging, especially for the weak-willed and undisciplined. For now, though, and for those whose work conditions allow for it, it’s a practical and effective solution for enhancing personal productivity.

As a part of my tentatively titled Kill Your Word Processor project, I’ve been trying to think of ways to expand the audience for it, particularly because I’ve more than once been frustrated by collaborators’ insistence on the use of clunky word processors and browser-based WYSIWYG editors which I’ve long since given up on.

I even posted a note on the Backpack forum, where users have been exposed to the Textile lightweight markup syntax, trying to get others’ opinions on the matter, but it seems I’m preaching to the choir over there.

I have, however, been encouraged by what appears to be an increased interest in lightweight markup that seems to be part of a broader paradigm shift led by the spread of "Web 2.0."

But ultimately tangible results will be the most effective means of persuasion–e.g., a beautifully formatted LaTeX document converted to PDF, and a crisp, clean Web page springing from the same source; a demonstration of the tools that make such results effortless, etc. Still, I wanted to at least lay out some of the rational arguments here, hopefully with the end result of compiling a concise list of the benefits of the moving to lightweight markup.

There are at least two technologies that have preceded the current wave of interest in alternatives to bloated word processing software–one rooted in the requirements of collaboration on document composition, the other in the benefits of separating the production of content from its presentation.

Wikis emerged on the strength of their usefulness for collaboration on documents. The limited utility of passing Word documents back and forth as attachments via email should be apparent to most by now to anyone with enough familiarity with the Internet to know that there are tools (e.g., Wikis) that make that technique unnecessary. Among other benefits, most Wiki software packages have built-in versioning features as well as warning features to deal with simultaneous edits.

Wikis in fact sprang from the "Extreme Programming" (XP) community and from one of the guiding principles of that community: "Use the simplest thing that could possibly work." For that reason, Wikis did in fact incorporate and promote the concept of separation of content from presentation through the use of exactly the kind of simplified markup languages that are under discussion here. From c2.com, the site of the original Wiki:

Wiki’s emphasis is on content, not presentation. The simple markup rules make people focus on expressing their ideas, not making them pretty. Some people have found that working in this minimal medium improves their writing.

Unfortunately, in many cases, more technically inclined users demanded the ability to include HTML formatting by typing angle brackets which make the raw input of a document difficult to read, while the less technically inclined demanded the inclusion of word processor-esque WYSIWYG editors. The degree of accommodation of such requests varies from package to package, but clearly not all Wiki users were convinced of the benefits of simplified markup. This may be because those markup languages were not particularly well thought-through.

The concept of separating content from presentation has a more venerable heritage, however: It’s embodied in the LaTeX typesetting system, which was derived from TeX, which in turn was developed by Donald Knuth in the 1970s. Ubergeek protestations notwithstanding, composing a document in LaTeX’s own syntax is far more painful than composing in a word processor, but the content/presentation concept is firmly embedded here, as is the benefit of being able to compose in any text editor.

LyX is an editor that takes a lot of the pain out of composing LaTeX documents. While I don’t consider it very useful for my purposes, its documentation does make some interesting points about the importance of content/presentation separation:

Part of the initial challenge of using LyX comes from the change in thinking that you, the user, must make. At one time, all we had for creating documents were typewriters, so we all learned certain tricks to get around their limitations. Underlining, which is little more than overstriking with the "_" character, became a way to emphasize text. You were forced to figure out column sizes and tab stops, and set them, before creating a table. The same applied for letters and other right justified text. Hyphenation at the end of a line required a careful eye and a lot of foresight.

In other words, we’ve all been trained to worry about the little details of which character goes where. Consequently, almost all word processors have this mentality. They still use tab stops for adding whitespace. You still need to worry about exactly where on the page something will appear. Emphasizing text means changing a font, similar to changing the typewriter wheel. This is the underlying philosophy of a WYSIWYG word processor: "What You See Is What You Get". Unfortunately, that paradigm often results in "What You See Is All You Get".

This is where LyX differs from an ordinary word processor. You don’t concern yourself with what character goes where. You tell LyX what you’re doing and LyX takes care of the rest, following a set of rules called a style. …

So, the basic idea behind LyX is: specify what you’re doing, not how to do it. Instead of "What You See Is What You Get," the LyX model is "What You See Is What You Mean" or "WYSIWYM." It’s a powerful idea that greatly simplifies the mechanics of writing documents. This is also why LyX isn’t so good for creating posters and flyers—in this case, you do want to specify exactly where everything goes, because there are no functional units like paragraphs, sections, etc.

(To be fair, most word processors do in fact have a similar "style" feature to help you concentrate more on the content and impose a uniform look on your documents, but almost nobody uses it, probably because the nature of the word processor discourages its use.)

In spite of the fact that LyX is an application and not a markup syntax, you could easily replace the word "LyX" in several parts of this passage with "lightweight markup" and gain a firmer understanding of the benefits of the approach I’m describing. This passage is also interesting for indicating the circumstances under which this approach is not appropriate (e.g., posters and flyers).

Finally, in a recent thread on David Heinemeier Hansson’s site titled "Why do people still use word processors?", Hansson responds to a challenge to his rhetorical question and makes us aware of the false dichotomy between "technical" and "nontechnical" users vis-a-vis the difficulty of using a lightweight markup like Textile:

Reader: "People" are generally not IT professionals. It’s hard enough for some to even understand the concept of a table. It’s very hard for quite a few to understand the simple concepts of HTML. LaTeX is harder still to learn and use. … Textile is a good start and leverages some users, but probably not my mother, anyone else in her generation, or a significant part of the entire private sector.

Hansson: I was puzzled that students attending HA.dat (computer science and business administration) would suffer to Word when there were plenty of alternatives that didn’t require a bachelor’s degree in CS. … Your mother is excused, they were not.

I.e., there’s no reason why lightweight markup shouldn’t be appealing at least to computer literate users, even if non-computer literate can’t reasonably be expected to live without the word processor’s conceptual link back to the pre-digital world of typewriters.

In a lengthy post earlier this month, I took a first stab at describing a word processor-less approach to document composition and formatting. A key component of that approach is the use of what has variously been referred to as "humane text," WYSIWYM (What You See Is What You Mean), and "lightweight markup"–formatting syntax that allows you to compose a document–in a completely human-readable manner, without angle brackets–in plain text in any text editor, browser text field, or device, and run it through a parser to turn it into a format more suitable for your audience, such as HTML or PDF.

Searching the Web for those who have gone before me–particularly on the matter of convincing the technologically disinclined of the benefits of weaning themselves off of their dependency on Microsoft Word (or any other word processor or even browser-based WYSIWYG editor for that matter)–I began to discern the emergence of an eerie pattern: Interest in lightweight markup seems to correspond with the rise of the so-called Web 2.0 technologies.

First, I (re)discovered the existence of other lightweight markup languages besides the one I’m used to working with, reStructuredText; the principal ones are Markdown and Textile. (I should note here that various Wiki software packages have long provided their own markup formats.) Later, I learned that Textile (or some variation of it) is used as the default formatting syntax for 37signals‘ excellent personal information management tool, Backpack. 37signals had previously created the application that made Ruby on Rails famous, and in fact employ the creator of this Web framework, David Heinemeier Hansson. Ruby on Rails’ exploitation of the Ajax technologies in large measure has marked it as a kind of technological standard bearer of Web 2.0, as suggested by the first product offering from Adaptive Path (a key promoter of the Web 2.0 idea), Measure Map, also created on Ruby on Rails.

As if that weren’t enough, the other major lightweight syntax project, Markdown, was created by Mac-oriented blogger John Gruber, who is employed by Joyent, a company whose work appears heavily to leverage Ruby on Rails and Ajax, and which had previously acquired TextDrive, the company whose founder is responsible for Textile. The circle seems complete.

So where does lightweight markup fit into Web 2.0 conceptually speaking? It might be worth explaining what I mean by Web 2.0, since there seems to be no commonly accepted definition of this relatively new industry buzzword. One intriguing definition I came across (I forget where): "Web applications that don’t suck." For those inclined to a more formal explanation, I recommend Tim O’Reilly’s (his company coined the phrase) treatment of it, as well as Paul Graham’s reflections (though in light of the present topic, I can’t help but think that his suggestion to create a Web-based version of Microsoft Office is misguided).

Aside from noting that it was made possible in part by the mainstreaming of broadband Internet access (Ajax-laden applications tend to take a long time to load otherwise), maybe the best way to gain an understanding is to list a few of the more prominent examples:

• Google Maps, but especially the so-called "mashups" like chicagocrime.org and housingmaps.com, which use the Google Maps open API map data from a third source onto Google Maps’ super-responsive, desktop app-like interface, which popularized Ajax overnight
• del.icio.us, the "social bookmarks" manager, which aggregates the "tags" that users apply to their favorite sites and makes the results of that tagging available to all other users, allowing them to take advantage of the network effects that redound as a result of mass use of the site.
• Wikipedia, the collaborative encyclopedia, which until recently was open to editing by all Internet users, even without requiring registration of a username. As Graham notes, "experts have given Wikipedia middling reviews, but they miss the critical point: it’s good enough."

One of the common threads of these applications is their embrace of openness. In many cases this means using Open Source software and participating in Open Source projects, in recognition of the effectiveness of that approach in creating quality software in a short period of time. But perhaps more importantly it means open standards, including Web standards like XHTML and CSS, as well various XML dialects for sharing data between applications via Web services.

It is probably this fundamental quality of openness which has motivated the interest in lightweight markup among the Web 2.0 cognoscenti. Plain text is among the most platform-neutral technologies imaginable, requiring no special software to use, no special browser plugins, very minimal hardware requirements. It’s agile; when used with appropriate parsers, input can be repurposed for any number of uses, "decoupling" it, so to speak, from dependency on any particular application, just as Web 2.0 technologies tend to do with their functionality.

There once was a time when I regarded the word processor as a milestone in the evolution of publishing technology that began with Gutenberg’s printing press. I was astonished at the results that could be achieved when using Microsoft Word in conjunction with TrueType fonts and a low-cost HP inkjet printer. For a few hundred dollars, virtually anyone could quickly produce professional-looking documents that just a few years earlier required expensive typesetting equipment and human resources skilled in its use. Unaware of the coming Web publishing revolution and the subsequent importance of online documents, I believed that surely the word processor represented a quantum leap not only in the history of publishing, but also made a substantial contribution to the empowerment of the individual.

Today, I have a contempt for all word processors that borders on dementia.

Like most of the company’s products, Microsoft Office voraciously devours disk space and memory. Here in Argentina, where I often rely on older computers in cybercafes and the university where I study, a machine with 128 MB of RAM running Windows XP and Microsoft Office 2003 can easily choke on any document you try to feed it, thrashing for minutes at a time on simple actions like sending the document to the printer. But the problem extends beyond Microsoft bloatware–even on the iron-clad Ubuntu Linux system I’ve got running on my IBM ThinkPad T23, every time I try to access the help system in OpenOffice (to figure out why it wasn’t properly encoding accented Spanish characters from an imported Microsoft Word document) the program freezes my machine so completely that the only way to restart it is to pull the plug. Without denigrating the achievement of OpenOffice as a high-quality Free Software office suite, this is the most severe crash issue (if not one of only two or three such issues, period) I’ve experienced in seven years of using Linux. Even when OpenOffice functions properly, it is far from the lightest application on my desktop in terms of load times and memory consumption.

To question the utility of the vast majority of the features of the contemporary word processor is nothing new. It was once (and perhaps still is) a commonplace to state that 90% (or some such proportion) of users of Microsoft Word use only 10% of the program’s features. Without denying that some users really need such features as mail merge, embedded spreadsheets, etc., I propose taking that line of thinking to a logical near-extreme. The 90% of users who use only 10% of the features of a word processor can live without a word processor entirely.

Every item that you read on this site was composed not by tediously applying HTML angle brackets to delineate the paragraphs appropriately or create hyperlinks, nor by selecting elements and clicking icons using a "WYSIWYG" editor that may or may not generate valid XHTML, but by using a plain old text editor (e.g., Notepad for Windows users) and a free text standard called reStructuredText (RST). Plone, the Content Management System (CMS) on which this site runs, has RST to HTML capability out-of-the-box, but it wouldn’t be difficult to include such functionality in any CMS as at all.

The short version: RST enables you to produce a 100% human-readable plain text document which is totally free of arcane markup, but which provides a set of features for determining the structure of a document as well as elements for applying formatting such as boldface and italics and tagging items with hyperlinks. reStructured Text emerged from the documentation efforts of the team working on the Python programming language–the idea was to be able to provide fully readable inline documentation for programmers while maintaining the ability to use the exact same text for producing pretty, printable manuals, or documentation in any other format that happens to be useful. But its utility is in no way restricted to technical projects. I’m planning to use it, in conjunction with a set of tools that I’m currently researching (notably LaTeX and related programs), to compose virtually everything I write, which currently consists mainly of papers for classes in international relations.

So what are the advantages?

First of all, reStructuredText allows you concentrate on the content and structure of your document rather than its final appearance. A related advantage follows from this: As suggested by the motivations, mentioned above, of the Python team, when used with the right tools, a reStructuredText document can serve as a single source: for an HTML page, a nicely formatted and highly readable PDF file, an article in a printed newsletter, etc. All derive from your one lone RST document. If you want to make a change, you only need to make the change to the RST source and you can automatically regenerate the other documents.

Additionally, programs that convert RST to other formats, such as HTML for Web pages, typically include parsers to validate your RST. In the case of the HTML converter, if you care about things such as Web Standards, this helps ensure that your documents will be valid XHTML when the conversion is performed. Indeed, the RST concept shares much in common with the separation of content from presentation that derives from the application of Web standards with XHTML and CSS.

Some other advantages may be more personal–speed of composition, for example. It goes without saying that composing raw HTML markup will slow down anyone’s composition considerably. But I personally find word processor-like WYSIWYG interfaces to be tedious as well–selecting text and clicking icons can end up being more time-consuming than simply sticking with the keyboard during the composition process. Not to mention the fact that you can compose your RST documents virtually anywhere where you have a plain text editor available–the same won’t always be the case if you’re reliant on a particular WYSIWYG editor.

Finally, I can’t help but feel that the philosophy behind reStructuredText contributes, in some way, to informational hygiene in a world made sloppy by the mindless dumping of content onto the global net. It’s just a hunch–I may have to take some time to further explore this notion in the pages of this site.

The links you see in the sidebar of the home page of this Weblog are generated by a Python script which is executed nightly and collects any links I’ve posted to my del.icio.us account that have been tagged with an information technology-related keyword. del.icio.us permits this kind of keyword aggregation via a relatively new feature called "bundles." Unfortunately, there’s currently no way to get del.icio.us to show you all the most recent posts for a particular bundle. So I access the del.icio.us Web API via a delicious.py that I’ve modified to be able to work with these bundles, plus my own script:

(NOTE: MISSING CODE HERE) Inline code samples discontinued until I can fix the CSS for the HTML "pre" tag not to expand the window in I.E.).

As you can see from the comments (if not the code itself), it’s a little ugly right now, but it works. Looking forward to sharpening my Python skills by working with del.icio.us and other Web APIs.

I’ve been waiting for something like the ScrapBook Firefox extension ever since I started using a Web browser. ScrapBook enables you to save a snapshot of any Web page, or selection of a Web page, as easily as you would bookmark it, and then allows you to highlight passages, and provide your own annotations.

With an agile and intuitive interface, ScrapBook is yet another astonishing testimony to the superiority of Firefox as a Web browsing platform. For the first time in my life, I’m willing to put aside my nostaglic attachment to dead-tree books to consider how great it would be if everything I wanted to read were online.

The only problem I see is: What if I want to share my ScrapBook online? Or don’t want it to be restricted to one local machine? Is there any way I could synchronize it with a remote server?

Some time ago, driven by the same need to annotate Web pages that ScrapBook seeks to address, I experimented with a Zope-based annotation server project, and the annotation features of the W3C’s experimental multipurpose Web client, Amaya. But the way the annotation server worked is that it would save the "coordinates" of your annotations for a particular page in a database (local or on a trusted server), while referencing the original URL of the document. The problem was that many of the items that I wanted to annotate were items from semi-free publications like the New York Times or the Economist, where I could never be sure for how long I would have access to the original document.

Additionally, if these publications were to change their site design, the "coordinates" of my annotations could easily be lost.

I toyed with the idea of writing some sort of product to accompany the annotation server that would make a snapshot of the page so that perpetual access would be ensured.

But currently I live a somewhat less-than-wired existence, carrying my laptop from place to place in Argentina, where my access to the Internet in general, let alone broadband coverage, is somewhat limited. For working with my collected documents, it’s ideal to have them available offline.

So for now, ScrapBook fits the bill far better than an annotation server does.

But it seems that there’s an opportunity here for extending this excellent product.