What makes an on-line community?  In the past two weeks I have received announcements of three new “communities” all interested in using open-source software to retrieve, share, and analyze data from or about governments.  Most of these announcements say the same thing: “A lot people seem to be working on this, but they aren’t talking to each other.”  Each group has a slightly different slant, but in my mind I lump them all under the heading “Semantic Government,” i.e. building the semantic web for government data.

I started casting out a few search queries, and quickly compiled a list of eight different mailing lists and/or wikis devoted to this subject.  That doesn’t include for-profits like Justia.com or larger non-profits like the Sunlight Foundation.

This is a problem.  Not only do I have to subscribe to half a dozen mailing lists to keep abreast of what others are doing, I also have to cross-post to several lists when I want to announce something myself.  So far, nothing I have posted to these lists has garnered as much response as private emails sent directly to people whom I know are subscribed to the lists.

Perhaps the very idea of a “web-based community” has become a victim of its own success.  Back in the olden days, when I was still learning how to type, creating an on-line community was hard.  You had to wrangle with BBS software, mailing list managers, or content management systems.  It took dedicated individuals willing to invest considerable time and money.  Now?  Just go to Google / Yahoo / Facebook / whatever flavor-of-the-month service, type in the name of your group and presto, you’re a “community.”

The problem is that it’s now easier to start a group than to join one.  Every project wants to be the center of its own community, but what most projects actually get is a lonely soapbox in the wilderness from which to cry, “Announcing version 0.x…”

I’m equally guilty in this trend, having founded one of the sites I referred to above (LawCommons) and built a wiki for another (IGOTF).  Once you’ve started a site it’s easier to leave it there than to formally announce “I am shutting down X and throwing my lot in with Y.”  It’s also a hedge against the (very likely) possibility that group Y won’t be around in a year.  But I worry that a broad movement (Semantic Government) fragmented into so many tiny sub-groups will never gather enough momentum to succeed.  The very thing we all want — to share information better — is lost through the scattered efforts to achieve it.

I’m sure I’m not the first to suggest this, but here goes.

Ever since somebody first thought of applying the Model-View-Controller paradigm to the web, we’ve had this:

The View is a conflation of HTML and JavaScript.  JavaScript is an afterthought, a gimmick to make pages “dynamic.”  All the real action is in the Controller, which talks to the database, processes the internal application logic, and renders templates before sending complete pages back to the client.

But what if we implement the Controller entirely in JavaScript?

Now we can put the Controller on the client, and build a RESTful HTTP interface to communicate with the database.

Obviously there are many issues to consider.  First and foremost is making sure that rogue clients cannot do anything to the database they’re not supposed to.  But that’s a manageable problem — Amazon S3 is a good example.  Apps that run entirely in the client can even be made more secure than their server-based counterparts, because encryption can be implemented entirely in the client, so that the server never sees the unencrypted data.  (Clipperz, an password-storage service, calls this a zero-knowledge web app.)

There are some interesting possibilities.  For example, the entire application, including the current state of the model, can be downloaded as a single web page for off-line use.  (Clipperz supports this.)  Also, the same application could connect to multiple data sources.  And as with any RESTful architecture, back-end scaling is relatively easy.

Update July 10, 2008: I’m always amazed when one of my posts show up on Reddit.  Maybe it was the diagrams.  In any case, thanks to everyone who sent in comments.  A couple responses:

1. Yes, in a sense I’ve described Ajax.  But most Ajax-related code around the web these days is still in the “dynamic view” mode rather than the “client-side controller” mode.
2. I like Sun’s MVC diagram in which the View takes an active role in rendering the model rather than being just a template.  It’s actually quite similar to what I’m suggesting here.
3. Some MVC frameworks, such as Ruby on Rails, insist that logic in the View is bad, but then they include all these Ajax view helpers, so it’s a bit of a mixed message.
4. I’m not insisting that all business logic be implemented client-side.  Rather, I’m assuming some kind of “smart” database, with a RESTful front-end, that’s capable of containing business logic.  Back in the day, these were SQL stored procedures.  Now it’s probably something like CouchDB.
5. Yes, this design is bad for search engines, bookmarks, and deep linking.  But there are plenty of cases where those don’t matter.  Look at Google Mail, for example.  It basically follows the design I’ve laid out here: the entire app is one HTML page (or a very few pages) with behavior implemented in client-side JavaScript.

Update August 7, 2008: This is an example of the code-on-demand style described in Roy Fielding’s REST thesis.  Link from Joe Gregorio.

The web is a strange beast.  Server logs reveal just how strange.  Someone’s crawling AltLaw.org, sending an HTTP Referrer of “http://www.nero.com/enu/downloads-nero8-trial.php” with a User-Agent identified as “MSIE 5.00; Windows 98″.  What the heck?

A cool new site, with the best possible slogan, “All the code that’s fit to printf().” Nice to see a giant media company getting into this.

After the New York Times’ premature announcement of AltLaw.org — I don’t mind, publicity is good — I discovered the downside of getting linked, even indirectly, from a major site. I woke this morning to find 632 bounced spam messages in my inbox sent from spoofed “@stuartsierra.com” addresses. Gotta update my catch-all email settings.

Wowsers. I just spent two nail-biting, hair-pulling days getting Ruby on Rails running on a new dedicated server. What’s the deal here? I spent the first six hours trying to get Capistrano to work with darcs. Then I gave up on Capistrano.

I didn’t know anything about Mongrel, nginx, Lighttpd, or any of that stuff, so I decided to try the simplest setup I could think of: Apache2 with FastCGI. I’d done this on a shared host before, so at least I had some vague idea of what to expect. Some things I learned:

• Do not use Ubuntu’s rubygems package; build from source. The Ubuntu package has a weird configuration that stores gems under /var, where nothing can find them.
• Do not use Ubuntu’s fcgi-ruby package. Install the gem (see below).
• Make sure your web-data user (or whatever user Apache runs as) has write privileges on RAILS_ROOT/log, RAILS_ROOT/tmp, RAILS_ROOT/index (if you’re using Ferret), and RAILS_ROOT/public (if you’re using page caching).
• Make sure there are no leftover files owned by other users in any of those aforementioned directories.
• Don’t rely on Apache to provide Rails with environment variables like RAILS_ENV and PATH. Set them in environment.rb.

My conclusion, after I finally got the app running: Rails has major work to do when it comes to startup error reporting. When dispatch.fcgi dies repeatedly without neither output nor log messages, there’s something wrong. Seriously, how hard could it be to dump a plain-text HTTP response with a stack trace? That way I’d at least know why it won’t start, instead of staring at “Rails application failed to start properly” for hours on end.

For the record, here was the installation recipe I came up with. This is on Ubuntu Feisty:

MySQL:
sudo aptitude install mysql-server mysql-client

Memcached:
sudo aptitude install memcached

Basic ruby essentials:
sudo aptitude install ruby libruby1.8-extras irb rdoc ri

Extra ruby libraries:
sudo aptitude install libreadline-ruby libopenssl-ruby libmysql-ruby

Essentials for building native ruby extensions:
sudo aptitude install build-essential ruby1.8-dev libfcgi-dev

The latest version of rubygems (NOT the Ubuntu package):

wget http://rubyforge.org/frs/download.php/20989/rubygems-0.9.4.tgz
tar -xzf rubygems-0.9.4.tgz
cd rubygems-0.9.4
sudo ruby setup.rb

Gems:
sudo gem install --include-dependencies --no-rdoc --no-ri rake rails acts_as_ferret acts_as_taggable asciify BlueCloth builder cached_model chronic feedtools ferret hpricot memcache-client mongrel RedCloth fcgi

Continuing on the theme of HTML’s flaws, consider the humble BLOCKQUOTE. While long used simply to indent text, it has a recognizable semantic meaning: a long quotation from another work.

A block quote may contain multiple paragraphs, so BLOCKQUOTE logically enough is a block-level element that contains other block-level elements like P.

But suppose I want to number the paragraphs in my document? I don’t want to count the P elements within the BLOCKQUOTE, because they’re part of a larger paragraph that contains the quote. Furthermore, I don’t want to count the P immediately after the BLOCKQUOTE either, because it is actually just the continuation of the paragraph before the quote. Except in those rare cases when a paragraph ends with a block quote.

So really, what I’d like to do is this:

<p>
  This is the beginning of my body paragraph.
  <blockquote>
    <p>First paragraph of the quotation.</p>
    <p>Second paragraph of the quotation.</p>
  </blockquote>
  Here the body paragraph continues.
</p>

But I can’t, because BLOCKQUOTE can’t appear inside a P. (Looser forms of HTML will allow it, but only by inserting an assumed </P> before the BLOCKQUOTE.)

To be fair, word processors don’t handle this correctly either. Even LaTeX requires you to flag the continuing paragraph with \noindent. The “correct” definition of BLOCKQUOTE, in my opinion, is awkward: BLOCKQUOTE itself should be an in-line element, but it should contain block-level elements.

If HTML had a FOOTNOTE tag, I would want it to work the same way.

Leonard’s comment on my post about XML and footnotes got me thinking about representing footnotes in HTML. Not the visual presentation — there are lots of options for that, using CSS, JavaScript, and internal links — but the semantic one. In other words, using nothing but semantically-meaningful HTML tags (DIV, SPAN, P, A), how should one mark up a footnote in a document?

I believe it’s a failing of HTML that it does not include a footnote tag. I’ve heard that earlier drafts did include it, but it was dropped for lack of interest. Clearly, early HTML users — math and comp. sci. types — weren’t as fond of footnotes as those in the humanities. Lawyers, for instance, are fanatical about footnotes. There are entire books on proper citation for legal documents.

So if you’re building, as I am now, a web-based database for legal documents, what do you do? An article called Scholarship on the Web: Managing & Presenting footnotes and Endnotes lists several possibilities. Basically, if you don’t want to use internal links, you can put the entire footnote inside a SPAN and use CSS and/or JavaScript to display it as a pop-up or a side note. This is great for on-screen reading, but there’s an edge case: multi-paragraph footnotes. Lawyers love footnotes so much they can go on for many paragraphs, even including block quotations.

It’s not valid XHTML to put a block-level element like P inside of a SPAN. So put the footnote in a DIV instead of a SPAN, I thought. No dice: you can’t put a DIV inside of a P. So in the end I have to put the footnote in a DIV entirely outside P that referenced it. I need some way to connect the footnote with the place in the text where it’s referenced, so at this point I might as well go back to internal links.

Perhaps I’m making this more complicated than it really is.

Working alongside legal academics, I hear a lot about a web site called SSRN, the Social Science Research Network. It’s a free service that hosts thousands of academic papers on law, economics, and business. It also tracks the number of times each paper is downloaded and publishes regular reports on the most-downloaded papers and authors.

A couple of years ago, some institutions began using SSRN download counts to evaluate faculty job candidates. Aspiring academics got wind of this and started asking all their friends to download their papers. SSRN responded by requiring users to register to download papers. Irritated professors then boycotted SSRN.

As some sensible people pointed out, the rankings aren’t that meaningful anyway. The real loser is the scholarship SSRN was meant to encourage.

I find this somewhat funny, because these professors have had to learn what web advertisers discovered in the nineties: hit counts are virtually meaningless.

Sometimes I feel like every time I come up with a good idea, I read about it somewhere else a week later. It least it’s nice to have some indication I’m not a raving lunatic. This time, A List Apart suggests Paper Prototyping, just what I was talking about a week ago.