The first draft of Project Posner was written in Common Lisp. I thought it would be fun to see how Common Lisp fared as a language for doing heavy text processing with a web front end. It worked well, and I’m convinced it made the process easier than it would have been with any other language. But everything I’ve done with it up till now is off-line. I used Lisp to statically generate the site on my desk, then uploaded the HTML pages to the server. Search is handled by ht://Dig, an old-school CGI app written in C.
I’d love to continue to develop Project Posner in Lisp, especially since I’m currently the only programmer working on it. But to add any more features I need server-side programming. I find myself wondering … do I dare try to use Lisp? First off, I’d have to get a new web host, probably a virtual server, since no shared-host server offers Lisp pre-installed. That would cost more. Then I’d have to set up and maintain the OS on the server, which I’d frankly rather not be bothered to do. Then there’s the multiple headache of getting Apache, mod_Lisp, PostgreSQL, and a CL implementation all running and talking to one another. Then, and only then, can I start work on the application itself. And then I don’t have much pre-written code to draw on. Sure, HTML and JavaScript generation is in the bag, but there aren’t any drop-in libraries for forums, guestbooks, user authentication, or any of that good stuff.
I could probably write that stuff myself in Lisp. But could I do it faster and better than the hundreds of other people who have already done it in Perl/PHP/Python/Ruby? I don’t think so. I’m not that good.
So there it is. Web application development is an evolving problem, but by and large a solved one. And it wasn’t solved in Lisp. When Paul Graham was creating Viaweb, no one else was even thinking of web applications, so he had to create his own tools. But the biggest recent poster child for Lisp on the Web, Reddit, gave up and switched to Python (to much gnashing of teeth in the Lisp world). It has nothing to do with the language itself. Lisp is still great. It’s all about the tools, the libraries, the “borrowablility” of other people’s code.
So I’ll continue using Lisp for off-line stuff, private projects and such. But for building Project Posner version 2.0, I’ll probably look elsewhere.
First, there are shared host services with lisp installed. See http://wiki.alu.org/Lisp-friendly_Web_Hosting I use zill.net (number five on that list), which uses a method I was going to mention below involving apache proxying. I probably should disclaim that I co-locate with zill.net and thus don’t actually use their lisp service. Now that I’m mentioning that service, I should see if I can get a guest lisp account to test out for better pushing on web blogs ;).
You could use PortableAserve, or some other web server written in Lisp, instead of using Apache and mod_lisp. For one thing, this would get you the benefit of a frame work in addition to just a web server (assuming that you like the framework). Another possible lisp web server is Hunchentoot.
Also, instead of having your lisp web server face the web directly, you can hide it behind Apache easily using the ProxyPass feature. By doing so you can offload things like images to what might be a more efficient platform for doing them, as well as still being able to make use of assorted libraries in other languages in Perl, PHP, Python, etc. Apache also has configurable caching so perhaps the lisp server wouldn’t always have to be connected to for operations like hitting the back button. Squid’s caching is more configurable. Many people use it for load management of web services.
On the stuff I run, I use some PHP stuff (via mod_php), some Perl (via CGI rather than mod_perl for security) stuff, and some Python CGI stuff. I also use the proxy service to talk to the Python web server for some stuff, and the proxy service to talk to the PLT Scheme web server for some stuff. Setting up all of it was mostly painless (open the tarball somewhere and add a few lines to httpd.conf to enable it). The python and scheme proxied stuff seems to be a lot faster than the mod_php stuff is.
I can’t swear that you won’t run into trouble trying such a combination on your own. Part of the reddit problem was that they were developing on Macs for FreeBSD. Developing on Linux for Linux, Solaris for Solaris, or OSX for OSX would have simplified things greatly. Also, using something more mainstream than FreeBSD for their main lisp server would likely have been more stable. They chose to change their language rather than their platform, so I guess we will never no if changing their problem might have fixed the problem. While I use myself as an example, I don’t do terribly high volume stuff since the code I write is for a small group of private users. I just would have to see you give up on lisp too quickly.
I would hate to give up on Lisp, too. I haven’t completely made up my mind, though, and I’ll certainly check out the Lisp-friendly web hosts.
Part of it will depend on where we decide to take Project Posner. Currently it’s about 2100 static documents with a primitive search feature. It might someday morph into something much larger, hundreds of thousands of documents with thousands of users.
My basic worry is that I need a unified web framework like Rails to help me integrate all the features. I want to make sure to provide a complete and polished service, not a grab-bag of little CGI forms. I’m curious to know: what has your experience been mixing different languages together on one site? Is it possible/practical to try to blend them together into a unified user experience?
[…] I continue to sweat (see previous entry) over the question of language choice for future iterations of Project Posner (and some as-yet-unnamed similar projects). Ruby on Rails is the obvious mainstream choice, mainstream at least compared to Lisp. But a part of me really wants to do it in Common Lisp, just to prove I can. […]