Comments on: List Processing and the Efficiency of CONS http://stuartsierra.com/2006/06/17/list-processing-and-the-efficiency-of-cons From programming to everything else Tue, 07 Oct 2008 04:16:57 +0000 http://wordpress.org/?v=2.6 By: Ruslan Abdulkhalikov http://stuartsierra.com/2006/06/17/list-processing-and-the-efficiency-of-cons#comment-52 Ruslan Abdulkhalikov Thu, 22 Jun 2006 13:43:49 +0000 http://stuartsierra.com/?p=30#comment-52 Recursion in Lisp should be used with caution. Most of the Lisp implementation do not support tail recursion by default so you more likely end up with stack overflow on long lists. loop macro and other iteration macros should be used in Lisp. Recursion in Lisp should be used with caution. Most of the Lisp implementation do not support tail recursion by default so you more likely end up with stack overflow on long lists. loop macro and other iteration macros should be used in Lisp.

]]> By: Ben http://stuartsierra.com/2006/06/17/list-processing-and-the-efficiency-of-cons#comment-46 Ben Thu, 22 Jun 2006 00:53:41 +0000 http://stuartsierra.com/?p=30#comment-46 As in any programming language, you should "just write" the code first, then profile. Often (not always, but often) you'll find some function that's consing (not necessarily literally `cons') some huge amount, possibly non-obviously, and you can get a dramatic speedup just by changing that one spot. The advice isn't unique to Lisp. The difference is that in Lisp this particular issue happens to be one of the easier ways to find and get improvements. And there's more awareness (with a catchy name) for this issue in Lisp. As in any programming language, you should “just write” the code first, then profile. Often (not always, but often) you’ll find some function that’s consing (not necessarily literally `cons’) some huge amount, possibly non-obviously, and you can get a dramatic speedup just by changing that one spot.

The advice isn’t unique to Lisp. The difference is that in Lisp this particular issue happens to be one of the easier ways to find and get improvements.
And there’s more awareness (with a catchy name) for this issue in Lisp.

]]> By: guy http://stuartsierra.com/2006/06/17/list-processing-and-the-efficiency-of-cons#comment-45 guy Tue, 20 Jun 2006 22:10:48 +0000 http://stuartsierra.com/?p=30#comment-45 Right - "shalt not cons <i>in vain</i>." That's hardly a condemnation of CONS (is it ever, in fact, good to do something in vain?). CONS is not "computationally inefficient." At least, I don't think it is, since it's not clear what that phrase is supposed to mean. CONS is a totally efficient function, but calling it <b>unnecessarily</b> will lead to a more painful GC. But this is only a problem when it's a problem. Avoiding CONS "just because" is silly. I find simple recursive patterns like PROCESS repulsive. Every time I look at a function like PROCESS, I have to walk through it in my head and then realize that, "Oh, it's just an instance of Pattern X. I've seen this a million times." I would try to use LOOP here, so that it's immediately apparent what you're doing. Right - “shalt not cons in vain.” That’s hardly a condemnation of CONS (is it ever, in fact, good to do something in vain?).

CONS is not “computationally inefficient.” At least, I don’t think it is, since it’s not clear what that phrase is supposed to mean. CONS is a totally efficient function, but calling it unnecessarily will lead to a more painful GC. But this is only a problem when it’s a problem. Avoiding CONS “just because” is silly.

I find simple recursive patterns like PROCESS repulsive. Every time I look at a function like PROCESS, I have to walk through it in my head and then realize that, “Oh, it’s just an instance of Pattern X. I’ve seen this a million times.” I would try to use LOOP here, so that it’s immediately apparent what you’re doing.

]]> By: Jon Philpott http://stuartsierra.com/2006/06/17/list-processing-and-the-efficiency-of-cons#comment-38 Jon Philpott Mon, 19 Jun 2006 21:24:12 +0000 http://stuartsierra.com/?p=30#comment-38 I agree, sometimes it is better to write a recursive procedure instead of mapcar'ing if you need to do something a little more funky. I agree, sometimes it is better to write a recursive procedure instead of mapcar’ing if you need to do something a little more funky.

]]> By: Joe Marshall http://stuartsierra.com/2006/06/17/list-processing-and-the-efficiency-of-cons#comment-37 Joe Marshall Mon, 19 Jun 2006 16:17:29 +0000 http://stuartsierra.com/?p=30#comment-37 These days, with modern generational garbage collectors, you shouldn't have to worry about consing. I rarely give it a second thought. These days, with modern generational garbage collectors, you shouldn’t have to worry about consing. I rarely give it a second thought.

]]> By: stuart http://stuartsierra.com/2006/06/17/list-processing-and-the-efficiency-of-cons#comment-36 stuart Mon, 19 Jun 2006 14:53:59 +0000 http://stuartsierra.com/?p=30#comment-36 Christophe Rhodes Says: <em>Do articles and books really say “CONS is expensive”, let alone “avoid [CONS] where possible”?</em> I can only find one specific <a href="http://www.cs.utexas.edu/~novak/lispeff.html" rel="nofollow">example</a> right now, which gives the "eleventh commandment" for Lisp programmers as "Thou shalt not cons in vain." I think it's an impression I got from newsgroup and mailing list discussions. One frequently hears "consing is expensive," which can lead the less-experienced Lisp programmer (e.g. me) to believe "consing is bad." Christophe Rhodes Says: Do articles and books really say “CONS is expensive”, let alone “avoid [CONS] where possible”?

I can only find one specific example right now, which gives the “eleventh commandment” for Lisp programmers as “Thou shalt not cons in vain.” I think it’s an impression I got from newsgroup and mailing list discussions. One frequently hears “consing is expensive,” which can lead the less-experienced Lisp programmer (e.g. me) to believe “consing is bad.”

]]> By: Janis Dzerins http://stuartsierra.com/2006/06/17/list-processing-and-the-efficiency-of-cons#comment-35 Janis Dzerins Mon, 19 Jun 2006 14:37:41 +0000 http://stuartsierra.com/?p=30#comment-35 Another thing is that in Lisp, to "cons" means to allocate memory, not necessarily by means of calling the CONS function. Actually, calling other functions, like MAKE-ARRAY will usually "cons" a lot more. And my advice: do things the functional way and create new lists -- it will save you a lot of debugging time. Another thing is that in Lisp, to “cons” means to allocate memory, not necessarily by means of calling the CONS function. Actually, calling other functions, like MAKE-ARRAY will usually “cons” a lot more.

And my advice: do things the functional way and create new lists — it will save you a lot of debugging time.

]]> By: Christophe Rhodes http://stuartsierra.com/2006/06/17/list-processing-and-the-efficiency-of-cons#comment-34 Christophe Rhodes Mon, 19 Jun 2006 14:36:17 +0000 http://stuartsierra.com/?p=30#comment-34 Do articles and books really say "CONS is expensive", let alone "avoid [CONS] where possible"? I can believe that they say that "consing" is potentially undesireable; the Lisp community uses the verb "cons" to mean heap-allocation, and typical lisp implementations with their garbage collectors will suffer latency at some later point when enough heap allocation has been done to trigger a collection; for time-critical or low-latency code, these collections can cause trouble. As you note, there's no need to optimize code beyond its speed requirements, and certainly no need to do so at the expense of clarity or correctness. But I don't think that anyone is actually advocating that, and it's a little disingenuous to suggest that they are. Do articles and books really say “CONS is expensive”, let alone “avoid [CONS] where possible”? I can believe that they say that “consing” is potentially undesireable; the Lisp community uses the verb “cons” to mean heap-allocation, and typical lisp implementations with their garbage collectors will suffer latency at some later point when enough heap allocation has been done to trigger a collection; for time-critical or low-latency code, these collections can cause trouble.

As you note, there’s no need to optimize code beyond its speed requirements, and certainly no need to do so at the expense of clarity or correctness. But I don’t think that anyone is actually advocating that, and it’s a little disingenuous to suggest that they are.

]]> By: Faré http://stuartsierra.com/2006/06/17/list-processing-and-the-efficiency-of-cons#comment-33 Faré Mon, 19 Jun 2006 14:32:33 +0000 http://stuartsierra.com/?p=30#comment-33 In Exscribe, I have a macro to recurse through the user-constructed tree of text and tags, side-effecting the structure as I go. It's very effective to rewrite the Scribe-like tree into HTML. In Exscribe, I have a macro to recurse through the user-constructed tree of text and tags, side-effecting the structure as I go. It’s very effective to rewrite the Scribe-like tree into HTML.

]]> By: Zach Beane http://stuartsierra.com/2006/06/17/list-processing-and-the-efficiency-of-cons#comment-32 Zach Beane Mon, 19 Jun 2006 14:05:05 +0000 http://stuartsierra.com/?p=30#comment-32 Lisp doesn't use recursion, people do. Part of the trick of programming in any language is learning what is available and when it's appropriate to employ it. Fortunately, CL gives you a lot of options. Lisp doesn’t use recursion, people do. Part of the trick of programming in any language is learning what is available and when it’s appropriate to employ it. Fortunately, CL gives you a lot of options.

]]>