This was the page that made me finally grasp why Lisp and s-expressions are relevant. After reading this about a year ago, I ended up picking up a Clojure book and now I'm enamored with s-expressions. I feel late to the party.
The last time I tried to learn Lisp (prior to reading this) was in like 2003/4 from some CLISP tutorial that scared me away from having nothing but math examples and zero explanation of the philosophy or design. It felt like an esoteric and purely academic language, and I brushed it off as not worth my time. (I was never really put off by the parens, and now I love them. Never understood that aspect of Lisp discourse. Once you learn ParEdit[1] you don't want to go back.)
Now I link this page whenever I want to explain to someone why there are "all these parens". It's dated, though, and associating s-expressions with JSON and JavaScript would probably be a more contemporary approach. Maybe I should rewrite this article for a modern developer audience.
There's a "lisp parens lol" thing among programmers that's so intellectually lazy. I once interviewed someone who I noticed had explicitly mentioned Lisp on their resume, and when as an aside I asked what their thoughts on Lisp development were, all they could muster was "Oh god the parens hyuck hyuck".
Other languages tend to get a free pass, but I take every chance I get to whine about JavaScript's three kinds of parens that I have to keep track of manually, and the convention of putting nearly every closing paren on its own line, wasting kilometers of space.
Yeah that screen space, truly a limited resource. You should cram everything together and don't forget to be extra clever in the middle of the dense unreadable forest of sigils as well. It was hard to write, it should be hard to read.
While I agree that in programming we should break our code in logical paragraphs, just like in prose, such conventions (placing closing brackets on their own line) are detrimental for that purpose. For me this doesn't have anything to do with wasting vertical space, as it has to do with aesthetics.
I'm not really speaking for LISP, but in my projects what really bothers me are people that don't have common sense when indenting / formatting their code and so they start using IDE or build plugins, using common conventions and formatters that will never be able to understand the meaning of the code being formatted, making a total mess of other people' carefully crafted code. And we go round and round on this.
I don't know that it is intellectually lazy. It makes a lot of sense.
If a lisp dialect had a large, vibrant ecosystem on the scale of Python or Javascript, then a lot of people would see the parens, see the advantage of having them, work their way through it and eventually get used to them and even embrace them because they see a real payoff.
That's not the case with lisp dialects. The major payoff with lisp dialects is macros. But for most people, it isn't obvious why they are important. They are something you have to use to really get.
So the intellectually reasonable position for most people is to either ignore lisp or laugh at it. Meanwhile, you see endless articles like this posted over the years (not knocking the article itself), trying to explain lisp in a kind of reverence and terminology usually associated with religious texts. "Hey, brother, you just need to see the light".
I've been doing quite a bit of Clojure coding lately and I love it. I just can't see a lisp ever catching on. Actually, the project I'm working on is a lisp that does not look like a lisp but compiles down to Clojure. The parens aren't really necessary for lisp's best feature, as seen by Elixir. Here's a bit of my language:
To "ignore or laugh at" a whole family of important and interesting languages because they aren't as popular as Python or JavaScript is exactly laziness, or at best pragmatic, but has nothing to do with intellectual reasonableness.
I'm not saying being "pragmatic" in this way is "bad" or "wrong," and since this is pretty much a total derail, I'll stop engaging in this thread. But, like, this is the actual thing you said that I was responding to:
"the intellectually reasonable position for most people is to either ignore lisp or laugh at it"
I said that such a position has nothing to do with "intellectual reasonableness" because it's barely an intellectual position at all. There is no propositional content; there is very little thought involved at all; it's a default "meh" position that is more accurately labelled "laziness."
One should not sneer or laugh at languages like Perl, COBOL, or similar, simply because there are others we like better, or are more popular.
I used to be a HUGE Perl fan, and discounted Python for years. I figured that I already knew Java, and since Python and Perl were effectively similar, it was a waste of my time to study both. I now regret that. As formative as that was for my growth as a programmer, I've loved coding in Python more than almost any other language -- and I delayed that discovery by a decade because I wrote it off as "effectively the same as" Perl.
Sure, it was pragmatic to focus on one language that I was learning and using. But the reasons I did so were flawed, IMO.
the parens seems pretty necessary for that expression, I can't even tell what it's doing...
it's joining the vector 1,2,3 with a space to make "1 2 3 ", right? so what is that count doing there? I'm not saying that lists are the end all be all of syntax (clojure has vectors for a lot of things specifically to reduce the parens), but you need something there to make it clearer
I get what you are saying, but when you are talking about programming languages as opposed to math or another domain, more people are familiar or comfortable with the former, hence the reason for numerous posts like the original having to explain why lisp's syntax is a good thing.
The mainstream may realize how syntax doesn't really matter. The FP push helps a lot already (map,filter,streamfusion instead of crazy for/while statements).
Why do you call for and while statements "crazy"? Because it's possible to write them wrong? Because you think it's crazy to have to write them at all? Why?
too easy languages to conflate a lot of logic into large side effectful statements. FP forces you to separate layers, and deforestation (when available) helps reclaiming space/time costs.
Alright, that was my own misunderstanding of loop invariants and scope showing. That gave opportunity for traumas when I started reading imperative code.
I think most people have an irrational dislike of s-expressions, but that at the same time, s-expressions are not as good as their proponents think they are. There's a certain aesthetic "purity" to them that's seductive, but from a practical standpoint, it is quite reasonable to adapt syntax to common patterns. The issue with mainstream languages isn't that they do that, it's that they do it too much.
But there are syntactic schemes besides s-expressions that allow for good flexibility. For instance, in my programming language Earl Grey[1], I use this very simple sugar: `a b: c` <=> `a(b, c)`. So you can use this todo syntax if you want, and it will just work:
todo "housework":
item priority(high): "Clean the house."
item priority(medium): "Wash the dishes."
item priority(medium): "Buy more soap."
Interestingly this has one advantage over s-expressions: whereas Lisp editors usually need to specify a special highlighting/indent policy for keywords like `let`, here editors can just highlight/indent `a b: c` indiscriminately.
I also think a limited number of infix operators are worth defining in most languages, and I don't mean arithmetic operators (although I'm okay with them). I mean ubiquitous or easily repurposed operators like assignment/declaration, lambda, pairing, or commas. In my view, Lisp-like languages hold to standards of purity that are unreasonable (don't get me wrong, they are quite usable, just not optimal according to any useful metric).
I don't think that's much clearer than a lisp equivalent:
(todo "housework"
(item (priority high) "Clean the house.")
(item (priority medium) "Wash the dishes.")
(item (priority medium) "Buy more soap."))
Yours looks nicer, but if I wanted to operate on your todo there, it's not obvious how just by looking.
With the lisp, it's obvious: There's a list of 5 elements. The last three are 'items', each of which is a list of 3 elements.
The benefit of s-expressions isn't just that they're flexible in what you can write with them, but that they're easy to manipulate programmatically, and their structure is immediately obvious.
S-expressions are not obvious "just by looking" either. Don't forget you had to learn what they were and how they worked. You know that `(a b c)` is a list of three elements, that the first element can be extracted with the `car` function, and the rest with `cdr`, and so on. You also know that `(a b)` is shorthand for `(a . (b . nil))`, which helps a lot in knowing how to manipulate it. The fact that you are already familiar with Lisp taints your perception.
Not to mention Lisps, especially the new ones, throw obviousness out the window with merry abandon. Let's look at Clojure. You think it's obvious how to operate on Clojure? Here: `(let [a 1, b 2] (+ a b))`. What is that? I've mostly coded in Scheme, I don't know what square brackets are supposed to do. Vectors? How do I manipulate them and what are they doing in my let? And what does the comma do? Oh, nothing? Then why is it there? And if the comma means nothing are you telling me that if I want to list the names that are being defined by this form, I need to extract the even-indexed elements of a vector? Sure I can do that, but don't tell me you care about ease of manipulation.
Anyway, sorry for the rant ;)
I think the point I really want to make here is that sure, s-expressions are quite simple, but if I can explain how s-expressions work in a minute, and how my expressions work in five minutes, this isn't really a matter of ease of manipulation, this is a matter of laziness. If a language's source-to-AST rules can be fully explained in one or two paragraphs or a small table, and can be easily remembered, then that's all that matters. I think that `a b: c` <=> `a(b, c)` is trivial enough that it does not have any meaningful impact on ease of manipulation.
I would say there are too many distracting "tags" in that which may be unnecessary ("XML disease"). For instance, look how you just indicated with a tag that the elements of a list are "items". In Lisp!
(todo "name" (item ...) (item ...) (item ...)).
Basically the symbol todo is enough of a clue about the structure, and the pure syntax can determine most of the rest, with possibly a modicum of keywords here and there.
(todo housework ;; symbols for naming!
(:high "Clean the house")
(:medium "Wash the dishes")
(:medium "Buy more soap"))
For reference, an S-expression based syntax that could be used for this example:
(todo "housework"
(item :priority high "Clean the house.")
(item :priority medium "Wash the dishes.")
(item :priority medium "Buy more soap"))
Your syntax certainly has advantages, but one can easily prefer S-expressions in this case without appealing to vague values like "aesthetic purity," nor are there any blatantly obvious "practical" downsides to using a more fully parenthesized syntax.
One of the aspects of the data-is-code and code-is-data duality that always bugged me surrounds security. Ever since we started embedded macro languages into documents, there have been classes of security problems around them. Even before the web there were macro viruses in abundance, and running spreadsheets with macros embedded in them is still a dangerous proposition today.
How does Lisp deal with the modern Writable Xor Executable philosophy? Or does it? Can a lispy program that exposes its lispyness to its users be secure (emacs, I'm looking at you)?
Generally, you just don't expose your Lisp evaluator to untrusted users. Lisp macros are a powerful language feature, and to allow your user to evaluate arbitrary code is exactly as dangerous as with any other system.
Emacs is not meant to be secure against user scripts. You could even say that's an integral part of its whole philosophy: the user can do whatever she likes, and her scripts too.
It does have some rudimentary safety. For example, color themes by default can't run Lisp code. They are written as Lisp data structures, but they aren't run through the evaluator unless the user explicitly allows it.
- Expose the entire language and then there's not much you can do to prevent something harmful.
- Expose a subset of the language perhaps through parsing for harmful patterns or by somehow making potentially harmful functions unreachable.
- Expose a more restricted language (scripting like c/lua) that you control and know its full capabilities.
The only difference I can think of is that it's easier to write flexible and dynamically modifiable software in lisp than in most mainstream languages.
All access to anything important (network, files, other commands, etc) have to happen through opaque handles called capabilities. Think of them as objects that you can't look for.
When you call code, you pass it a set of capabilities that are available. That is all it can ever access.
But the whole language and environment needs to be defined from the bottom up to enable this.
The end result has been years of (practical) research going into an actually safe, actually sandboxed, actually secure cross-platform platform: https://developer.chrome.com/native-client
It's up to the programmer to decide what code/data is trusted and untrusted. So you wouldn't `eval` any S-expressions that could be submitted by a user. It's really no different in terms of security implications than any other dynamic language like Ruby, Python, Perl, PHP, Javascript...
AFAIUI you write restricted evaluators to deal with external code. You can dismiss or rewrite any form you want, avoid IO, system calls and such. I never looked into it so maybe it's possible to escape that, it's been used on an irc bot for #clojure, so it's not stupid; but there's always a possibility for crazy ideas like crafting memory patterns and trying to DDoS the evaluator in order for these to become code (complete speculation based on Return Oriented Programming).
Jonathan Rees's 1995 paper "A Security Kernel Based on the Lambda Calculus" was enlightening. Not only was it capability-based security in Scheme -- but capabilities are lambdas!
Don't use read or eval willy nilly.
If you have to use read, at the very least turn off read-eval. I give up on trying to get an asterisk on each side of that variable. No form of escaping seems viable.
I started trying to learn FP this last year. I started trying to understand Clojure and then watched the SCIP lectures and worked through the problems in the book with the class. At this point, I think I _get_ LISP.
What I don't get is how I build an application using LISP. From the article, the todo list wasn't really a list, it was a todo function call that took tasks as arguments unless there was a missing quote. That seems to be a point missed by the author. With the macro, he was able to format the tasks themselves, but after applying the macro you'd still be left with a todo function and done arguments. I don't see where that gets called.
LISP seems to work best as an evolving environment. I can see where a LISP machine could be powerful for defining your own DSL as you try to parse through some data. I also saw the power in how you abstract the problem, really taking advantage of code reuse in a powerful way. I don't see how you write an application.
Is there some resource or there that can help me bridge this gap? I feel like this is like trying to use Mathematica to write a program. Mathematica is great for calculating as LISP seems great for processing data, but I'm stuck in seeing how I can utilize it for anything beyond a contained process. At least with Clojure or F#, I could still use Java or C# for the entry point and call into an FP block of code for calculations and processing. How would I do something with Haskell or Scheme?
Eventually, some of your code will make a call to IO: reading from or writing to disk, responding to a web query, updating a DB, drawing something on the screen, etc, etc.
For example, you could put up a web server written entirely in Scheme or some other dialect. This would process and respond to queries entirely in Scheme, including some sort of engine for generating and serving html. Hacker News itself is an example of this: it's written in a lisp dialect called Arc.
Incidentally, Haskell is functional, but it isn't a lisp.
I've read and heard the commonly echoed sentiments about Macros being used to build a language for your application. I'm a bit confused in how this is philosophically different than just writing domain specific classes and functions? Can someone point to an example where the syntax allowed by Ruby or JS produced obfuscated code and a LISP macro produced something really simple comparatively?
It's the difference between introducing new nouns and verbs, and introducing new grammar.
With new classes, I add new nouns (Circle, for example). With functions, I add new verbs (move, for example). And I can add adjectives and adverbs (attributes and arguments). This lets me build a "new language" (for handling shapes, in my example).
But macros let you go much further. They let you change the syntax, so that instead of saying "shape.move(newCoords)", you can say "move shape newCoords", for example, even though that was not legal syntax in the language you're writing in. That's where it really becomes a new language - not just new words.
But if you've got "good enough" syntax in the language you're writing in, you might not see that as particularly useful. That is, the less impoverished your out-of-the-box implementation language syntax is, the less you need macros. (Most of the examples say "it's awkward to write this; I can write a macro to make it easier". Well, that's great and all, but awkwardness of writing something that I don't write very often in the first place does not make me want to run out and adopt a macro language.)
Statically compiled regular expressions. Efficient pattern matching implementations. Generating a boilerplate for an ORM. Embedding languages with arbitrarily complex syntax. There are hundreds of things that you simply cannot do with functions but are trivial with macros.
With macros being so easy to write, and so idiomatic to the way the language is used, isn't it very difficult to understand a Lisp codebase that you are unfamiliar with?
While I like Lisp a lot for my own projects (well Racket in Dr. Racket to be specific), I don't feel very comfortable trying to understand other people's code.
Other languages like Go, for example, seem a lot more tractable when you have to grok a large foreign codebase.
I very often find it difficult to understand code in non-Lisp languages, too. For example, you may think you know Ruby, but that has nothing to do with understanding Rails or its myriad metalinguistic tricks.
Abstractions are often difficult to understand, and require explanations. Introducing a new abstraction, regardless of implementation language, puts the burden on you to define it with clarity, to document its purpose and meaning, and so on.
Lisp macros let you define abstractions in a certain way that's quite powerful. Fortunately, Lisp systems also tend to have good provisions for online documentation, and make available macro-expansion facilities that immediately explain exactly what a given macro is doing.
In general, the more you can package up into single abstractions, the more someone will have to learn to thoroughly understand your code base. If those abstractions fit the problem domain well, hopefully they will ease the overall cognitive load of working in the system, even somewhat early on, but there will always be some ramp-up time. I'm not at all sure it's a big problem, just something you need to be aware of.
If macros are used properly, i.e., for implementing nice, multi-staged, composable eDSLs, instead of just another anaphoric if or similarly useless "extension", then a Lisp codebase is much easier to read than anything that does not allow implementing macro-based eDSLs.
> Lisp is a way to escape mediocrity and to get ahead of the pack. Learning Lisp means you can get a better job today, because you can impress any reasonably intelligent interviewer with fresh insight into most aspects of software engineering. It also means you're likely to get fired tomorrow because everyone is tired of you constantly mentioning how much better the company could be doing if only its software was written in Lisp. Is it worth the effort? Everyone who has ever learned Lisp says yes. The choice, of course, remains yours.
The last time I tried to learn Lisp (prior to reading this) was in like 2003/4 from some CLISP tutorial that scared me away from having nothing but math examples and zero explanation of the philosophy or design. It felt like an esoteric and purely academic language, and I brushed it off as not worth my time. (I was never really put off by the parens, and now I love them. Never understood that aspect of Lisp discourse. Once you learn ParEdit[1] you don't want to go back.)
Now I link this page whenever I want to explain to someone why there are "all these parens". It's dated, though, and associating s-expressions with JSON and JavaScript would probably be a more contemporary approach. Maybe I should rewrite this article for a modern developer audience.
[1]: http://danmidwood.com/content/2014/11/21/animated-paredit.ht... "GIFs of ParEdit in action"