Show me the tutorial where I can build, _in C_ a bare bones working (tail recursion) implementation of the lambda calculus with type inference and dependent (algebraic?) types
Why would you want to do that?
Type inference and dependent types don't go well together (already System F doesn't have type-inference), although it depends on the details of what you mean by dependent types.
Tail recursion is a concept of compilation and does not show up in an interpreter.
Note also that C has higher-order functions (via function pointers). The only relevant difference from lambda-calculus is that C functions must be named, cannot be nested and must exist at compile time. But conceptually these restrictions make little difference to the understanding of what lambda does: it creates a function.
Finally writing something like that (to the extent that the requirements make sense) is gonna be really painful in C. If you understand the lambda-calculus well enough to know what dependent types are, the implementation language should not make much of a difference.
I think the easiest way to learn the lambda-calculus
program in a functional language. The second easiest way
is to read the theory, after all the lambda-calculus as a calculus is incredibly simple, and you can hand-wave away the problems of bound-variable renaming. A slow-paced textbook like Hankin's
"An Introduction to Lambda Calculi for Computer Scientists", and doing the exercises in the first few chapters should do the job.
I don't think the grandparent wants all of these features at once necessarily in a single tutorial, but an introduction to them in a language that they already know.
Also, the things you say that function pointers don't have are exactly the things that make lambdas interesting. Lambdas are anonymous and, crucially, can be nested. It's the very nesting that is what makes them difficult to get right, as they close over their containing scope. Variables mentioned in lambda bodies that are not parameters get their values from the definition scope, not the scope where they are used. In C function pointers, this issue doesn't even come up.
Indeed, you have put it more succinctly and cogently that I ever could.
I would like an introduction to these advanced topics in the language that I already now. For instance, for a Haskell module I was studying we used a teeny tiny interpreter that had been created just for the task of learning the basics of Haskell. This interpreter was written in Lua. It was small, and I could follow the logic of it. It lacked type inference though, and if I had seen a tutorial _in C_ about how to bolt on a Hindley Milner type inference engine to a lambda calc core I bet I could have made a stab at improving this toy project.
Even with Ruby now I bet I code Ruby as if it were a really dynamic C. I hardly ever use map() or collect(), for instance. I just don't have it in my mental toolbox. I'm not total loser though, I do like my {|o| blocks!}
I need to emphasize this. I am fluent in C, and can think in it. When I open Pierce TaPL and see immediately ML my heart sinks.
When I open Pierce TaPL and see immediately ML my heart sinks.
I strongly recommend that you ignore this feeling and learn a modicum of ML coding. If you want to learn TAPL, just learn a bit of Ocaml. You don't need much. If you already speak C, a bit of Haskell and Ruby, you should be able to pick up enough Ocaml in a few hours/days. Programming in Ocaml is way easier than in Haskell or C. The only two things that are a bit difficult in Ocaml are modules and the object system. You can completely ignore objects. Nobody ever uses them, and for TAPL you can also ignore modules (although you occasionally have to open a module, which is trival, something like: open Support.Pervasive).
I feel like C fits my brain, know what I mean? I don't even think that it was because I learnt it early on. It's a simple language, I feel.
Why is programming way easier in Ocaml than in C? Setting aside that for me programming is way easier in C than in Ocaml because I know C and don't know Ocaml, what is it about Ocaml that is way easier than C?
Anyway, my original point is that it is _cheating_ to explain how to build a Lisp by saying, "First reach for your nearest functional programming language". Useless car analogy. Today I'm going to learn how to build a car. First, take the engine from my Toyata... oh... um...
I've written vast amounts of Ocaml and C/C++, certainly over 100K LoCs in each. Programming in ML dialects is so much easier than C/C++, it's not funny. I'd estimate it takes me about 10 times longer to implement the same functionality in C/C++ than in an ML dialect. The three key features enabling this difference are the following.
* Garbage collection, removes all memory bugs, avoids tons of ugly and distracting (de)allocation commands and keeps your head free to think about more important stuff.
* Pattern matching, enables you to code up complex decision making code (and most functinality needs this) in a readable and maintainable way.
* higher-order functions allow you to parameterise code with behaviour in a neat way.
(There is one caveat, which is code for extremely high performance, I'd probably write that in C/C++, but this is not an issue concerning a learner.)
As to useless car analogies, I think implementing lambda in C is like saying: Today I'm going to learn how to build a car. First, let's do this while riding a uni-cycle ... oh... um...
Are you sure C is simpler then ML? If you really get into the nitty gritty details (which is what you do when interacting with programming language theory), the definition of the lambda calculus fits in a single page and an ML-lite that is powerful enough for real programming is just a little larger than that.
On the other hand, C needs a manual with hundreds of pages and has lots of dark corners and undefined behaviour. IMO, if you really want to keep things simple you need to stick to assembly language instead of C.
You think that assembly needs less of a manual than C? You think assembly has less dark corners? You think it's going to make things simpler? My mind boggles.
Oh, and also, which assembler? 68000? x86? ARM? (One of these is not like the others in terms of complexity...)
Maybe talking about undefined behavior wasn't the best approach I could take :) But if you stick to simple machine languages then it shouldn't be a big problem. Something that is closer to Knuth's MIX than to x86 won't have much undefined behavior.
The thing that I was thinking about was that many operations that are defined in assembly language, such as integer overflow are undefined in C (and this undefined behavior is often abused by optimizing compilers) and C also abstracts over a lot of control flow conventions.
> Today I'm going to learn how to build a car. First, take the engine from my Toyata... oh... um...
To the extent that the analogy works, it doesn't help your case: nobody these days builds or learns how to build a car from scratch, without access to an existing car. It's not only that you're typically making a modified version of an existing model: you also use a car to get to the factory, you use existing vehicles to transport raw materials and tools to the factory, etc.
In any case, as others have pointed out, it's not like you need to use a full {Lisp, Haskell, ML} compiler to implement (say) tail recursion. It's that you bootstrap: as soon as you have a minimal viable {Lisp, Haskell, ML} compiler, it makes sense to use that instead of C to implement other features.
I agree with you it'd be useful to see how you might implement lambdas in a language like C. But after that first example, implementing other more complicated language constructs in C, instead of learning ML/haskell is going to be more time fighting C, and less time learning anything new. The lack of side-effects and equational reasoning in functional languages let you get a lot of details out of your way.
That being said, this is if you're interested in the theory. If you're interested in building compilers, this list won't help too much.
The lack of side-effects and equational reasoning in functional languages let you get a lot of details out of your way.
Actually ... You (kind-of/sort-of) need side effects when implementing a language with higher-order functions. You at least need it for generating fresh variables to avoid free name capture. Yes, I know you can do this with a monad, but this is not going to help a beginner. It's much easier to maintain a global counter that you increment every time you want a fresh variable name.
The way I did it was just recursively pass down an accumulator argument to generate De Bruijn index-like variable names, rather than keeping a global counter
Point taken though that it's a bit simpler with global state
> I would like an introduction to these advanced topics in the language that I already now.
The problem with this may be best expressed by Whorf[1]:
"Language shapes the way we think, and determines what we can think about."
A philosophical school of thought concerned with this is known as Linguistic Relativity[2]. But I digress.
Part of internalizing foreign concepts is approaching them with minimal prior prejudices. Trying to relate concepts inexpressible in C to C idioms is extremely difficult, if not impossible, before being able to _translate_ those concepts into what can be represented in C. It's the classic "chicken and egg" problem.
The best thing I can think to recommend is to pick a language which is generally considered capable of supporting the concepts you wish to learn and approach it as if you have never seen a programming language before.
> When I open Pierce TaPL and see immediately ML my heart sinks.
IMHO, this reinforces the fact that tools which you have used before (likely to great success) are influencing your view of those tools which you are not as familiar. IOW, you learned BASIC, x86 assembler, and C... Why not ML?
Best of luck and hopefully the path you walk is fun!
It's actually a lot easier to learn ML and then read/write the interpreter in ML than to read/write the interpreter in C. An interpreter written in C would be too convoluted.
I did point out the nesting and the requirement to name in C.
The scoping follows immediately from beta-reduction
(lambda x.M)N --> M[N/x]
which I find very intuitive. I never had trouble understanding this, but maybe I'm overgeneralising from my own learning process. The real difficulty, or rather the real subtlety, is not nesting but name binding and the automatic renaming of bound variables where necessary. Having taught lambda-calculus to generations of students, that causes confusion with some students.
And I think implementing a type-inferencer in C is going to be confusing because beauty of e.g. the W type-inference algorithm is drowned by orthogonal issues that C forces on you like memory management. Interpreters are best written in languages with pattern matching. So I prefer to bootstrap one's learning: get the basics of lambda-calculus, then implement a lambda-calculus interpreter and type-inferencer in a modern FP language. (As an aside, that's what SICP also does.)
The real difficulty, or rather the real subtlety, is not nesting but name binding and the automatic renaming of bound variables where necessary.
If you're trying to understand what a closure does, perhaps that's true.
If you're trying to understand how it does it, and you come from an imperative programming background where you know the ABIs for your languages like the back of your hand and are used to the kind of function set-up that always allocates space for local variables on the stack, there is a huge question of how all that lovely, neat theory is actually implemented in practice.
Some of this discussion reminds me of the jokes about professors who prove that a solution to some elegantly expressed problem must exist, but then lead it to the grad students to actually find it...
Type inference and dependent types don't go well together (already System F doesn't have type-inference), although it depends on the details of what you mean by dependent types.
Tail recursion is a concept of compilation and does not show up in an interpreter.
Note also that C has higher-order functions (via function pointers). The only relevant difference from lambda-calculus is that C functions must be named, cannot be nested and must exist at compile time. But conceptually these restrictions make little difference to the understanding of what lambda does: it creates a function.
Finally writing something like that (to the extent that the requirements make sense) is gonna be really painful in C. If you understand the lambda-calculus well enough to know what dependent types are, the implementation language should not make much of a difference.
I think the easiest way to learn the lambda-calculus program in a functional language. The second easiest way is to read the theory, after all the lambda-calculus as a calculus is incredibly simple, and you can hand-wave away the problems of bound-variable renaming. A slow-paced textbook like Hankin's "An Introduction to Lambda Calculi for Computer Scientists", and doing the exercises in the first few chapters should do the job.