Clojure is a general-purpose functional programming language based on the Lisp-1 model. Its reference implementation runs on the JVM, but there are also editions that work in other environments – for example, the popular ClojureScript, which compiles to JavaScript. Clojure is a Lisp created with concurrent processing and the Java ecosystem in mind.

The Clojure language was created in 2007 by Rich Hickey, who based it on four premises:

1. A (not merely) functional language that emphasizes data immutability.
2. A dialect of Lisp.
3. Adapted for concurrent execution.
4. Running on a host platform.

The reference implementation of Clojure runs on the Java Virtual Machine (JVM) and in this case its so-called host platform is the JVM. This means that while programming we have access to all built-in classes and libraries of that environment, and the source form of a program is transformed into bytecode.

Clojure is written in Java, although certain specific constructs, after compilation, cannot be directly translated back into Java source code due to the optimizations employed.

We will begin by acquainting ourselves with the characteristics of the progenitor of the language family to which Clojure belongs – that is, Lisp – and then smoothly transition to descriptions specific to the dialect that this manual is devoted to.

Lisp

Clojure is a dialect of Lisp (once written as LISP). The name is an acronym for List Processor (also known as List Processing Language), since the most commonly used data structure in the language is the list. It was created in 1958 by John McCarthy with help from Steve Russell.

Clojure belongs to the Lisp-1 family. This means that we are dealing with a single kind of so-called namespace. Namespaces will be discussed in detail later, but broadly speaking they are structures used to store mappings from identifiers (naming e.g. functions or variables) to corresponding in-memory objects (e.g. subroutines or values). This is one of the ways of managing name visibility in computer programs.

When a programming language supports only one kind of namespace, the same name in a given lexical context can unambiguously point to an in-memory object of only one kind. If we assign an identifier (e.g. x) to some variable, then – depending on the language – assigning the same identifier to a function will either produce an error or overwrite the previous value. This is exactly the situation we encounter in Clojure, where in a given namespace (named e.g. user) both function mappings and so-called global variable mappings reside. There do exist, however, dialects of Lisp in which functions and variables are placed in namespaces of different kinds (Lisp-2), and even ones where there are significantly more kinds of namespaces (Lisp-3, Lisp-4, etc.). Peter Norvig, in his book “Paradigms of Artificial Intelligence Programming”, cites as many as 7 possible namespace families.

Lisp is one of the oldest programming languages still actively used today; it is second only to Fortran in this respect. It was created to perform mathematical computations, process natural language, and explore problems related to artificial intelligence in the so-called symbolic model (fuzzy logic, genetic algorithms, inference based on collections of experiences, simulations of specific processes).

Lisp is a concise and expressive language, although it requires longer deliberation on a problem before one begins formulating it as code. Conciseness means that compared to other programming languages, its syntactic constructs occupy less space, and expressiveness means that the grammar allows describing a problem’s solution in fewer of them.

Features

Untyped lambda calculus

Lisp is based on the so-called untyped lambda calculus – a formal system created by the mathematicians Alonzo Church and Stephen Cole Kleene. It is a mathematical notation that allows expressing computations using expressions that are single-argument functions. Both the return value of each such function and the argument it accepts are also single-argument functions. More complex (e.g. multi-argument) functions can be expressed using currying. If an algorithm can be expressed using the lambda calculus, then it can certainly be implemented on a Turing machine – that is, as a computer program.

Church wanted the lambda calculus to serve as an alternative to set theory for specifying the foundations of mathematics, but that did not come to pass. Its value in the theory of computation and in the design of functional programming languages, however, is beyond measure.

It is worth noting that by design Lisp was not and is not an exact computer implementation of the lambda calculus, where everything is a function. In Lisp we deal with expressions that can effectively be either functions or constant values.

Another difference concerns the expression evaluation strategy. In the lambda calculus, computations proceed in the so-called normal order, where the outermost reducible expressions are always reduced first, and functions are applied (computed) before their arguments are evaluated. They therefore operate on the functions passed as arguments, rather than on the values those functions return. In Lisp, on the other hand, call by value is used, where each argument’s value is computed recursively first, and only then are those values passed to the invocation.

One can therefore say that Lisp is an implementation of the untyped lambda calculus with support for constant values and call-by-value evaluation.

Functional language

Lisp is a multi-paradigm language with a strong functional emphasis – the fundamental building blocks for constructing programs are functions or constructs that behave like functions (subroutines that accept arguments and return results). As mentioned earlier, the focus is on computing their values, i.e. evaluation.

It is worth noting that Lisp dialects are not purely functional languages – that is, they contain constructs which, in addition to computing results, can trigger so-called side effects: for example, displaying something on screen, exchanging data with input/output devices, or making direct modifications to in-memory structures. The responsibility for distinguishing purely functional operations from those whose behavior depends not only on the arguments passed but also on the environment falls on the programmer.

Function calls in Lisp are frequently nested or operate on other functions passed as arguments. Programs therefore do not execute from top to bottom as in the imperative model, but rather inward. Operators, loops, and conditional statements familiar from other languages have functional counterparts in Lisp. This preserves a simple syntax and facilitates learning through understanding rather than memorization.

Immutable data

In Clojure, similarly to purely functional languages – and unlike other Lisp dialects – the aim is for data stored in memory to be immutable. When we perform computations on a single value or a composite structure, the result is not a change to the memory area it occupies, but the creation of an entirely new object. This new object is an unmodifiable value, not a mutable structure assigned to a fixed location.

We can capture and name the results of function calls – in that case we are dealing with so-called bindings, not variables. Bindings are, in their simplest form, symbolic names that identify values stored in memory.

There are cases, however, where one needs to work with mutable data – data whose content can and must be directly modified. A purely functional program would be a calculator that receives some input data at the start, passes it to the main function as arguments, which in turn calls further subroutines until it obtains a value constituting the result of the computation. The problems programmers face are somewhat more complex, and the need often arises to represent changing states in programs, identified by stable names. The amount of funds available in a bank account, a player’s position and score, the representation of the audio track currently being processed – these are examples of stable identities that can assume different states from moment to moment, expressed as immutable values.

The above need can be met by introducing variables, but then we risk their overuse and abandon the assumption that all values are constant by default. Such inconsistency would lead to even greater confusion than treating all data as potentially subject to change. For this reason, Clojure employs so-called reference types. Their instances do not themselves store values, but refer to existing ones. By analogy, this somewhat resembles pointers known from C or references from C++, although Clojure’s reference types additionally provide built-in concurrency semantics. When a new value appears as the result of an operation (e.g. a function call), the reference is updated rather than the previous data being overwritten. The previous data continues to reside in memory, albeit without an explicit identity – the reference object now points to the new result. As a consequence, parts of a program that are still operating on the previous value at any given time can continue to do so predictably. The programmer does not need to employ constructs that create local copies or establish locks, as is the case in, for example, the conventional multithreaded programming model.

Let us note that most programming languages in use today are rooted in the imperative paradigm, in which the language constitutes an advanced stage in the evolution of machine language programming – that is, it is based on the way the hardware works. In a sense, we are dealing with more or less abstracted interfaces to operational memory and the central processing unit. Data – which may be individual values, structures, or other objects – are placed in memory cells identified by variable names, and sequences of instructions read results of computations from these areas and modify them, storing the results of new ones. The application influences what ends up in specific, accessible RAM locations.

The above does not seem unusual and for many is the obvious (if not the only) approach to solving problems with a computer. Since the computer is equipped with memory, programming should consist of accessing it; of writing, reading, and replacing the information stored there. This will certainly be a good strategy when designing operating systems, device drivers, or local databases – wherever a program must directly operate on in-memory structures for the sake of performance and resource conservation. An object placed at a specific location can then be quickly modified or replaced with an entirely different one. But “with great power comes great responsibility” – besides devising a solution to the problem, the programmer must keep in mind that, for example, simultaneous writes to the same memory space by more than one subroutine can cause a bug that is difficult to detect.

The point here is not even the need to manually manage processes that a compiler or interpreter should handle, but rather the low level of abstraction. Every problem modeled imperatively must be adapted to a reality in which most objects can undergo changes, and once planned changes occur, the previous structures assigned to specific locations are not remembered but overwritten. The exception will be situations where the programmer explicitly handles such cases and takes care to create, say, a change history or copies of certain data.

Unfortunately the world does not work exactly like a computer, and most of the problems for which we use programming languages can be expressed without the need to directly operate on modifiable areas of memory.

Programming firmly rooted in the functional paradigm frees us from managing memory resources – not only from allocating and freeing space (as encountered in, say, C or C++), but even from the fact that objects occupy any specific location at all! Decoupling the identity of information from its location means fewer problems, especially in the context of concurrent processing.

Laziness

Another feature of functional languages is the laziness of expression evaluation. This feature means that evaluation does not happen immediately, but only when a request to read the final result occurs. Lisp dialects are not lazy by nature (we are dealing with call-by-value), but Clojure supports so-called lazy sequences – abstract data structures that allow lazy access to the values of collection elements or values returned by recursively invoked generator functions. Deferring individual computations in time can also be achieved using higher-order functions or certain reference types.

Innovations

Lisp had a large influence on the shape of many programming languages. Mechanisms that were introduced in so-called modern languages just a few or a dozen years ago – and described as groundbreaking and innovative – were implemented in Lisp as early as the 1960s–80s.

From Lisp originate, among others, such technologies as:

• garbage collection,
• the eval construct,
• recursion,
• conditionals,
• function type,
• first-class functions,
• higher-order functions,
• closures (lexical closures as such appeared in Scheme, a Lisp dialect),
• symbols,
• variable binding,
• expressions,
• macros,
• syntax tree,
• homoiconicity,
• metaprogramming,
• domain-specific languages (DSL).

Popularity

One might ask why a language of such expressive power and capabilities has not dominated the market and become the most popular way to solve problems using computers. The reasons are not strictly technical.

First, Lisp had to contend with the marketing of Unixes and the C language, and it lost that battle. There did exist computers specialized in running Lisp code, equipped with operating systems written in Lisp, but these were ecosystems that were less open and less portable than Unixes.

Second, when years later it had a chance to make a comeback, the boom in object-oriented languages (C++ and Java) arrived, followed by the boom in web technologies and modern multi-paradigm languages such as Ruby or Python and their frameworks, Ruby on Rails and Django. [I personally became interested in Lisps when my Ruby code started becoming increasingly functional and declarative – author’s note.]

The list as a fundamental data structure and a running garbage collector mean not only greater RAM usage, but also somewhat greater CPU load. Proliferating memory structures (and removing unused ones) does require a certain amount of work. Programs written in Lisp can be noticeably slower than their counterparts written in statically typed imperative languages. This used to be an obstacle, but today most CPUs are equipped with multiple cores. For imperatively oriented programs, this means having to use workarounds to perform operations concurrently – a consequence of the multitude of mutable structures they employ. By contrast, the functional approach works very well in this domain, and the load on a single core ceases to be as critical – especially when we take into account the cost of maintaining and managing changes in the software.

Another reason for the “hibernation” of Lisps is that while the syntax is simpler than in most other languages (fewer rules, fewer ready-made constructs to memorize, fewer operators and special instructions with peculiar syntax – practically no syntax at all), mapping problems to code can be troublesome for beginners. This is because in many cases one must abandon sequential, imperative thinking, where instructions are executed and the results of previous ones are passed to subsequent ones using variables. Instead, we build small functions or macros that visually resemble tree roots more than a ladder. Often, instead of loops we use recursion, and instead of storing a function’s intermediate result in a variable, we simply pass along the function itself!

Why Lisp, why Clojure?

Some believe that functional languages – or multi-paradigm languages with good support for functional mechanisms (such as Scala) – are the future of the industry and will become very popular. I do not share this optimism when it comes to Lisp dialects. Although using them can confer a competitive advantage, they may turn out to be too hard. I do not mean to offend anyone here – it is not about the potential ability to understand them, but rather about the determination and time required to do so. Learning a functional language for someone who has spent most of their life programming imperatively can be somewhat shocking and demand the right conditions. Ideally, these conditions would also exist in the workplace.

For Clojure to become popular, it must therefore be used in business, and there it may turn out to be cheaper to hire two or three people of modest expectations and skills rather than a single programmer who will do it better in Clojure. The problems that the statistical majority of programmers face today are not of such caliber that the teams working on them would announce a paradigm and technology shift.

If, however, we are not a corporation but a small or medium-sized team, and moreover are working on a project with a potentially longer development cycle, then Clojure can be tremendously helpful. This concerns the changeability of programs, which directly translates into productivity. Software written in a functional style is less tangled. We do not mean the structure of the source code, but rather the separation of concerns (SoC), which fits the functional style naturally.

The Clojure programming community places great emphasis on ensuring that individual components of software systems are not complected with each other – that is, that a change made in one does not necessitate changes in all the others. At the outset, such an approach requires thorough consideration of the system’s architecture, but later it repays itself through savings of hundreds or thousands of person-hours that would otherwise have to be spent on testing and fixes.

Observing Clojure programs published online, we can notice that keywords, maps, and vectors are used very frequently. For someone coming from the world of object-oriented programming, this may look like poverty and perpetual experimentation. Nothing could be further from the truth. First, maps and vectors in Clojure are structures based on the world’s first practical implementation of high-performance Hash Array Mapped Tries (HAMT), known for short as Ideal Hash Trees. Second, the Clojure community adheres to a principle formulated by Alan Perlis, the first recipient of the Turing Award, who said:

It is better to have 100 functions operating on one data structure than 10 functions operating on 10 structures.

Thanks to putting this motto into practice, we can program more generically and over the years expand a base of useful functions regardless of the business or (to some extent) implementation specifics of projects that come and go. We thus abandon the creation of pointless dictionaries of classes and methods that every participant in a project must learn, even though these typically perform similar operations on data of similar characteristics, differing only in the business-domain names of the data being processed.

It is also worth learning Clojure (or another Lisp dialect) for the sake of its formalism and expressive power. Right from the start, while writing simple example programs, one comes into direct contact with software engineering knowledge that is universal and broadens one’s perspective when devising solutions to problems.

When starting out with Perl, PHP, Ruby, or even C, one can for some time afford not to concern oneself with how the interpreter or compiler processes source code to carry out the program’s tasks. Over time, one may delve deeper into how the chosen language actually works and employ more advanced constructs – for example, using function objects (or function pointers in C), closures, reference types, or writing subroutines that execute lazily. Learning more, one may also wish to program more elegantly: controlling the visibility and scope of identifiers, choosing types and data structures judiciously, and ultimately adapting the application to make use of multiple processors or cores.

In Lisps, a good deal of this knowledge is needed from the very beginning, lest one program blindly – like someone writing C for the first time, adding and removing asterisks before pointer variables until the program stops crashing with a segmentation fault.

The point is not even that Lisp is demanding and inclines one to study the details of how the language works. It is rather about what happens to the programmer in the process. Being compelled by the language’s character to better understand certain mechanisms, one looks at programming in other languages in an entirely different way.

Beyond intersubjective reasons, Lisp is worth using for objective technical reasons. Besides its accentuated concurrency support (in the case of Clojure), Lisp is a language that, thanks to its elaborate macro system and a property called homoiconicity, allows the creation of syntactic layers of abstraction tailored to solving specific problems and accelerating development. This means that we have a real influence on how we handle complexity management, being able to add constructs to the language that will not merely be procedures grouping existing calls, but elements that change the rules of how the language operates.

The syntactic flexibility of Lisps means that by using macros and higher-order functions we can add entirely new constructs (loops, conditional expressions, operators) that in other languages would have to be implemented by their creators. In that case, a programmer wanting some needed mechanism could at most propose changes and wait for a new version of the compiler or interpreter to be released. In Clojure, many commonly used language constructs are in fact programmed as macros – for example, when, or, and and.