Clojure is a general-purpose functional programming language based on the Lisp-1 model. Its reference implementation runs on the JVM, but there are also editions that work in other environments – for example, the popular ClojureScript, which compiles to JavaScript. Clojure is a Lisp created with concurrent processing and the Java ecosystem in mind.

The Clojure language was created in 2007 by Rich Hickey, who based it on four premises:

1. A (not merely) functional language that emphasizes data immutability.
2. A dialect of Lisp.
3. Adapted for concurrent execution.
4. Running on a host platform.

The reference implementation of Clojure runs on the Java Virtual Machine (JVM) and in this case its so-called host platform is the JVM. This means that while programming we have access to all built-in classes and libraries of that environment, and the source form of a program is transformed into bytecode.

The reference implementation is written in a mixture of Java and Clojure and compiles Clojure forms to JVM bytecode. The resulting classes need not have a useful one-to-one translation back to Java source code.

We will begin by acquainting ourselves with the characteristics of the progenitor of the language family to which Clojure belongs – that is, Lisp – and then smoothly transition to descriptions specific to the dialect that this manual is devoted to.

Lisp

Clojure is a dialect of Lisp (once written as LISP). The name is an acronym for List Processor (also known as List Processing Language), since the most commonly used data structure in the language is the list. It was created in 1958 by John McCarthy with help from Steve Russell.

Clojure belongs to the Lisp-1 family. In this terminology, “namespace” means a name-resolution domain: a symbol in an expression has one value position, whether that value happens to be a function or something else. This is distinct from Clojure’s concrete Namespace objects, which maintain mappings from symbols to Vars and classes.

Consequently, one symbol in a given lexical context cannot simultaneously denote one value when called and another when merely read. Clojure may bind x to a function or to another value, but both uses resolve through the same value namespace. Other Lisp dialects, commonly called Lisp-2, maintain separate function and value namespaces. Some systems distinguish still more name-resolution domains. Peter Norvig, in his book “Paradigms of Artificial Intelligence Programming”, cites as many as 7 possible namespace families.

Lisp is one of the oldest programming languages still actively used today; it is second only to Fortran in this respect. It was created as a notation for computing with symbolic expressions and became an important language for early artificial-intelligence research, including theorem proving and natural-language processing.

Lisp is a concise and expressive language, although it requires longer deliberation on a problem before one begins formulating it as code. Conciseness means that compared to other programming languages, its syntactic constructs occupy less space, and expressiveness means that the grammar can describe a problem’s solution in fewer of them.

Features

Untyped lambda calculus

Lisp was strongly influenced by the untyped lambda calculus – a formal system developed by Alonzo Church and further studied by, among others, Stephen Cole Kleene. Its terms are variables, abstractions, and applications. An abstraction binds one parameter, while multiple parameters can be represented through nesting, commonly called currying. Lambda calculus and Turing machines characterize the same class of computable functions, although neither formalism by itself is an executable computer programming environment.

Church wanted the lambda calculus to serve as an alternative to set theory for specifying the foundations of mathematics, but that did not come to pass. Its value in the theory of computation and in the design of functional programming languages, however, is beyond measure.

Lisp is not a literal implementation of the lambda calculus. Lisp dialects add data types, named bindings, control constructs, input/output, state, and explicit rules for evaluation. The lambda calculus itself does not mandate one reduction strategy; normal order and applicative order are alternatives. Clojure normally evaluates the operator and arguments of a function call before invocation, while special forms and macros define different rules. It is therefore safer to say that Lisp borrowed lambda abstraction and function application as central ideas, not that it merely implements lambda calculus with constants.

Functional language

Lisp is a multi-paradigm language with a strong functional emphasis – the fundamental building blocks for constructing programs are functions or constructs that behave like functions (subroutines that accept arguments and return results). As mentioned earlier, the focus is on computing their values, i.e. evaluation.

It is worth noting that Lisp dialects are not purely functional languages – that is, they contain constructs which, in addition to computing results, can trigger so-called side effects: for example, displaying something on screen, exchanging data with input/output devices, or making direct modifications to in-memory structures. The responsibility for distinguishing purely functional operations from those whose behavior depends not only on the arguments passed but also on the environment falls on the programmer.

Function calls in Lisp are frequently nested or operate on other functions passed as arguments. Evaluation still follows a defined order: subexpressions are evaluated according to the rules of their enclosing form. The nested syntax makes data flow explicit, but does not replace sequencing with an informal “inward” execution model. Operators, loops, and conditional statements familiar from other languages have functional counterparts in Lisp.

Immutable data

In Clojure, similarly to purely functional languages – and unlike other Lisp dialects – the aim is for ordinary values to be immutable. Operations on a persistent collection return a value representing the requested result instead of modifying the input collection. Implementations commonly share unchanged structure, and an operation may even return the original object when the resulting value is unchanged; immutability is a semantic contract, not a promise of full copying.

We can capture and name the results of function calls – in that case we are dealing with so-called bindings, not variables. Bindings are, in their simplest form, symbolic names that identify values stored in memory.

There are cases, however, where one needs to work with mutable data – data whose content can and must be directly modified. A purely functional program would be a calculator that receives some input data at the start, passes it to the main function as arguments, which in turn calls further subroutines until it obtains a value constituting the result of the computation. The problems programmers face are somewhat more complex, and the need often arises to represent changing states in programs, identified by stable names. The amount of funds available in a bank account, a player’s position and score, the representation of the audio track currently being processed – these are examples of stable identities that can assume different states from moment to moment, expressed as immutable values.

The above need can be met by introducing variables, but then we risk their overuse and abandon the assumption that all values are constant by default. Such inconsistency would lead to even greater confusion than treating all data as potentially subject to change. For this reason, Clojure employs so-called reference types. Their instances coordinate access to a current state value. The indirection resembles pointers known from C or references from C++, but Clojure’s reference types additionally provide explicit concurrency semantics. When a new state is produced, the reference is updated to refer to that value rather than mutating the previous one. Code that still holds the previous value can continue to use it predictably. Once no live reference reaches that value, however, the garbage collector may reclaim it: persistence does not create an automatic change history. These mechanisms reduce the need for explicit locking, but do not eliminate concurrency design.

Let us note that most programming languages in use today are rooted in the imperative paradigm, in which the language constitutes an advanced stage in the evolution of machine language programming – that is, it is based on the way the hardware works. In a sense, we are dealing with more or less abstracted interfaces to operational memory and the central processing unit. Data – which may be individual values, structures, or other objects – are placed in memory cells identified by variable names, and sequences of instructions read results of computations from these areas and modify them, storing the results of new ones. The application influences what ends up in specific, accessible RAM locations.

The above does not seem unusual and for many is the obvious (if not the only) approach to solving problems with a computer. Since the computer is equipped with memory, programming should consist of accessing it; of writing, reading, and replacing the information stored there. This will certainly be a good strategy when designing operating systems, device drivers, or local databases – wherever a program must directly operate on in-memory structures for the sake of performance and resource conservation. An object placed at a specific location can then be quickly modified or replaced with an entirely different one. But “with great power comes great responsibility” – besides devising a solution to the problem, the programmer must keep in mind that, for example, simultaneous writes to the same memory space by more than one subroutine can cause a bug that is difficult to detect.

The point here is not even the need to manually manage processes that a compiler or interpreter should handle, but rather the low level of abstraction. Every problem modeled imperatively must be adapted to a reality in which most objects can undergo changes, and once planned changes occur, the previous structures assigned to specific locations are not remembered but overwritten. The exception will be situations where the programmer explicitly handles such cases and takes care to create, say, a change history or copies of certain data.

Unfortunately the world does not work exactly like a computer, and most of the problems for which we use programming languages can be expressed without the need to directly operate on modifiable areas of memory.

Programming firmly rooted in the functional paradigm lets application code reason mostly in terms of values rather than storage locations. Garbage collection handles allocation lifetime, while immutable values reduce the number of places where identity, time, and memory location must be considered together. Resource lifetime and performance still matter, especially for files, sockets, native memory, and large retained structures.

Laziness

Some functional languages use lazy evaluation, in which a computation is deferred until its result is demanded. Lisp dialects are not lazy by nature, and Clojure normally uses eager argument evaluation, but it supports lazy sequences – abstract data structures that allow lazy access to the values of collection elements or values returned by recursively invoked generator functions. Deferring individual computations in time can also be achieved using higher-order functions or certain reference types.

Innovations

Lisp had a large influence on the shape of many programming languages. Mechanisms that were introduced in so-called modern languages just a few or a dozen years ago – and described as groundbreaking and innovative – were implemented in Lisp as early as the 1960s–80s.

Lisp pioneered, popularized, or gave an early practical home to mechanisms such as:

• garbage collection,
• the eval construct,
• pervasive use of recursion and expression-oriented conditionals,
• function type,
• first-class functions,
• higher-order functions,
• closures (lexical closures as such appeared in Scheme, a Lisp dialect),
• symbols,
• explicit symbolic variable binding,
• macros,
• representing programs as list-shaped data, commonly discussed as homoiconicity,
• metaprogramming,
• extensible domain-specific languages (DSLs).

This is a history of influence rather than a claim that Lisp invented every item in isolation. Several ideas had mathematical or programming-language predecessors, while Lisp made their combination unusually programmable.

Popularity

One might ask why a language of such expressive power and capabilities has not dominated the market and become the most popular way to solve problems using computers. The reasons are not strictly technical.

First, Lisp had to contend with the marketing of Unixes and the C language, and it lost that battle. There did exist computers specialized in running Lisp code, equipped with operating systems written in Lisp, but these were ecosystems that were less open and less portable than Unixes.

Second, when years later it had a chance to make a comeback, the boom in object-oriented languages (C++ and Java) arrived, followed by the boom in web technologies and modern multi-paradigm languages such as Ruby or Python and their frameworks, Ruby on Rails and Django. [I personally became interested in Lisps when my Ruby code started becoming increasingly functional and declarative – author’s note.]

Runtime cost depends on the implementation, workload, data representation, and compiler. Persistent structures and garbage collection have costs, but they also share structure, reduce copying in some workflows, and simplify safe concurrency. A Lisp program may be slower or faster than an imperative counterpart on a particular task; language family alone does not determine CPU or memory use. The practical trade-off also includes development time, correctness, and the cost of coordinating mutable state.

Another reason for the “hibernation” of Lisps is that while the syntax is simpler than in most other languages (fewer rules, fewer ready-made constructs to memorize, fewer operators and special instructions with peculiar syntax – practically no syntax at all), mapping problems to code can be troublesome for beginners. This is because in many cases one must abandon sequential, imperative thinking, where instructions are executed and the results of previous ones are passed to subsequent ones using variables. Instead, we build small functions or macros that visually resemble tree roots more than a ladder. Often, instead of loops we use recursion, and instead of storing a function’s intermediate result in a variable, we simply pass along the function itself!

Why Lisp, why Clojure?

Some believe that functional languages – or multi-paradigm languages with good support for functional mechanisms (such as Scala) – are the future of the industry and will become very popular. I do not share this optimism when it comes to Lisp dialects. Although using them can confer a competitive advantage, they may turn out to be too hard. I do not mean to offend anyone here – it is not about the potential ability to understand them, but rather about the determination and time required to do so. Learning a functional language for someone who has spent most of their life programming imperatively can be somewhat shocking and demand the right conditions. Ideally, these conditions would also exist in the workplace.

For Clojure to become popular, it must therefore be used in business, and there it may turn out to be cheaper to hire two or three people of modest expectations and skills rather than a single programmer who will do it better in Clojure. The problems that the majority of programmers face today are not of such caliber that the teams working on them would announce a paradigm and technology shift.

If, however, we are not a corporation but a small or medium-sized team, and moreover are working on a project with a potentially longer development cycle, then Clojure can be tremendously helpful. This concerns the changeability of programs, which directly translates into productivity. Software written in a functional style is less tangled. We do not mean the structure of the source code, but rather the separation of concerns (SoC), which fits the functional style naturally.

The Clojure programming community places great emphasis on ensuring that individual components of software systems are not complected with each other – that is, that a change made in one does not necessitate changes in all the others. At the outset, such an approach requires thorough consideration of the system’s architecture, but later it repays itself through savings of hundreds or thousands of person-hours that would otherwise have to be spent on testing and fixes.

Observing Clojure programs published online, we can notice that keywords, maps, and vectors are used very frequently. For someone coming from the world of object-oriented programming, this may look like poverty and perpetual experimentation. Nothing could be further from the truth. First, Clojure’s hash maps and sets are implemented using persistent Hash Array Mapped Tries ([HAMTs]), while vectors use a related wide persistent tree. These structures build on earlier work on Ideal Hash Trees and adapt it to Clojure’s value semantics. Second, the Clojure community adheres to a principle formulated by Alan Perlis, the first recipient of the Turing Award, who said:

It is better to have 100 functions operating on one data structure than 10 functions operating on 10 structures.

Thanks to putting this motto into practice, we can program more generically and over the years expand a base of useful functions regardless of the business or (to some extent) implementation specifics of projects that come and go. We thus abandon the creation of pointless dictionaries of classes and methods that every participant in a project must learn, even though these typically perform similar operations on data of similar characteristics, differing only in the business-domain names of the data being processed.

It is also worth learning Clojure (or another Lisp dialect) for the sake of its formalism and expressive power. Right from the start, while writing simple example programs, one comes into direct contact with software engineering knowledge that is universal and broadens one’s perspective when devising solutions to problems.

When starting out with Perl, PHP, Ruby, or even C, one can for some time afford not to concern oneself with how the interpreter or compiler processes source code to carry out the program’s tasks. Over time, one may delve deeper into how the chosen language actually works and employ more advanced constructs – for example, using function objects (or function pointers in C), closures, reference types, or writing subroutines that execute lazily. Learning more, one may also wish to program more elegantly: controlling the visibility and scope of identifiers, choosing types and data structures judiciously, and ultimately adapting the application to make use of multiple processors or cores.

In Lisps, a good deal of this knowledge is needed from the very beginning, lest one program blindly – like someone writing C for the first time, adding and removing asterisks before pointer variables until the program stops crashing with a segmentation fault.

The point is not even that Lisp is demanding and inclines one to study the details of how the language works. It is rather about what happens to the programmer in the process. Being compelled by the language’s character to better understand certain mechanisms, one looks at programming in other languages in an entirely different way.

Beyond intersubjective reasons, Lisp is worth using for objective technical reasons. Besides its accentuated concurrency support (in the case of Clojure), Lisp is a language that, thanks to its elaborate macro system and a property called homoiconicity, allows the creation of syntactic layers of abstraction tailored to solving specific problems and accelerating development. This means that we have a real influence on how we handle complexity management, being able to add constructs to the language that will not merely be procedures grouping existing calls, but elements that change the rules of how the language operates.

The syntactic flexibility of Lisps means that by using macros and higher-order functions we can add entirely new constructs (loops, conditional expressions, operators) that in other languages would have to be implemented by their creators. In that case, a programmer wanting some needed mechanism could at most propose changes and wait for a new version of the compiler or interpreter to be released. In Clojure, many commonly used language constructs are in fact programmed as macros – for example, when, or, and and.