References Are Good

Last night I received an e-mail in which one of my visitors asked me a few questions after reading the post about parsing phone numbers in Clojure. The question was seemingly simple and concerned a technical detail of software construction, but in reality the reader touched on a very important subject that I had been pondering years ago, and I decided to summarise it here.

At the outset I want to stress that I use the word reference in a fairly general sense that goes beyond the reference mechanism known from programming languages, although I will rely mostly on technical comparisons to illustrate its essence.

Etymology

What is a reference? Let me refer (sic!) to the Online Etymology Dictionary, where we can read that the word comes from French, which borrowed it from Latin, and that since 1580 it has been used to denote the act of referring to someone or something. A bit later (around 1602) the term also began to be used in the sense of recommending someone (e.g. for a job).

The Latin verb referre is a compound of the prefix re-, which conveys the idea of doing something again or in response to something, and the verb ferre meaning to bear, to carry or to bring. So we have something that is bought back, carried again, etc.

It is worth mentioning that Proto-Indo-European contains the root bher- (visible in Latin as fer-) conveying care, upbringing, or authority. This gives us some insight into the character of the relationship expressed by the term: it includes some kind of source of truth or rule.

Thus, the “pointing” or “relating” meaning of the term does not merely carry information about making a connection but also conveys a dependency on the other party.

In this article I shall call reference any act of relating a person or object to another person or object within a relationship marked by reliance on that person or object as a source of decisive value.

Reference as a tool

In everyday life we have plenty of examples of referring to something, for instance in publications, reviews, descriptions, and even conversations. Social-media comments often contain references to authoritative statements or scientific publications in order to rationalise the validity of the theses being made.

Let us try to look at reference in a more utilitarian way, ignoring the cases where authors cite sources of knowledge. In most programming languages, for example, we have a mechanism called a reference or pointer whose purpose is to refer to a value or operation indirectly, using an identity created for that purpose.

Identity?! Are we still talking about computers?

Identity

The word “identity” comes from the Latin idem, meaning “the same”. The concept expresses similarity of objects, and at the level of process the ability to distinguish (and group) entities with common properties.

Thanks to the ability to constitute identity we function in the world without continuously pondering what relationship to form with the people we meet or everyday objects. We simply recognise them; they are familiar to us. This ability is owed to parts of our mind that can transparently form identities and operate on them, building models of the world that help us react appropriately to stimuli. A kind of reducer.

How does this relate to references? Patience. First, we’ll distinguish two kinds of identity according to their foundation, and then we’ll tie this back to references.

Identity can be extensional, in which case it rests on the object’s attributes (for example its shape, location, or suitability for specific purposes). We may loosely call this objective. It’s the kind of identity we rely on in everyday life-for instance, recognising a friend by appearance and character. The same holds for things. Two objects are equal if their selected properties are the same (2 is 2, April is April etc.).

There is also intensional identity, meaning one that does not depend on the properties of the identified object. Two objects are equal as long as their function within a system is the same - for example, Paul’s salary, Alexandra’s account balance, or the current number of birds in the sky. What we observe here are values that can change over time and space. This is where reference comes in. Reference is the fundamental means of constructing identity, because it allows us to single out any entity and create an abstraction capable of tracking a stream of changing values.

Abstractions are useful because they let us represent complex things in forms simplified just enough for a given purpose or for understanding in a particular context. A camera snaps a photo of a skyscraper hundreds of times larger than the print. The print is an abstract version of the building and is sufficient to judge its appearance. Without abstracts there would be no modelling of world processes in computers, and thus no problem-solving.

We can also view the difference between intensional and extensional identity through the lens of operations. For extensional identity (called here functional extensionality) what matters is the operation’s result. For intensional identity what matters is the way the operation is carried out, regardless of the resulting values.

When distinguishing intensional identities, we will focus on their purpose (role) within the system, whereas for extensional identities we will focus on values.

Identity in IT

In programming, an identity is a construct that allows the programmer (and the program) to differentiate certain data from others easily.

• In typically imperative languages we will have extensional identity (defined by a place in memory), and attempts to build intensional identity on that basis are often fraught with the risk of leaky encapsulation (the control aspect is weakened).

• In multiparadigm languages with functional roots we can create intensional identities (e.g. using reference types), and because the data are independent of their location such constructs are in fact independent of value.

• In purely functional languages intensional identity may prove unnecessary due to the absence of accidental shared state and the lack of need to manage it, although under certain conditions we can still build intensional identities (see reify and runST in Haskell).

• In object-oriented languages identity is intensional, created by a special property of every object. Extensional elements can creep in when the language is based on mutable memory areas, even if referencing them does not require using addresses or pointers. In addition, some languages introduce an extra family of so-called value objects whose identity depends on value - thereby producing intensional identities as an exception. The programmer’s problem will be recognising which kind of identity is in play.

On the World Wide Web, identity is expressed by a Uniform Resource Locator (URL). This is an intensional identity, because the URL does not depend on the resource itself (e.g., an HTML document). We must remember, however, that the resource can change or disappear while the URL remains the same - and that is still fully consistent with the idea of intensionality.

Reference examples in programming

A reference in programming is a mechanism for accessing data through an indirect construct that stores the appropriate pointer to it. We can usually change that pointer.

Take a pointer in the C language:

#include <stdio.h>

const char text[]  = "Mary sowed poppy-seed.";
const char other[] = "She did not know how.";

int main(int argc, char *argv[])
{
void const *reference = text;
printf("%s\n", (const char*)reference);

reference = other;
printf("%s\n", (const char*)reference);
}

Or a reference object in C++:

#include <iostream>
#include <functional>

int main()
{
const char *text  = "Mary sowed poppy-seed.";
const char *other = "She did not know how.";
auto reference    = std::ref(text);

std::cout << reference << std::endl;

reference = std::ref(other);
std::cout << reference << std::endl;
}

And a global variable based on the Var reference type from Clojure:

(def text "One two.")
(println text)

(alter-var-root #'text (constantly "Three four."))
(println text)

Forget for a moment that identity can be dual (because programming languages are rooted in imperative ideas) and focus on the demonstrated ability to point to values indirectly.

Each example contains an element (a pointer, a reference object, a binding) that creates an intensional identity. What can we say about it?

• It does not depend on the value it points to.
• We decide what it means inside the program.
• At different moments in time it can point to different values.

Reference in programs is not a value but an abstraction that points to values, often changing over time.

Why references are good?

Let me recall the features of a good intensional identity:

• abstraction (it can point to any value),
• dynamism (it can point to different values over time),
• specificity (it can be identified within the system),
• uniqueness (it does not repeat within the system).

I would add one more, very important:

ability to control.

In programming we can include within that control also more subtle features of identity, in particular the ability to encapsulate values, but that is a topic for a separate article.

Moreover, in some programming languages the control aspect of identity is so common and/or transparent that we might not appreciate it, although in other fields it is the property we care about most.

When we operate directly on values - on original resources - changing them (e.g., to adapt to new conditions or requirements) can be difficult and time-consuming (modifications in many places) and sometimes impossible (original resources outside our control).

Real-life examples

References are not only for programming. They are useful wherever we need to identify something so that others can make use of it. It is then worth checking who controls the reference, and if it is not us, trying to change that.

People are divided into those
who use references
and those who are going to use them.

Take a video hosted on a popular video platform as an example. When we upload our own material to such a service, we get the illusion of control. For various reasons the provider can stop serving this particular work or all works. We might also have limited ability to change it - for instance, to apply a small correction. If the service does not let us replace the file under the original URL, we cannot manage updates. Sharing links with other people becomes risky, because the pointers to the resources (URLs) can stop identifying our works overnight.

If, however, we created a simple service under our own domain name containing links to the videos (either in documents presented on the site or via redirects), we would have a pretty good, reference-based intensional identity for each of our publications - a truly universal identifier. If something bad happened to the “warehouse” holding our works, we could quickly switch to another one by changing the current values of the existing references (the URL associations). Users would still visit our site, because it is that domain we would share with people. This principle is used, for instance, by podcast RSS feeds: although episodes may live on different servers, they all point back to a single canonical feed.

Another common example is extensional identity in the domain of authentication data and contact data. This is a more serious matter.

It is very easy, having a user account in a popular internet service, to use it (if supported) to log into other services. The problem is that losing access to the account - or more typically to the e-mail address that identifies it - becomes catastrophic. Losing access to e-mail means losing our digital identity and having to go through tedious updates everywhere that relies on the e-mail address. Some services are not very sensible here and require confirmation sent to the current address, which no longer works. We are then stuck.

Does that mean we should run our own mail server? Not necessarily - a reference is enough. In this case it is an e-mail address. The address itself also contains a reference, because it consists not only of a local part but also of a domain name. In practice it is enough to register your own domain with a reputable registrar and set up mail forwarding. This means that e-mails sent to the address under our control will be forwarded to our current mailbox hosted by one of the popular providers. Should that provider cause trouble, we can reconfigure the forwarding address within minutes and remain reachable under the same canonical e-mail.

It is worth taking a moment to examine how your e-mail service provider defends itself against unsolicited messages and to fine-tune the mechanism you are building-for instance, by using SPF records.

References can streamline change management and reduce risk across many domains-especially wherever we are modelling concrete, material problems. If we rely solely on values, our processes become tied to the data’s shape, the way we access it, and its intrinsic properties. By introducing references and shifting identification to abstract intermediary elements, we can regain control and decouple the design of the interpretive system from the things it interprets.

The e-mail question

The question that inspired me to share these thoughts was very specific. It concerned the reason why in the Makefile of the phone-number library. I place calls to small scripts from the application’s bin subdirectory instead of invoking the Clojure compiler directly with the appropriate options.

Here is an example of such a script whose role is to start the documentation-building process:

#!/bin/sh

echo "[phone-number] Building docs, please wait..."

And below are lines from the Makefile that use the docs script (instead of invoking clojure with the appropriate options):

docs:
    bin/docs

In this case the shell scripts play a referential role: they are the entry points for compiler invocations. If something needs to change (e.g., modify the options for clojure or clj), I will do it right there. There will be no need to adjust the configuration for every other tool.

References are good. It is not even about the mechanism itself but about a problem-solving approach in which we leave ourselves a unified point of control over access to data and use that point wherever possible.