.. title: Introduction to Makefile .. slug: introduction-to-makefile .. date: 2017-10-27 02:59:26 UTC .. updated: 2018-01-05 04:01:51 UTC .. tags: make .. category: .. link: .. description: .. type: text Are you in the same boat as I was only a few weeks ago? That is, does writing a Makefile intimidates you beyond belief? Fear not! Makefiles appear to be insurmountable (and some of the complicated ones really are) but if you start with simple steps, they are really not that difficult. .. TEASER_END: Read more .. contents:: The first thing you should do is read `Makefiles for Golang `_. It does an excellent job of introducing the basic concepts of a Makefile. I'll attempt the same thing but with a different approach. Rules ----- Makefiles are composed of one or more rules. A rule has three parts: * Target * Dependencies (or prerequisites) (zero or more) * Recipe (one or more steps The format of a rule is, :: target: dependency recipe All lines in the *recipe* need to be single-indented with a *tab* (not spaces!). Let me repeat: use a tab to indent. Otherwise, ``make`` complains. Beware: code snippets in this post may be rendered with spaces instead of tabs. I apologize that so far I've been unable to find a fix. Variables --------- Makefiles can declare one or more variables. They can also use environment variables as if they were declared in the Makefile. For our purposes, a variable is declared in this manner, :: VAR := some_value Notice the use of *:=*. In some Makefiles, you'll also see *=*. The difference is that *:=* binds quickly while *=* binds lazily (when it's actually used for the first time). This is what I understand. I may be wrong. In that case, read the official documentation for ``make``. In any case, I've seen it recommended that we always prefer *:=* over *=*. To use a variable, reference it as *$(VAR)*. Environment variables are referenced exactly like a variable declared in a Makefile i.e. $(ENVVAR). I like to use environment variables extensively so my Makefiles are customizable. Recipe ------ A recipe is zero or more commands to run when the target is invoked. Target ------ There are two kinds of targets: file (or directory) or not-a-file (also called *phony*). Let's see some examples to understand this concept a bit more. :: temp_file: touch temp_file .PHONY: ls ls: ls -a Save the above as *Makefile* in some empty directory. This *Makefile* has two targets. One target (*temp_file*) is a file (called *temp_file*) and the other (*ls*) is not a file (but is a *command*? and is thus marked with *.PHONY*. Let's see which files are in the current directory. :: $ ls Makefile Run ``make`` like so, :: $ make temp_file touch temp_file Let's see which files are in the current directory. :: $ ls Makefile temp_file We ran the *temp_file* target and it ran the recipe for it. The recipe basically created a file called *temp_file* (the name of the target). Now let's run the *ls* target, :: $ make ls ls -a . .. Makefile temp_file This time the target *ls* did what its recipe state i.e. run ``ls -a``. No file was created. Let's run target *temp_file* again, :: $ make temp_file make: `temp_file' is up to date. Since *temp_file* (the file) was not modified since the last time we ran ``make temp_file``, ``make`` recognizes this and does not run its recipe. We could repeat the same command again and again and as long as *temp_file* remains unmodified, ``make`` will not run its recipe. It must be noted that ``make`` uses last modified time of a file to determine whether it was modified or not. This is unlike ``git``, which tracks content to determine if a file was modified. Edit *Makefile* and add a new target, *no_file*, :: temp_file: touch temp_file .PHONY: ls ls: ls -a no_file: touch temp_file Run the new target, :: $ make no_file touch temp_file Run it again for good measure, :: $ make no_file touch temp_file Unlike when we repeated ``make temp_file`` -- and ``make`` didn't re-run the recipe because it didn't need to -- repeating ``make no_file`` runs the recipe every time. The reason is that ``make`` does not see a file called *no_file* appear after running the target's recipe. So unless the recipe is changed that results in the creation of a file called *no_file*, ``make`` will *always* run its recipe. In other words, the *no_file* target is a *phony* target, i.e. running its recipe does not create a file of the same name as the target name. In this sense, the *ls* target and *no_file* target are functionally equivalent. For the sake of being proper, we should really mark *no_file* target with *.PHONY*, like we did with the *ls* target. Dependencies ------------ A target can depend on other target(s). In this case, recipes of all the dependency targets are run before the recipe of the invoked target. For example, we could have a *Makefile*, :: one: touch one two: one touch two In this *Makefile*, the target *two* has target *one* as a prerequisite (or dependency). We'll expect ``make`` to always run the recipe of target *one* (when needed) before running the recipe of target *two* (when needed). Run the target *two*, :: $ make two touch one touch two As is visible, ``make`` ran the recipe of target *one* before it ran the recipe of target *two*. Let's run the same target (*two*) again, :: $ make two make: `two' is up to date. Since file *two* was not modified, ``make`` did not run the target *two*. Let's just update file *one* and run target *two* again. :: $ touch one $ make two touch two Here, as expected, ``make`` ran target *two* because it recognized that its dependency had been updated. Why did it not run target *one*? I don't know. Let's update file *two* and run target *two* again. :: $ touch two $ make two make: `two' is up to date. This is weird. We ``touch``-ed file *two* but ``make`` did not run its recipe unlike when we had ``touch``-ed file *one*. I don't know why this is. Let's explore it a bit more. Let's run target *one* and then target *two*, :: $ make one make: `one' is up to date. $ make two make: `two' is up to date. Let's touch file *one* and run target *one*, :: $ touch one $ make one make: `one' is up to date. Now let's run target *two*, :: $ make two touch two From observing these outcomes, I assume that if the last touched time of a file is updated and that file is a dependency of another target, the target's recipe is run. However, if the touched time of a file is updated and the file's own target is invoked, its recipe is not run. This is something worth exploring. This also illustrates the sometimes odd-seeming behavior of ``make`` and why many people are not very excited to adopt it. Default Target -------------- The first target in the *Makefile* is the default target. When you run ``make`` without specifying a target, it invokes the default target. Usually, people call the default target *all*. This is why many instructions for many projects ask users to call ``make configure && make all``. Let's create an example *Makefile*, :: .PHONY: all all: echo "Default target" .PHONY: clean clean: echo "Not the default target" Let's run ``make`` without any target specified, :: $ make echo "Default target" Default target Let's now run it with the *all* target, :: $ make all echo "Default target" Default target In this case ``make`` and ``make all`` are functionally equivalent. Alias ----- Creating an alias for a target is pretty easy. Create the alias target, with a dependency on the original target, but no recipe. This is mostly useful in *phony* targets. The *Makefile* could look something like this, :: original: recipe .PHONY: alias alias: original List All Targets ---------------- ``make`` has no way to cleanly list all targets in a *Makefile*. Fear not, though, since there are awesome people who have figured out ways to work around such inconveniences. The source of this "magical" solution is `How do you get the list of targets in a makefile? `_. Add the following to any *Makefile* and run ``make list`` to get a clean list of all targets in a *Makefile*. :: .PHONY: list list: @$(MAKE) -pRrq -f $(lastword $(MAKEFILE_LIST)) : 2>/dev/null | awk -v RS= -F: '/^# File/,/^# Finished Make data base/ {if ($$1 !~ "^[#.]") {print $$1}}' | sort | egrep -v -e '^[^[:alnum:]]' -e '^$@$$' | xargs What if a Target has a Directory as a Dependency? ------------------------------------------------- Let's start with a single, empty directory. :: $ ls Now create a *Makefile* in that empty directory that could look something like this: :: files: mkdir -p files files/mykey: files ssh-keygen -N "" -f files/mykey -t rsa -b 4096 Here, *files* is the name of a directory created by the recipe of the *files* target. Similarly, *files/mykey* is the name of the file created by the recipe of the *files/mykey* target. Run the *files/mykey* target, :: $ make files/mykey mkdir -p files ssh-keygen -N "" -f files/mykey -t rsa -b 4096 Generating public/private rsa key pair. Your identification has been saved in files/mykey. Your public key has been saved in files/mykey.pub. Note that first the dependency target, *files*, is run and then the invoked target *files/myfiles*. The output above has been truncated because it's not relevant to our purposes here. Run the same target again, :: $ make files/mykey make: `files/mykey' is up to date. Create a file in the directory *files*, :: $ touch files/tmp Run the *files/mykey* target again, :: $ make files/mykey ssh-keygen -N "" -f files/mykey -t rsa -b 4096 Generating public/private rsa key pair. files/mykey already exists. Overwrite (y/n)? n make: *** [files/mykey] Error 1 Since there was a change in the *files* directory, and it is a dependency of the *files/mykey* target, the recipe for *files/mykey* target is run. The *files* target was not run since the directory already exists. Why did ``make`` run the recipe of the invoked target when that target already exists and was not modified? As I understand it, ``make`` runs any targets that depend on a directory target if there have been any changes in the directory (such as adding another file). This is clearly not what we want. We know that *files* directory already exists and that no changes were made to *files/mykey*. We expect ``make`` to not run recipe of either target. ``make`` differentiates between normal prerequisites and order-only prerequisites (`Types of Prerequisites `_). What we have learned so far are normal prerequisites (dependencies). When a target's dependency is updated, the target is updated (its recipe is run). In special cases, like the one described here, you don't want to run a target if its dependency is updated (it'll still run the target if the dependency is *created*. The syntax for specifying an order-only prerequisite is to add a pipe (|). Any dependencies to the left of the pipe are normal prerequisites and those to the right are order-only prerequisites. Our *Makefile* can be modified as below, :: files: mkdir -p files files/mykey: | files ssh-keygen -N "" -f files/mykey -t rsa -b 4096 Create another file in the directory *files*, :: $ touch files/tmp2 Run the *files/mykey* target again, :: $ make files/mykey make: `files/mykey' is up to date. It worked! Let's just run through this modified *Makefile* yet again. :: $ rm -rf files $ make files/mykey mkdir -p files ssh-keygen -N "" -f files/mykey -t rsa -b 4096 Generating public/private rsa key pair. Your identification has been saved in files/mykey. Your public key has been saved in files/mykey.pub. $ make files/mykey make: `files/mykey' is up to date. $ touch files/tmp $ make files/mykey make: `files/mykey' is up to date. Embed Shell Script in Makefile ------------------------------ A *Makefile* can call any shell script. Sometimes, though, you just want a quick way to embed a small shell script in the *Makefile* itself. A primary reason for doing this could be that each command in a recipe is run in its own separate shell. This way any output from one command is difficult to use in a subsequent command. A workaround, you may think, is to save the output in a variable. That doesn't always work. Let's say you have a target with a recipe that stores the output of a command into a variable. Store this in a *Makefile* that is in an otherwise empty directory. :: .PHONY: shell-script shell-script: touch tmp owner = $(shell ls tmp) echo $(owner) What does ``$(shell ls tmp)`` mean? It means we're explicitly invoking the shell to run something for us, the output of which we wish to store in a variable. Invoke the *shell-script* target, :: $ make shell-script ls: tmp: No such file or directory touch tmp owner = make: owner: No such file or directory make: *** [shell-script] Error 1 What the huh? Why did ``ls`` run before ``touch``? We even used the lazy evaluation version of variable assignment (*=*) instead of the recommended version (*:=*). ``make`` runs any ``$(shell foo)`` pieces in the *Makefile* before running any targets or their dependencies, no matter where they occur in the file. ``make`` also runs them exactly once. Given this new information, let's review our *Makefile*. We create a file, run ``ls`` and store its output in a variable, and finally print the contents of the variable. The way ``make`` looked at the file and decided to execute it is different from what we would (quite logically, I think) expect. ``make`` first executes ``$(shell ls tmp)`` but comes back with an error because the file *tmp* does not exist yet. Its return value is an empty string. Then it executes ``touch tmp`` and a file called *tmp* is created. But that's too late for our purposes. Next, an empty string is assigned to the variable called *owner*. ``make`` then believes that *owner* is a file and since there is no file called *owner* in the directory throws another error saying the same. What a mess! Unless you are ok with the behavior of running ``$(shell foo)`` before anything else, you will want to embed shell scripts in a more cumbersome but ultimately successful way. Let's rewrite our *Makefile*, :: .PHONY: shell-script shell-script: touch tmp { \ owner=$$(shell ls tmp) ; \ echo $$owner ;\ } Here we have started a shell script block within which we handle all our logic. Unfortunately, I have yet to find a way to use a variable declared in that block outside of the block in the rest of the recipe. There are serious limitations in embedding shell scripts directly in a *Makefile* but in certain circumstances where it's needed, it's certainly doable. Although, of course, you may want to write separate shell scripts and just execute them from within the *Makefile* instead. Using ``$(shell foo)`` is usually done in variables at the beginning of a *Makefile* (and *foo* is replaced by something more useful and meaningful) with the expectation that anything that replaces *foo* here acts upon information and artifacts that exist prior to running any (or all) targets of the *Makefile*. For example, :: CWD := $(shell pwd) SOMEDIR := $(CWD)/somedir .PHONY: all all: echo $(CWD) echo $(SOMEDIR) Hide Command ------------ You'll have noticed that each step in the recipe is printed on stdout. To suppress this behavior, prepend each step with *@*. Now that line will not be printed before it's executed. Your *Makefile* could look like this, :: .PHONY: cmd cmd: @echo "Only the message is printed" Run ``make`` and see that only the message is printed while the command is not, :: $ make Only the message is printed Let's remove the *@* in our *Makefile*, :: .PHONY: cmd cmd: echo "Only the message is printed" Run ``make`` again and see how the command is printed as well, :: $ make echo "Only the message is printed" Only the message is printed Default Shell ------------- ``make`` uses ``/bin/sh`` as the default shell on Linux/UNIX(-like) systems. This behavior can be overriden by overriding the *SHELL* variable. For example, your *Makefile* could look like, :: SHELL := /bin/bash .PHONY: all all: echo "I'm using $(SHELL)"