Let’s play with make (part 2)

Context, reloaded

As you might remember from the previous post, the idea is to produce pdf documents from tex sources. The sources for the different types of documents are named after the problem and the source type, e.g. problem_name-answers.tex.

In addition, the same output might be published for different classes or courses: this only changes the title headers and is called a “flavor”. The flavored header is included upon compilation, from any directory where LaTeX might automatically find it [1], usually ~/texmf/tex/latex/helpers/ for me. In the previous post, these files were not managed as prerequisites, though they obviously are. We will address this problem in this post using the makefile include directive, but let us first introduce this directive with something simpler.

DRY: Don’t Repeat Yourself

Most programming projects make heavy use of the include directive, for example “calling” makefiles in subdirectories to make subparts of the project, or allowing to share a common set of variables.

We will use it to factor out some parts of the makefile which might be used in another ones, like the helpers, to avoid repeating ourselves in every makefile. We defined in part one this helper target:

print-%:
    @echo '$*=$($*)'

It is obvious that it can prove useful in any makefile we could write. We can then put it in another makefile, with maybe other useful and common targets, and include it in all makefiles where we would need it. As a convenience, if the included makefile does not start with a slash (i.e. if it is not a full path), and it is not found in the current directory, then make will search for it in other directories. The most usual one is probably /usr/local/include, see the include directive documentation for the other ones, or for the include-dir option.

I thus created a helpers.mk in /usr/local/include, and replaced the above snippet with:

include helpers.mk

You might consider replacing include with -include, the latter failing silently if helpers.mk, or any of its prerequisites did not exist or could not be remade.

Prerequisites

Speaking of prerequisites, let us come back to the ones we did not manage in the previous post. As I explained, and again to avoid repeating myself, most of the target problem tex files are structured that way:

%================================================
%  abc-text.tex
%===============================================

\input{helpers/preamble-en}

\input{meta}

\begin{document}

\markright{\thetitle}
\pagestyle{myheadings}

\input{helpers/title-FLAVOR}
\maketitle

\input{src}

\end{document}

This example file includes 4 other tex files, namely helpers/preamble-en.tex, meta.tex, helpers/title-FLAVOR.tex and src.tex. In the makefile we built so far, only the ones which are local to the current directory are managed, meta.tex and src.tex.

What I want to achieve is managing the chain of all included tex files, i.e. the 4 ones listed above, but also the ones included in those [2].

As it happens, this is not a very original idea: including files in sources is rather ubiquitous in all programming languages and building systems. The odds were great that something existed to manage this, and was documented: we will indeed replicate what is described in Generating Prerequisites Automatically, but for tex files.

The idea is to have an external program generate for us a line with the target file and the whole chain of its prerequisites. The external program actually exists for c sources, as the cc compiler can do this, but I could not find any equivalent for LaTeX sources, so I wrote one. It is a short python script which recursively builds a list of those files, and then outputs it in the required format [3].

On the example file above, the python script works this way [4]:

$ latex-depends.py abc-text.tex
$(BUILD)/abc-text.tex $(BUILD)/abc-text.d : helpers/preamble-en.tex \
helpers/preamble-standard-packages.tex helpers/preamble-newcommands.tex \
$(BUILD)/meta.tex helpers/title-std.tex $(BUILD)/src.tex

You probably noticed that FLAVOR was replaced by its current value, which was the default one (std) as we did not provide any. You also notice that the script actually recursed, as helpers/preamble-standard-packages and helpers/preamble-newcommands are not directly included in abc-text.tex.

In addition, there are two targets: the source file itself, and a file with the same name but with the .d suffix (a common suffix for files describing “depends”). The latter contains the prerequisites line itself, and is the one we will include in the makefile. Making it depend on all the prerequisites ensures that it will be rebuilt if any of those changes.

Finally, the target and local source names are prepended with the $(BUILD) variable, as they should be: the pdf final output actually depends on the sources being in the $(BUILD) directory.

How do we use this now?

The makefile will know how to generate a .d depends file (the prerequisites line) from a tex file through the use of this pattern rule:

$(BUILD)/%.d: %.tex | $(BUILD)
    FLAVOR=$(FLAVOR) $(BIN)/latex-depends.py $< > $@

As any intermediary target needed for the final compilation, the depends file is generated in the $(BUILD) directory.

We now require a depends file to be generated and included for each of the $(SOURCES) files through a regular include:

-include $(patsubst %.tex,$(BUILD)/%.d,$(SOURCES))

Now that we have a good way to track all dependencies for the $(SOURCES) files, we can simplify the makefile we did in part 1, as some variables and sources are no longer needed. Instead of considering any tex file in the source directory which is not in the $(SOURCES) as a dependency, we now have an exact list of dependencies (the .d file) for each of the source files.

A final touch: the final version of the makefile contains a cleanall phony target, which simply destroys (rm -rf) the $(DEST) and $(BUILD) directories. As the above include will rebuild the depends files if necessary whatever the target, invoking make cleanall will make sure that $(BUILD)/*.d files exist, right before removing them. Ahem.

This small problem is common enough to be used in the documentation as an example for the special variable MAKECMDGOALS, see Arguments to Specify the Goals. The trick is to test that the goal (the target specified on the command line) is not cleanall before including the depends files:

ifneq ($(MAKECMDGOALS),cleanall))
-include $(patsubst %.tex,$(BUILD)/%.d,$(SOURCES))
endif

That’s all for today, happy make-ing!