Vizmake: Debugging Makefiles via Visualization
Wenbin Fang
Daniel Crowell
wenbin@cs.wisc.edu
danieljc@cs.wisc.edu
Abstract
Build tools (e.g., GNU “make”) automate the tedious work of repeatedly invoking compilers in the
correct order with the right flags and input files by using developer-provided build description files (e.g.,
Makefiles). However, build description files can easily become very complex and buggy, as a project
evolves. There is no good built-in debugging support for build description files. Developers tend to
resort to manual inspection of the build description file.
In this project, we conducted a comprehensive Makefile bug study on open source projects that use
GNU “make” and discovered that the two common root causes for most Makefile bugs are using variables
with incorrect values and specifying incorrect dependencies in Makefile rules (i.e., missing dependencies
or extra dependencies). We developed a tool, Vizmake, to automatically visualize variable (e.g., variable
value, source of definition ...) and dependency information (e.g., potential dependency problems for
certain targets) during build time, which provides a good starting point for developers to interactively
debug problems related to incorrect variables and dependencies in Makefiles. Vizmake leverages an
instrumented GNU “make” and an unmodified strace to collect trace data during build time, and then
it analyzes the trace data and generates web pages that visualize variable and dependency information.
We conducted a case study on using Vizmake to debug real problems of open source projects including
GCC, GDB and Apache Httpd, which demonstrates the ability of vizmake to allow developers to visually
identify the root causes of bugs.
1
Introduction
The build tool plays an important role in the software development cycle, where it automates the tedious
work of repeatedly invoking compilers in the correct order with the right flags and input files by using
developer-provided build scripts. Make [4] is such a build tool that uses description files, or Makefiles,
to produce the desired artifacts. Developers write rules in Makefiles to explicitly capture “dependencies”
between “build targets” (e.g., executables, object files ...), and to specify “executable commands” to produce
1
“build targets”. Incremental compilation is supported via a timestamp-based heuristic that ensures that a
rule’s target is only rebuilt if at least one of its dependees is newer. The following text shows an example
rule in a Makefile, where “prog” is a build target that depends on three files “x.o”, “y.o”, and “z.o”. To
produce “prog”, the command in the second line should be executed.
prog : x . o y . o z . o
c c x . o y . o z . o −l S −o prog
However, the build description file can easily become very complex and buggy. The complexity of the
build description file grows as the project evolves. For example, when adding new source files, deleting
outdated source files, and supporting new platforms in a project, the build description file needs to be
updated accordingly. Build description files tend to be buggy, because they are not as well tested as source
code files that typically have well-established testing frameworks, and there are no good built-in debugging
tools for build description files. Developers tend to resort to manual inspections of build description files,
which use declarative-like language and statements that do not execute in linear order.
This project is to build tools to help developers debug build description files. Although many alternatives
for “make” and GNU “make” have been proposed, none of them have been able to overthrow the original.
Therefore, we focus on the GNU “make” program and its description files (or Makefiles) in this project.
We first conducted an in-depth study in bug databases and commit logs of open source projects (i.e.,
MySQL, Firefox, GCC and GDB), trying to understand characteristics and root causes of Makefile bugs.
We found that there are primarily two root causes for most bugs in Makefiles, incorrect variable assignments
and incorrect dependencies. First, incorrect variable assignments in Makefiles leads to abnormal behaviors
at build time. Developers often have a misconception of variable precedence in Makefiles, given the complex
variable assignment approaches (e.g., macro definition via command line overwrites variable definition in
Makefiles). Second, an incorrect dependency results in either 1) adding unnecessary dependencies that cause
unnecessary rebuilding or 2) missing necessary dependencies that break parallel building.
The common practice of debugging Makefiles is to manually inspect contents in Makefiles or trace data
from running the make program (i.e., make -pn). However, both the contents of a makefile and trace data from
running make can be voluminous and not well-structured and in these cases a manual inspection is inefficient
and ineffective. However, manual inspection of a Makefile is still indispensable for debugging, because a
Makefile is produced by human developers or generated from templates written by human developers, which
means there are tons of idioms that are hard to correctly verify through automated tools. As applied in
many subareas of software engineering [1], we believe that visualization is also able to guide the process of
manual Makefile debugging and help developers better understand Makefiles. Therefore, we designed and
2
implemented vizmake, a tool that automatically visualizes a Makefile’s build-time variable and dependency
information, which can be used to effectively guide the Makefile debugging process.
In vizmake, build-time variable information includes values of variables and the relationships between
variables (i.e., whether a variable uses another). This helps developers identify variable bugs visually, which
is also suggested in some literature, e.g., “displaying values could give developers the ability to identify
suspicious values” [8]. Build-time dependency information is used to check if the claimed dependencies in a
Makefile’s rules are consistent with the actual files accessed when executing those rules. If a dependency is
not in a certain rule but is accessed in the commands (e.g., running compiler) in that rule, then this may
mean that there is a missing dependency that should be added. On the other hand, if a dependency is in a
rule but is not accessed in the commands for that rule, then this may mean that there is an extra dependency
that should be removed.
In our implementation, vizmake automatically analyzes trace data from running a modified make program
and running the Linux system call tracing tool strace. Then, vizmake visualizes variable information and
dependency information in well organized web pages.
Finally, we evaluated vizmake by conducting a case study of using our tools to diagnose real bugs in
Makefiles of real open source software. The result shows that our tools are both efficient and effective.
This report is organized as follows. Section 2 surveys related topics on build systems. Section 3 studies
bug databases and commit logs of open source software. Section 4 describes the design and implementation
of vizmake. The case study on using vizmake to find Makefile bugs is in Section 5. We conclude in Section 6.
2
Related Work
In this section, we survey Makefiles debugging and build systems in general.
Tamrawi et al. [9] developed SYMake to help developers detect code smells and errors in makefiles.
SYMake does static analysis on makefiles and visualizes symbolic dependency graphs for targets, files, and
variables. The key departure points in our vizmake in comparison to SYMake is that we analyze and visualize
trace data collected during build time, which faithfully reflect the actual values of variables in a makefile at
build time.
Adams [2] developed MAKAO to understand the build system. MAKAO reads the trace output from
“make -p”, and it visualizes the dependency graph for targets and files. By contrast, our vizmake utilizes
the trace data from our modified version of the “make” program, which provides more accurate data that
fit our needs. Moreover, our vizmake visualizes variables in Makefiles, which enables finding variable bugs.
Bernstein [3] developed Remake which can be used to debug Makefiles. Remake helps with identifying the
3
specific place of build errors by adding line number information on outputs as well as a chain of commands
in the makefile that led to an error. GNU “make” has simple error checking for semantic errors, but its
output is usually limited or not helpful. Another way Remake assists in debugging is by expanding variables
and other data in a Makefile to make understanding it easier. Unlike Remake, we will directly present visual
representations as well as potential errors in the dependences of a Makefile rather than providing tools for
others to manually find this information.
Johnson et al. [5] looked at ways to improve the build process. They used Makepp
1
that provided more
accurate builds and dependency info compared to a similar build process with GNU “make”. This was
especially beneficial for making rebuilding more reliable with fewer portions retouched with the same input
rules. The visualization provided by our vizmake can also help developers understand the build process, thus
achieving a similar goal as Johnson et al.
Jorgensen [6] provides a theoretical foundation for verifying that Makefiles are safe and produce the same
output when partially rebuilding a project compared to building it from scratch. This includes ways to check
for soundness and completeness of dependencies. In the future, we can consider incorporating their work
into vizmake.
3
Bugs Study
As a feasibility study for developing debugging tools for Makefiles, we conducted a study on four open source
projects, including Apache httpd, GCC, Firefox, and GDB, which are real projects with a wide sample of
bugs covering many years of changes and commits. From these projects we analyzed their bug databases and
commit logs for Makefiles to identify a variety of bugs that have been encountered and the corresponding
root causes. This gives us a better idea of what are widely encountered Makefile bugs along with some
examples with which to test the tools we developed.
Through the study, we discovered that there are two main types of causes for Makefile bugs. The first
type is incorrect variables where a wrong value or no value is assigned, leading to abnormal behaviors at
build time. The second type is incorrect dependencies that are either 1) adding unnecessary dependencies
that cause unnecessary rebuilding or 2) missing necessary dependencies such as those that show up in parallel
building. Table 1 shows a sample of Makefile bugs that were found in this study as well as the root causes
of them. Five bugs are due to incorrect variables and three are dependency bugs.
The problem with undefined or invalid variables was seen repeatedly in our study, which lead to abnormal
behavior of the build process (e.g., a generated file is not deleted via “make clean” in Bug #26552). Finding
1 http://makepp.sourceforge.net/
4
Project
Apache
Apache
Apache
GCC
GCC
GCC
Firefox
GDB
GDB
Bug
A generated file is not deleted
by “make clean”
Copy files from a wrong path
Abnormal build behavior
Rebuild the CNI header files
multiple times with with -j
builds.
Generating invalid man page
Does not recompile source files
that include io.h
Abnormal clean behavior
Refactoring request
Source
Bug Database #26552
Bug Database #7790
Commit Log on Jan 9 2004
Bug Database #29278
Bug Database #256608
Bug Database #27107
Commit Log on March 14 2008
Bug Database #10649
Unexpected DESTDIR variable
Bug Database #7786
Root Cause
Variable CLEAN TARGETS
doesn’t include that file
a NULL $(top blddir) variable
Redundant
PHONY TARGETS
variable definition
Incorrect dependency
Missing dependency on gccvers.texi
Make dependency on io/io.h
broken
Missing dependencies on clean
Undefined
variable
GDB DATADIR PATH
DESTDIR defined via command line argument is not used
Table 1: Sample Makefile Bugs and Root Causes
Project
GCC
Firefox
Bug
Dead targets
Abnormal build behavior
Bug Source
Commit Log July 18 2004
Bug Database #255872
Firefox
Unused directory created
Commit Log on on Sept 17 2009
Root Cause
Targets never be used
Typo on a path (not in variable)
Extra remove command usage
Table 2: Other Makefile Bugs
variable-related bugs requires the knowledge of developers, because variables contain semantics and only
developers know what values are correct at a given scenario. We developed vizmake to visualize variable
values and the relationships between variables, which helps developers manually and visually inspect the
value of variables and locate erroneous variables.
Examples we found where developers corrected dependency bugs always involved adding missing dependencies, or removing unnecessary dependencies. A common case where dependency-related bugs are
encountered is when building a project in parallel where alternate ways of stepping through the commands
can uncover faulty dependencies. To inspect dependency-related bugs, developers can start with automatic
approaches by following simple rules, e.g., if main.o relies on foo.c, then foo.c must be used to build main.o,
and then verify the rule violations. Therefore, we developed vizmake to automatically detect potential
incorrect dependencies in Makefiles.
In addition to the two major types of root causes for Makefile bugs listed in Table 1, three other types
were found, which are shown in Table 2.
The first type is a dead target in a Makefile that is never used, where it has nothing to do with dependencies, so it does not belong to dependency bugs. The second type is a simple typo in a dependency
list of a rule in Makefile, where this typo is not a value assigned to variables, so it does not belong to the
variable-related bugs. The last type is the creation of a blank directory in a make file that was never used,
5
due to an extra directory creation command in the rule. Building tools to inspect them would be great
future work.
4
VizMake
This section first describes an overview of vizmake, and then we present the design and implementation
details.
4.1
Overview
Through the bug study in Section 3, we know there are two major root causes for Makefile bugs, namely,
incorrect variables and incorrect dependencies. Therefore, we design vizmake to make finding these two types
of root causes easy.
Vizmake is aimed to automatically produce visualization that faithfully displays Makefile variable and
dependency information to developers, allowing developers to effectively and efficiently identify abnormal
variables and dependencies.
To achieve this goal, vizmake collects trace data during build time, using a modified GNU “make”
(denoted as make’ ) and unmodified strace. The visualization is automatically generated right after the
build process is complete via make’. Figure 1 summarizes the major workflow of vizmake.
Data Collection
Run make`
and strace
Visualization
Trace Data
Visualization Engine
Developer
Figure 1: Workflow of vizmake.
We implemented Vizmake in Python. Vizmake launches make’ and strace to collect trace data, and
then it generates web pages to display visualization. The debugging process is highly interactive, so our
visualization allows developers to interactively manipulate the visualization inside a browser. Developers
run vizmake in the same way they run GNU “make”. They can pass any GNU “make” command line
arguments to vizmake. Vizmake performs normal build process of GNU “make” with trace data generated
to /tmp directory. At the end, trace data is analyzed and visualized by generating web pages with D3 [7]
javascript code. Vizmake starts a simple web server to host the visualization web pages, and developers use
6
a web browser to view and interact with the visualizations.
Figure 2 shows an example index page for the generated visualization. The index page mainly visualizes
the hierarchy of processes involved in the build process, so developers can have a basic idea of how the build
process works. A “make” process is also associated with three links for showing the complete command line
used for this process (CMD), the variable visualization (VAR), and the dependency visualization (DEP).
Figure 2: Workflow of vizmake.
In the following two subsections, we will describe the details about visualizing variables and dependencies,
respectively.
4.2
Visualizing Variables
Makefile variables are complex in how they are defined. There are two ways for a variable to be expanded.
The first way is to recursively expand a variable that is defined by lines using “=” or by the “define”
directive. Such a variable recursively expands all referenced variables only when itself is referenced, that is,
the value expansion is on demand. The second way is to simply expand a variable that is defined by lines
using “:=”. The value of a simply expanded variable is scanned once, expanded immediately, and used for all
references. It is further complicated when a variable is defined outside a Makefile (e.g., environment variable
or command line arguments) or is overrode by other definitions (e.g., variable definition from command line
takes higher precedence than from Makefile). Makefile-writers’ misconceptions about variable definitions are
typically the source of these bugs.
The result of our study on Makefile bugs in open source projects in Section 3 guided us to design vizmake
in a way that provides enough variable information to facilitate developers in visually identifying abnormal
variables. Specifically, vizmake should help developers answer the following questions:
7
• What is the value of a particular variable being used in a particular place? In common
programming practice, we oftentimes use a debugger only to inspect the value of some variables.
Similarly, faithfully displaying variable values at build time is highly desirable.
• Where is a referenced variable defined? This helps developers determine whether or not the
variable definition comes from the source they expect.
• For a variable, how does its value expand? In other words, what other variables are referenced
in a variable’s definition.
4.2.1
Data Collection
Similar projects that visualize Makefiles use the trace data either from running GNU “make” (e.g., make
-p) [2] or from statically analyzing the Makefile itself [9]. However, trace data from running “make p” or
statically analyzing the Makefile is incomplete and does not expand values for recursively expanded variables
(i.e., variables assigned by “=”). In addition, neither approach is able to determine the source of a variable
definition (i.e., from a Makefile, command line or environment variable).
To faithfully reflect Makefile variables at build time, we decided to modify the GNU make program to
meet our needs, which now outputs trace data of various variable information such as when a variable is
referenced, including variable name, variable value, what other variables are referenced by a variable, which
line in a Makefile references what variables, and where the referenced variable is defined (e.g., at which line
in which Makefile, or from command line).
4.2.2
Visualization
We use an indented tree as the major visualization style in vizmake, because indented trees are widely-used
to represent hierarchical structures like file systems, and what vizmake shows is inherently hierarchical too:
a Makefile contains lines of texts; a line in the Makefile references some variables; a variable may reference
other variables too. Although indented trees require much vertical space, they do allow efficient interactive
exploration of the tree to find a specific node. In addition, the indented layout allows rapid scanning of node
labels, making bug finding easy.
Figure 3 shows the visualization for a Makefile involved when building an open source project lighttpd.
The root node (in blue) in the indented tree displays the full path of a Makefile in its label. There are two
child nodes (in white) of the root node, where the subtrees rooted at them represent information of included
files and lines of text that reference variables, respectively. The orange node represents a line of text, where
the label displays the line number and the text. The children of the “line” node represent variables referenced
8
Figure 3: Visualizing Makefile variables.
at the current line, whose label shows the name of the referenced variable, the corresponding value, and the
source of definition for this variable. If a referenced variable can be simply expanded, then its value is shown
in the node’s label directly; otherwise, its value is shown inside a yellow floating tooltip, and all referenced
variables are child nodes, if there is any. A red node represents a variable whose value is not from within
Makefile, which means a variable is either undefined or defined outside the Makefile (e.g., from command
line). We highlight such a node (in red) because if a variable has a value not from the Makefile, then it may
not be anticipated by Makefile-writers, which may possibly be a bug.
In the same way that debug points are mainly used to inspect variable values , the visualization in vizmake
provides developers an effective way to visually inspect Makefile variables.
4.3
Visualizing Dependencies
Vizmake is also aimed to provide suggestions to developers about possible dependency problems in the Makefile. Specifically, vizmake holds an invariant: given a rule, the set of files accessed in the command processes
should be the same as the set of files the target depends on. Vizmake should report extra dependencies,
missing dependencies and other related information to developers, who have the final word on the correctness
of dependencies.
9
4.3.1
Data Collection
To report potential dependency problem, vizmake requires two sets of files, files that are actually read in the
commands to build that target (“accessed files”) and files that are declared as prerequisites for building a
target (“dependencies”).
Vizmake uses strace to trace system call information for running “make” and for all processes created to
build targets, in order to get data about “accessed files”. Specifically, we monitor the open system call to
know all files accessed. We also monitor process creation system calls (i.e., fork, vfork, and clone to get all
child PIDs, which are used in a later analysis stage to group all accessed files for the same rule.
Vizmake uses make’ to collect data about “dependencies”. The make’ also outputs data of child processes
that are created to build a particular target, which is used to match the data collected by strace.
4.3.2
Visualization
Dependency problems include missing dependencies and extra dependencies. Let the set of “dependencies”
be D and the set of “accessed files” be A, then the set of missing dependencies M = A \ D (the set difference
between A and D), the set of extra dependencies E = D \ A (the set difference between D and A), and the
set of correct dependencies C = D ∩ A (the set intersection of D and A).
The set of “dependencies” can be obtained accurately from running make’. However, it is challenging to
define the set of “accessed files”. It is tempting to simply define “accessed files” as all files ever successfully
opened in child processes of the make process. However, this definition would yield many noises that lead to
false missing dependencies or false extra dependencies. After our careful investigation, we list all noises of
“assessed files” and the corresponding alleviations to make the “accessed files” definition accurate in practice.
• Intermediate files. A rule may generate intermediate files, which would be accessed but not be part
of the dependency list in that rule. If we count these files as “assessed files”, then we would get false
missing dependencies. Therefore, vizmake excludes those files that are ever opened as write-only files
from the set of “assessed files”.
• Standard libraries and header files. Standard libraries, header files and other files in standard
linux directories are typically read-only and are not specified as dependencies in a Makefile rule. If
we count them as “assessed files”, then we would get false missing dependencies. Therefore, vizmake
excludes files with prefixes like “/lib”, “/usr/lib”, “/proc” etc.
• Directories. Some commands may access directories using the open system call. Typically, directories
are not specified as dependencies in a rule. If we count these directories as “assessed files”, then we
10
would get false missing dependencies. Thus, vizmake excludes “files” ending with “/”, “.” and “..”.
• Other phony targets as dependencies. A rule may have other phony targets as dependencies. If
we count them as “assessed files”, then we would get false extra dependencies, because they are not
actual files and are not accessed. There is not any good way to distinguish non-existing files from
phony targets, so we resort to letting developers manually discriminate this case.
• Empty dependencies of phony targets. A rule may have no dependency for a phony target. For
example, it’s very common to have an “all” target with no any dependency. We would always get false
missing dependencies for such rules. There is not a good way to distinguish non-existing files from
phony targets, so we resort to letting developers manually discriminate this case.
• Executable files. Executable files that are executed in commands are tricky to handle. Oftentimes,
they are not accessed by the open system call, so we do not count them as “assessed files”, which may
lead us to get false extra dependencies. Therefore, we also need to use strace to record system calls like
exec. However, if we count them as “assessed files”, then we may get false extra dependencies, since
the executable dependencies may not be specified in Makefiles but rather in configuration scripts. To
alleviate the problem, we count executable files as “assessed files” only when false extra dependencies
occur.
Although false negatives are significantly reduced after applying these heuristics, they cannot be totally
eliminated. However, the result from running vizmake can be used as a good starting point for manual
sophisticated analysis. Vizmake provides as much information as possible for developers. For example, we
provide separate CMD pages to show what commands are executed to generate certain targets, and what
arguments are passed to the commands.
Figure 4 shows the visualized dependency analysis result. An orange node represents a rule that generates
a target, whose child nodes are the dependency analysis result. A red node is a missing dependency, a yellow
node is an extra dependency, and a green node is a correct dependency.
5
Case Study
In this section, we conducted a case study of evaluating our tools for debugging. In the evauation, we wanted
to answer this main question: can vizmake help find root causes of real bugs in real open source projects?
We used real bugs from GCC, GDB and Apache Httpd for our evaluation, which are summarized in
Table 3.
11
Figure 4: Visualizing Makefile Dependencies.
5.1
GDB VAR1
This “bug” is in fact reported as a refactoring request to replace an undefined variable GDB DATADIR PATH.
Vizmake can be used to help verify this refactoring request, inspecting whether or not GDB DATADIR PATH
references are undefined.
First, we looked for the references of GDB DATADIR PATH in all Makefiles involved, because we wanted
to know where this variable is referenced by which other variables or in which lines in a Makefile. Therefore,
we can have a better idea on how this variable is used in the Makefile at build time. Vizmake provides an
ID
GDB VAR1
Project
GDB 6.8
Bug Description
Bug 10649: refactoring request
GDB VAR2
GDB 5.2
Bug 7786: DESTDIR variable
from command line is not used
APACHE VAR1
Apache Httpd 2.0.48
GCC DEP1
GCC 4.2.0
Bug 26552: The file server/export files is generated during
the build process, but neither a
“make clean” nor a “make distclean” will remove it.
Bug 27107: don’t recompile
source files for libfortran component that include io/io.h
when io/io.h is modified
Table 3: Bugs Used in Evaluation
12
Root Cause
GDB DATADIR PATH
is
used, but never defined
DESTDIR value is not passed
from command line argument
to child make process
Wrong
value
in
CLEAN TARGETS variable
Missing dependency on io/io.h
intuitive indented tree structure to present variable reference information.
Second, we verified whether or not GDB DATADIR PATH is undefined. Vizmake highlights those variables that are not defined from Makefiles. Therefore, it is easy to identify that all GDB DATADIR PATH
references are undefined.
Figure 5: Screenshot for GDB VAR1, identifying undefined GDB DATADIR PATH reference.
Figure 5 shows a screenshot for an undefined GDB DATADIR PATH reference. This test case suggests
a good use of vizmake in refactoring Makefile, which is often reported in bug databases.
5.2
GDB VAR2
This bug is that when invoking “make DESTDIR=/some/dir install”, the value of “/some/dir” is not
assigned to the variable DESTDIR when it is referenced. The variable assignment precedence of command
line argument is higher than the assignment in Makefiles, so the correct result should be that DESTDIR is
assigned to “/some/dir”.
To debug this problem using vizmake, we performed these steps:
First, we looked for the references to DESTDIR and checked the value of DESTDIR and the source
of its definition when it was referenced. As shown in Figure 6, we can see that the value for a reference
of DESTDIR was clearly not “/some/dir” and it was defined in the Makefile instead of the command line
argument.
Second, we questioned whether the definition of DESTDIR used “override” directives, which provide the
13
way to override a definition from command line arguments. So, we looked into Line 35 in the Makefile to
check the definition of DESTDIR, which didn’t use the “override” directive.
Third, we suspected that command line argument was not passed to the child make process. So, we
checked the command line arguments for all make processes via CMD pages described in Section 4.3.2, and
confirmed our suspicion – child make processes did not pass “DESTDIR=/some/dir” via command line
arguments.
Figure 6: Screenshot for GDB VAR2, identifying wrong value of DESTDIR reference.
5.3
APACHE VAR1
This bug is that the file server/export files generated during the build process is not deleted by a “make
clean” nor a “make distclean”. We diagnose this problem using vizmake by the following steps:
First, we know that the “make clean” error appears within the directory server, so we only run “vizmake
clean” in the directory server.
Second, we check the rm commands invoked by “make clean” to remove generated files, and find that
“export files” is missing in the command line arguments of rm commands.
Third, we check the command line specified in the Makefile from the web browser, and find that the text
references a variable CLEAN TARGETS, which does not include “export files”. By this point, we find the
root cause for this bug.
14
Figure 7: Screenshot for APACHE VAR1, identifying wrong value of CLEAN TARGETS reference.
5.4
GCC DEP1
This bug is that when the file io.h in GCC’s libfortran component is modified after an initial build, the
modifications to io.h do not cause the libfortran component to be rebuilt. We diagnose this problem using
vizmake through the following steps:
First, we know “make” does not work correctly when building the libfortran component, so we only run
vizmake in the build directory of libfortran.
Second, we look at the dependency visualization page and search for io.h. We find that io.h is a missing
dependency for several targets, as shown in Figure 8.
Third, at the bottom of this dependency visualization page, we find the line number in the Makefile that
specifies a particular rule with missing dependencies. Therefore, we can go back to the Makefile and edit
the corresponding lines to add the missing dependency on io.h.
6
Conclusion
In this project, we developed vizmake to facilitate debugging Makefiles via visualization. Our initial evaluation result is encouraging, which proves that vizmake is able to visually present the root causes for real bugs
in real open source projects.
Vizmake can be easily extended to visualize more information that helps debugging Makefiles, in addition
to variable and dependency information. For example, we can visually highlight dead rules that are never
15
Figure 8: Screenshot for DEP VAR1, identifying missing dependency on io.h .
executed, or we can detect parallel building bugs by examining the order of rules executed.
In addition to bug finding, we envision that Vizmake can be applied to many other applications. First, it
can be used in the process of writing Makefiles, so that developers can examine and understand a Makefile
they write, reducing the chance of introducing bugs. Second, it can be used to help refactor Makefiles, as
we demonstrated in the first test case in the case study. Third, it can be used to understand a big project’s
structure for security analysis or other purposes.
The source code of vizmake is available online: https://github.com/wenbinf/vizmake .
References
[1] ACM. SoftVis. http://www.st.unitrier.de/d̃iehl/softvis/org/index.php.
[2] B. Adams. Co-evolution of Source Code and the Build System: Impact on the Introduction of AOSD in Legacy Systems.
PhD thesis, Ghent University, 2008.
[3] R. Bernstein. Debugging makefiles with remake. In 25th Large Installation System Administration Conference, 2011.
[4] S. I. Feldman. Make — a program for maintaining computer programs. Software: Practice and Experience, 9:255–265,
1979.
[5] A. Johnson and G. Holt. s is for source: The role of the build system in configuration management. Technical report,
NVIDIA, 2005.
[6] N. Jørgensen. Safeness of make-based incremental recompilation. FME 2002: Formal MethodsGetting IT Right, pages
126–145, 2002.
[7] Michael Bostock. D3.js. http://d3js.org/.
16
[8] C. Parnin and A. Orso. Are automated debugging techniques actually helping programmers? In Proceedings of the 11th
International Symposium on Software Testing and Analysis, 2011.
[9] A. Tamrawi, H. A. Nguyen, H. V. Nguyen, and T. N. Nguyen. Build code analysis with symbolic evaluation. In Proceedings
of the 2012 International Conference on Software Engineering, 2012.
17