Source Code Management
Rob Baxter, Project Manager
Source Code Management
4Code management
– Why bother?
4Build management using ‘make’
– The basics
– The art of writing makefiles
4Revision control
– What it is, why we should use it
– Common RC tools: SCCS, RCS, CVS, SourceSafe, ClearCase
2
Advanced Programming: Tools and Techniques - Session 2
1
Code management
4Even a modest-sized program is a complex thing
4Complexity enters at a number of levels
–
–
–
–
Understanding the workings of the algorithm
Designing a software project to make the algorithm work
Mastering the syntax of the programming language
Managing the program module dependencies and compiling the
right bits
– Managing the source code tree
– Delivering working programs to your “customers”
4By “code management” we mean ways and
means of addressing the last three
3
Advanced Programming: Tools and Techniques - Session 2
Code management
4The two most important things to do before
writing code:
1. Set up a Revision Control system
2. Write a makefile
4Revision control and make should be at the
heart of your software project
– Control of software changes
– Control of software compilation and building
– Control of software release mechanisms
4
Advanced Programming: Tools and Techniques - Session 2
2
Build management using ‘make’
4What is ‘make’?
– A tool to “maintain, update and regenerate related programs
and files”
4What does it do?
– Manages dependencies between source, object and
executable files
– Regenerates files, according to customised or default rules,
when other files upon which they depend change
4Dependencies between files, and rules for
regenerating files, are stored in a makefile
4Writing makefiles is an art, but well worth it
5
Advanced Programming: Tools and Techniques - Session 2
Why use ‘make’?
4Compiling a small program is easy
– cc -o hello hello_world.c
4Compiling larger programs is harder
– cc -o scheduler main.o schedule.o optimise.o io.o
utils.o -I./include -L./lib
-lm -lblas -llapack -lrng
– Where io.c depends on ./include/io.h, and schedule.c
and optimise.c depend on ./include/schedule.h
– (which depends on ./include/structs.h, as does main.c)
– and librng.a is a custom library built from Fortran source with
f90 and ar, upon which everything depends
– and...
6
Advanced Programming: Tools and Techniques - Session 2
3
Why use ‘make’?
4Building a computer program soon gets complex
– Even typing make is easier than cc -o hello hello_world.c
4‘Make’ allows you to write these instructions once
and use them many times
– Instructions on how to build the executable from source, objects,
libraries etc.
– Instructions on which source must be recompiled when a certain
header file changes
– Instructions on how to build independent libraries
– Instructions on multistage compilations (eg. .F → .f → .o,
or .tex → .dvi → .ps → .pdf)
7
Advanced Programming: Tools and Techniques - Session 2
Why use ‘make’?
4Only rebuilds what’s necessary
– Important for 1000-subroutine programs...
4Will rebuild what’s necessary
– No linking against old object code and wondering why your
program is misbehaving
4Reduces overhead of edit-compile-run
4Means others can understand how to build
your program if you’re hit by a bus...
8
Advanced Programming: Tools and Techniques - Session 2
4
The basics of using ‘make’
4make [target]
– target is something defined in the makefile that you want to build
• A program, a library, a document, a “configuration”, anything
• No specified target means make looks for the first one in the makefile
– By default, make looks for a makefile called makefile, or
Makefile
– Can use alternatives with make -f Makefile_new target
4The makefile contains build instructions of
essentially three types
– Macro definitions
– Target definitions
– Implicit rules
9
Advanced Programming: Tools and Techniques - Session 2
Macro definitions
4Simply define symbols for later use
– OBJ = main.o schedule.o optimise.o
4Refer to macros using ${...} or $(...)
– OBJ = ${FOBJ} ${COBJ}
4Many macros predefined
– CC, CFLAGS, FC, FLAGS, LDFLAGS...
4Special pattern substitution macros
– SRC = ${OBJ:.o=.c}
• “SRC = everything in OBJ with the .o’s replaced by .c’s”
– Be warned: these can vary from ‘make’ flavour to flavour
10
Advanced Programming: Tools and Techniques - Session 2
5
Macro definitions
4Numerous special symbols with mystic meanings
– Called ‘dynamic macros’
– Definitions can vary from ‘make’ to ‘make’
• Check the man pages!
4Solaris ‘make’ defines these common ones:
– $@
• The name of the current target
– $<
• The name of the current dependency file in an implicit rule (qv)
– $*
• The basename of the current target in an implicit rule (i.e. with any
suffix and prefix stripped off)
Advanced Programming: Tools and Techniques - Session 2
11
Target definitions
4A target is a “thing to make”
4Target definitions can indicate
– “when to make” – the dependency clause
– “how to make” – the build rule
4Example:
scheduler: ${OBJ} Makefile
${CC} ${CFLAGS} ${OBJ} ${LDFLAGS} -o $@
4Generally:
NB: Hard tab
target: dependencies
_______build rule
12
Not eight spaces, a
hard tab!
Advanced Programming: Tools and Techniques - Session 2
6
Target definitions
4No dependencies = always do this
clean:
rm -f ${OBJ} ${TMP}
4No build rule = dependency chain
kernels.o: numerics.h structs.h
${OBJ}: ${INC}
${OBJ}: Makefile
– If the RHS files change, the LHS must be updated
– The ‘how’ for the LHS may be defined elsewhere, or by implicit
rules
13
Advanced Programming: Tools and Techniques - Session 2
Implicit rules
4Implicit rules define general ways of rebuilding
files of type A from files of type B
4Two types: pattern matching and suffix rules
4Suffix rules are generally more common:
– General form:
.fromSuffix.toSuffix:
_______build rule
– Example:
.c.o:
${CC} ${CPPFLAGS} ${CFLAGS} -c $<
14
Advanced Programming: Tools and Techniques - Session 2
7
Implicit rules
4Pattern matching rules are more flexible:
– General form:
tp%ts: dp%ds [other dependencies ...]
_______build rule
– Example:
%.html: src/%.xhtml ../dtd/main.dtd Makefile
xparse $< > $*.html
– Key here is that general dependencies for all .html files can be
included
4Many implicit rules are predefined by ‘make’
– Typically in /usr/share/lib/make/make.rules
15
Advanced Programming: Tools and Techniques - Session 2
A sample makefile
CFLAGS = -g
CPPFLAGS = -I./include -DDIAGNOSTICS
LDFLAGS = -L./lib -lrng -lm
OBJ = main.o schedule.o optimise.o io.o utils.o
INC = schedule.h io.h structs.h
SRC = ${OBJ:.o=.c}
${OBJ}: ${INC} Makefile
scheduler: ${OBJ}
${CC} ${CPPFLAGS} ${CFLAGS} ${OBJ} ${LDFLAGS} -o $@
clean:
rm -f scheduler ${OBJ}
.c.o:
${CC} ${CPPFLAGS} ${CFLAGS} -c $<
16
Advanced Programming: Tools and Techniques - Session 2
8
Adding complexity
4Makefiles can start simple and grow as needed
LDFLAGS = -L./lib -lrng -lm
LIB = lib/librng.a
scheduler: ${OBJ} ${LIB}
${CC} ${CPPFLAGS} ${CFLAGS} ${OBJ} ${LDFLAGS} -o $@
LOBJ = lib/uni.o
${LIB}: ${LOBJ}
ar -r $@ ${LOBJ}
.c.o:
${CC} ${CPPFLAGS} ${CFLAGS} -o ${<:.c=.o} -c $<
17
Advanced Programming: Tools and Techniques - Session 2
Adding complexity
4And thus...
$ make scheduler
cc -DDIAGNOSTICS -g -o main.o -c main.c
cc -DDIAGNOSTICS -g -o schedule.o -c schedule.c
cc -DDIAGNOSTICS -g -o optimise.o -c optimise.c
cc -DDIAGNOSTICS -g -o io.o -c io.c
cc -DDIAGNOSTICS -g -o utils.o -c utils.c
cc -DDIAGNOSTICS -g -o lib/uni.o -c lib/uni.c
ar -r lib/librng.a lib/uni.o
cc -DDIAGNOSTICS -g main.o schedule.o optimise.o io.o utils.o
-L./lib -lrng -lm -o scheduler
$
4If unsure of what ‘make’ will do, use ‘make -n’
18
Advanced Programming: Tools and Techniques - Session 2
9
The benefits of ‘make’
4Using ‘make’ gives you control over your
program
4Also gives others control when necessary
4Learn to understand others’ makefiles
4Learn to write your own makefiles
4A makefile is a “first point of contact” for people
coming new to a big program
19
Advanced Programming: Tools and Techniques - Session 2
‘make’ and project-based IDEs
4“I use MS Developer Studio; I don’t need to
care about make”
– Typical IDEs use “project files” and “workspaces”
– Underneath they usually use ‘make’
• e.g. MS Developer Studio → nmake
– Makefiles are generated automatically from the project files
and settings
– Normally you don’t have to care about them...
– ...unless cross-platform portability is an issue
– Then the project file is useless and you need to understand the
makefile
• Often non-standard, but essential structure is there
20
Advanced Programming: Tools and Techniques - Session 2
10
A word on Configuration
Management
4Configuration management includes
–
–
–
–
Revision control
Change & bug tracking
Build procedures
Release mechanisms
4 “Configuration Management is a discipline applying technical and
administrative direction and surveillance to identify and document
the functional and physical characteristics of a configuration item,
control changes to those characteristics, record and report change
processing and implementation status, and verify compliance with
specified requirements.”
IEEE-Std-610 (revision and redesignation of IEEE-Std-729-198)
21
Advanced Programming: Tools and Techniques - Session 2
Revision control
4Revision control ≠ configuration management,
but it is an important aspect of it
4RC maintains a history of your source files
– Records changes over time in a retrievable way
– Developers can (and should!) add comments to each revision,
giving a documented history of the source code
4Uses concept of code Repository
– A safe storage area away from day-to-day coding
– Repository is “public”; working copies are “private”
22
Advanced Programming: Tools and Techniques - Session 2
11
Revision control
4Working code is “checked out”
– Developer works on checked out versions
– Checked out versions are “private” and “local”
4Revisions to be saved are “checked in”
– Checked in versions are “published” in the Repository
– Revisions can be commented and documented at check-in
4Multiple developers can work together
– Can work on different parts of code straightforwardly
– Can work on same part of code
• Different RC systems handle this differently
• File locking, change merging and conflict resolution, etc.
• Talking to co-developers is still a must!
23
Advanced Programming: Tools and Techniques - Session 2
Revision control tools
4SCCS
4RCS
4CVS
4Sun Microsystems’ TeamWare
4Microsoft Visual SourceSafe
4Rational ClearCase
4Many others
– PRCS, Bitsccs, PVCS, JavaSafe, ......
– Often embedded in development environments
24
Advanced Programming: Tools and Techniques - Session 2
12
SCCS
4Source Code Control System
4Venerable UNIX revision control system
–
–
–
–
Originally written by AT&T
Still found as part of most proprietary UNIX flavours
Fundamental part of Single UNIX Specification (Open Group)
Free versions do exist, as do SCCS → RCS converters
4Very like RCS (q.v.) in structure and use
– Uses “local” repository, operates on single files, etc.
4http://www.cyclic.com/cyclic-pages/sccs.html
25
Advanced Programming: Tools and Techniques - Session 2
RCS
4Revision Control System
4SCCS-like, GNU-based, free, widely available
4Works with “local” repository
– Safe copies kept in ./RCS or .
– Can fake global repository using, eg., symbolic links
4Operates on single files
– Users check in/out one file at a time
4Stores latest file plus sequence of diffs
4Allows file “locking” to minimise conflicts
26
Advanced Programming: Tools and Techniques - Session 2
13
RCS
4Basic RCS commands:
– ci - “check in” a revision or initial file to the repository
• Places file in ./RCS or in .
• Repository files are stored as <filename>,v
– co - “check out” a repository file
• Creates working copy, although with read-only permission
• To enable editing for a file, lock it using co -l <filename>
• Retrieve specific versions with co -r<rev-number> <filename>
– rcsdiff - compare working version with last repository version
– rlog - retrieve the history log for all checked-in versions of a file
– rcs - various administrative command options
4http://www.cyclic.com/cyclic-pages/rcs.html
27
Advanced Programming: Tools and Techniques - Session 2
CVS
4Concurrent Versions System
4GNU-based, free, widely available
4Built on top of RCS
– Repository files are standard RCS <filename>,v files
4Uses global repository
– Set by environment variable CVSROOT
– Usually $HOME/CVS
4Operates on multiple files in “modules”
– A module is a directory, set of directories, set of named files...
4Does not use locking
28
Advanced Programming: Tools and Techniques - Session 2
14
CVS
4More complicated than RCS, but more powerful
4In-built client-server mode for remote use
– Maintain single repository on network
4WinCVS client for Windows/NT and 9x
– Working directory on NT system can talk to remote repository on
UNIX
– Does the right thing with EOLs etc...
4More to come on CVS...
Advanced Programming: Tools and Techniques - Session 2
29
TeamWare
4Sun WorkShop commercial product
4Combined version control / build system
–
–
–
–
Command line or GUI driven
Graphical version histories
Graphical file diffs
Integrated make/build
4Uses concept of “workspaces”
– Like CVS modules
4Built on top of SCCS and make
– “Standard” foundations usable outside TeamWare
30
Advanced Programming: Tools and Techniques - Session 2
15
Visual SourceSafe
4Microsoft commercial product
4Version control system for projects
–
–
–
–
“Windows Explorer”-style GUI
Graphical file and project diffs
Point-and-click conflict merging
Integrates with MS development tools
4Uses own internal database for repository
– Multiple client model
• Typically uses OS file locking to minimise conflicts
– Incompatibilities between versions 4.0a, 5.0, 6.0
• Take care in upgrading VSS products
31
Advanced Programming: Tools and Techniques - Session 2
Visual SourceSafe
4Uses Microsoft Source Code Control Interface
– Similar tools are “SCC-compliant”
4SCC-compliant versions for UNIX and Mac
– MainSoft Corporation’s VSS for UNIX
– Metrowerks Inc’s VSS for MacOS
4 http://msdn.microsoft.com/ssafe/prodinfo/default.asp
4 http://www.mainsoft.com/products/visual/over.html
4 http://www.metrowerks.com/desktop/MWVSS/
32
Advanced Programming: Tools and Techniques - Session 2
16
ClearCase
4Rational Software commercial product
– “The world’s market-leading configuration management solution”
4Full configuration management system
–
–
–
–
Project tracking features
Change tracking features
Developer workspaces
Global repository for distributed development
• Like transparent CVS client-server
– Uses SQL databases for code repositories
4Industry-standard “sledgehammer”
4 http://www.rational.com/products/clearcase/index.jtmpl
33
Advanced Programming: Tools and Techniques - Session 2
Revision control: summary
4Revision control software is widely available
and even free
– There is no excuse for not using it!
4For bigger projects, a “module” or “workspace”
system is better than a single-file system
4Time invested in learning an RC system is time
well spent...
34
Advanced Programming: Tools and Techniques - Session 2
17
Summary
4Writing a computer program is hard
– Anyone who says any different is lying!
4Revision control and ‘make’ are essential for the
successful management of a software project
–
–
–
–
Bring order out of chaos
Save you time
Save others time
Help smooth the process from development to delivery to
maintenance
4Time invested in learning these tools will reap
dividends further down the line
35
Advanced Programming: Tools and Techniques - Session 2
18