International Journal of Engineering Trends and Technology (IJETT) - Volume4Issue5- May 2013
TDD for Embedded Systems: Feasibility and Issues
Ashish Soni#1, Nikhil Saxena#2, Narayan Sharma#3,Sapan Kumar Gupta#4
#
Electronics and Communication Deptt.
JIIT, Noida
Abstract— The paper provides a deep insight regarding the test
driven development (TDD). The paper discusses the challenges
and some solutions to adapt it in embedded system. Increasing
complexity and the need to forever-shorter development cycles
are pushing embedded development teams to the limit. To gain a
competitive edge, companies are looking towards agile methods,
such as Test Driven Development to help prevent defects by
ensuring requirements are understood at the earliest stage. TDD
has not yet applied to the embedded applications widely and
therefore embedded system designers face a lot of problems. The
paper provides a good awareness regarding the use of TDD in
embedded system. The paper also deals with these issues and
their solution through test driven development.
following list are based on Kent Beck’s description in his
book Test-Driven Development.
 Add a small test.
 Run all the tests and see the new one fail, maybe not
even compile.
 Make the small changes needed to pass the test.
 Run all the tests and see the new one pass.
 Refactor to remove duplication and improve
expressiveness.
Keywords— TDD, microcycle, MCH, Dual Target Testing,
Testing with hardware
I. INTRODUCTION
Test-Driven Development is a technique for building
software incrementally. Simply put, no production code is
written without first writing a failing unit test. Tests are small.
Tests are automated. Test-driving is logical. Instead of diving
into the production code, leaving testing for later, the TDD
practitioner expresses the desired behavior of the code in a test.
The test fails. Only then do they write the code, making the
test pass.
TDD is a part of a software development methodology
called Extreme Programming. It stresses customer satisfaction.
Instead of delivering everything engineers could possibly
want on some date far in the future, this process delivers the
software they need as they need it. Extreme programming
empowers developers to confidently respond to changing
customer requirements, even late in the life cycle. TDD began
to receive publicity in the early twenty-first century as an
aspect of Extreme Programming. TDD is based on the idea to
create tests for the program before you develop the program
code. This is the opposite of what is usual in current software
development methodologies. The availability of tests before
actual development ensures rapid feedback after any change.
Practitioners emphasize that test-driven development is a
method of designing software, not merely a method of testing.
The TDD practice starts with thoughts on how to test the
required functionality. After writing automated test cases that
generally will not even compile, the programmers write
implementation code to pass these test cases.
Figure 1. TDD Microcycle state machine flow
III. EMBEDDED TDD TECHNIQUE
Although TDD has a lot of advantage but still embedded
developers have some issues for its implementation
A. Dual Target Testing
Dual targeting means to design a code considering both
final target hardware and development system from starting.
Dual-targeting solves several problems. It allows you to test
code before the hardware is ready, and you can avoid the
hardware bottleneck throughout the development cycle. You
also avoid the finger pointing that goes with simultaneous
hardware software debugging. It is a practice that keeps you
moving fast. Due to differences between the development and
target environments some risks are also inherited with it such
II. TDD MICROCYCLE
A TDD state machine simple process is shown in the figure. as runtime libraries may be different, include filenames and
At the core of TDD is a repeating cycle of small steps known features may be different, primitive data types might have
as the TDD microcycle. The steps of the TDD cycle in the
ISSN: 2231-5381
http://www.ijettjournal.org
Page 2141
International Journal of Engineering Trends and Technology (IJETT) - Volume4Issue5- May 2013
different sizes, Byte ordering and data structure alignments
may be different.
B. Testing With Hardware
The use of interfaces and isolation allows us to develop
and test significant system behavior without access to the real
target. Automated tests to exercise the hardware are valuable
to both hardware and software developers because they
perform independent tests of hardware subsystems. Testing
with hardware falls into the following categories:
 Automated hardware tests
 Partially automated hardware tests
 Automated
hardware
tests
with
external
instrumentation
There should be many tests like this focused on specific
scenarios. Much of the core functionality of a system can be
tested well before the target hardware is available.
Furthermore, when the target becomes available, bring up
time and subsequent integration and testing can be
substantially reduced because the core system functionality
has already been well tested. The isolation approach just
described is not new; it simply comprises the application of
the well-known practices of abstraction and encapsulation and
the separation of concerns that are essential for TDD. This
technique is independent of any computer language or tool; it
is simply a matter of applying good modular design practices.
In C++, polymorphism is exploited for creating interfaces
between modules. In C or C++, we can use the linker to
substitute in test stubs. An example for a home guard system
is shown in the Appendix A.
C. Embedded Model Conductor Hardware Example: MCH in
a C based environment
Creating mocks and tests in an embedded C
environment is accomplished through compiled mock.o
implementations of header file function declarations. For
example, suppose hardware.h declares all functions for
interfacing the hardware features of a particular
microcontroller. In this example, Conductor tests will verify
that the Conductor makes specific calls on the hardware with
appropriate parameters. As such, a mock hardware.c definition
file will be written containing otherwise empty functions that
store individual function call parameter values or return
specific values – both as defined by global variables. Object
files for mock and functional code are linked together, and
tests access the previously mentioned global values to verify
the Conductor calls to the mock hardware.h interface.
The code blocks in Appendix B are examples drawn from a
real world project developed for Savant Automation of Grand
Rapids, Michigan. Savant builds Automated Guided Vehicles.
These samples pertain to a dedicated speed control board; the
functions shown set output drive voltage. In this testing
scenario, we illustrate a Conductor under test. The example
tests verify that the Conductor is correctly using the hardware
interface.
ISSN: 2231-5381
IV. CHALLENGES ISSUES AND SOLUTIONS
Although TDD has a lot of advantage but still embedded
developers have some issues for its implementation. To write
all these test codes more time is needed. It’s hard to shift from
DLP to TDD, and this is a common reaction: “Let’s just write
the tests after.” This practice has a name, Test-After
Development. But still TDD may be preferred. But in it, it is
needed to maintain the tests. We get value from having the
tests that makes the maintenance effort worth the investment.
TDD will not prevent all bugs, but that does not make a
case for not doing TDD. We will still need integration tests,
acceptance tests, exploratory tests, and load tests. TDD will
eliminate many of the problems so that the higher-level tests
are finding appropriate problems. Integration tests should find
integration problems, acceptance tests should show that the
code meets its requirements, and load tests should help
determine whether the system can meets its design limits.
When changes occur, there will be implications at the unit test
level, and the TDD tests will help assure that changes have
only the intended consequences.
Constrained memory is the reality for many embedded
developers. Running tests in the development system won’t
reveal the same memory constraints found in the target. Use
dual-targeting so the bulk of your code is tested off-target,
Make a lab version of your target system with plenty of
memory to hold all production code and test cases and finding
small test harness may limit this problem upto an extent.
When TDD is applied on a real project during the
development of an embedded system application, then it is to
be developed with custom hardware which takes a long time
to be developed, nearly months. There are issues with timing,
concurrent processing, and throughput. Also, there are many
unknowns, as one has to settle a lot many of the architecture
issues, including same code. This is very powerful as it helps
to greatly reduce one of the software developer's biggest
problems, side effect defects. The unit test suite for a module
also provides an unambiguous detailed specification of the
module under test. In contrast, acceptance tests operate on
integrated groups of modules to show that the software meets
its requirements .Modules are bound together and various test
scenarios are fed into them, demonstrating the system or
subsystem behavior. Ideally, nonprogrammers write these
tests in an application-specific test language. These tests
become part of the engineering specifications. This paper is
mainly concerned with unit testing. But many developers and
some people from managerial people think that these are just
the crippling uncertainties, and hence think that the only
productive work to be done was to write design documents in
preparation for when one could start the implementation. To
one’s surprise, developers are actually able to start design and
implementation within days on the areas in which the
requirements are understood. One can employ TDD to test
core behavior and use the best practices of object-oriented
design to isolate areas of uncertainty. Work on requirements
continued in parallel, reducing the uncertainty in the process.
After very few months of development, one usually has only
http://www.ijettjournal.org
Page 2142
International Journal of Engineering Trends and Technology (IJETT) - Volume4Issue5- May 2013
two or three bugs that required more than a few minutes of
debugging. The total time spent in debugging is
approximately only about a tenth of the project duration.
V. CONCLUSIONS
The paper discusses about the issues and some solutions of
implementation of TDD in the embedded application. TDD is
important software development practices that can help
embedded developers deliver higher quality products. The
embedded TDD cycle can help take hardware availability off
the software critical path, enabling steady progress with or
without hardware. TDD can be used for embedded
development in C and C++. Java may also be an option for
some embedded systems. Applying TDD improves test
coverage and makes it possible to find defects earlier in the
development life cycle. Highly coupled designs and
implementations, as found in many embedded systems, make
automated test a huge challenge. To be testable, modules have
to be independent because automated tests operate on
independent modules. A commitment to TDD will result in
lower coupling and higher cohesion, which are key attributes
of solid designs.
APPENDIX A
The example code for the home guard system is shown:
TEST(HomeGuard, WindowIntrusion)
{
MockAlarmPanel* panel = new MockAlarmPanel();
HomeGuard hg(panel);
hg.arm();
hg.windowIntrusion();
CHECK(true == panel->isArmed());
CHECK(true == panel->isAudibleAlarmOn());
CHECK(true == panel->isVisualAlarmOn());
CHECK(panel->getDisplayString() == "Window
Intrusion");
}
APPENDIX B
hardware.h
/// Get feedback from analog drive output.
millivolts Hardware_GetFeedbackVoltage(void);
/// Set the drive output voltage
void Hardware_SetOutputVoltage(
millivolts output);
/// Set the error flag.
void Hardware_SetError(bool err);
model.h
typedef struct _ModelInstance {
millivolts FeedbackVoltage;
millivolts OutputVoltage;
bool Error;
} ModelInstance;
/// Set the feedback voltage.
void Model_SetFeedbackVoltage(
millivolts feedback);
ISSN: 2231-5381
/// Get drive output voltage for hardware.
millivolts Model_GetOutputVoltage(void);
/// Get the error state.
bool Model_GetError(void);
conductor.h
/// Callback for hardware feedback voltage.
void Conductor_HandleFeedbackVoltage(void);
/// Control loop called by main() forever
void Conductor_Run(void);
mockhardware.h
/// Feedback voltage to return
extern millivolts
Hardware_InputFeedbackVoltage;
/// Output voltage set by conductor.
extern millivolts
Hardware_OutputDriveOutputVoltage;
/// Error flag
extern bool Hardware_OutputError;
mockhardware.c
#include "mockhardware.h"
#include "hardware.h"
#include "conductor.h"
millivolts Hardware_InputFeedbackVoltage;
millivolts Hardware_OutputDriveOutputVoltage;
bool Hardware_OutputError;
millivolts Hardware_GetFeedbackVoltage(void) {
return Hardware_InputFeedbackVoltage;
}
void Hardware_SetOutputVoltage(
millivolts output){
Hardware_OutputDriveOutputVoltage = output;
}
void Hardware_SetError(bool err) {
Hardware_OutputError = err;
}
mockmodel.h
/// Modeled system feedback voltage
extern millivolts Model_FeedbackVoltage;
/// Modeled system output voltage
extern millovolts Model_OutputVoltage;
/// Modeled error state
extern bool Model_Error;
mockmodel.c
// Linked with testconductor.o in place of
// model.o to allow conductor tests
// independent of logic in actual model.
#include "mockmodel.h"
millivolts Model_FeedbackVoltage;
millovolts Model_OutputVoltage;
bool Model_Error;
void Model_SetFeedbackVoltage(
millivolts feedback) {
Model_FeedbackVoltage = feedback;
}
millivolts Model_GetOutputVoltage(void) {
return Model_OutputVoltage;
}
http://www.ijettjournal.org
Page 2143
International Journal of Engineering Trends and Technology (IJETT) - Volume4Issue5- May 2013
bool Model_GetError(void) {
return Model_Error;
}
model.c
#include "model.h"
ModelInstance Model;
void Model_SetFeedbackVoltage(
millivolts feedback) {
Model.FeedbackVoltage = feedback;
if(feedback != Model.OutputVoltage) {
// realistically use nominal value
Model.Error = true;
}
}
millivolts Model_GetOutputVoltage(void) {
return Model.OutputVoltage;
}
bool Model_GetError(void) {
return Model.Error;
}
conductor.c
#include "model.h"
#include "hardware.h"
void Conductor_HandleFeedbackVoltage(void) {
Model_SetFeedbackVoltage(
Hardware_GetFeedbackVoltage());
}
void Conductor_Run(void) {
Hardware_SetError(Model_GetError());
if(!Model_GetError()) {
Hardware_SetOutputVoltage(
Model_GetOutputVoltage());
}
}
testconductor.c
#include "conductor.h"
#include "mockhardware.h"
#include "mockmodel.h"
static void testHandleFeedback(void) {
Hardware_InputFeedbackVoltage = 7;
Conductor_HandleFeedbackVoltage();
TEST_ASSERT_EQUAL_INT( 7,
Model_FeedbackVoltage);
}
static void testConductorRun(void) {
Model_Error = false;
Model_OutputVoltage = 78;
Conductor_Run();
TEST_ASSERT_MESSAGE(
Hardware_OutputError == false,
"Error set incorrectly");
TEST_ASSERT_EQUAL_INT( 78,
Hardware_OutputDriveOutputVoltage);
Model_Error = true;
Model_OutputVoltage = 99;
ISSN: 2231-5381
Conductor_Run();
TEST_ASSERT_MESSAGE(
Hardware_OutputError == true,
"Error not set");
TEST_ASSERT_EQUAL_INT(78,
Hardware_OutputDriveOutputVoltage);
}
ACKNOWLEDGMENT
The authors thank the assistance provided by Sanjeev
Kumar of NIT Kurukshetra, Rahul Singh M.P.E.C. Kanpur
and Aditya Pathak of YLM Tube Solutions & Service India
Pvt. Ltd. Additional thanks go to Himanshi Sharma of Sharda
University, Ragini Rai of HCL Technologies, Rahul Pandey
of M.P.E.C.Kanpur and Pushkar Mishra of TCS India Ltd. for
their great efforts and inspiring me for this paper.
REFERENCES
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
S. M. Metev and V. P. Veiko, Laser Assisted Microtechnology, 2nd ed.,
R. M. Osgood, Jr., Ed. Berlin, Germany: Springer-Verlag, 1998.
K. Beck, "Extreme programming eXplained: embrace change",
Addison-Wesley, Reading (2000)
L. Huang, M. Holcombe, "Empirical investigation towards the
effectiveness of Test First programming", Inf. Softw. Technol. 51,
2009
O. Salo, P. Abrahamsson, "Empirical Evaluation of Agile Software
Development: A Controlled Case Study Approach", Product Focused
Software Process Improvement, 2004
A. Höfer, M. Philipp, "An Empirical Study on the TDD Conformance
of Novice and Expert Pair Programmers", XP, 2009
M. M. Müller, O. Hagner, "Experiment about test-first programming",
IEE Procedings-Software. v149, 002
E. M. Maximilien, L. Williams, "Assessing test-driven development at
IBM", 25th international Conference on Software Engineering, ICSE
2003
B. George, L. Williams, "A Structured Experiment of Test-Driven
Development", Information and Software Technology 46(5), pp. 337342, 2003
J. C. Sanchez, L. Williams, E. M. Maximilien, "On the Sustained Use
of a Test-Driven Development Practice at IBM", AGILE, 2007
G. Canfora, A. Cimitile, F. Garcia, M. Piattini, C. A. Visaggio,
"Evaluating advantages of test driven development: A controlled
experiment with professionals", International Symposium on Empirical
Software Engineering, 2006
L. Cao, B. Ramesh, "Agile Requirements Engineering Practices: An
Empirical
Study",
Software,
IEEE
,
vol.25,
2008
Curriculum",International Symposium on It in Medicine and Education,
2008
L. R. Chien, D. J. Buehrer, C. Y. Yang, C. M. Chen, "An Evaluation of
TDD Training Methods in a Programming with Professionals",
Product-Focused Software Process Improvement (PROFES), 2006
L.-O. Damm, L. Lundberg, "Results from introducing component level
test automation and Test-Driven Development", Journal of Systems
and Software, 2006
L.-O. Damm, L. Lundberg, "Quality impact of introducing fault
detection effectiveness of unit test suites", Softw. Process 13, 2008
M. A. Domino, R. W. Collins, A. R. Hevner, "Controlled
experimentation on adaptations of pair programming", Inf. Technol.
and Management 8, 2007
D. Janzen, H. Saiedian, "Does Test-Driven Development Really
Improve Software Design Quality?", IEEE Software, 25, 2008
http://www.ijettjournal.org
Page 2144