Category Archives: Uncategorized

Test Seams in C: Function pointers vs. Preprocessor hash defines vs. Link-time substitution

When replacing production C code with test doubles (or fakes), there are three basic approaches: function pointers, preprocessor hash defines, and link time-time substitution.  All three are examples of what Michael Feathers calls a seam, or a way to change the behavior of a program without editing in that place. We use seams when we unit test software to decouple and localise the software under test.  Each of these types of seams has different trade-offs for testing embedded C.  Here’s what I think.

Function Pointers (run-time substitution)

  • + Easy to substitute at runtime for test functions
  • + Makes for more decoupled design
  • – Harder for IDE to jump to definitions, etc.
  • – Can make static code analysis mot work so well

Preprocessor hash defines (preprocessor-time substitution)

  • + Good for getting legacy code under test
  • – Needs discipline to structure the hash defines well
  • – Can make code harder to read and understand
  • – Definitions can be hard to locate because they can be set on the command line (-D), include path, header files, etc and can be overridden at any location

Link different function implementations (link-time substitution)

  • + Good for getting legacy C under test (limited changes needed to production code)
  • – Need to check the link definitions to understand what is running for any given executable
  • – Can’t have multiple implementations of a function in one executable
  • – Can create the need for multiple test executables

You can use the fake function framework to generate the fake functions for testing, whatever type of seam you choose.  It just creates a function and the supporting structure to capture arguments and provide return values. You can find more information about the fake function framework here:

https://github.com/meekrosoft/fff#readme

Advertisement

10 Comments

Filed under Uncategorized

Have Your Own Object Calisthenics Refactoring Dojo

We have been having lots of fun at work by having regular coding dojos.  In particular, we have been using refactoring dojos to hone our skills at improving existing code guided by tests.  For this weeks dojo, Juven and I created a particularly crappy implementation of TicTacToe to be the starting point.  Feel free to use the code and associated slides to host your own.  Fun guaranteed!

Further reading:

Leave a comment

Filed under Uncategorized

Hierarchy of Software Needs

What happens when I apply Maslow’s hierarchy of needs to software?
Maslow's Hierarchy of Needs

Physiological

  • Breathing: There is a user for this software.
  • Sex: It compiles.  Software that does not compile has no future.

Safety

  • Morality: It adds value.  Software that does not add value does not survive.
  • Security of body: The software is under version control.
  • Security of health: There are no compiler or static analysis warnings.
  • Security of resources: There are limited signs of broken windows.  Tests, if any, run successfully.  Commented out code is minimal.

Love/Belonging

  • Friendship: The software meets the needs and expectations of the users
  • Family: Developers enjoy working on the software

Esteem

  • Achievement: Developers are proud of the software
  • Confidence: tests and quality assurance mean making changes is a low risk proposition.
  • Respect of others: Future developers needs are considered
  • Respect by others: Users and stakeholders trust in the software to meet their needs.

Self-actualization

  • Morality: The code is “Good”.
  • Acceptance of facts: The software has executable Acceptance Tests to define it’s requirements.
  • Lack of prejudice: The problem domain is perfectly expressed in the source
  • Creativity, Spontaneity: It is easy and cheap to experiment with the code.
  • Homoiconic?

This may be a flawed extension of a flawed model, but the central point is that before the basic physiological and safety needs of the software are meet it is really difficult to focus on the higher motivations of software development.  This might also provide some clues as to where to start cleaning house on legacy renovation project.

1 Comment

Filed under legacy, quality, software, Uncategorized

Triptych: Red, Green, Refactor!

Triptych: Red, green, refactor!

Triptych: Red, green, refactor!

Inspired by James Grenning.

Leave a comment

Filed under Uncategorized

Less Architecture, More Microtecture

Microscope at Bioscience Center

I’m going to jump straight onto my soapbox: the world needs fewer architects. Am I suggesting that engineers of the world unite and overthrow these unjust overlords and reclaim software by the people, for the people? Well, yes, actually.  Let’s round up all the architects and send them to a dark place with no whiteboards.

Is there room for reform in my plan? Maybe. I’m not hopeful though.  It is a long and tortuous penance that must be exacted, and it will take all the concentration and commitment that an architect can muster.  It may also include humiliation and a few slices of humble pie.

You might ask yourself, why has this bozo got a chip on his shoulder? Well let me tell you.  The world is full of huge systems built on the watch of a network of highly talented architects performing due diligence and risk control.  Yet, 5 years later all that remains is a few MLOCs of hell that not even consultants can be tempted to work on.

How does this happen?  Was there a lack of boxes and arrows written down in some document? Was poor requirements management to blame? Hardly.  Codebases become unmanageable not because of the high-level design decisions, but because of the low-level details.  No UML drawing ever caused a maintainence clusterfuck (apologies to my mother for dropping the f-bomb). What destroys software is engineers.  We do this. It’s us.

We write 100-line functions. We inherit for code reuse. We write 10-line comments instead of taking 15 minutes to discuss naming a variable. We don’t write tests. We rush to deadlines. We put third-party framework calls in the middle of our production code. We think multi-hour-long build times are acceptable. We copy/paste follow patterns.

No more. I’m out. Done. Finished. I no longer wish to be associated with this crap. But I have a cunning plan.  Just like the architects, I want to solve this problem with job title. From now on, you can call me a microtect.  You are welcome to join me on this crusade, of course. Viva la revolution!

Of course no revolution is complete without a manifesto, so without apology I offer my Microtecture Manifesto:

  • Sweat the small stuff
  • Small, loosely coupled over large, monolithic
  • Prefer composition over inheritance
  • Code is king
  • Optimise for change
  • Tight feedback loops
  • Value people, not technology stacks
  • Value reducing lines of code over adding more code
  • Name stuff good [sic]

Remember kids, take care of the pennies and the pounds take care of themselves.

2 Comments

Filed under Uncategorized

Embedded Software – The next 10 Years

It is with a hint of sadness that I say my 7-year adventure in embedded systems is coming to an end.  I’ve had a great time and I’ve learned a lot but now it is time for me to try something new. 

For me, the great joy of working in embedded software is that the magic of computation is completely exposed; you learn every layer of the onion from the registers to the operating system, operating system to userspace, instruction sets to cross-compilation, linkers and debuggers.  And while you see through the layers, you learn the absolute necessity of abstraction in finding understanding through the chaos.

Over my (brief) career in embedded software, I have seen a huge shift in approaches to develop software.  This last decade has seen the rise of Agile practices and the adoption of Test Driven Development.  While these practices are established at the major software houses, they are only starting to be recognized within the embedded.  So, what can we learn from the current leading edge in software development that might predict the future we in the embedded industry are headed towards (and perhaps adopt earlier than the competition)?

It’s always a bit risky to predict the future, but I figure what the hell, why not?  If you can’t laugh at yourself there’s no hope.  :-)  So here are my three big bets for the coming years in embedded software:

1. Executable Specifications (Automated Acceptance Tests)

This is already a hot ticket in mature software houses, but automated acceptance testing is just starting to take hold in the embedded arena.  I predict that tools very much like Cucumber and their ilk will become the de-facto standard for requirement specification and progress measurement in embedded projects in the coming years. 

 A natural consequence of this is that automated hardware-in-the-loop testing will become necessary and commonplace.

2. Executable Data Sheets (Automated Hardware Tests)

If hardware becomes as flexible as software (which with the latest FPGA’s and tools they are) then the media breakage that plagues software will come home to roost with the hardware guys ‘n’ gals as well.  Also, as cores are getting softer, the need to verify the hardware works as software expects it to will increase.  

Perhaps it will be the hardware engineers who create these test suites, or maybe it will be left to the software guys, but one thing’s for sure: datasheets will become executable.

3. Open Source Toolchains

Gone are the days where one chip has one compiler, but that doesn’t mean that the compilers we have are ideal.  In fact I would go as far to say that most of the toolchains we have are downright lousy, especially with regard to automation and scriptability.

We have all kinds of hassle with licence management, incompatible upgrade paths (TI, I’m looking at you), vendors going out of business (or bought up by the big hitters only for the technology to be buried), closed source RTOS’s with critical but hidden or unfixable bugs.  These all represent an unacceptable risk for the future of the products that we bring to market.

Of course, for this to happen requires that the software engineers have a bigger say in the selection of silicon for the projects.  So in that respect it is up to us as professionals to get involved in defining the hardware requirements as early as possible and have a stronger voice in highlighting the significant risks involved in hardware and toolchain selection.

Conclusion

I don’t think there is anything particularly outrageous about these predictions. There are examples of all these practices in embedded projects now.  People in conferences are presenting on these topics, and we are starting to see books and blog posts appearing.  What is perhaps most tragic is my prediction that it will take 10 years before we see widespread adoption of these techniques. So it goes..

Historically, every change we make in the industry is concerned with increasing the quality of software or reducing project risk.  The first two predictions are concerned with increasing quality, but that is not the true motivation for them.  The real reason is to increase the agility and speed of development by closing the feedback loops. The only way to go fast is to go well.

The final prediction is all about project risk.  It’s time we as an industry acknowledge that the tools we use represent project risks and take time to mitigate them to protect the future viability of our work.

So, what are your predictions?

Leave a comment

Filed under Uncategorized

Global Day of Coderetreat, Beijing!

Beijing is participating in the Global Day of Coderetreat 2011!

Coderetreat is a day-long, intensive practice event focused on the fundamentals of software development and design.The unique coderetreat format (which eliminates the pressure of ‘getting things done’ and focuses on practicing basic principles of good design) has proven to be a highly effective (and fun) means of skill improvement. Check out to Corey Haines’s coderetreat site if you want further details on what to expect.

On December 3rd, Coderetreats will be held at cities all over the world. To celebrate this significant event, we’ll be participating in some global activities in addition to our normal Coderetreat activities. Afterwards, stick around and we’ll all go out to dinner or for a few drinks. It’s going to be a lot of fun!

As with other Global Day of Coderetreat sites, registration will open on Thursday, November 3rd, 2011. You can register at:

http://beijingcoderetreat.eventbrite.com/

You only need to bring a laptop with the development tools you require to write code using your chosen programming languages. A continental breakfast and lunch will be provided.

Attached is a flyer for the event. Feel free to print it out, post it in approved locations at work, share it via email, etc.

This event is hosted and sponsored by Schlumberger Beijing Geoscience Center. See the Google Map for directions and Parking suggestions. If you have any problems registering, or have any other questions, feel free to contact Mike (mikelong2005@gmail.com).

Leave a comment

Filed under Uncategorized

Ye Gong Hao Long (Lord Ye Loves Dragons)

Dragon

An executive software manager recently told me a tale of Agile adoption within an organization. There was a management team that has discussed, promoted, and advocated agile techniques to the engineering population for years. When the tide finally turned in favour of agile in the company the same management team then became the focal point of resistance. The reality of the consequences self-organization and “Individuals and interactions over processes and tools” were too alarming to be accepted.

He then went on to tell me the tale of Ye Gon Hao Long:

In ancient times there was a man called Ye Gong who was very fond of dragons. In his home everything, including the walls, windows, doors and even articles of daily use, were decorated with dragon designs. A real dragon was quite impressed when it heard about this, so it went to visit Ye Gong. However, when it struck its head through the window Ye Gong was frightened and ran away.

The reality of Agile is that it is vastly easier to talk about than to implement, and to fully commit means to change the organization. So beware the manager who loves dragons…

Leave a comment

Filed under Uncategorized

Code Retreat: An Interview with Corey Haines

Code Retreat at WesternGeco offices in Norway
In June this year, we were lucky enough to have Corey Haines come by the Westerngeco offices in Oslo to host a code retreat with the seismic embedded software engineers. We had a great time and learned a lot. Corey kindly agreed to give a short interview for our software newsletter, and here it is:

Q. Could you tell us a little about who you are and what you do?

I’m an independent software developer trainer, specializing in working
with teams to improve their fundamental software design skills. In
2008, I lost my job and decided to go on a pair-programming tour,
traveling around writing code with people in exchange for room and
board. Over the past few years, I’ve been working on developing and
augmenting a training format called code retreat. When I’m not
traveling, I live in Chicago with my girlfriend and a cat named Zak. I
spend my time at home working on my application, MercuryApp, as well
as advising non-technical founders of startups.

Q. What is a code retreat? Where did the idea of code retreats come
from? What can you expect when you go to your first code retreat?

A code retreat is a day-long, practice-oriented training event. It
stems from the idea that, as software developers, we spend most of our
performing on live code. Since we are always trying to ‘get things
done’, we necessarily have to cut some corners in our code. We often
neglect some of the basics of our craft, such as good abstractions.
Using a focused practice approach, we can make ourselves a bit more
effective at designing and developing software. So, the format is
designed to give the opportunity to really work on writing better
code.

During a person’s first code retreat, the morning is usually spent
getting intimately acquainted with the problem. Then, in the
afternoon, we really push hard on practicing some variants of design,
focusing on aspects of the 4 Rules of Simple Design. Depending on the
group, we might then move into pairing games, design constraints (no
if statements, no loops, ultra-small methods) or a
TDD-as-if-you-meant-it exercise. And, lots and lots of intense coding.

Q. You have been facilitating code retreats all over the world. What
are some of the surprising things you’ve experienced during these
sessions?

A major thing that has surprised me over the past couple years is just
how much progress a person can make in their understanding when
working under the code retreat constraints. Having the opportunity to
delete your code every 45 minutes allows you to really try new things
out and not have to live with the mistakes of the past. In our
day-to-day coding, we often write production code before we really
have a great grasp on the problem. By giving ourselves the freedom to
explore, the solution we come up with at the end of the day is so much
better than at the beginning.

Another surprising thing has been the number of people who get hooked
on working on Conway’s Game of Life as a regular practice problem. I
often hear from people later that they can’t stop working on the
problem, approaching it from different directions, trying out new
abstractions, etc.

I’ve also had more than one company tell me that they’ve begun
integrating the idea of ‘do it once and throw it away’ in their
regular routine. This has a surprising effect on the codebase, as it
decreases the amount of technical debt in a codebase, as spiking the
solution first gives a better understanding of the domain.

Q. First there was OO, then there came patterns, followed closely by
agile. It seems that the movement du jour is software craftsmanship.
What does software craftsmanship mean to you? Is it important? Is
there anything new in it?

Software Craftsmanship is a natural partner to all those things. It is
an outgrowth and evolution of our understanding of how to develop
software. Just like OO and the agile methodologies were culminations
of previous work, Software Craftsmanship brings together a lot of the
lessons that have been learned over the past decades around building
and delivering software products.

For me, Software Craftsmanship is about working to build a programming
industry that can be proud of its professionalism, delivering the
right product for the right situation at the right time. This involves
the whole range of activities, from effectively mentoring new
developers to working closely with our client businesses to understand
the values that a modern software development process can provide.

Q. What excites you about software development now? What technologies
do you have on your radar?

I’m quite excited about the push to get kids interested in
programming. I’ve been spending a lot of time in Scratch
(scratch.mit.edu) lately, as I’m helping teach a class for teenagers.
There is a dearth of developers, especially experienced ones, so it
would be good to start working with the next generation.

I think it is cool to see some of the more fundamental languages
coming back into vogue, such as lisp and smalltalk, emphasizing
functional and pure object-oriented paradigms. The emphasis on being
adept in different language paradigms is an encouraging trend.

Q. How do you see the future of software development?

First, I’d like to see a continued renewed energy and joy in the
younger generation. It is exciting to see the push in the past couple
years to focus on showing kids just how exciting our craft can be. We
wake up in the morning, start with nothing, and have created something
new by the end of the day. It is an amazingly heady feeling to be a
maker.

Second, I’d like to see the industry focus back on fundamentals of how
to deliver quality software for the businesses we are part of. Rather
than always looking at the latest shiny toy, let’s also take some time
to really work on improving our core skills as software developers:
abstractions, modular design, naming, etc. It is important to keep
innovating, but the past has seen a push to focus too much on the next
great thing. For me, code retreat is one of the core activities in the
quest to improve our fundamental skills.
—-

Corey is organizing a global day of Code Retreat, find out more at the Code Retreat Blog.

1 Comment

Filed under Uncategorized

Hardware is a global variable.

PCB

There, I’ve said it.  No going back now. But when you think about memory mapped IO, a device register is just a globally available address that any Joe can read or write, right?  It is only convention that keeps us from accessing the hardware willy-nilly.

Why are global variables considered harmful?

Reasoning about the behaviour of software with global variables is hard. The potential scope for state change is unbounded.   The reason is poor locality.  Software that is hard to reason about is also hard to test.

But a memory mapped IO register differs from a standard global variable in these ways:

  • It is necessary (there are no formal methods to localize the scope of hardware in software)
  • By definition it is volatile.  Even if the software doesn’t write to the address the value that is read can change unpredictably.
  • It is not a memory location.  What you write might not be what you read.

Enter the Device Driver

So given that memory-mapped IO is just a fancy type of global variable, how do minimise the impact of this necessary evil?  The standard approach is the device driver.

A device driver is just a software module that is used to create an abstraction of the hardware to the rest of the software system.  Some drivers can be simple translations of function calls into hardware access, but there is usually a lot more going on behind the scenes in a typical device driver.  Devices such as flash memory might require complex communication protocols and timing considerations, and interrupt handling routines might have to delegate to program mode functions with queue worker threads.  With this much complexity in the software, it is a good idea to make an automated test suite for drivers.

The challenge then becomes: how do I test software that is so intimately tied to memory mapped IO?

How to unit test device drivers

There are many approaches to testing driver software, but I want to share my favorite.  I like it because it has zero runtime overhead in production, but gives full flexibility during testing.

In production code use a compile-time macro to access hardware registers.  In test builds replace this with a fake function.  This is quite simple to do with a header file:

#ifndef HARDWARE_ABSTRACTION
#define HARDWARE_ABSTRACTION

#include <stdint.h>

#ifndef TESTING
#define IO_MEM_RD8(ADDR)  (*((volatile uint8_t *)(ADDR)))
#define IO_MEM_WR8(ADDR, VAL_8)   (*((volatile uint8_t *)(ADDR)) = (VAL_8))
#else
/* In testing use fake functions to record calls to IO memory */
uint8_t IO_MEM_RD8(uint32_t reg);
void IO_MEM_WR8(uint32_t reg, uint8_t val);
#endif

#endif /* Include guard */

Then, it’s simply a matter of implementing the driver in a standard way:

#include "HardwareAbstraction.h"

#define DRIVER_OUTPUT_REGISTER 0xFFAA
#define DRIVER_INPUT_REGISTER  0XFFAB

void write_to_driver(uint8_t val)
{
    IO_MEM_WR8(DRIVER_OUTPUT_REGISTER, val);
}

uint8_t read_from_driver()
{
    return IO_MEM_RD8(DRIVER_INPUT_REGISTER);
}

Now I know this is a contrived example, but it illustrates the point.  In a production compilation the preprocessor substitutes this with direct memory access.  However, in the tests there is a link seam which can be overriden as you please.

Capture and Assert

So what would a test of this code look like?  Well, there has to be fake implementations for the IO_MEM_RD8 and IO_MEM_WR8 functions.  They should capture the arguments passed to them so we can verify the driver has the correct behaviour.  So an initial stab at the problem might look like this:

extern "C"
{
#include "driver.h"
#include "registers.h"
}
#include <igloo/igloo.h>
#include <map>
#include <iostream>

using namespace igloo;

extern "C"
{
    static uint8_t readVal;
    static int readCalled;
    static uint32_t readRegister;
    uint8_t IO_MEM_RD8(uint32_t reg)
    {
        readRegister = reg;
        readCalled++;
        return readVal;
    }

    static uint32_t writeRegister;
    static uint8_t writeVal;
    static int writeCalled;
    void IO_MEM_WR8(uint32_t reg, uint8_t val)
    {
        writeRegister = reg;
        writeVal = val;
        writeCalled++;
    }
}

Context(AHardwareDriver)
{
    Spec(WritesDataToCorrectRegister)
    {
        driver_write(0x34);
        Assert::That(writeCalled, Equals(1));
        Assert::That(writeVal, Equals(0x34));
        Assert::That(writeRegister, Equals(DRIVER_OUTPUT_REGISTER));
    }

    Spec(ReadsDataFromCorrectRegister)
    {
        readVal = 0x55;
        uint8_t returnedValue = driver_read();
        Assert::That(readCalled, Equals(1));
        Assert::That(returnedValue, Equals(readVal));
        Assert::That(readRegister, Equals(DRIVER_INPUT_REGISTER));
    }

};

These tests are written using the Igloo C++ testing framework, but of course any testing framework will do.  These tests are clear and verify that the driver performs the reads and writes that are expected of it.  But there are a couple of things missing here.  The capture variables and return variables maintain state across tests so there needs to be additional reset code run before every test.  If a function of the driver performs several reads and writes it is impossible to capture them without making very complex implementation of the fake function.  And with the current implementation it is not possible to track the order of function calls, so we can’t tell if (for example) a read came before a write.  To make this work for more interesting scenarios we need to write a lot more code.

Fake Functions

Thankfully there is an alternative to handwriting these complex fake functions by using the Fake Function Framework.  All the faking code in the previous example:

extern "C"
{
    static uint8_t readVal;
    static int readCalled;
    static uint32_t readRegister;
    uint8_t IO_MEM_RD8(uint32_t reg)
    {
        readRegister = reg;
        readCalled++;
        return readVal;
    }

    static uint32_t writeRegister;
    static uint8_t writeVal;
    static int writeCalled;
    void IO_MEM_WR8(uint32_t reg, uint8_t val)
    {
        writeRegister = reg;
        writeVal = val;
        writeCalled++;
    }
}

Can be replaced with this:

FAKE_VOID_FUNC(IO_MEM_WR8, uint32_t, uint8_t);
FAKE_VALUE_FUNC(uint8_t, IO_MEM_RD8, uint32_t);

Of course that is much less typing.  But there are additional benefits with using the framework.  All the tricky testing scenarios involving multiple call sequences, return values, and argument histories are taken care of by the framework.

A less contrived example

Imagine that there a particular revision of the hardware requires a peripheral to be enabled before it can be initialized (this is quite common in low power hardware), how can I write a test for this?

I need to be able to simulate the hardware revision, verify that the enable register is written to before the initialize register, and verify that the correct values have been written  It turns out that by using the fake function framework this is a quite simple operation.  The frameworks captures all this information for us, so all we need to do is assert our expectations in the tests:

Context(DuringSetupOfRevisionBHardware)
{
    Spec(EnablesPeripheralBeforeInitializingIt)
    {
        IO_MEM_RD8_return_val = HARDWARE_REV_B;
        driver_init_device();
        // Gets the hardware revision
        Assert::That(call_history[0], Equals((void*) IO_MEM_RD8));
        Assert::That(IO_MEM_RD8_arg0_history[0], Equals(HARDWARE_VERSION_REGISTER));
        // Enables Peripheral
        Assert::That(call_history[1], Equals((void*) IO_MEM_WR8));
        Assert::That(IO_MEM_WR8_arg0_history[0], Equals(DRIVER_PERIPHERAL_ENABLE_REG));
        Assert::That(IO_MEM_WR8_arg1_history[0], Equals(1));
        // Initializes Peripheral
        Assert::That(call_history[2], Equals((void*) IO_MEM_WR8));
        Assert::That(IO_MEM_WR8_arg0_history[1], Equals(DRIVER_PERIPHERAL_INITIALIZE_REG));
        Assert::That(IO_MEM_WR8_arg1_history[1], Equals(1));
    }
};

How does this work? The Fake Function Framework is contained in a single header file which needs to be included into the test code.  When a fake is defined, all the boilerplate code is created by the framework. That’s all there is to it.

Summary

  • Hardware is a global variable.
  • Device drivers are a means of localizing their influence.
  • Device drivers can be complex, and therefore should be tested.
  • Device drivers produce side effects not seen in software.
  • We can test these side effects with zero runtime overhead using the right techniques.
  • The Fake Function Framework can significantly reduce the amount of boilerplate code required to test C code.

The code examples for this article can be found on GitHub:
https://github.com/meekrosoft/driver_testing
The Fake Function Framework and documentations can be found here:
https://github.com/meekrosoft/fff


12 Comments

Filed under Uncategorized