Hardware is a global variable.

PCB

There, I’ve said it.  No going back now. But when you think about memory mapped IO, a device register is just a globally available address that any Joe can read or write, right?  It is only convention that keeps us from accessing the hardware willy-nilly.

Why are global variables considered harmful?

Reasoning about the behaviour of software with global variables is hard. The potential scope for state change is unbounded.   The reason is poor locality.  Software that is hard to reason about is also hard to test.

But a memory mapped IO register differs from a standard global variable in these ways:

  • It is necessary (there are no formal methods to localize the scope of hardware in software)
  • By definition it is volatile.  Even if the software doesn’t write to the address the value that is read can change unpredictably.
  • It is not a memory location.  What you write might not be what you read.

Enter the Device Driver

So given that memory-mapped IO is just a fancy type of global variable, how do minimise the impact of this necessary evil?  The standard approach is the device driver.

A device driver is just a software module that is used to create an abstraction of the hardware to the rest of the software system.  Some drivers can be simple translations of function calls into hardware access, but there is usually a lot more going on behind the scenes in a typical device driver.  Devices such as flash memory might require complex communication protocols and timing considerations, and interrupt handling routines might have to delegate to program mode functions with queue worker threads.  With this much complexity in the software, it is a good idea to make an automated test suite for drivers.

The challenge then becomes: how do I test software that is so intimately tied to memory mapped IO?

How to unit test device drivers

There are many approaches to testing driver software, but I want to share my favorite.  I like it because it has zero runtime overhead in production, but gives full flexibility during testing.

In production code use a compile-time macro to access hardware registers.  In test builds replace this with a fake function.  This is quite simple to do with a header file:

#ifndef HARDWARE_ABSTRACTION
#define HARDWARE_ABSTRACTION

#include <stdint.h>

#ifndef TESTING
#define IO_MEM_RD8(ADDR)  (*((volatile uint8_t *)(ADDR)))
#define IO_MEM_WR8(ADDR, VAL_8)   (*((volatile uint8_t *)(ADDR)) = (VAL_8))
#else
/* In testing use fake functions to record calls to IO memory */
uint8_t IO_MEM_RD8(uint32_t reg);
void IO_MEM_WR8(uint32_t reg, uint8_t val);
#endif

#endif /* Include guard */

Then, it’s simply a matter of implementing the driver in a standard way:

#include "HardwareAbstraction.h"

#define DRIVER_OUTPUT_REGISTER 0xFFAA
#define DRIVER_INPUT_REGISTER  0XFFAB

void write_to_driver(uint8_t val)
{
    IO_MEM_WR8(DRIVER_OUTPUT_REGISTER, val);
}

uint8_t read_from_driver()
{
    return IO_MEM_RD8(DRIVER_INPUT_REGISTER);
}

Now I know this is a contrived example, but it illustrates the point.  In a production compilation the preprocessor substitutes this with direct memory access.  However, in the tests there is a link seam which can be overriden as you please.

Capture and Assert

So what would a test of this code look like?  Well, there has to be fake implementations for the IO_MEM_RD8 and IO_MEM_WR8 functions.  They should capture the arguments passed to them so we can verify the driver has the correct behaviour.  So an initial stab at the problem might look like this:

extern "C"
{
#include "driver.h"
#include "registers.h"
}
#include <igloo/igloo.h>
#include <map>
#include <iostream>

using namespace igloo;

extern "C"
{
    static uint8_t readVal;
    static int readCalled;
    static uint32_t readRegister;
    uint8_t IO_MEM_RD8(uint32_t reg)
    {
        readRegister = reg;
        readCalled++;
        return readVal;
    }

    static uint32_t writeRegister;
    static uint8_t writeVal;
    static int writeCalled;
    void IO_MEM_WR8(uint32_t reg, uint8_t val)
    {
        writeRegister = reg;
        writeVal = val;
        writeCalled++;
    }
}

Context(AHardwareDriver)
{
    Spec(WritesDataToCorrectRegister)
    {
        driver_write(0x34);
        Assert::That(writeCalled, Equals(1));
        Assert::That(writeVal, Equals(0x34));
        Assert::That(writeRegister, Equals(DRIVER_OUTPUT_REGISTER));
    }

    Spec(ReadsDataFromCorrectRegister)
    {
        readVal = 0x55;
        uint8_t returnedValue = driver_read();
        Assert::That(readCalled, Equals(1));
        Assert::That(returnedValue, Equals(readVal));
        Assert::That(readRegister, Equals(DRIVER_INPUT_REGISTER));
    }

};

These tests are written using the Igloo C++ testing framework, but of course any testing framework will do.  These tests are clear and verify that the driver performs the reads and writes that are expected of it.  But there are a couple of things missing here.  The capture variables and return variables maintain state across tests so there needs to be additional reset code run before every test.  If a function of the driver performs several reads and writes it is impossible to capture them without making very complex implementation of the fake function.  And with the current implementation it is not possible to track the order of function calls, so we can’t tell if (for example) a read came before a write.  To make this work for more interesting scenarios we need to write a lot more code.

Fake Functions

Thankfully there is an alternative to handwriting these complex fake functions by using the Fake Function Framework.  All the faking code in the previous example:

extern "C"
{
    static uint8_t readVal;
    static int readCalled;
    static uint32_t readRegister;
    uint8_t IO_MEM_RD8(uint32_t reg)
    {
        readRegister = reg;
        readCalled++;
        return readVal;
    }

    static uint32_t writeRegister;
    static uint8_t writeVal;
    static int writeCalled;
    void IO_MEM_WR8(uint32_t reg, uint8_t val)
    {
        writeRegister = reg;
        writeVal = val;
        writeCalled++;
    }
}

Can be replaced with this:

FAKE_VOID_FUNC(IO_MEM_WR8, uint32_t, uint8_t);
FAKE_VALUE_FUNC(uint8_t, IO_MEM_RD8, uint32_t);

Of course that is much less typing.  But there are additional benefits with using the framework.  All the tricky testing scenarios involving multiple call sequences, return values, and argument histories are taken care of by the framework.

A less contrived example

Imagine that there a particular revision of the hardware requires a peripheral to be enabled before it can be initialized (this is quite common in low power hardware), how can I write a test for this?

I need to be able to simulate the hardware revision, verify that the enable register is written to before the initialize register, and verify that the correct values have been written  It turns out that by using the fake function framework this is a quite simple operation.  The frameworks captures all this information for us, so all we need to do is assert our expectations in the tests:

Context(DuringSetupOfRevisionBHardware)
{
    Spec(EnablesPeripheralBeforeInitializingIt)
    {
        IO_MEM_RD8_return_val = HARDWARE_REV_B;
        driver_init_device();
        // Gets the hardware revision
        Assert::That(call_history[0], Equals((void*) IO_MEM_RD8));
        Assert::That(IO_MEM_RD8_arg0_history[0], Equals(HARDWARE_VERSION_REGISTER));
        // Enables Peripheral
        Assert::That(call_history[1], Equals((void*) IO_MEM_WR8));
        Assert::That(IO_MEM_WR8_arg0_history[0], Equals(DRIVER_PERIPHERAL_ENABLE_REG));
        Assert::That(IO_MEM_WR8_arg1_history[0], Equals(1));
        // Initializes Peripheral
        Assert::That(call_history[2], Equals((void*) IO_MEM_WR8));
        Assert::That(IO_MEM_WR8_arg0_history[1], Equals(DRIVER_PERIPHERAL_INITIALIZE_REG));
        Assert::That(IO_MEM_WR8_arg1_history[1], Equals(1));
    }
};

How does this work? The Fake Function Framework is contained in a single header file which needs to be included into the test code.  When a fake is defined, all the boilerplate code is created by the framework. That’s all there is to it.

Summary

  • Hardware is a global variable.
  • Device drivers are a means of localizing their influence.
  • Device drivers can be complex, and therefore should be tested.
  • Device drivers produce side effects not seen in software.
  • We can test these side effects with zero runtime overhead using the right techniques.
  • The Fake Function Framework can significantly reduce the amount of boilerplate code required to test C code.

The code examples for this article can be found on GitHub:
https://github.com/meekrosoft/driver_testing
The Fake Function Framework and documentations can be found here:
https://github.com/meekrosoft/fff


12 Comments

Filed under Uncategorized

12 responses to “Hardware is a global variable.

  1. Jeanne

    The fake function framework looks handy. Your examples all show native types passed in by value, which makes perfect sense in the context of faking register access. I wonder how useful it is beyond that… Can it also handle user defined types like enums or structs, when passed in or returned? I assume no, because you don’t describe a way to instruct fff what other header functions it might use, but thought I’d ask.

  2. Jeanne

    Now that I’ve taken a closer look, I have a few comments. First, nice work! Next… fff won’t build out of the box. For one thing it depends on Google Test, so you have to install that first. It also depends on gtest-1.5.0 while the current one is gtest-1.6.0. There may not be an appreciable difference but your test/Makefile looks for libgtest.a, which (I don’t think) is part of gtest-1.6.0. (The gtest README led me to build libgtest.a and gtest_main.a). Is an update available? Next, the test/Makefile has hardcoded paths for your local computer, e.g. “-I/home/mlong/tools/gtest/gtest-1.5.0/include”. And finally, fff_test_cpp.cpp has no main() so it won’t build an executable. With enough Makefile tweaks I was able to get the C version of fff built, but the cpp version is incomplete. Thanks!

  3. meekrosoft

    Hi Jeanne,
    Thanks for taking a look! It might be a bit misleading from looking at the project on github, but fff doesn’t depend on googletest or any framework at all. The only thing you need to do to use fff is include the fff.h header file into your test module. The makefile in the project is used to build the test suite that makes sure fff.h behaves as I expect. You can actually use the test modules as examples of how it works:
    https://github.com/meekrosoft/fff/blob/master/test/fff_test_cpp.cpp

    And the test suite for ansi C uses no test framework at all, only assert:
    https://github.com/meekrosoft/fff/blob/master/test/fff_test_c.c

    The examples from this article used the Igloo testing framework:
    https://github.com/meekrosoft/driver_testing

    If you have any more questions please don’t hesitate to get in touch. I’d love to know how it goes.

    • Jeanne

      Can I ask, why Igloo? I know you like Google Test. Also, is there a way I can get fff to give me actual C or C++ files containing the code it generates, such that I can modify them? I’m thinking if I want to fake a function that passes in an enumerated data type, since fff only fakes native types (right?), I could use fff to fake it with a native type and then edit and save the file to use during unit testing.

      • meekrosoft

        Sure, I just used Igloo in this example because I wanted to try it out. I quite like the expressiveness it gives, but I also like the googletest framework with googlemock. I don’t think one is better than the other, just different.

        With fff you should be able to mock with enumerated types. I haven’t checked it, but it works with tyoedef’d types and structs so I don’t see why this would be a problem.

        When you define a fake function with fff, what it actually does is expand a macro in your source to create the fake functionality, so there is no “generated code” so to speak. If you’d like to see how the macro gets expanded you should ask your compiler to generate the output of the preprocessor. Refer to this to see how to do it:
        http://stackoverflow.com/questions/277258/c-c-source-file-after-preprocessing

  4. Jeanne

    Now that I’ve played with fff a bit, the only complaint I have is that it seems to choke when a pointer is passed in. Otherwise, it’s a great compliment to the likes of Google Test, because Google Mock can’t mock standalone functions; all functions must be members of a class. Thank you for writing it and making it available.

  5. Jeanne

    Well, it doesn’t, unfortuantely. I sent you an email to a gmail account you own with details. I have a pointer to a structure which isn’t handled gracefully by fff. The struct contains an unsigned 8 bit number, an unsigned 16 bit number, and a pointer to another struct of the same type.

    More specifically,
    FAKE_VOID_FUNC(StartSWTimer, *pTimer);

    fails to build this way:
    g++ –DMY_FLAG -O0 -g -Wall -Wno-write-strings –I./my/include/paths -I ../../../utils/fff -Wextra -c utest_MyFile.cpp
    utest_MyFile.cpp:107: error: expected initializer before `StartSWTimer_arg0_val’
    utest_MyFile.cpp:107: error: expected initializer before `StartSWTimer_arg0_history’
    utest_MyFile.cpp:107: error: variable or field `StartSWTimer’ declared void
    utest_MyFile.cpp :107: error: `pTimer’ was not declared in this scope
    utest_MyFile.cpp:107: error: expected primary-expression before `arg1′
    utest_MyFile.cpp:111: error: expected `}’ at end of input
    utest_MyFile.cpp:107: warning: `StartSWTimer_arg1_val’ defined but not used
    utest_MyFile.cpp:107: warning: `StartSWTimer_arg1_history’ defined but not used

    Email me if you have questions.

  6. meekrosoft

    Thanks for the feedback! I think the syntax you need is like this:
    FAKE_VOID_FUNC(sw_timers_StartTimer, Timer *, Uint16);

  7. shreyasbharath

    Great work, this makes mocking so easy!

    How does your framework handle return-by-reference outputs?

    For example –

    int GetIndexOfKeyInList(const int array*, unsigned int arraySize, int keyValue, bool* findResult);

    How do we set *findResult to TRUE/FALSE in the fake/mocked function?

    • meekrosoft

      @shreyasbharath
      This is something I’m working on right now. There are some recent updates to fff that I haven’t written about yet but you can try them out.
      https://github.com/meekrosoft/fff/blob/master/fff3.h

      With this version of fff you can attach a custom fake to your fake. In this custom fake you can hand-roll any side-effects you need.

      Another update is that the data for the fakes is stored in a struct to avoid polluting the global namespace.

  8. Pingback: Fake Function Framework – Request For Comments! | Mike Long’s Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s