Channels ▼
RSS

Testing

Testing Python and C# Code


In part one of this three-part series on testing complex systems, I covered the practical aspects of deep testing and demonstrated how to test difficult areas of code, such as the user interface, networking, and asynchronous code. In part two, I discussed some techniques and utilities I have used to successfully test complex systems in C++.

In this final installment, I discuss similar complex testing for Python (with Swig C++/Python bindings) and high-volume, highly available Web services in C# and .NET.

Python

Python is strongly typed and dynamically typed. It is also an interpreted language (compiled just-in-time). This means that type errors are discovered at runtime, rather than compile time. For example, if you pass an int to a function that expects a string, your program will still run and maybe even run successfully to completion if your code never reaches the bad call:

def sayHello(name):
    print 'Hello, ' + name

def sayHelloToMyLittleFriend():
    sayHello(5)
	
sayHello('Jebediah')

Output:
Hello, Jebediah

The sayHelloToMyLittleFriend() function containing the bad call to sayHello(5) is never invoked, so the program runs fine. So, should you do something different than when writing programs in a statically typed language? No! If you didn't discover the bad call, it means you didn't test the code path.

If you don't test a code path, you can't tell whether it's broken. And it really doesn't matter whether it's broken due to type errors or some other kinds of errors. (By the way, in statically typed languages, there is a very similar problem that crops up a lot. The dreaded NULL/null/nill/None or whatever you want to call it. If you pass a NULL pointer to a C++ function that tried to dereference it, you will cause a crash: This is actually much more common than passing an int to a function expecting a string.)

With Python, if you are really paranoid, you can test the type of arguments. This may be useful in libraries that are used a lot and where you want to provide a good error message to the user instead of just crashing with a type error. You can either wrap the suspicious code in a try-except block and catch TypeError, or you can use the instanceof() function to dynamically determine the type of arguments (but this second option is pretty brittle). Here is how it looks with try-except:

def sayHello(name):
    try:
        print 'Hello, ' + name
    except TypeError:
        print 'Oh, no!', name, 'is not a string'

def sayHelloToMyLittleFriend():
    sayHello(5)

sayHelloToMyLittleFriend()

Output:

Oh, no! 5 is not a string

Monkey Patching

Testing Python code (and other dynamic languages) is a pleasure. There is no need to design testability into your code or create special interfaces or fight with legacy code. With Python, you can just reach into the guts of any object and replace its members with whatever you like. You can even replace functions. It doesn't get any better than that. This time honored practice is called monkey patching. Monkey patching works even if you don't use dependency injection because you can replace the dependencies at any time. In the following code example, class A instantiates its own Dependency:

class Dependency(object):
    def __init__(self):
        self._state = 'complex state'


    def bar(self):
        print 'Doing something expansive here...'
        print 'state: ' , self._state

class A(object):
    def __init__(self):
        self._b = Dependency()

    def foo(self):
        self._b.bar()

a = A()
a.foo()

Output:
Doing something expansive here... state: complex state

If you want to test A, but you don't want it to invoke the real expansive Dependency.bar() method you have two options:

  1. You can replace the _b member of A with a mock object that has a bar() method
  2. You can replace the bar() method of the Dependency

Method 1 is useful if you want to mock the entire dependency, often with multiple methods and internal state. Method 2 is useful if you want to use the real Dependency object with its logic and initialized state, but just replace a few methods.

Method 1:

class MockDependency(object):
    def __init__(self):
        """No state"""

    def bar(self):
        print 'Doing something cheap here...'

a = A()
a._b = MockDependency()
a.foo()

Output:
Doing something cheap here...

Method 2:

def mock_bar(self):
    print 'Doing something cheap here...'
    print 'state: ' , self._state

a = A()
Dependency.bar = mock_bar
a.foo()

Output:
Doing something cheap here... state: complex state

You'd be wise to restrict your usage of monkey patching to testing and other special circumstances. A lot of people get excited about the power of monkey patching and use it willy-nilly all over the place, which leads to buggy and unmaintainable code.

Python Test Frameworks

Python has multiple test frameworks. The xUnit-like unittest package has been part of the Python standard library since Python 2.1, but it was never as powerful and user-friendly and some other test frameworks until Python 2.7. I like nose a lot. With nose, you don't need to write test classes derived from a base class, but can instead write simple test functions and nose will discover and run them. In Python 2.7, the standard unittest package acquired many features available in third-party test frameworks and became a much more formidable contender (although nose has evolved, too, and there is nose2).

There is a lot of material available around the Web on unittest, so I'll just quickly list the main features of the Python 2.7 (and Python 3.2) unittest package:

  • Abundance of assert functions
  • Test discovery
  • Test organization
  • Reporting
  • Command-line interface
  • Fine-grained control on running tests

Python also has an interesting module called doctest, which allows you to write tests embedded in your docstrings. I don't use doctest because I like to separate the code under test from the code that tests it. I also aim for more comprehensive testing, which is difficult to do in a doc string (not to mention that it will clutter the code).

Python Extensions

Like any self-respecting dynamic language, Python (CPython) has an extension and embedding mechanism. That means you can extend Python with native C libraries and even embed Python in a native program. Other Python implemenations that target specific runtime environments, like Jython on the JVM and IronPython on .NET, provide similar facilities. Here, I'll discuss only CPython.

First things first — Python is slow. It is, of course, slow compared with native languages, but it is also slow compared with other dynamic languages. I will not delve into the particulars (super-flexible, super-dynamic, GIL, multithreading issues). There have been various efforts over the years to improve Python's performance. The most notable is the PyPy project. In the CPython ecosystem, the recommended way to extend Python with native code is to use Cython, which is a Python-like language that lets you write extensions to Python mostly in Python. If you need to interface with existing C/C++ code, there are several other tools available such as Boost.Python and Py++.

Dealing with Language Bindings

My language binding experience mostly involves using Swig. Swig is a veteran project that can create C/C++ bindings for almost any programming language. I have used it to create language bindings for the NuPIC 2.0 API for Python, Java, and C#. I have also had to debug through its interface code for NuPIC 1.0 Python bindings. Testing such code is no fun. It is all about marshaling data types, matching calling conventions, and adapting error-handling mechanisms. The best advise I can give here is to keep the binding code as lean as possible. Don't give in to the temptation to add syntactic sugar or try to make the binding code follow the idioms of the target language. Use a layered approach — the bindings themselves, whether hand coded or autogenerated, should closely match the native code. Later, once you are back in your target language world, write a wrapper or syntactic sugar layer on top of it. You can write your tests using the wrapper layer.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Comments:

ubm_techweb_disqus_sso_-d282dffe76fccdf5d72ad71339e97ea4
2013-02-20T00:22:59

CPython has been shown to be slow compared to other dynamic languages (not to mention Java,C#/C++ etc) in myriad benchmarks and in the field. It is mostly due to its highly dynamic nature. CPython also can't take advantage directly of multiple cores due to the GIL, which becomes a more serious issue. But, it is a great language (my favorite) and eco-system and there are lots of ways to compensate for its innate slowness (either other implementations like PyPy, Jython, IronPython or native C extensions a-la NumPy and SciPy mentioned in the article you linked to). It is absolutely fit for large enterprise applications, web apps, scientific tools and what not. It's just not suitable for the inner loop/number crunching part. That's all.


Permalink
ubm_techweb_disqus_sso_-eb143e6d5f6d149c968e253a5efe5875
2013-02-08T01:46:08

So you state "First things first — Python is slow". Slow in what way? How does your statement fit with news such as that of DARPA funding a company to the tune of 3 million dollars to add to Pythons use with "big data" http://www.itworld.com/big-dat....

Slowness is complex. Maybe Python *is* slow for you. But not necessarily everyone else.


Permalink

Video