Code integrity is not only essential to quality software, but also to meet regulatory requirements. Your software must do what it's specified to do, and the development organization needs to prove this to regulators, as part of its risk mitigation program. Processes, such as developer testing, should be in place to ensure that requirements are met. They help ensure that your software was designed with these concerns in mind and provide verification that your code follows the logic that it was designed for.
There are a multitude of use cases that must be tested for to meet regulatory requirements. Because of Linux's configurability, and extensibility, and hundreds of distributions, it's difficult to prove to regulators that your code can uniformly meet regulatory requirements and that the conditions tested are replicable under alternate configurations.
Many regulations mandate security and data privacy. As "open source" is a frequently found on Linux, there are unique security issues to consider when your project uses open source code. On one hand, because of open source code and a community of developers, holes tend to be identified and patched quickly. At the same time, open vulnerabilities do not remain unexploited for long. There is also a tendency of some within the Linux community to ignore these security threats, viewing only Microsoft software as vulnerable to threats. However, according to Security Strategies in Linux Platforms and Applications, "While crackers attack Microsoft software more frequently, they also attack open source software." Testing for potential security breaches is essential, even in Linux.
As part of reporting code quality metrics, testing can measure "code coverage" how much of your code is tested. Coverage tools can also verify that every instruction was executed and document the coverage. Because coverage alone is not a metric for quality, unit testing forces review of the program logic, and can detect coding, compiler, and design errors. Collecting multiple metrics helps ensure regulatory compliance; still, you may need a set of different tools to report compliance.
Without the ability to unit test existing code, organizations find it hard to prove they have covered the entire code base. As a result, they throw a lot of money on QA and post-development testing. Other organizations generate reports that do not specify coverage metrics, because their tools do not provide it. Regulated industries understand that to move to better, safer products, they should move to better tooling that can offer more exact coverage reporting.
To ensure standardization and the fulfillment of regulatory requirement, it's ideal to use a consistent toolkit that ensures that you can design a system that can meet regulatory compliance across all the different Linux distributions you intend to ship on.
Testing Best Practices
Despite the challenges discussed previously, there are best practices that can provide confidence and assurance to developers testing code in Linux. Some tips include:
- Use consistent tooling, which can provide streamlined development and verifiable results across platforms and infrastructure.
- Use productivity tools: Not everything needs to be hand coded. By using appropriate tooling, such as mocking frameworks, you can test even difficult-to-test code.
- Pick the right test: There is no single type of test that is always appropriate. There are times in which unit tests should be performed while sometimes integration tests are the appropriate path to take.
- Automate, automate, and automate automation gives a way to repeat operations and releases us from trying to remember all scenarios. Automation also means that new developers or developers who are returning to their legacy code can work on existing code without worry that they forgot something.
- Use isolated tests wiselyIsolation in testing is good, but you don't need to isolate everything outside the method. Isolate where things slow tests down (such as databases, file systems, or Web servers). Also, the user needs to interact when third-party components are involved. Isolation may not be necessary if the tests run quickly.
Linux's role as a platform for software development is well established, and it is likely to only increase. Linux presents unique challenges to test code due to the fluid nature of many community development projects, scattered toolsets, and extensive legacy code. However, these challenges are not insurmountable. By using best practices, such as integration and unit testing, and proper tools, even the most difficult code can be tested.
Gil Zilberfeld is Product Manager of Typemock, The Unit Testing Company. Avi Hein is Typemock's Community Manager.