Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Channels ▼
RSS

JVM Languages

Measuring Java Reuse, Productivity, and ROI


David is a senior consultant with CrossLogic Corp. and coauthor of Enterprise Java Programming with IBM WebSphere (Addison-Wesley, 2003). He can be contacted at [email protected].


Java has moved the once-niche object technology into mainstream application development—even competing .NET technology leverages object technology. The now cliché marketing pitch for object technology is the ability to create "real-world type objects." Nontechnical folks in the field can easily grasp the power of treating software like widgets (another word for objects) and fitting these widgets together like LEGOs to build applications. In this vision, a whole new marketplace of prebuilt (reusable) widgets would exist to produce custom applications quickly and with few defects. Instead of programmers, end users or domain experts could assemble applications. This is the elusive reuse panacea, which still drives marketing for some tooling.

Indeed, the technology has matured enough to support this vision, especially from the available processing horsepower and graphic sophistication available in most workstations and laptops of even low-end machines. Products are available that allow software to be created in a WYSIWIG fashion, and they can be productive when generating standalone applications. However, other issues come into play—maintenance, modification, and integration—which require programming activities. Also, these solutions are typically vendor driven, and therefore, don't benefit from the economy of scale that standardization provides.

As it turns out, the pursuit of an idealistic plug-and-play vision of object technology has provided a windfall of productivity and quality in another way. The ability to bind data and operations in a single unit, and then generalize these implementations through abstraction and polymorphism, has closed the chasms between analysis, design, and construction. The ability to iteratively turn domain-specific elements into software has increased enthusiasm for object technology as seen in the current Java marketplace. However, most organizations only perceive productivity and reusability efforts when applying Java. In this article, I present a way to measure productivity and reusability that occurs in Java development projects, and examine why these are important measurements to capture. Also, the openness of Java and the current open-source movement has provided many ready-to-use frameworks. This article also explains how to determine cost savings so that return on investment (ROI) can be determined.

Any given IT organization has to make decisions on technology, training, consulting services, and tooling. These decisions are validated by customer satisfaction and reinforced by being within schedule and budget constraints. In theory, as experience occurs and best practices are applied, an organization should become more productive and deliver higher quality applications. Designers and developers exploit object-oriented principles for agility and for producing reusable frameworks, components, utilities, and so on. But how does an organization determine if it is becoming better at what it does? Relying on customer satisfaction and budget/schedule parameters indicates success, but does not reflect potential efficiencies either good or bad. By measuring reuse and productivity, an organization can prove that practices and approaches are paying off by making them more efficient. So how can reuse and productivity be measured?

Many elements are generated from an IT project, but arguably the most important element is the source code. This is the end product of the project endeavor. Therefore, how productively source code is produced reflects the efficiency of an organization. Likewise, the proportion of source code produced to source code reused provides a way to measure reuse. Increased reusability should lead to increased productivity.

Counting Source (ESLOC)

An accepted way to determine the size of a software application is to report the amount of "effective source lines of code" (ESLOC) for a given application. "Effective" means whitespaces and comments are not considered. Assume expressions, keywords, and a control statement encountered until a line break occurs constitute a line of source code. Consider Example 1, which illustrates an ESLOC value of 9.

In Java, a line of code does not necessarily contain just one keyword or expression of logic. Because it's free form, a single line of code could contain many keywords and expressions. Likewise, a control block statement could occupy a single line of code. It could be argued that only expressions of keywords should be considered, where control block statements are ignored. Also, package statements aren't considered because they are usually applied automatically by an IDE without consideration, as opposed to import statements, which do require developer decision even with code-assist mechanisms.

Instead of delimiting a line of code with a carriage return, semicolons could be counted. Another more fine-grained approach could also be taken by counting each object receiver (message) expression. These are valid counting approaches; however, consistency is the key—as long as all applications are counted using the same methodology, an accurate measurement for comparison can be assumed when comparing project ESLOC values. Of course, developers have varying styles, but you can assume individual style is consistently applied through all projects particular developers participate in. For this article, I take the carriage return approach.

Counting Reused ESLOC (Reuse Percentage)

The size of a development project is reflected in its ESLOC value—the proportion of ESLOC generated by the project compared to the ESLOC reused brought to the project; in other words, "Reused" indicates the reuse percentage that has taken place (Reuse % = [Reused ESLOC/Total ESLOC]).

How is the reused ESLOC count determined? Assume sets of reused classes are defined in their own packages, and that classes are not considered reused if they occupy a package defined for the application. In other words, if effort has not been put in separating classes into standalone package boundaries, then credit for reuse should not be given. Figure 1 depicts application classes and reused class relationships.

In Figure 1, two application classes reference reused classes defined in the reused package, com.example.test.reused. Reuse ESLOC is determined by counting the ESLOC for the classes referenced by the application. If a class resides in a reused package that is not referenced, then it is not considered; this is the case for ReusedClassE. Therefore, its ESLOC value of 10 is not included. However, classes referenced through inheritance or aggregation/reference associations are considered in counting ESLOC. Also, ESLOC counts for a reused class are considered only once, even if referenced multiple times.

In Figure 1, ESLOC values are shown next to each class, and the reuse percentage is calculated by totaling referenced classes ESLOC (40) and dividing this value by the totaled application ESLOC values (130). This yields a reuse percentage of 30 percent. The only reused class not considered is ReusedClassE, which is not referenced directly by the application classes or indirectly by a reused class.

Reused ESLOC Class versus Method

You could argue that if all the methods in a reused class are not exercised by the application, considering all the ESLOC for a class is not accurate. While this might be true, this logic would mean expressions in a method appearing in control flow blocks may or may not be executed during execution and would also not provide an accurate ESLOC amount. Instead, the level of granularity for reuse is at the class level. Because methods can't stand by themselves, ESLOC counts consider all the methods in a class, which reflects its potential reusability. If a class has many methods that are infrequently used, then a possible design problem exists. Even though it may contribute to an inflated reuse percentage, you are kidding yourself with a maintenance issue due to a design that is not functionally decomposed.

ESLOC Counting Mechanisms

Manually counting ESLOC is not an advised endeavor because any productivity gains would be quickly nullified by the amount of time it would take to count lines of code. Commercial and open-source metric tools commonly report ESLOC values. For instance, a utility specifically built to scan source and report reuse and ESLOC values for a set of applications is freely available at http://www.crosslogic.com/esloc/. The graphs and ESLOC values I present in this article were generated from this utility.

What Not to Count for Reuse

Everyone developing in Java is experiencing reuse. Java is object based, and therefore, a core set of packages are supplied and reused when any application is developed. Likewise, Java's maturity has lead to a healthy number of ready-to-use frameworks. Java-base classes and third-party framework ESLOC counts could be considered. However, if an organization is striving for reuse, a decision should be made on which frameworks to consider. Classes defined in the java.lang package should not be considered because they are omnipresent and immutable. Things get a little gray when it comes to third-party frameworks that perform horizontal-type functionality (such as persistence) or cross-cutting functionality (such as security or logging). Because persistence frameworks such as Hibernate (http://www.hibernate.org/) provide a definite reuse benefit, time spent on rolling your own persistence framework can be wasted, and ESLOC amounts should be considered when computing reuse percentage. The mechanism used to compute ESLOC values for reused elements should allow for a way to be selective when obtaining ESLOC numbers so comparisons can be made when computing productivity numbers.

JSPs and Other Resources

Most J2EE applications are constructed with elements other than just Java source code. These can include database schemas, files to hold external properties for an application, and XML files for deployment configuration information. In most cases, these elements support an application or reusable framework and don't stand alone as reusable elements, but are considered as a piece of a given framework or component. This means they don't need to be considered when determining ESLOC values.

On the other hand, web-based applications could utilize a significant amount of dynamic HTML produced using Java Server Pages (JSP) technology. JSP elements are files that contain both HTML document tags and embedded Java source. JSPs could be scanned and ESLOC amount tallied for blocks of embedded Java source. Inline expressions could count as a single ESLOC amount. Of course, if tag libraries were completely utilized, source would not appear in the JSP.

Another approach is not to consider JSP elements at all in ESLOC counts. If an organization uses an approach that minimizes application logic from appearing in JSP elements, then the little that does can be considered inconsequential.

What Can These Measurements Tell Us?

Gathering and comparing ESLOC amounts on an application basis can provide information that benefits management and developer decisions. Some might look at this exercise with a "big brother is watching" attitude. However, if you believe in the productivity of Java, then why not have the ability to prove it and see this efficiency. Here are some of the things you can elicit from capturing ESLOC amounts:

Size. The resulting total ESLOC for a given application can be used to reflect the size of a given project. Figure 2 shows the size (ESLOC) of a set of applications.

Reuse. Capturing reused ESLOC helps validate whether productivity is increasing; if not, the choice of reuse efforts may not be paying off. Also, reuse percentages should be somewhat consistent between projects. Big discrepancies in reuse percentages between projects could indicate lack of communication in common frameworks or components, or inconsistent development approaches. Most organizations have discovered the value of using a common application architecture; therefore, all projects should gain similar reuse benefits. Figures 3 and 4 present reuse percentages for some example applications. You'll notice that the CRM Rewrite application has 0.0 percent reuse. This should be investigated; it may not be using available architectural frameworks or components.

Productivity. All the man-hours expended for a given project (including analysis, design, and testing) spread across a projects total ESLOC reflect the productivity of a given project. Monitoring this number helps validate whether project development is becoming more efficient. The ESLOC per man-hour should improve with each project. If the amount of ESLOC per hour is decreasing, then why is this happening? Is analysis paralysis occurring—too much time revamping requirements or analysis? Are incomplete specs being provided? Are developers not adequately unit-testing code—lots of bugs generated during system or acceptance tests? Are opportunities to reuse frameworks or generalized solutions causing the wheel to be reinvented each time?

Table 1 shows productivity (ESLOC/man-hour) amounts for some sample projects. It's easy to see which project has been more productive. The billing system produced 30 ESLOC per hour, while the CRM Rewrite project only produced 10 ESLOC per hour. One clue to productivity may be reuse: The billing system had a reuse percentage of 40 percent, which is a 66-percent reuse increase over the CRM rewrite. The billing system was so much more productive appears to be due to the lower reuse percentage. This should be investigated; it could be a simple matter of lack of communication, where solutions were built when a reusable solution already existed.

The billing system's overall productivity produced 30 lines of code every man-hour. twelve lines of code were reused per man-hour (reuse 40 percent), leaving 18 to be effectively produced per man-hour by the development team. Knowing the effective line of application ESLOC produced for an application can help gauge a team's productivity outside of productivity given due to reused ESLOC.

Return on Investment. Java developers have numerous choices in tools, frameworks, and practices. Some of these require investment in time; some require an investment of hard dollars. In any case, the motivation should always be to gain more efficiency, and not just falling into a cult of "framework du jour."

ESLOC per man-hour measurements presented here reflect the impact of a new development tool or the introduction of a new persistency framework. Hopefully, the new persistency framework allows mapping object models to data sources to happen quicker and more accurately than the replaced mechanism. This should be reflected in an increased ESLOC/man-hour number and maybe an increased reuse percentage.

Return on investment or savings that the new framework provides is calculated by multiplying the ESLOC reused in the framework by the effective cost per ESLOC man-hour determined for the project. The cost per line of code is determined by dividing the effective ESLOC produced for the application by the total dollars expended for the project.

When calculating cost savings of a framework, the cost calculation does not include the total reused ESLOC, only the ESLOC produced during the project. This accurately reflects the effectiveness of a given team's analysis, design, construction, and testing abilities. Considering reused ESLOC would jade the actual cost for each line of code:

Cost = (ESLOC Produced for Application) / Project Dollars

Savings = Cost × (Framework ESLOC Utilized for Application)

Apply this to the example persistency framework previously presented. Assume the cost of this framework entailed $5000 in resource time to learn and apply to the application. The framework software itself was open source, so it did not have a hard dollar cost. Of course, if it were purchased, this would be included. The total cost from analysis through testing for the billing system was $100,000.

The billing system application utilized 7000 ESLOC of the persistency framework. Based upon the total ESLOC values and reuse percentage for the billing application, Figure 5 details the savings delivered by this framework. This example shows that the persistency framework provided a cost savings of $1470 based on the number of framework ESLOC utilized multiplied by the productivity cost (ESLOC per dollars expended). The return on investment of the $5000 expended for integrating the framework was $3530. Future projects that reuse the persistency framework and integration experience will continue to whittle down against the initial framework expense until the return on investment becomes positive, effectively paying for the investment and then some.

Outsourcing. Outsourcing projects are commonly assumed to be less expensive due to the lower cost per man-hour of IT resources. However, the process of developing a project still involves internal personnel to communicate requirements, manage development, and verify and test this outsourced development. Low quality means more bugs and maintenance issues. One way to verify an outsourcing decision is to obtain the cost of a single ESLOC produced for a project. When determining this cost, all dollars spent on the project have to be factored in, including analysis and testing issues, not just the money spent on the outsourced effort. This can be compared against the ESLOC cost of internal projects. Outsourced projects that don't produce better cost-per-line of ESLOC may need to be examined for logistic issues, quality, or type of project. In any case, this measurement can help validate outsourcing decisions.

Estimation. Estimating development projects is a difficult task. Many approaches are applied, and if a perfect method were found, it would revolutionize the IT industry. Instead, we are forced to extrapolate efforts across prototypes, screen mockups, use cases, and requirement documentation. Capturing ESLOC per man-hour for projects, then correlating these numbers with use cases or requirements provides a reference point for use in future projects. For instance, Table 1 shows a productivity of 30 ESLOC per man-hour for the billing system. A project with similar functionality and requirements can assume a similar production rate, and therefore, can be used to validate an estimated cost of development.

Quality. If defects during the testing phases of the project are tracked, then comparing defects to the number of resources will reflect the quality of development efforts. As reuse increases and developers gain efficiency, the ratio of bugs to ESLOC should improve.

Conclusion

Although reuse is a noble goal, many organizations have unrealistic expectations, and therefore, are often disappointed. Others do experience the productivity of Java, but still leave further efficiencies on the table or lack consistency: Some projects are successful, others are not. In this article, I've explained how reuse and productivity can be measured to eliminate a perception of productivity with a proven measurement approach.

DDJ


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.