|
Porting 300,000 lines of 32-bit code to nearly a dozen 64-bit platforms requires careful planning.
February 01, 2006
URL:http://www.drdobbs.com/high-performance-computing/parallel/multiplatform-porting-to-64-bits/184406427
The authors are senior software engineers for Visual Numerics. They can be contacted at http://www.vni.com/.
One project we were recently involved in was the port of a large 32-bit application, which supported 11 platforms to a 64-bit environment. The number of lines of code in this application exceeded 300,000 lines. Considering that the 32-bit application had parts developed several years ago, there was every likelihood that the code had been modified by a variety of developers. For this and other reasons, we suspected that, among other problems, type mismatches that cause problems for a 64-bit port were likely introduced as modules were added or removed over time. We ported the 32-bit application to 64-bit to take advantage of the benefits of 64-bit technology—large file support, large memory support, and 64-bit computation, among other features. Our overall approach was an iterative one that alternated between zooming in on detailed issues such as byte order and refining compiler flags, to stepping back to look at global issues, such as ANSI compliance and future portability of source-code base. Our first step was to research 64-bit resources to learn about each of the 11 operating system's compiler switches, memory models, and coding considerations. To define our starting point, we turned on the compiler warnings for one platform, ran a first build, and examined the build log's messages. With these initial builds and later use of tools such as Parasoft's Insure++ (http://www.parasoft.com/), lint, and native debuggers, we developed a road map of the issues we would encounter. From there, we proceeded to perform a complete inventory of the source code and examine every build configuration.
After initial code modifications, debug sessions, and passes through build messages, we had enough information to sort out and prioritize realistic milestones and the specific tasks required to get there. We reached a significant milestone when we had a running application with enough basic functionality that it could be debugged by running it through our automated test suite, which consists of backward compatibility tests in addition to new tests built to exercise 64-bit features.
If you have several 64-bit platforms as part of your conversion project, you might be tempted to work on one platform at a time. Once the application is running properly on the first platform, you might move on to the next platform, and so on. However, we found significant advantages to working on all platforms at the same time because:
A final consideration in approaching this project was to plan ahead for time required for the final release testing phase. Because our newly modified code base is shared across multiple 32-bit and 64-bit platforms, each 32-bit platform would need to be retested as thoroughly as our newly ported platforms, thereby doubling testing time and resources.
There are a number of issues, ranging from compiler warnings to reading/writing binary data, that you can face when porting 32-bit applications that run on multiple 64-bit operating systems. Luckily, compilers can assist in determining 64-bit porting issues. Set the warning flags of the compilers to the strictest level on all platforms, paying close attention to warnings that indicate data truncation or assignment of 64-bit data to 32-bit data. However, one problem with compiler warnings is that turning on stricter warning levels can lead to an overwhelming number of warnings, many of which were automatically resolved by the compiler. The problem is that major warnings are buried within the mass of minor warnings, with no easy way to distinguish between the two. To resolve this issue, we enabled the warnings on multiple platforms and performed concurrent builds. This helped because different compilers give different warnings with different levels of detail. We then filtered the warnings using information from multiple compilers and were able to determine which warnings needed to be fixed.
Some application requirements call for binary data or files to work with both 64-bit and 32-bit applications. In these situations, you have to examine your binary format for issues resulting from larger longs and pointers. This may require modifications to your read/write functions to convert sizes and handle any Little- or Big-endian issues for multiple platforms. To get the correct machine endianess, the larger data sizes in 64-bit applications require extended byte swapping. For example, a 32-bit long:
Big Endian = (B0, B1, B2, B3)
can be converted to:
Little Endian = (B3, B2, B1, B0)
while a 64-bit long:
Big Endian = (B0, B1, B2, B3, B4, B5, B6, B7)
is converted to:
Little Endian = (B7, B6, B5, B4, B3, B2, B1, B0).
Most compilers will find mismatched types and correct them during the build. This is true for simple assignments as well as most parameters passed to other functions. The real problems lay in the integer-long-pointer mismatches that are invisible to the compiler at compile time, or when an assumption the compiler makes at compile time is what produces a mismatch. The former concerns pointer arguments and function pointers, while the latter primarily concerns function prototypes.
Passing integer and long pointers as arguments to functions can cause problems if the pointers are then dereferenced as a different, incompatible type. These situations are not an issue in 32-bit code because integers and longs are interchangeable. However, in 64-bit code, these situations result in runtime errors because of the inherent flexibility of pointers. Most compilers assume that what you are doing is what you intended to do, and quietly allow it unless you can enable additional warning messages. It is only during runtime that the problems surface.
Listing One, for example, compiles without warnings on both Solaris and AIX (Forte7, VAC 6) in both 32-bit and 64-bit modes. However, the 64-bit version prints the incorrect value when run. While these problems may be easy to find in a short example, it may be more difficult in much larger code bases. This sort of problem might be hidden in real-world code and most compilers will not find it.
Listing One works properly when built as a 64-bit executable on a Little-endian machine because the value of arg is entirely contained within the long's four least-significant bytes. However, even on Little-endian x86 machines, the 64-bit version produces an error during runtime when the value of arg exceeds its four least-significant bytes.
With function pointers, the compiler has no information about which function will be called, so it cannot correct or warn you about type mismatches that might exist. The argument and return types of all functions called via a particular function pointer should agree. If that is not possible, you may have to provide separate cases at the point at which the function is called to make the proper typecasts of the arguments and return values.
The second issue concerns implicit function declarations. If you do not provide a prototype for each function that your code calls, the compiler makes assumptions about them. Variations of the compiler warning "Implicit function declaration: assuming extern returning int" are usually inconsequential in 32-bit builds. However, in 64-bit builds, the assumption of an integer return value can cause real problems when the function returns either a long or a pointer (malloc, for example). To eliminate the need for the compiler to make assumptions, make sure that all required system header files are included and provide prototypes for your own external functions.
There are, of course, issues that may not be readily apparent at the beginning of the project. For instance, in 64-bit applications, longs and pointers are larger, which also increases the size of a structure containing these data types. The layout of your structure elements determines how much space is required by the structure. For example, a structure that contains an integer followed by a long in a 32-bit application is 8 bytes, but a 64-bit application adds 4 bytes of padding to the first element of the structure to align the second element on its natural boundary; see Figure 1.
To minimize this padding, reorder the data structure elements from largest to smallest. However, if data structure elements are accessed as byte streams, you need to change your code logic to adjust for the new order of elements in the data structure.
For cases where reordering the data structures is not practical and the data structure's elements are accessed as a byte stream, you need to account for padding. Our solution for these cases was to implement a helper function that eliminates the padding from the data structure before writing to the byte stream. A side benefit to this solution was that no changes were required on the reader side; see Listing Two.
64-bit long type arrays and arrays within structures will not only hold larger values than their 32-bit equivalents, but they may also hold more elements. Consider that 4-byte variables previously used to define array boundaries and allocate array sizes may also need to be converted to longs. (For help in determining whether existing long arrays should be reverted to integer type for better performance in your 64-bit application, see http://developers.sun.com/prodtech/cc/articles/ILP32toLP64Issues.html.)
In addition to following the standard 64-bit coding practices recommended in your operating system's compiler documentation and noted in the resources listed in the Resources section, here are a few considerations and coding tips that will help when planning a 64-bit migration project:
% isainfo -v
64-bit sparcv9 applications
32-bit sparc applications
Performing a source-code inventory for a large code base shared across several platforms for 32-bit to 64-bit migration and assessing the scope of each change, however trivial, can prove to be a daunting task. The potential to overlook conversion problems and introduce new errors is high. However, by using a small arsenal of 64-bit tools and techniques, many of these potential problems can be caught during the precompilation stage, at compile time, and at runtime. Some of the tools available are:
Taking the time to do up-front planning and investigation is worth the effort. Don't get discouraged when nothing in your application is working correctly. Methodical and careful passes through the code will uncover the problem areas. With available memory and dataset sizes growing tremendously each year, the benefits of a 64-bit application are worth the pain of conversion.
DDJ
#include <stdlib.h>
#include <stdio.h>
int Func1(char *);
int main()
{
long arg, ret;
arg = 247;
ret = Func1((char *)&arg);
printf("%ld\n", ret);
return(0);
}
int Func1(char * input)
{
int *tmp;
tmp = (int *)input;
return(*tmp);
}
Back to articletypdef struct demo{
int i;
long j;
} DEMO;
DEMO test;
/*pout_raw outputs raw bytes to a file */
/* output each element of a structure to avoid padding */
pout_raw ((int) file_unit, (char *) test.i, sizeof (test.i));
pout_raw ((int) file_unit, (char *) test.j, sizeof (test.j));
/* the following line of code includes padding */
pout_raw ((int) file_unit, (char *) test,sizeof(test));
Back to article
warning: implicit function declaration: main
warning: argument does not match remembered type: arg #1
warning: passing 64-bit integer arg, expecting 32-bit integer:
MyProc(arg 7)
warning: assignment of 64-bit integer to 32-bit integer
warning: function argument ( number ) used inconsistently
warning: comparing 32-bit integer with 64-bit integer
Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.