In a previous blog post entitled Android Native Libraries for Java Applications, we discussed how to integrate native C++ code libraries with Java applications for Android. In that post we described how to use the Android Open Source Project to accomplish the integration.
Since that post was written in September 2009, the Android team has released the Android Native Development Kit (NDK). This extracts relevant portions of the Android Open Source Project into an easier to use system for working with C++ native code on Android. The NDK has been updated several times and is now a very useful tool for C++ development on Android.
Android developer support for Java is quite robust. The developer tool set allows writing and debugging with relative ease. With the addition of the JIT (just in time) compilation feature in Android 2.2, the performance of Java on Android has improved several fold.
Android also offers the NDK to support C++ native code. Although this was initially offered mostly to improve performance for computationally intense portions of your application, that need is greatly reduced by the JIT Java compilation capability offered in Android 2.2. However for extreme gaming performance, C++ is still faster than even JIT compiled Java code.
For the largest number of mobile OS platforms, C++ is the common denominator. Of the currently available major platforms, only RIM Blackberry does not allow applications written in C++ code. So when deciding to port an existing mobile application to Android, reusing a large body of existing C++ code can be a major consideration.
That is certainly true for our Aton Connect application as we ported it to Android from Windows Mobile. This application contains about 40,000 lines of C# managed code for the GUI, and about 100,000 lines of C++ code related to communication protocols, security algorithms and screen display processing. The prospect of rewriting 100,000 lines of field tested C++ code into Java was simply not an option.
In this blog post, I will review some of the issues and resolutions we used in porting our Windows Mobile application to Android. Hopefully, it will give you some insights in how you can port your application to Android.
Our Windows Mobile application was developed with Visual Studio along with the Windows Mobile SDK. This combination provides the ability to integrate the C# and C++ code along with the application installer into a single “solution” definition that encompasses multiple project types, C#, C++, and installer. Overall, this a very powerful development environment for mobile application development. One gets very enamored with the ability to visually step through C++ code, holding the mouse over variable names to get their current value in the mobile device, etc.
The Android development tools environment provides capabilities equivalent to Visual Studio for Java development. However, capabilities for Android C++ code are primitive at best. It is back to the 1980’s gdb command line for debugging C++ code. And even setting up for debugging Android C++ code with gdb is an exercise in command line minutia.
So rather than attempting to do initial C++ debugging in the Android environment, we elected to perform primary debugging in the full Linux desktop (Ubuntu) environment. Once that was completed, we moved to debugging what remained directly in Android using the much clumsier gdb debugger.
We elected to use NetBeans rather than Eclipse on Ubuntu for first round debugging, mostly because the GUI was more similar to Visual Studio. One can also do the work using Eclipse if desired. The idea here is to use the Linux standard tool chain and run time libraries for initial debugging.
The Android run time environment is a subset of the full Linux environment, so as we ported code, we needed to continually check to see if the function we anticipated using was actually supported in Android as well as Linux. This is accomplished by searching the folder tree …android-ndk-r4buildplatformsandroid-8arch-armusrinclude for the function declaration. Where several alternatives exist for mapping a Windows API function to Android, we selected the Posix method over more historical alternatives.
The overall application design is to encapsulate the C++ code into shared libraries that interface with Java via JNI (Java Native Interface). In our case, this means creating C++ wrapper code to interact with JNI for the Android code and a Platform Invoke wrapper for Windows Mobile.
We used conditional compilation to allow a single body of C++ source code to be utilized for both Android and Windows Mobile. The Linux OS is supported as a guest client on the Oracle (Sun) VirtualBox VM environment running on Windows 7. Native Android development was later done using the Android NDK running over Cygwin on the Windows 7 host machine. The shared folders feature of VirtualBox is used to allow NetBeans to access the single source code base located on the Windows 7 host. This allowed parallel use of Visual Studio and NetBeans on the same source code base, greatly easing searching for relevant data in the process of adapting Windows Mobile OS calls to the Android/Linux OS API.
Once we adjusted for all of the C++ and OS API differences between Android and Windows Mobile, we got a clean build in the standard GNU C++ Linux development environment under the NetBeans IDE, and a clean build in Visual Studio on the same source code base. With the techniques described in the Android NDK HOWTO.TXT file in the Docs folder we also get a clean build using the Android NDK tool chain. Testing using the Windows Mobile build helps validate that none of the application logic was broken in the process of adding in the Android porting code. Using the desktop Linux build, we can use NetBeans to do the majority of the debugging on the C++ code. Finally we do the remainder of the debugging in the Android environment itself.
Next step is to write a simple test harness in Java to exercise the ported application on Linux. The test harness and the ported C++ code are then debugged using the NetBeans IDE and its visual debugging facility for C++ code. This is especially helpful to debug the arcane and error prone JNI interfaces. Writing a test harness in Java is essential to test the complex JNI interfaces. Note that the Java classes used to develop the Android GUI for your application are substantially different from those available on desktop Linux. So a small, custom Java test harness to exercise the JNI interfaces is all the Java that makes sense to do on desktop Linux.
The last step is move all debugging and testing over to the Android environment. Presumably most of the C++ errors have been caught while testing on Linux. Any remaining errors can be identifies using the classic “printf” technique, or setting up and using the command line gdb debugger. The Java code on Android can be tested and debugged using the visual debugging features of Eclipse.
PORTING ISSUES AND FIXES
There are a variety of issues when porting a large body of code to Android. We go through some of the major ones here. For a number of the solutions, we have borrowed code from the Wine open source project and from the Android open source project itself. We will be publishing these derivations as a separate open source project on github.
- C++ exceptions, RTTI and Standard C++ Library
The Android team chose to not support these features because they felt that the impact on performance and memory footprint would be excessive. That is a reasonable assessment, but the lack of these features can be a real headache for developers wishing to port large bodies of C++ code that utilize them. At least one developer is attempting to make these features available for Android. You can review his offering at http://www.crystax.net/android/ndk.php. We do not utilize these features and have not tested this offering.
- Structured Exception Handling (SEH)
This Microsoft C++ feature is not supported in Linux. We made some use of this and replaced the __leave directive with the C language goto statement. Exceptions must be handled using the Linux signal mechanism which is quite a bit different from SEH.
- Standard Template Library (STL)
This is a widely used template library but not shipped with Android. There are open source versions of STL available, for instance STLPort and uSTL. One STLPort distribution for Android (but which we have not tested), is available here: http://www.anddev.org/viewtopic.php?p=29939.
- Wide and Narrow Character String Conversions
This is quite a complex issue when porting an application because of the multiple methods and standards in existence. Windows Mobile (Windows CE) standardized on the two byte per character unit UTF-16 and with rare exception, the ANSI or one byte per character unit native APIs were eliminated. The C# language and .NET Compact Framework also utilize UTF-16.
The Linux and Android native API’s rely on single byte per character unit, null terminated strings. A wide C++ character on Linux is 4 bytes per character as opposed to 2 bytes per character unit on a Microsoft platform. One effect is to double the length of all wide character strings including string literals preceded by the L character.
One possibility is to translate UTF-16 including surrogate pairs to UTF-8 multi-byte strings which can require one to four bytes for each character, and can contain embedded zero bytes. The Java Native Interface (JNI) provides routines to translate Java UTF-16 into “Modified” UTF-8. The modifications result in a narrow character string containing no embedded zeros, only the zero at the end of the string. Another modification is to translate a four byte UTF-16 surrogate pair into two UTF-8 characters, each three bytes long instead of a single UTF-8 character, four bytes long.
The end result of using JNI routines to translate between wide and narrow strings is that the wide UTF-16 string format is compatible both with Java and Windows Mobile (CE) and the narrow Modified UTF-8 string is compatible with Android / Linux OS API and C run time library.
The Android C run time library (Bionic) contains a wchar.h to implement functions such as wcslen, wcscpy, etc., but as noted in the comments in the header, no actual wide char functions are implemented in the Android C run time library. We resolve this by using the GNU C++ compiler option “-fshort-wchar” which forces the compiler to treat wide characters as two bytes instead of four bytes. This makes the L”string” literal two bytes per character and compatible with UTF-16. We have extracted the actual wide character run time library from the Wine open source project.
It is possible to use JNI as delivered in Android to translate between native C++ wide and narrow strings. This involves a round trip through the Java environment and thus is not very efficient. We elected to extract the core wide/narrow conversion routines from the Android implementation of JNI in the Android open source project so the conversion runs entirely in native code.
The handling of time is substantially different between Linux and Windows. One issue is that the operator changing the time of day can disrupt timers used by your application. A new clock type “montonic” was introduced which allows accurate implementation of functions like the Windows GetTickCount. Android also implements this newer clock type.
- Windows Message Queue
The Windows Message Queue is a very powerful and lightweight mechanism widely used for communication in threads and processes. The native equivalent on Linux has more features but can be considerably slower as it relies on the file system for its implementation. We elected to implement a subset of the Windows message queue functions that also include the Windows timer message and this code will be included as part of the open source project we are placing on github.
- Enums and Structs
The C++ native function often can refer to an enum or struct as a parameter. To work through JNI, you must define equivalent enums and structs in Java. Java does provide language support for Enums, but it is a bit awkward to use unless you use the default itemization that has the first member with a value of zero and the rest monotonically increasing by one. I find it easier to define enums as a class, define the named and initialized int values as “public static final”, and then return the particular enumerated value as class.name.
Since Java has no language support for structs, use a class as a replacement. As with classes implementing enums, you can avoid unnecessary object generation and resulting load on the garbage collector by utilizing public fields rather than traditional getter/setter functions.
- By Reference Parameters for Functions
Although Java objects in theory are passed by reference to a function, the reality is that the pointer to the object is passed by value to the native function. If the object is immutable, such as an instance of the Integer class, then the native function cannot legally pass back integer data to that object. We created a set of classes parallel to classes like Integer that contain public fields of the appropriate type that are not declared final. Native code can then legally return data back to Java through these objects. In general, you can return data to the Java program via the return value of the function, or via non-final fields in a class passed as a function parameter.
This is the command line utility provided in the JDK to generate C/C++ headers from Java source code. One of the command line options can reference the CLASSPATH environment variable. Unfortunately, installing or updating Apple QuickTime can change this environment variable unexpectedly. This can cause Javah to break. Setting the CLASSPATH variable back to its original setting fixes the Javah problem and does not affect QuickTime.
- Registry Functions
The registry functions could be replaced by functions that manipulate INI style configuration files. A much better choice for Android is to use the “preferences” classes implemented in Java. The means to do this is via JNI calls back into the Java preferences classes. The code to implement the Windows registry functions interface with JNI is part of the open source project we have created. By using the preferences mechanism instead of independent configuration files means that both native C++ and Java code can operate on a consistent set of configuration information.
Other posts in this series:
- Developing An Android Mobile Application
- Android Software Development Tools – What Do I Need?
- Android Native Development on Ubuntu 9.04 (Jaunty Jackalope)
- Android Native Development Using the Android Open Source Project
- Android Native Libraries for Java Applications
- Porting Native Code to Android: From Business Case to Coding
- Pure Native App or Java Shell for Android Market?
- Integrating the Android NDK C++ Native Support into Eclipse Using Sequoyah
With 20+ years as a top software and firmware developer, Charles Wilde has acquired a combination of proven business smarts, mobile development skills and device engineering expertise that is hard to match. Charles is available to consult with you and your team about native code development in Android, Windows Mobile or Windows CE. Wilde is author of the e-book, Porting Native Code to Android. He can be reached at AtonMail ( at ) aton (dot)com. © 2010 Aton International, Inc.