The NetBeans Helper
The NetBeans Helper class is responsible for implementing all the code that is NetBeans-specific. The generic build_system_generator.py script is using this helper to generate all the NetBeans project files (inside the nbproject directory) for each project and a project group that includes all the generated project. Let's take a closer look at this class. The first thing it does is import some useful system modules and then import the BaseHelper and Template classes from the build_system_generator module (as well as the 'title' function for debugging purposes):
#!/usr/bin/env python
import os, sys, string
from pprint import pprint as pp
sys.path.insert(0, os.path.join(os.path.abspath(os.path.dirname(__file__)), '../'))
from build_system_generator import (BaseHelper,
Template,
title)
Then the Helperclass is defined. This is the class that the build system generator module is using to customize the build system generation for each specific target (NetBeans 6.x in this case). The __init__() method accepts the templates_dir, which is the path to the root of the templates directory used to generate all the build files. It also initializes the separator ('/') and line separator ('\n') to Unix values to make the generated files fit well in their intended environment. The skip_dir is used to tell the recursive drill-down code that looks for projects in sub-directories to ignore directories called 'nbproject' (which is the special sub-directory used by NetBeans to store the build files). The other methods this class implements are: get_templates(), prepare_substitution_dict(), and generate_workspace_files().
class Helper(BaseHelper):
"""NetBeans6 helper
"""
def __init__(self, templates_dir):
BaseHelper.__init__(self, templates_dir)
self.sep = '/'
self.linesep = '\n'
self.skip_dir = 'nbproject'
def get_templates(self, template_type):
...
def prepare_substitution_dict(self, ...):
...
def generate_workspace_files(self, name, root_path, projects):
...
get_templates()
The get_templates()method is pretty simple. For each build file there is a corresponding template file. These template files are just skeleton of real build files, with some place holders. You will see all the template files soon enough. The get_templates() method just iterates over all the template files (located in the nbproject) and adds a template for the Makefile in the project directory itself. For each such build a file a Template object is generated. Finally the list of Template objects is returned.
def get_templates(self, template_type):
result = [Template(os.path.join(self.templates_dir,
template_type,
'Makefile'),
'Makefile',
template_type)]
nb_project = os.path.join(self.templates_dir, template_type, 'nbproject')
assert os.path.isdir(nb_project)
for f in os.listdir(nb_project):
project_file_template = os.path.join(nb_project, f)
if not os.path.isfile(project_file_template):
continue
filename = os.path.join(nb_project, f)
relative_path = '/'.join(['nbproject', f])
result.append(Template(filename, relative_path, template_type))
return result
prepare_substitution_dict()
This method is the heart of NetBeans6_Helper class. It is responsible for creating a substitution dictionary that contains all the values to be substituted into the templates of each build file. This is not so trivial because some place holders are supposed to be replaced by dynamic content that is generated on the fly. In addition, as you saw earlier NetBeans has quite a few build files. The prepare_substitution_dict() method has several nested function to assist in prepare the substitution dictionary for each one of them. The nested functions are:
- prepare_makefile() for generating the Makefile-Debug.mk and Makefile-Release.mk files
- prepare_configurations_xml() for generating configurations.xml
- prepare_project_properties() for generating project.properties
- prepare_project_xml() for generating project.xml
The substitution dict for the project's main Makefile is empty because it is a generic file that doesn't have any place holder and the substitution dict for the Makefile-impl.mk file contains only the name of the project so no helper function is necessary. Here is the code of the method (without the nested functions). It accepts a long list of arguments that the various nested functions use to generate the proper values. The operating system and the dynamic library extension are also determined here. This method is called multiple times with different template names (each template_name corresponds to a build file) and prepare_substitution_dict() calls the proper nested function or generates the dict directly (for Makefile and MakeFile-Impl.mk).
def prepare_substitution_dict(self,
project_name,
project_type,
project_file_template,
project_dir,
libs_dir,
dependencies,
source_files,
header_files,
platform):
if platform.startswith('darwin'):
operating_system = 'MacOSX'
ext = 'dylib'
elif platform.startswith('linux'):
operating_system = 'Linux'
ext = 'so'
temaplate_name = os.path.basename(project_file_template)
if temaplate_name == 'Makefile':
return {}
if temaplate_name == 'Makefile-Debug.mk':
return prepare_makefile('Debug', operating_system)
if temaplate_name == 'Makefile-Release.mk':
return prepare_makefile('Release', operating_system)
if temaplate_name == 'Makefile-impl.mk':
return dict(Name=os.path.basename(project_dir))
if temaplate_name == 'configurations.xml':
return prepare_configurations_xml(operating_system)
if temaplate_name == 'project.properties':
return prepare_project_properties()
if temaplate_name == 'project.xml':
return prepare_project_xml(dependencies)
assert False, 'Invalid project file template: ' + temaplate_name
return {}
Now, let's examine one of nested functions. I chose the prepare_makefile() function because it is not trivial. The keys in its substitution dictionary are: 'ObjectFiles', 'CompileFiles', 'LinkCommand', 'LDLIBSOPTIONS', 'BuildSubprojects', 'CleanSubprojects', 'OperatingSystem' and 'DynamicLibExtension'. Some of these are simple strings like 'OperatingSystem' and 'DynamicLibExtension'. Others are much more complicated like 'CompileFiles', which is a list of compile commands where each command itself requires a template with substitution values such as 'File', 'CompileFlag', 'Platform' and 'FPIC'. The link command depends on the project type and ldliboptions depends on the platform. Here is the code:
def prepare_makefile(conf, operating_system):
compile_flag = '-g' if conf == 'Debug' else '-O2'
d = dict(Name=project_name)
object_file_template = ' ${OBJECTDIR}/%s.o \\\n'
object_files = ''
for f in source_files:
f = os.path.splitext(os.path.basename(f))[0]
object_files += object_file_template % f
# Flag for dynamic libraries
fpic = '-fPIC ' if project_type == 'dynamic_lib' else ''
# Get rid of last forward slash
if len(object_files) > 2:
object_files = object_files[:-3]
d['ObjectFiles'] = object_files
compile_file_template = \
'$${OBJECTDIR}/${File}.o: ${File}.cpp \n' + \
'\t$${MKDIR} -p $${OBJECTDIR}\n' + \
'\t$$(COMPILE.cc) ${CompileFlag} -I../.. ${FPIC}-o $${OBJECTDIR}/${File}.o ${File}.cpp\n\n'
t = string.Template(compile_file_template)
compile_files = ''
for f in source_files:
f = os.path.splitext(os.path.basename(f))[0]
text = t.substitute(dict(File=f,
CompileFlag=compile_flag,
Platform=platform,
FPIC=fpic))
compile_files += text
# Get rid of the last two \n\n.
compile_files = compile_files[:-2]
d['CompileFiles'] = compile_files
link_command = ''
if project_type == 'dynamic_lib':
if platform.startswith('darwin'):
link_command = '${LINK.cc} -dynamiclib -install_name lib%s.dylib' % project_name
else:
assert platform.startswith('linux')
link_command = '${LINK.c} -shared'
d['LinkCommand'] = link_command
ldlibsoptions = ''
if dependencies != []:
ldliboption_template = '../../hw/%s/dist/%s/GNU-%s/lib%s.a'
ldlibsoptions = ' '.join([ldliboption_template % \
(dep.name, conf, operating_system, dep.name)
for dep in dependencies])
if operating_system == 'Linux':
ldlibsoptions += ' -ldl'
d['LDLIBSOPTIONS'] = ldlibsoptions
build_subproject_template = '\tcd ../../hw/%s && ${MAKE} -f Makefile CONF=%s'
clean_subproject_template = build_subproject_template + ' clean'
build_list = [build_subproject_template % (dep.name, conf) for dep in dependencies]
clean_list = [clean_subproject_template % (dep.name, conf) for dep in dependencies]
d['BuildSubprojects'] = '\n'.join(build_list)
d['CleanSubprojects'] = '\n'.join(clean_list)
d['OperatingSystem'] = operating_system
d['DynamicLibExtension'] = ext
return d
Note, that there are better ways to accomplish this task. There are several third-party template languages like Genshi, Mako, Tempita and Jinja. These template engines can handle the nested templates that prepare_makefile() generates manually in a much more natural way. The code could have been much shorter and concise. I made a deliberate decision to use only standard Python libraries in the interest of keeping the scope of this project limited. Choosing a particular template language/engine would have made the code shorter, but required the reader to understand an additional language and might antagonize fans of other template languages.
The other prepare_XXX() nested functions are all very similar to make_makefile() although some of them generate XML files and another one generate a properties file (INI file like).
generate_workspace_files()
This method is responsible for generating the project groups in the user account. The reason the method is called generate_workspace_files() is that the method is defined in the generic Helper base class and NetBeans6_Helper is just overriding it. So, the NetBeans-specific term "Project Group" is not used here. The code itself is pretty simple. It either creates or updates the proper .properties files that dictate the contents of the project groups as explained earlier:
def generate_workspace_files(self, name, root_path, projects):
"""Generate a NetBeans project group for all the generated projects
"""
base_path = \
'~/.netbeans/6.7/config/Preferences/org/netbeans/modules/projectui'
base_path = os.path.expanduser(base_path)
if not os.path.exists(base_path):
os.makedirs(base_path)
# Create a project group
groups_path = os.path.join(base_path, 'groups')
if not os.path.exists(groups_path):
os.makedirs(groups_path)
text = """\
name=%s
kind=directory
path=file\:%s"""
group_filename = os.path.join(groups_path, name + '.properties')
open(group_filename, 'w').write(text % (name, root_path))
# Make it the active project
text = 'active=' + name
open(os.path.join(base_path, 'groups.properties'), 'w').write(text)
The NetBeans Project Templates
The substitution dictionaries are very important of course, but they can't do much by themselves. Each build file is generated by substituting the values from the proper dictionary into the proper template file.
As you recall NetBeans can build three types of projects: static library, dynamic library and a program. for each one of them there are templates of all the build files. A few templates are the same for some or all project types, so an identical copy is kept for each one. The templates are organized in the following file system structure:
project_templates
NetBeans_6
dynamic_lib
Makefile
nbproject
configurations.xml
Makefile-Debug.mk
Makefile-Impl.mk
Makefile-Release.mk
project.properties
project.xml
program
Makefile
nbproject
...
static_lib
Makefile
nbproject
...
This regular structure mimics the structure of the build files inside a project directory and allows the generic part of ibs to apply the substitution dicts to the templates blindly and end up with the correct build file in the correct place. Note, the project.properties file that wasn't mentioned earlier. This is an empty file that doesn't seem to have a role in C++ projects, but I keep it there to be consistent with NetBeans.
To create the template files I simply took the various NetBeans build files and replaced anything that was project-specific (like the source files or list of dependencies) with a place holder.Let's examine a couple of template files. Here is the main part of the Makefile-Debug.mk of the 'program' project type:
# Link Libraries and Options
LDLIBSOPTIONS=${LDLIBSOPTIONS}
# Build Targets
.build-conf: $${BUILD_SUBPROJECTS} dist/Debug/GNU-${OperatingSystem}/${name}
dist/Debug/GNU-${OperatingSystem}/${name}: $${BUILD_SUBPROJECTS}
dist/Debug/GNU-${OperatingSystem}/${name}: $${OBJECTFILES}
$${MKDIR} -p dist/Debug/GNU-${OperatingSystem}
$${LINK.cc} -o dist/Debug/GNU-${OperatingSystem}/${name} $${OBJECTFILES} $${LDLIBSOPTIONS}
${CompileFiles}
# Subprojects
.build-subprojects:
${BuildSubprojects}
# Clean Targets
.clean-conf: $${CLEAN_SUBPROJECTS}
$${RM} -r build/Debug
$${RM} dist/Debug/GNU-${OperatingSystem}/${name}
# Subprojects
.clean-subprojects:
${CleanSubprojects}
The placeholder are expressions of the form ${Place holder}. This is the format used by the string.Template class. Unfortunately, this convention is used by make files a lot too for environment variable, defined symbols and make variables. So, when a $ sign is part of the make file it is escaped by additional $ sign. For example, $${MKDIR} will not be treated as a place holder by the ibs.
Here is a simpler template of the project.xml file. It is just a bunch of XML with two place holders for the name of the project and its dependencies:
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://www.netbeans.org/ns/project/1"> <type>org.netbeans.modules.cnd.makeproject</type> <configuration> <data xmlns="http://www.netbeans.org/ns/make-project/1"> <name>${Name}</name> <make-project-type>0</make-project-type> ${MakeDepProjects} </data> </configuration> </project>
