Why this post ?
In this post, I will explain how I build an RCP application using eclipse PDE build. There are several samples on the web but quite often for toy applications with only one plugin and with no tests.
This is not a ready-made template but you will perhaps find some good ideas to use in your build project.
The sample RCP application has
- many features and plugins
- tests, some of them use SWTBot for testing UI.
We want the build to :
- set up development environment (provision source and binary artifacts)
- create an executable application to be used in production
- launch junit tests (some of which use swtbot) during the build and produce junit report and coverage report
- create an executable test application that will run all the tests when launched and will produce a junit-tests.xml file with results (this can be useful to test the product in a specific environment)
Choosing a build system
We have the choice between :
However, I chose Eclipse PDE build and not Buckminster for the following reasons :
- we use ivy at work and ivy is not supported by Buckminster (maven is supported though)
- although documentation is better than before thanks to the Buckminster book, the examples are still quite basic
- I do not know how to run tests using Buckminster (SWTBot tests, code coverage). This seems to be a work in progress (see https://bugs.eclipse.org/bugs/show_...)
That said, PDE build system is hard to use and give sometimes cryptic error messages.
Others links
Some other related projects that may be worth trying :
Some doc/presentations :
SwtBot releng project that can be used as sample :
- http://github.com/ketan/swtbot/tree/master/org.eclipse.swtbot.releng/
Creating products and features
I created two products
- one for production
- one for test. This way, we can run the tests no only on the build machine and the developpement machine but also run them in a specific environment.
The production product (defined in cetl-product.product file) contains the following features :
<features>
<feature id="net.entropysoft.cetl.feature"/>
<feature id="net.entropysoft.transmorph.feature"/>
<feature id="net.entropysoft.dashboard.feature"/>
<feature id="net.entropysoft.jmx.feature"/>
[...]
<feature id="org.eclipse.wst.xml_ui.feature"/>
<feature id="org.eclipse.sdk"/>
<feature id="org.eclipse.gef"/>
<feature id="org.eclipse.emf.edit.ui"/>
<feature id="org.eclipse.emf.edit"/>
<feature id="org.eclipse.emf.ecore"/>
<feature id="org.eclipse.emf.common"/>
<feature id="org.eclipse.emf.common.ui"/>
<feature id="org.eclipse.emf.ecore.edit"/>
<feature id="org.eclipse.xsd.edit"/>
<feature id="org.eclipse.xsd"/>
<feature id="org.eclipse.wst.common.fproj"/>
</features>
The test product (defined in cetl-product-test.product) contains the same features except that it adds the test plugins, swtbot and everything else needed to run the tests :
<feature id="net.entropysoft.cetl.test.feature"/>
<feature id="net.entropysoft.cetl.testcomponents.feature"/>
<feature id="org.eclipse.swtbot"/>
<feature id="org.eclipse.swtbot.eclipse"/>
<feature id="org.eclipse.swtbot.eclipse.test"/>
Note that we include org.eclipse.swtbot.eclipse.test feature here. This feature comes from the swtbot "Headless Testing Framework".
Checkout of the plugins and features
All our plugin and features are at the same level
net.entropysoft.cetl.feature
net.entropysoft.cetl.plugin
net.entropysoft.cetl.plugin.test
net.entropysoft.cetl.product
net.entropysoft.cetl.releng
net.entropysoft.cetl.test.feature
...
You can use a team project set to checkout your projects easily. You can then use svnGetProjectSet task from ant4eclipse if you want to checkout your projects from the team project set.
the releng project
the releng project will contain (most of the directories are created during the build) :
baseLocation/ : the target platform (created during the build)
features/
org.eclipse.rcp_3.5.0.v20090519-9SA0FwxFv6x089WEf-TWh11/
...
plugins/
org.eclipse.core.runtime_3.5.0.v20090525.jar
...
eclipse/ : an eclipse install (created during the build)
ivy/
apache-ivy-2.1.0-rc2 : an ivy install (created during the build)
lib/ : the dependencies retrieved by ivy (created during the build)
bundles/
slf4j-api.jar
...
test
cobertura.jar
...
...
product/
buildConfiguration/ : configuration directory
build.properties : copied from org.eclipse.pde.build and modified according to our needs
buildDirectory/ : directory the build will take place in (created during the build)
product-test/
buildConfiguration/ : configuration directory for test product
build.properties : copied from org.eclipse.pde.build and modified according to our needs
buildDirectory/ : directory the build will take place in (created during the build)
test/
cetl-studio-test/
junit-results/
workspace/
build-product-test.xml
build-product.xml
build.properties
build.xml
dependencies.xml
ivy.xml
materialize.xml
test.xml
I used the same terms (baseLocation, buildDirectory, buildConfiguration) than in
PDE product build
build-product-test.xml, build-product.xml, build.xml, dependencies.xml, materialize.xml and test.xml are ant build files. Build.xml import all the others. This way, we have clear and short xml files instead of one big ant file.
Materializing target platform
The target platform (aka baseLocation) will be used to build the product but will be also used during developpement.
It contains all the pre-built features and plug-ins that our product requires.
ant target
The following ant target will provision the binary artifacts against which to build.
<target name="materializeTargetPlatform" description="materialize the target platform" depends="retrieve-ivy-dependencies">
<delete dir="${baseLocation}" />
<unzip dest="${baseLocation}" overwrite="true" src="${eclipse.zip}">
<mapper type="glob" from="eclipse/*" to="*" />
</unzip>
<unzip dest="${baseLocation}" overwrite="true" src="${eclipse-RCP-delta-pack.zip}">
<mapper type="glob" from="eclipse/*" to="*" />
</unzip>
<unzip dest="${baseLocation}" overwrite="true" src="${gef-runtime.zip}">
<mapper type="glob" from="eclipse/*" to="*" />
</unzip>
<unzip dest="${baseLocation}" overwrite="true" src="${emf-runtime.zip}">
<mapper type="glob" from="eclipse/*" to="*" />
</unzip>
[...Some other dependencies from eclipse.org...]
<unzip dest="${baseLocation}" overwrite="true" src="${swtbot.eclipse.zip}">
<mapper type="glob" from="eclipse/*" to="*" />
</unzip>
<unzip dest="${baseLocation}" overwrite="true" src="${swtbot.eclipse.headless.zip}">
<mapper type="glob" from="eclipse/*" to="*" />
</unzip>
<!-- bundles we depends on and that are in ivy repository -->
<copy overwrite="true" todir="${baseLocation}/plugins">
<fileset dir="lib/bundles" />
</copy>
<!-- don't need other directories than plugins and features. And if we create a .target from this dir it will not be correct if
we keep other directories -->
<delete includeemptydirs="true">
<fileset dir="${baseLocation}">
<exclude name="plugins/**" />
<exclude name="features/**" />
</fileset>
</delete>
</target>
It just uncompress eclipse zip file, RCP-delta-pack, SWTBot and some other packages that our product requires.
${eclipse.zip}, ${eclipse-RCP-delta-pack.zip}, ${swtbot.eclipse.zip} ... refer to a corresponding file. I put all these files in a shared directory (And I do not delete them when I use a newer version so that I can rebuild an old version of the product).
Some things that are worth noting :
- The RCP delta pack is mandatory as it includes the org.platform.launchers.feature which contains the launchers and root files necessary for a product.
- swtbot.eclipse.zip and swtbot.eclipse.headless.zip are necessary as we use the target platform to build both production product and test product
- I remove all directories other than plugins and features as it caused problems when I created a .target file
ivy
"materializeTargetPlatform" target depends on "retrieve-ivy-dependencies" because we use ivy at work and some of our dependencies here are managed by ivy.
For the curious persons, here is an excerpt of the ivy file :
<ivy-module version="1.0">
<info organisation="entropysoft" module="net.entropysoft.cetl.releng">
<description homepage="http://www.entropysoft.net">Content ETL</description>
</info>
<configurations>
<conf name="bundles" visibility="private" transitive="false" description="osgi bundles required for compiling the module. non transitive"/>
<conf name="components-plugin-libs" visibility="private" transitive="false" description="libraries required for net.entropysoft.components.plugin. non transitive"/>
<conf name="test" visibility="private" description="dependencies required for the test compilation and execution phases."/>
[...]
</configurations>
<publications>
</publications>
<dependencies>
[...]
<dependency org="entropysoft" name="components" rev="latest.integration" conf="components-plugin-libs->default"/>
<dependency org="org.slf4j" name="jcl-over-slf4j" rev="1.5.8" conf="bundles->default"/>
<dependency org="org.slf4j" name="slf4j-simple" rev="1.5.8" conf="bundles->default"/>
<dependency org="org.slf4j" name="slf4j-api" rev="1.5.8" conf="bundles->default"/>
<!-- we use cobertura to instrument our plugins -->
<dependency org="net.sourceforge.cobertura" name="cobertura" rev="1.9.2" conf="test->default"/>
</dependencies>
</ivy-module>
We use slf4j in our product. slf4j is already OSGI compatible. But if you need an an OSGI-ready version of a library, you can try at either :
Using the target platform during developement
See the following article to know why creating a target platform is a good thing :
Why create a custom target platform?
I created a .target file and set it as target platform.
As location, I choose "installation" and ${project_loc}/../net.entropysoft.cetl.releng.baseLocation (as I put the target file in net.entropysoft.cetl.product)
Updating the workspace
This step is very specific to your product but you probably have some things to copy, create or modify to have your workspace up and ready.
Here we copy some jars to a plugin and update the help (we use DocBook for the documentation and we have to convert it to eclipse help and to pdf ...)
<target name="materializeWorkspace" description="materialize workspace" depends="retrieve-ivy-dependencies">
<copy overwrite="true" todir="../net.entropysoft.components.plugin">
<fileset dir="lib/components-plugin-libs" />
</copy>
<antcall target="updateHelpPlugin" />
[...]
</target>
The developement environment
<target name="setup" depends="materializeTargetPlatform,materializeWorkspace" />
This time, we have all we need to setup our environment. Once eclipse, subclipse and swtbot have been installed, all we need is to import the team project set and to launch the ant build with setup target. After selecting the .target and refreshing all the projects, they should all be able to compile.
Building the product
Create the build directory
First we create the build directory that is the directory the build will take place in. We copy our plugins and features respectively in "plugins" and "features" subdirectories.
The createBuildDirectory macro is defined as :
<macrodef name="createBuildDirectory">
<attribute name="buildDirectory" />
<attribute name="buildVersion"/>
<sequential>
<delete failonerror="false" includeemptydirs="true">
<fileset dir="@{buildDirectory}">
<exclude name="plugins/**/**.*" />
<exclude name="features/**/**.*" />
</fileset>
</delete>
<sync todir="@{buildDirectory}/features" includeemptydirs="true">
<fileset dir="../">
<include name="net.entropysoft.*.feature/**" />
[...]
</fileset>
</sync>
<sync todir="@{buildDirectory}/plugins" includeemptydirs="true">
<fileset dir="../">
<include name="net.entropysoft.*.plugin/**" />
<include name="net.entropysoft.*.plugin.test/**" />
<include name="net.entropysoft.cetl.product/**" />
<include name="net.entropysoft.cetl.help/**" />
[...]
<!-- also exclude the generated class files -->
<exclude name="*/bin/**" />
</fileset>
</sync>
</sequential>
</macrodef>
using productBuild.xml from pde build
We then invoke the productBuild.xml provided by pde build (see Building an RCP application from a product configuration file)
<property name="product.buildDirectory" value="${basedir}/product/buildDirectory" />
<property name="product.buildConfiguration" value="${basedir}/product/buildConfiguration" />
<!-- make sure to call targets materializeTargetPlatform and materializeWorkspace before -->
<target name="product-build" depends="failIfBaseLocationUnavailable,extract-eclipse" description="creates product">
<createBuildDirectory builddirectory="${product.buildDirectory}" buildversion="${buildVersion}"/>
<java classname="org.eclipse.equinox.launcher.Main" fork="true" failonerror="true" dir="${basedir}">
<arg value="-application" />
<arg value="org.eclipse.ant.core.antRunner" />
<arg value="-buildfile" />
<arg value="${eclipseDirectory}/plugins/org.eclipse.pde.build_${pdeBuildPluginVersion}/scripts/productBuild/productBuild.xml" />
<arg value="-DbaseLocation=${baseLocation}" />
<arg value="-Dbuilder=${product.buildConfiguration}" />
<arg value="-Dbasedir=${basedir}" />
<arg value="-DbuildDirectory=${product.buildDirectory}" />
<arg value="-DJ2SE-1.5=${jdk15Dir}/jre/lib/rt.jar" />
<arg value="-DJavaSE-1.6=${jdk16Dir}/jre/lib/rt.jar" />
<arg value="-DbuildVersion=${buildVersion}"/>
<!-- there is a binary cycle due to slf4j -->
<arg value="-DallowBinaryCycles=true" />
<classpath>
<fileset dir="${eclipseDirectory}/plugins">
<include name="org.eclipse.equinox.launcher_*.jar" />
</fileset>
</classpath>
</java>
</target>
build configuration
product/buildConfiguration/build.properties is a modified copy of the template build.properties from org.eclipse.pde.build/templates/headless-build
product=net.entropysoft.cetl.product/cetl-product.product
runPackager=true
# The prefix that will be used in the generated archive.
archivePrefix=cetl-studio
# The location underwhich all of the build output will be collected.
collectingFolder=${archivePrefix}
# The list of {os, ws, arch} configurations to build. This
# value is a '&' separated list of ',' separate triples. For example,
# configs=win32,win32,x86 & linux,motif,x86
# By default the value is *,*,*
configs=win32, win32, x86
# Type of build. Used in naming the build output. Typically this value is
# one of I, N, M, S, ...
buildType=I
# ID of the build. Used in naming the build output.
buildId=cetl-studio-${buildVersion}
# Label for the build. Used in naming the build output
buildLabel=cetl-studio
# Timestamp for the build. Used in naming the build output
timestamp=007
#Os/Ws/Arch/nl of the eclipse specified by baseLocation
baseos=win32
basews=win32
basearch=x86
filteredDependencyCheck=false
resolution.devMode=false
skipBase=true
skipMaps=true
skipFetch=true
# Specify the output format of the compiler log when eclipse jdt is used
logExtension=.log
# Whether or not to fail the build if there are compiler errors
javacFailOnError=true
# Enable or disable verbose mode of the compiler
javacVerbose=false
extraVMargs=-XX:MaxPermSize=512m
# produce a properly installed, fully p2 enabled, product
p2.gathering=true
extract-eclipse target just creates eclipse directory with a full eclipse installation we use to run the build.
Build the test product
productTest-build target
productTest-build is very similar to product-build. baselocation is still the same but buildDirectory and builder need to be changed.
<target name="productTest-build" depends="failIfBaseLocationUnavailable,extract-eclipse" description="creates test product">
<createBuildDirectory builddirectory="${product-test.buildDirectory}" buildversion="${buildVersion}" />
<java classname="org.eclipse.equinox.launcher.Main" fork="true" failonerror="true" dir="${basedir}">
<arg value="-application" />
<arg value="org.eclipse.ant.core.antRunner" />
<arg value="-buildfile" />
<arg value="${eclipseDirectory}/plugins/org.eclipse.pde.build_${pdeBuildPluginVersion}/scripts/productBuild/productBuild.xml" />
<arg value="-DbaseLocation=${baseLocation}" />
<arg value="-Dbuilder=${product-test.buildConfiguration}" />
<arg value="-Dbasedir=${basedir}" />
<arg value="-DbuildDirectory=${product-test.buildDirectory}" />
<arg value="-DJ2SE-1.5=${jdk15Dir}/jre/lib/rt.jar" />
<arg value="-DJavaSE-1.6=${jdk16Dir}/jre/lib/rt.jar" />
<arg value="-DbuildVersion=${buildVersion}" />
<!-- there is a binary cycle due to slf4j -->
<arg value="-DallowBinaryCycles=true" />
<classpath>
<fileset dir="${eclipseDirectory}/plugins">
<include name="org.eclipse.equinox.launcher_*.jar" />
</fileset>
</classpath>
</java>
<antcall target="removeSwtbotJunit3Plugins" />
<antcall target="modifyTestProductIni" />
</target>
build configuration
What is different is the product-test/buildConfiguration/build.properties file :
product=net.entropysoft.cetl.product/cetl-product-test.product
runPackager=true
archivePrefix=cetl-studio-test
collectingFolder=${archivePrefix}
configs=win32, win32, x86
buildType=I
buildId=cetl-studio-test-${buildVersion}
buildLabel=cetl-studio-test
timestamp=007
baseos=win32
basews=win32
basearch=x86
filteredDependencyCheck=false
resolution.devMode=false
skipBase=true
skipMaps=true
skipFetch=true
logExtension=.log
javacDebugInfo=true
javacFailOnError=true
javacVerbose=false
# Extra arguments for the compiler. These are specific to the java compiler being used.
compilerArg=-g
extraVMargs=-XX:MaxPermSize=512m
p2.gathering=true
javacDebugInfo is set to true and compilerArg is set to -g (this seems to be necessary for cobertura reports to be useful)
Removing swtbot junit3 feature & plugin
The ant target removeSwtbotJunit3Plugins is used to remove org.eclipse.swtbot.eclipse.junit3.headless and org.eclipse.swtbot.ant.optional.junit3 plugins as I use Junit4 in my tests and that these plugins must not be used together with org.eclipse.swtbot.eclipse.junit4.headless and org.eclipse.swtbot.ant.optional.junit4.
This is needed as there is one feature for both swtbot junit3 and junit4 plugins. This may change in next swtbot release.
<target name="removeSwtbotJunit3Plugins">
<!-- JUnit 3.x and 4.x don't play well together. We remove org.eclipse.swtbot.eclipse.junit3.headless and org.eclipse.swtbot.ant.optional.junit3 from the plugins dir. -->
<!-- see http://wiki.eclipse.org/SWTBot/Ant -->
<zip destfile="${product-test.buildDirectory}/tmp.jar">
<zipfileset src="${product-test.buildDirectory}/cetl-studio-test/cetl-studio-test-${buildVersion}-win32.win32.x86.zip">
<exclude name="**/org.eclipse.swtbot.eclipse.junit3.headless*/**" />
<exclude name="**/org.eclipse.swtbot.ant.optional.junit3*.jar" />
</zipfileset>
</zip>
<move file="${product-test.buildDirectory}/tmp.jar" tofile="${product-test.buildDirectory}/cetl-studio-test/cetl-studio-test-${buildVersion}-win32.win32.x86.zip" />
</target>
Modifying product ini file
This step is necessary if you want that your test product run all the tests when launched and produce a junit-tests.xml file with results. I think this can be useful to test the product in a specific environment.
First I think I could set the program arguments in cetl-product-test.product :
-application
org.eclipse.swtbot.eclipse.junit4.headless.swtbottestapplication
formatter=org.apache.tools.ant.taskdefs.optional.junit.XMLJUnitResultFormatter,junit-results.xml
-testPluginName
net.entropysoft.cetl.plugin.test
-className
net.entropysoft.AllTestsSuite
but I got an exception during build :
java echo !MESSAGE Invalid action syntax:
addProgramArg(programArg:formatter=org.apache.tools.ant.taskdefs.optional.junit.XMLJUnitResultFormatter,junit-results.xml).
java echo !STACK 0
java echo java.lang.IllegalArgumentException: Invalid action syntax:
addProgramArg(programArg:formatter=org.apache.tools.ant.taskdefs.optional.junit.XMLJUnitResultFormatter,junit-results.xml).
java echo at
org.eclipse.equinox.internal.p2.engine.InstructionParser.parseAction(InstructionParser.java:97)
It seems that we cannot have a coma in the arguments otherwise we got this exception (at least with eclipse-3.5, this may be corrected in newer versions).
So I generate the ini file and include it in the test product zip file :
<target name="modifyTestProductIni">
<!-- We should set the arguments in cetl-product-test.product but we have an error during the build because of the coma in formatter line-->
<echo file="${product-test.buildDirectory}/cetl-studio-test/cetl-studio.ini">
-application
org.eclipse.swtbot.eclipse.junit4.headless.swtbottestapplication
formatter=org.apache.tools.ant.taskdefs.optional.junit.XMLJUnitResultFormatter,junit-results.xml
-testPluginName
net.entropysoft.cetl.plugin.test
-className
net.entropysoft.AllTestsSuite
-vmargs
-XX:MaxPermSize=512M
-Xms40m
-Xmx512m
</echo>
<zip destfile="${product-test.buildDirectory}/cetl-studio-test/cetl-studio-test-${buildVersion}-win32.win32.x86.zip" update="true">
<zipfileset file="${product-test.buildDirectory}/cetl-studio-test/cetl-studio.ini" prefix="cetl-studio-test" />
</zip>
</target>
Running the tests
For the tests, we can use http://github.com/ketan/swtbot/blob... from swtbot. However, I chose here not to use it.
Install the test product
We just uncompress the test product we just created :
<target name="install-product-test">
<delete dir="${testDirectory}" />
<unzip src="${product-test.buildDirectory}/cetl-studio-test/cetl-studio-test-${buildVersion}-win32.win32.x86.zip" dest="${testDirectory}" />
</target>
Instrument the classes to be tested
<target name="instrument-product-test" depends="taskdef-cobertura">
<mkdir dir="${testDirectory}/cobertura-data" />
<cobertura-instrument maxmemory="128M" datafile="${testDirectory}/cobertura-data/cobertura.ser">
<includeClasses regex="net\.entropysoft\..*\.plugin\..*" />
<instrumentationClasspath>
<fileset dir="${testDirectory}/cetl-studio-test/plugins">
<include name="net.entropysoft.*.jar" />
<include name="net.entropysoft.*/**/*.class" />
[...]
</fileset>
</instrumentationClasspath>
</cobertura-instrument>
</target>
Generate another ini file for our test
We need to modify the ini file of the product to take the instrumentation into account. All instrumented classes will need to be able to access classes from cobertura.jar, we can do that by adding cobertura.jar to the bootclasspath.
I also set the vm to use.
<target name="generateTestIniFile">
<echo file="${testDirectory}/cetl-studio-test/cetl-studio.ini">
-data
${testDirectory}/workspace
-application
org.eclipse.swtbot.eclipse.junit4.headless.swtbottestapplication
formatter=org.apache.tools.ant.taskdefs.optional.junit.XMLJUnitResultFormatter,${testDirectory}/junit-results/junit-results.xml
formatter=org.apache.tools.ant.taskdefs.optional.junit.PlainJUnitResultFormatter,${testDirectory}/junit-results/junit-results.txt
-testPluginName
net.entropysoft.cetl.plugin.test
-className
${testSuiteClass}
-nosplash
-suppressErrors
-consolelog
-vm
${jdk15Dir}/bin/java.exe
-vmargs
-Xbootclasspath/p:${basedir}/lib/test/cobertura.jar
-Dnet.sourceforge.cobertura.datafile="${testDirectory}/cobertura-data/cobertura.ser"
-XX:MaxPermSize=512M
-Xms40m
-Xmx512m
</echo>
</target>
Running the tests
Just execute the test product with the modified ini file :
<target name="run-tests" depends="generateTestIniFile">
<delete dir="${testDirectory}/workspace" />
<mkdir dir="${testDirectory}/junit-results" />
<exec executable="${testDirectory}/cetl-studio-test/cetl-studio.exe" dir="${basedir}" logError="true" failonerror="false" output="${testDirectory}/output.txt"/>
</target>
reports
Nothing special here :
<target name="junit-report">
<mkdir dir="${testDirectory}/junit-report" />
<junitreport todir="${testDirectory}/junit-report">
<fileset dir="${testDirectory}/junit-results">
<include name="*.xml" />
</fileset>
<report format="frames" todir="${testDirectory}/junit-report" />
</junitreport>
</target>
<target name="cobertura-report" depends="taskdef-cobertura">
<mkdir dir="${testDirectory}/coverage-report" />
<cobertura-report format="html" datafile="${testDirectory}/cobertura-data/cobertura.ser" destdir="${testDirectory}/coverage-report">
<fileset dir="../net.entropysoft.cetl.plugin/src" />
[...]
</cobertura-report>
</target>
Putting all together
I defined an "all" target to be called by CruiseControl, luntbuild, hudson, continuum or whatever continuous build tool you use :
<target name="all" depends="
materializeTargetPlatform,
materializeWorkspace,
product-build,
productTest-build,
install-product-test,
instrument-product-test,
run-tests,
junit-report,
cobertura-report,
publish,
fail-if-errors,
createSvnTag" />
<target name="fail-if-errors">
<loadfile property="test.results" srcFile="${testDirectory}/junit-results/junit-results.xml" />
<fail message="JUnit tests failed!">
<condition>
<not>
<contains string="${test.results}" substring='errors="0" failures="0"' />
</not>
</condition>
</fail>
</target>
