UIMA project logo
UIMA's Eclipse Update Site
Apache UIMA

Search the site

 UIMA's Eclipse Update Site

Introduction

The Apache UIMA™ project maintains an Eclipse update site as part of its prebuilt artifacts. The information on this page is mainly for developers, and documents how this update site is managed.

Over time, the update site ends up containing various Eclipse features and their plugins, at different "versions". It uses the (evolving) Eclipse standard mechanisms to install whatever version a user might require.

Because of this, the update site itself keeps "older" versions. Every time a new release is made which includes one or more new versions of some Eclipse features/plugins, the new artifacts are added to the set of existing (and perhaps, older versions of) feature and plugins.

P2

At some point in the evolution of the Eclipse update site mechanisms, the "P2" approach for Eclipse update sites was developed, and became the preferred way to package update sites. It has some changes (among others):

  • It adds to the normal metadata about components, the detailed version and dependency information contained in the manifest of the OSGi component JARs. This is a finer-grained versioning system in which version information is associated with Java Packages, in contrast to the entire contents of a Jar being the entity which is versioned.

    Some tooling, such as http://eclipse.org/tycho/Tycho, need this finer-grained information to resolve dependencies.

  • It continues the previous approach of putting a copy of the metadata in a separately downloaded file (formerly, the digest.zip). In P2 this is now in 2 files: content.xml and artifacts.xml (these files are actually made into compressed Jars).
  • The former update site files: site.xml, and digest.zip are no longer needed, and are no longer provided.

Some of the UIMA plugin projects now need to have the "high-fidelity" resolution of software dependencies that comes with the P2 repositories.

As of January 2013, we converted to the P2 style of update sites, and are no longer building the update site for pre-P2 style because P2 support has been in Eclipse for several years (since mid 2008).

Composite Update Sites

Update sites can be composite. A composite update site merely includes pointers to other update sites. The installation procedure combines all of the composite update sites mentioned in the top level composite description into one, for purposes of displaying and working with it. We use the composite mechanism to make maintenance of independently developed features more isolated; when a new version of a sub-site is developed, updates are localized to just that sub-site. The composite update site can also be separately maintained - it needs to change only when new sub-sites are added.

Our subsites are designed to be kept in named subfolders of the main update site.

We currently have sub-sites for

  • uimaj - base Java UIMA SDK tools and runtime
  • uima-as - addons for the base tooling to add the deployment descriptor editor, and uima-as runtime
  • ruta - the features for ruta

The update site looks like:

.../dist/uima/eclipse-update-site
                                 /compositeArtifacts.jar
                                 /compositeContent.jar
                                 /uimaj
                                       /artifacts.jar
                                       /content.jar
                                       /features/.... all features
                                       /plugins/.... all plugins
                                 /uima-as
                                       /artifacts.jar
                                       /content.jar
                                       /features/.... all features
                                       /plugins/.... all plugins                                      
                                 /ruta-2.0.1
                                       /artifacts.jar
                                       /content.jar
                                       /features/.... all features
                                       /plugins/.... all plugins                                      

In addition to plain files, the jars have checksums and signature files.

Layout and Versioning

The main Eclipse update site (which only has composite pointers to subsites) is kept with the general UIMA build tooling in http://svn.apache.org/repos/asf/uima/build/trunk/uima-eclipse-composite-update-site.

Feature and Plugin projects associated with particular parts of the UIMA project (such as the Rule Engine Workbench, or Java Base SDK, etc.), reside within those major components.

Each subsite is typically built with its associated component. It is therefore reasonable to version these together with the component. So, for example, the UIMA Java SDK, released at version 2.4.0, will have the plugin and feature projects for it also at 2.4.0.

The project in the build tools, uima-eclipse-composite-update-site, is only rebuilt when a new subsite is added; it has incrementing version numbers.

The update-site packaging of released components is released as part of the underlying release of the components it packages; it doesn't (normally) have a separate "vote". When doing a component release, the release manager should build and make available for testing the update (sub) site (and a new version of the composite update site, if that is being added to). When the release is accomplished, the release action is then a simple SVN switch from the dev/ to the release/ spot in the distribution SVN for the update project's subsite, followed by a commit. This guarantees the signed artifacts voted on are identical to what's put up for release.

Releasing an update to the Eclipse Update Site

To run the build for any of the Eclipse update sites, you must have maven property variables set to identify an accessible Eclipse installation (4.2 or later) so the Eclipse packaging tooling and Ant support can be located. These are typically put into your .m2 setttings.xml file like this:

<uima-maven-build-eclipse-home>C:/x/y/z/eclipse</uima-maven-build-eclipse-home>
<eclipse-equinox-launcher>
  ${uima-maven-build-eclipse-home}/plugins/org.eclipse.equinox.launcher_1.1.1.R36x_v20101122_1400.jar
</eclipse-equinox-launcher>
<uima-eclipse-jar-processor>
  ${uima-maven-build-eclipse-home}/plugins/org.eclipse.equinox.p2.jarprocessor_1.0.200.v20110808-1657.jar
</uima-eclipse-jar-processor>

The first decision when doing an update is to decide if you need to add a new sub-update-site to the set managed by the Composite collection. If so, update buildCompositeRepository.xml in the build/trunk/uima-eclipse-composite-update-site to add the sub-site.

To update a sub-site, go to that subsite's project for its update-site. For example, for uimaj, this is the project uimaj-eclipse-update-site. In that project, there is a file under src/main/resources, cagetory.xml, which has the information for the features in this update site.

You probably won't need to update this file, unless you're adding some completely new features. If all you're doing is releasing new versions of existing features, you can leave this file alone, because the version information is added during resource filtering based on the the update site's own POM version.

Any new features must be added to one (or more) categories, for them to be "visible". If you are changing categories, this is where that is done, also; but that is probably a rare occurrence.

How the update site build works

The build process packs (just the) new plugin Jars, and combines these and the new feature Jars with the existing update site (which probably has older versions of the features and plugins), and generates new metadata for the new plugins and appends this to any existing metadata.

  1. The distribution SVN's .../dev/... spot is updated so that the part of it which corresponds to the Eclipse update sub site is replaced by an SVN copy of the .../release/... spot.
  2. The existing subsite is checked out of the (now current) .../dev/... spot in the distribution SVN.
  3. The new features and plugins are collected using the maven dependency plugin from "official" levels using maven artifact coordinates
  4. The new plugin jars are packed
  5. New metadata for the features and plugins is generated and appended to the the existing metadata for the older versions
  6. New catogory information for the features is generated and appended to the existing category information
  7. The new features and plug jars are checksummed and (if the Apache Release profile is in effect) signed
  8. The result is left in the target/eclipse-update-site/[subsite] directory of the update site project.
  9. (Optional) You may want to do an SVN checkin on all the artifacts under target/eclipse-update-site/[subsite] - they will go into the distribution SVN .../dev/... spot. If you expect to do a lot of release candidates, you may want to avoid doing this and instead just copy the results to some other spot for PMC members to check; this is because multiple releases will take up extra space in the distribution SVN.

If the update consists of just new versions of the the existing features and plugins, it may not be necessary to update the POM for the update-site generation (other than to insure its version information is what you want the feature/plugin versions to be).

Releasing a new Composite Update Site

Normally, the composite update site doesn't change. It changes when new sub-sites are created, usually for new projects having Eclipse components.

To change the Composite site, update the POM to a new version number (it's numbered like build artifacts, with a simple incrementing integer). Then update the buildCompositeRepository.xml file at the top level to reflect the changes.

The build is done using the same techniques as the other update site: the dist .../release/... part is copied to the .../dev/... and that's svn-checked-out. Then the build runs the special ant tasks to generate the composite Jars from the buildCompositeRepository.xml. If you use the -Papache-release parameter on the mvn command line, the build will generate the checksums and do the gpg signing of the artifacts. The results are left in the tartet/eclipse-update-site for this project, ready to be svn-switched back to the dist .../release/... point and the changes committed.

Lifecycle for development operations

Plugins are the smallest unit of development. During development, these are rebuilt and can be launched into a sub-eclipse for debugging as an Eclipse application.

During development, the goal is quick turn-around, so the m2e capabilities available in Eclipse 4.2 and later can be used to run tests using the .class files produced by incremental Eclipse builds of the plugins, without running Maven builds of the plugin.

When the developer is finished with debugging the plugin and wants to proceed with more formal packaging, they can run mvn install to produce the plugin Jar (and eventually, the pack.gz form of the Jar).

Feature projects are used to define the Features that include the plugins.

Each major UIMA component (e.g., uimaj, ruta, etc.) defines its own independent Eclipse update site.

Earlier Optimizations (Historical)

The first UIMA Eclipse update sites were optimized by the then-current approach to optimization. This included 2 special things:

  • "super-duper" packing of the jar files using some new compression utilities that came with Java 5.
  • A separate copy of all the metadata needed for initial processing by the Eclipse update mechanism - put into a compressed "digest.zip".
  • See http://wiki.eclipse.org/Update_Site_Optimization, which describes the design goals for this.

    But note that the information on this page is now out of date, and has been superseded by a second version of update site packaging. In fact, the previously available headless Eclipse application, org.eclipse.update.core.siteOptimizer, is no longer part of Eclipse releases.

Links and References

Here are links to more information; there are many (partially) out-of-date websites which are misleading but these links (as of January, 2013, seem up to date.)