Making a Release
This is a guide for the creation of a release of Apache Accumulo.
There are number of things that are required before attempting to build a release.
- Use gpg-agent, and be sure to increase the gpg-agent cache timeout (via .gnupg/gpg-agent.conf) to ensure that the agent doesn’t require re-authentication mid-build, as it will cause things to fail. For example, you can add
default-cache-ttl 6000to increase the timeout from the default of 10 minutes to over an hour. If you do not have a GPG key, reference the very thorough ASF release signing documentation.
- Ensure that you’re using the correct major release of Java (check javadoc too).
- Ensure that you’re building Apache Accumulo with a username that has the same name as your Apache ID (this is due to the maven-release-plugin and staging the release candidate).
- Update the CHANGES file so that it’s in sync with Jira (manual process).
- Ensure that you have a texlive distribution installed so you are able to build the documentation.
- Have a clean workspace before starting.
Given all of this, it’s recommended that you only attempt making a release from a GNU/Linux machine.
Before creating a release candidate, all open issues with a fix version of the release candidate should be triaged. During the transition from JIRA to GitHub issues, anyone doing triage will need to search for open issues in both locations. Searching in both locations is only necessary for branches that have not released since the transition started.
Create the candidate
./assemble/build.sh --create-release-candidateto make the release candidate
git tag $version $version-rcNto create an RC tag from the actual tag
git tag -d $versionmake sure you don’t accidentally push a “release” tag
git push origin $version-rcNpush the RC tag
git checkout -b $version-rcN-branchsave off the branch from the Maven release plugin
- If vote fails, fix the original branch and start over.
- If vote passes,
git merge $version-rcN-branchback into the original branch you released from.
git tag -s $version-rcN $versionmake a GPG-signed tag
git push origin $versionpush the signed tag.
You should use the provided script assemble/build.sh to create the release candidate. This script is desirable as it activates all necessary maven profiles in addition to verifying that certain preconditions are met, like RPM signing availability and the ability to sign files using GPG. The –test option can be used as a dry run for creating a release candidate. The –create-release-candidate option should be used to create the actual release candidate.
When invoking build.sh with the –create-release-candidate option, the majority of the work will be performed by the maven-release-plugin, invoking release:clean, release:prepare, and release:perform. These will guide you through choosing the correct versions. The default options provided should be what you choose. It is highly recommended that an ‘RC’ suffix is not appended to the release version the plugin prompts you for, as that will result in that version string being placed into the poms, which then would require voting to occur on artifacts that cannot be directly promoted. After the build.sh script finishes (this will likely take at least 15 minutes, even on recent hardware), your current branch will be on the “next” version that you provided to the release plugin.
One unwanted side-effect of this approach is that after creating this branch, but before invoking release:perform, you must edit the release.properties to add the -rcN suffix to the value of scm.tag. Otherwise, the release plugin will complain that it cannot find the branch for the release. With a successful invocation of mvn release:perform, a staging repository will be made for you on the ASF Nexus server which you can log into with your ASF credentials.
After you log into Nexus, click on Staging Repositories in the Build Promotion toolbar on the left side of the screen. Assuming your build went according to plan, you should have a new staging repository made for you. At this point, you should inspect the artifacts that were staged to ensure that they are as you expect them to be. When you’re ready to present those artifacts for voting, you need to close that repository which will make it publicly available for other members to inspect.
At this point, you should have a closed repository that’s ready to vote on. Send a message to the dev list and get the ball rolling. Developers should test and verify the release candidate on their own. Accumulo has a guide for verifying releases.
Lazy consensus is not sufficient for a release; at least 3 +1 votes from PMC members are required. All checksums and signatures need to be verified before any voter can +1 it. Voting shall last 72 hours. Voters SHOULD include with their vote details on the tests from the testing section they have successfully run. If given, said details for each test MUST include: the number of worker nodes in the cluster, the operating system and version, the Hadoop version, and the Zookeeper version. For testing done on a version other than the release candidate that is deemed relevant, include the commit hash. All such gathered testing information will be included in the release notes.
If the vote ultimately fails, you delete the staged repository, clean up the branch you created (or wait until the release ultimately passes if you choose), and fix what needs fixing.
If the vote passes, follow the steps below.
Promote the artifacts
Promote that staged repository using Nexus which you can do with the click of a button. This will trigger a process to get the release out to all of the mirrors.
Create the final Git tag
The Git repository should also contain a tag which refers to the final commit which made up a release. This tag should also be signed with your GPG key. To ensure proper retention on release (stemming from ASF policy requirements), This final tag must being with “rel/”. For example, a release of 1.7.0 should have a corresponding tag name of “rel/1.7.0”.
Copy artifacts to dist.apache.org
An SVN server is running at https://dist.apache.org/repos/dist/release/accumulo. You need to upload the release tarballs, the GPG signatures and checksum files to the correct directory (based on the release number). If you are releasing a bug-fix release, be sure to delete the previous release in the same line (e.g. if you release 1.6.2, remove 1.6.1). The old tarballs removed from dist.apache.org will still be preserved in archive.apache.org automatically.
Fill out the add release form to update the projects website.
Update the Accumulo project website
After a successful vote, this website needs to be updated with the new artifacts.
- Update downloads page
- Create a post in
_posts/release/containing release notes
- Remove previous bug-fix release (if applicable)
- Update doap/accumulo.rdf
Starting with 2.0.0, the source code for the Accumulo documentation was moved to the accumulo-website repo except for two markdown files that should be changed in the Accumulo repo and copied/mirrored to the website repo for releases.
client-properties.mdshould be copied after it is generated by the Accumulo build.
cp /path/to/accumulo/core/target/generated-docs/*-properties.md /path/to/accumulo-website/_docs-2-x/configuration/
For major releases, follow the steps below to create docs for next release:
Create a new documentation collection for the new major release (i.e
3.x) using the collection of the last release. Avoid using a dot
.in the directory name:
cp -r _docs-2 _docs-3
Create a new doc layout using the previous layout. Update the reference to
site.docs-3(if creating 3.x docs):
cp _layouts/docs-3.html _layouts/docs-3.html vim _layouts/docs-3.html
Point Jekyll to the new documentation collection by modifying
_config.yml. Follow what was done for previous releases.
client-properties.md, and mirror
Once a collection is created for a major release, developers can make documentation updates like normal website updates.
For 2.x minor & bugfix releases, copy
client-properties.md, and mirror
For 1.x minor & bugfix releases, copy
accumulo_user_manual.html generated for release to the
1.x/ directory in the accumulo-website repo.
Javadocs are easy to update. Using the latest JDK7 or later (at least JDK 7u21 to avoid known vulnerabilities), follow these steps:
- Unpack the source release tarball and change to its root directory, or checkout the SCM tag for the release
- Build the javadocs with
mvn clean package javadoc:aggregate -DskipTests -Paggregate-javadocs
- Take note that the javadocs you will need to copy are the entire contents of
- In a different directory, checkout the
masterbranch of the accumulo-website repo
- Remove any existing apidocs from the appropriate version folder (e.g. 1.6/apidocs for a 1.6.x release)
- Copy the entire contents of the new apidocs directory (identified in step 3) to the destination in the website branch (e.g. to 1.6/apidocs)
- Continue updating the site content, as needed
- Commit the changes
- Update the site using jekyll with
./_devtools/git-hooks/post-commit(if you don’t have the commit hook already configured)
- Don’t forget to push both the
asf-sitebranches back to the accumulo-website repo
- Verify that javadocs have been updated on the production site (e.g. https://accumulo.apache.org/1.6/apidocs/)
Update Accumulo Examples
After the release has been made, the Accumulo version used by Accumulo Examples should be updated if this is the latest release of Accumulo.
- Update the
mvn clean verifyto confirm that nothing breaks
- Run one of the examples for additional testing.
Some good references that explain a few things: