Release Process¶

Outcomes¶

By the end of the release process we will have:

• a tag in version control,
• a Python wheel on Amazon S3,
• CentOS 7 RPMs for software on the node and client,
• Ubuntu 16.04 DEBs for software on the node and client,
• Ubuntu 14.04 DEBs for software on the node and client,
• documentation on flocker-docs.clusterhq.com, and

For a maintenance or documentation release, we will have:

• a tag in version control,
• documentation on flocker-docs.clusterhq.com.

Prerequisites¶

brew tap stepanstipl/noop
brew install createrepo dpkg

sudo apt-get update
sudo apt-get install -y dpkg-dev createrepo

sudo yum install -y dpkg-dev createrepo

Preparing For a Release¶

1. Confirm that the release and the proposed version number have been approved.

   The release must have been approved, unless it is a weekly development release. Refer to the ClusterHQ Flocker Releases and Versioning policy document.

   The version number must adhere to the Flocker version numbering policy.

2. Create an issue in JIRA:

   This should be a “Feature” with “Release Flocker [VERSION]” as the title, and it should be assigned to yourself. The issue does not need a design, so move the issue to the “Coding” state.

3. Create the release repository, environment and branch from within an existing Flocker development environment:

   admin/initialize-release --flocker-version=1.6.2
   

   Execute the commands output by the initialize-release script:

   export VERSION=1.6.2;
   cd /home/developer/flocker-release-1.6.2;
   source flocker-1.6.2/bin/activate;
   

4. Ensure the notes in docs/releasenotes/index.rst are up-to-date:

   Note

   git log can be used to see all merges between two versions.

   # Choose the tag of the last version with a "Release Notes" entry to compare the latest version to.
   OLD_VERSION=1.6.1
   
   BRANCH=$(git rev-parse --abbrev-ref HEAD)
   git log --first-parent ${OLD_VERSION}..${BRANCH}
   

   • Update the “Release Notes” document.
   • (optional) Add a version heading. If this is a Major or Minor Marketing (pre-)release, the “Release Notes” document should have a heading corresponding to the release version. If this is a weekly development release, add a “Next Release” heading instead.
   • Refer to the appropriate internal release planning document on Google Drive for a list of features that were scheduled for this release, e.g. Product > Releases > Release 0.3.1, and add bullet points for those features that have been completed.
   • Add bullet points for any other important new features and improvements from the git log above,
   • and add links (where appropriate) to documentation that has been added for those features.

   Finally, commit the changes:

   git commit -am "Updated Release Notes"
   

5. Push the changes:

   git push --set-upstream origin $(git rev-parse --abbrev-ref HEAD)
   

6. Ensure all the required tests pass on Jenkins:

   To run the tests on Jenkins, first run setup_ClusterHQ-flocker-release using the release branch as the parameter to the job. This will generate two sets of test jobs for the release branch which can be accessed from the releases view. For the following steps, use the results of the jobs within “Release release/flocker-<VERSION>” as these do not perform any pre-build merging with master.

   To run the tests, force a build of the __main_multijob job. Some of the tests will not be triggered by this (such as the acceptance tests), so these will also need to be started.

   Discuss with the team whether the release can continue given any failed tests outside of expected failures. Some jobs may have to be run again if temporary issues with external dependencies have caused failures.

   In addition, review the link-check step of the documentation builder to ensure that all the errors (the links with “[broken]”) are expected.

7. Make a pull request on GitHub:

   The pull request should be for the release branch against master, with a [FLOC-123] summary prefix, referring to the release issue that it resolves. Add a note to the pull request why any failed tests were deemed acceptable.

   Wait for an accepted code review before continuing.

Pre-tag Review Process¶

A tag must not be deleted once it has been pushed to GitHub. This is a policy and not a technical limitation, as removing tags can cause problems for anyone who has updated a cloned copy of the repository.

It is important to check that the code in the release branch is working before it is tagged.

1. Check the changes in the Pull Request:

   • The release notes at docs/releasenotes/index.rst should be up to date.
   • The build should be passing to the team’s satisfaction. See “Ensure all the required tests pass on Jenkins” in Preparing For a Release.

   For some releases the Pull Request may include bug fixes or documentation changes which have been merged into the branch from which the release branch was created, for example a previous pre-release. These fixes can be ignored in this review.

2. Update GitHub and JIRA:

   If there were no problems spotted while checking the changes, comment on the Pull Request that the release engineer can continue by following the Release section. Do not merge the Pull Request as this should happen after the branch has been tagged. Accept the JIRA issue, and add a comment that the release process can continue.

   If a problem was spotted, add comments to the Pull Request for each problem, and comment that they must be resolved before repeating this review process. Reject the JIRA issue and assign it to the release engineer.

Release¶

1. Tag the version being released:

   BRANCH=$(git rev-parse --abbrev-ref HEAD)
   RELEASE_BRANCH_PREFIX="release\/flocker-"
   TAG=${BRANCH/${RELEASE_BRANCH_PREFIX}}
   git tag --annotate "${TAG}" "${BRANCH}" -m "Tag version ${TAG}"
   git push origin "${TAG}"
   

2. Go to Jenkins and force a build on the release branch to test the latest commit.

   Currently, jobs cannot be created for git tags so the latest commit must be tested instead. This must be the same commit as the tag. The git commit that was used can be seen on the summary page for any build. To test this commit, force a build of the __main_multijob job and any other jobs which are not triggered by this.

3. Go to the BuildBot web status and force a build on the tag.

   Although the tests are run on Jenkins, we still use Buildbot to build the packages.

   Force a build on a tag by putting the tag name (e.g. 0.2.0) into the branch box (without any prefix).

   Note

   Although there would not have been any changes since the branch was built during the Preparing For a Release process, we need to build on the tag as the packages that were built before pushing the tag won’t have the right version.

   Wait for the build to complete successfully.

4. Set up AWS Access Key ID and AWS Secret Access Key Amazon S3 credentials:

   aws configure
   

   Enter your access key and secret token when prompted. The other configurable values may be left as their defaults.

5. Update the CloudFormation installer template.

   The following commands will generate new AWS AMI images with this version of Flocker pre-installed. The new AMI images will be used in the CloudFormation template used in the Docker, Swarm, Compose installation instructions.

   FLOCKER_VERSION="${TAG:?}"
   
   DOCKER_VERSION=1.11.2
   SWARM_VERSION=1.2.3
   
   export FLOCKER_VERSION DOCKER_VERSION SWARM_VERSION
   
   admin/ami-search-ubuntu --ubuntu-name xenial  --ec2-image-type ebs-ssd > ami_map_ubuntu.json
   
   admin/publish-installer-images \
       --copy_to_all_regions \
       --template=docker \
       --source-ami-map="$(<ami_map_ubuntu.json)" > ami_map_docker.json
   
   admin/publish-installer-images \
       --copy_to_all_regions \
       --template=flocker \
       --source-ami-map="$(<ami_map_docker.json)" > ami_map_flocker.json
   
   admin/create-cloudformation-template \
        --client-ami-map-body="$(<ami_map_docker.json)" \
        --node-ami-map-body="$(<ami_map_flocker.json)" \
        > "flocker-cluster.cloudformation.${FLOCKER_VERSION}.json"
   
   aws --region us-east-1 \
       s3 cp --acl public-read \
       "flocker-cluster.cloudformation.${FLOCKER_VERSION}.json" \
       s3://installer.downloads.clusterhq.com/
   

6. Publish artifacts and documentation:

   admin/publish-artifacts
   admin/publish-docs --production
   

7. Check that the artifacts are set up correctly:

   Note

   Ensure that Docker is installed and running, and can be controlled from the current user account. Run docker ps to check for any problems.

   The following command tests that the client packages can be installed on a number of platforms. This helps to identify any problems with the published artifacts that may not be evident in the regular tests (e.g. S3 permissions or packaging problems). This test can take about 30 minutes, especially if Docker images need to be pulled.

   admin/test-artifacts
   

   If an error occurs for any tests, create a JIRA issue and raise it with the team. In any case, continue with the release.

8. Remove the release virtual environment:

   VIRTUALENV_NAME=$(basename ${VIRTUAL_ENV})
   deactivate
   rmvirtualenv ${VIRTUALENV_NAME}
   

9. Remove the release Flocker clone:

   Warning

   rm -rf can be dangerous, run this at your own risk.

   rm -rf ${PWD}
   

10. Merge the release branch into master:

    If there are no conflicts, merge the pull request. If there are conflicts; create a new branch, merge forward and create a pull-request of that branch against master.

    git checkout -b merge-release-${VERSION}-FLOC-XXX release/flocker-${VERSION}
    git pull origin master
    

    Merging this pull-request will also close the release pull request. The merge-release-*-FLOC-XXX branch should be deleted once the pull-request has been merged.

    Unless this is a development release, do not delete the release branch because it may be used as a base branch for future releases.

Improving the Release Process¶

The release engineer should aim to spend up to one day improving the release process in whichever way they find most appropriate. If there is no existing issue for the planned improvements then a new one should be made. Look at existing issues relating to the release process. The issue(s) for the planned improvements should be put into the next sprint.