Contributing to Flocker
Introduction
ClusterHQ develops software using test-driven development, code reviews and per-issue branches.
- Each unit of work is defined in an issue in the issue tracker and developed on a branch.
- Code is written using test-driven development.
- The issue is closed by merging the branch (via a GitHub pull request).
- Before a branch is merged it must pass code review.
- The code reviewer ensures that the pull request:
- Follows the coding standard (Python’s PEP 8).
- Includes appropriate documentation.
- Has full test coverage (unit tests and functional tests).
- The tests pass in the continuous integration system (Buildbot).
- Resolves the issue.
- The code reviewer can approve the pull request for merging as is, with some changes, or request changes and an additional review.
Talk to Us
Have questions or need help?
- If you have problems running Flocker, please read our debugging documentation.
- If you want to follow our development plans, our main issue tracker is JIRA.
- You can open an account there to file issues, but we’re also happy to accept a GitHub issue with feature requests or bug reports. Security issues should be reported directly to our security team.
- You can also join us on the
#clusterhq channel on the irc.freenode.net IRC network or on the flocker-users Google Group.
Development Environment
You will need Python 2.7 installed on your development machine.
To run the complete test suite you will also need Docker installed.
Install Flocker’s development dependencies in a virtualenv by running the following commands:
mkvirtualenv flocker
pip install --requirement dev-requirements.txt
CentOS 7
sudo yum install git python-virtualenv libffi-devel openssl-devel gcc enchant-devel
Ubuntu
sudo apt-get install git virtualenvwrapper python-dev libffi-dev libssl-dev enchant
Running Tests
You can run all unit tests by doing:
tox
You can also run specific tests in a specific environment:
tox -e py27 flocker.control.test.test_httpapi
Functional tests require Docker to be installed and, in the case of Docker, running.
In addition, tox needs to be run as root:
sudo tox
Since these tests involve global state on your machine (filesystems, iptables, Docker containers, etc.) we recommend running them in a virtual machine.
Running Lint Tests
You can run flake8 and pylint tests by doing:
tox -e lint
Documentation
Documentation is generated using Sphinx and stored in the docs/ directory.
You can build it individually by running:
tox -e sphinx
You can view the result by opening docs/_build/html/index.html in your browser.
Contributing to Flocker
If you have any feature requests or suggestions, we would love to hear about them.
At a minimum you can simply submit a GitHub Pull Request with your changes.
In order to maximize your chances of getting your code accepted, and to keep you from wasting time:
- Discuss your ideas with us in advance by filing a GitHub issue.
- Explain the purpose of your PR, and why these changes are necessary.
- Limit your PR to fixing a single problem or adding a single feature.
- See the merge requirements below for details about our testing and documentation requirements.
Make sure your PR adds your name to AUTHORS.rst if you’ve never contributed to Flocker before.
Once your pull request is merged, as a small thank you for contributing to Flocker we’d like to send you some ClusterHQ swag.
Just send an email to thankyou@clusterhq.com with your t-shirt size, mailing address and a phone number to be used only for filling out the shipping form.
We’ll get something in the mail to you.
Merge Requirements
While we’re happy to look at contributions in any state as GitHub PRs, the requirements below will need to be met before code is merged.
All code must have unit test coverage and to the extent possible functional test coverage.
Use the coverage.py tool with the --branch option to generate line and branch coverage reports.
This report can tell you if you missed anything.
It does not necessarily catch everything though.
Treat it as a helper but not the definitive indicator of success.
You can also see coverage output in the Buildbot details link of your pull request.
Practice test-driven development to ensure all code has test coverage.
All code must have documentation.
Modules, functions, classes, and methods must be documented (even if they are private).
Function parameters and object attributes must be documented (even if they are private).
All user-facing tools must have documentation.
Document tool usage as part of big-picture documentation.
Identify useful goals the user may want to accomplish and document tools within the context of accomplishing those goals.
Documentation should be as accessible and inclusive as possible.
Avoid language and markup which assumes the ability to precisely use a mouse and keyboard, or that the reader has perfect vision.
Create alternative but equal documentation for the visually impaired, for example, by using alternative text on all images.
If in doubt, particularly about markup changes, use http://achecker.ca/checker/index.php and fix any “Known Problems” and “Likely Problems”.
Project Development Process
The core development team uses a JIRA workflow to track planned work.
Issues are organized by sprints, and can reside in various states:
- Backlog
- All issues start in the backlog when they are filed.
- Design Backlog
- The issue requires a design, and will be worked on soon.
- Design
- The issue is currently being designed.
- Design Review Ready
- The design is ready for review.
This often involves submitting a GitHub pull request with a sketch of the code.
- Code Backlog
- The design has been approved and is ready to code.
- Coding
- The issue is currently being coded.
- Code Review Ready
- The code is ready for review.
This typically involves submitting a GitHub pull request.
- Code Review
- The code is being reviewed.
- Done
- The issue has been closed.
Some final work may remain to address review comments; once this is done and the branch is merged the GitHub PR will be closed.
Reporting Security Issues
Please report security issues by emailing security@clusterhq.com.
Flocker bugs should normally be reported publicly, but due to the sensitive nature of security issues, we ask that they not be publicly reported in this fashion.
Instead, if you believe you have found something in Flocker (or any other ClusterHQ software) which has security implications, please send a description of the issue via email to security@clusterhq.com.
Your message will be forwarded to the ClusterHQ security team (a small group of trusted developers) for triage and it will not be publicly readable.
Once you have submitted an issue via email, you should receive an acknowledgment from a member of the security team within 48 hours, and depending on the action to be taken, you may receive further follow up emails.
Maintenance Branches
Note
As bugfix releases aren’t currently being produced,
the following instructions are only relevant for documentation fixes.
Occasionally, issues will be discovered that want to be fixed before the next full release.
The following is the procedure for fixing them.
File an issue in JIRA.
For example, “FLOC-1234: Fix a bug.”.
If the issue affects in multiple versions (including master),
create a sub-task for each affected, supported version:
- FLOC-1235: Fix a bug in 1.2.3.
- FLOC-1236: Fix a bug in 1.2.4.
- FLOC-1237: Fix a bug in master.
Create branch off the oldest affected, supported release.
git checkout -b release-maintenance/flocker-1.2.3/fix-a-bug-FLOC-1235 origin/release/flocker-1.2.3
Fix the bug.
ed ...
git commit -m'Fixed the bug.'
Push the branch to GitHub.
git push origin --set-upstream release-maintenance/flocker-1.2.3/fix-a-bug-FLOC-1235
Create a pull-request against the release branch.
https://github.com/ClusterHQ/flocker/compare/release/flocker-1.2.3...release-maintenance/flocker-1.2.3/fix-a-bug-FLOC-1234?expand=1
Note in the pull request that the branch shouldn’t be deleted until every affected release and master have received the fix.
Otherwise, commits are liable to be lost between branches.
Wait for the pull-request to be accepted.
For each other affected release, create a branch against that release, merge-forward, then create a pull-request.
git checkout -b release-maintenance/flocker-1.2.4/fix-a-bug-FLOC-1236 origin/release-maintenance/flocker-1.2.3/fix-a-bug-FLOC-1235
# The following command is only necesary if there are merge conflicts to resolve
git merge origin/release/flocker-1.2.4
git push origin --set-upstream release-maintenance/flocker-1.2.4/fix-a-bug-FLOC-1236
If master is affected, create a branch against master, merge-forward, then create a pull-request.
git checkout -b fix-a-bug-FLOC-1236 origin/release-maintenance/flocker-1.2.3/fix-a-bug-FLOC-1235
# The following command is only necesary if there are merge conflicts to resolve
git merge origin/master
git push origin --set-upstream fix-a-bug-FLOC-1236
Delete all the merged branches.
Pre-release Branches
Similarly to maintenance-branches, bug fixes and improvements may need to be applied to pre-releases.
These changes should be the only changes between pre-releases for the same marketing release, and the only changes between the last pre-release and the final marketing release.
Follow the procedure for merging fixes into maintenance branches, merging fixes into the last pre-release.
Testing Code on Nodes
CentOS 7
Start with some nodes which are configured correctly for Flocker.
A simple way to do this is to run the acceptance test runner with the --keep option.
Log in to each node in the cluster, forwarding the authentication agent connection:
ssh -A root@${NODE_IP}
At this point it is possible to install packages built by Buildbot.
This would be a more complete testing of code on a node.
However, it takes some time for packages to be built, and for development purposes the trade-off of a fast development cycle may be worthwhile.
Those trade-offs include the ability to test new or changed Python dependencies.
The following instructions replace just some of the code used by Flocker, but enough that can be useful.
On each node, run the following commands.
Install git:
sudo yum install -y git
Clone Flocker somewhere to use later:
mkdir /flocker-source
cd /flocker-source
git clone git@github.com:ClusterHQ/flocker.git
Change the Flocker code in the checkout to what needs to be tested:
cd /flocker-source/flocker
git checkout BRANCH-NAME
Make a backup of the code and unit files which will be replaced:
mkdir /backup
cp -r /opt/flocker/lib/python2.7/site-packages/flocker/ /backup
cp -r /etc/systemd/system/multi-user.target.wants/ /backup
Replace the node services with the new code:
# Move Python code from the Git clone to where they are used
rm -rf /opt/flocker/lib/python2.7/site-packages/flocker/
cp -r /flocker-source/flocker/flocker/ /opt/flocker/lib/python2.7/site-packages/
SYSTEMD_SOURCE_DIR=/flocker-source/flocker/admin/package-files/systemd/
SOURCE_SERVICE_FILES=$(
ls ${SYSTEMD_SOURCE_DIR}/*.service |
xargs -n 1 -I {} sh -c 'basename {} .service'
);
# Stop systemd units before they are changed
for service in ${SOURCE_SERVICE_FILES};
do
systemctl stop ${service}
done
# Move systemd unit files from the clone to where systemd will look for them
# This uses /bin/cp instead of cp because sometimes cp is aliased to cp -i
# which requires confirmation
# This overwrites existing files (-f)
/bin/cp -f ${SYSTEMD_SOURCE_DIR}/* /etc/systemd/system/multi-user.target.wants
# Reload systemd, so that it can find new or changed units:
systemctl daemon-reload
# Start systemd units
for service in ${SOURCE_SERVICE_FILES};
do
if [ "$(systemctl is-enabled ${service})" == 'enabled' ]
then
systemctl start ${service}
fi
done
The services will take a short amount of time to start.
Then the new code should be running on the node.
From then on, change the files in /flocker-source/flocker (perhaps using git pull on each node) and run the above commands to replace the node services with the new code.
