Submitting a bug report¶
Please report any bugs to the GitLab (internal) or GitHub (public) issue trackers. Your issue will be resolved more quickly if you are able to provide a minimal working example, including an example concrete data file if applicable.
Ensure an issue has been created for your new feature/bugfix on GitLab (internal) or GitHub (public).
If you are adding a new feature, create a stub (placeholder implementation) of the desired argument/function/class/etc.
Write a test for your new feature/bugfix and run it, ensuring that it fails on the current implementation:
NameErrors, ImportErrors, SyntaxErrors, etc. do not count (they indicate the API is wrong).
Implement your new feature/bugfix.
Run the test again, ensuring that it now passes.
Run all tests and style checks, ensuring that they pass:
Optionally, run integration tests (you must have Redis server version 2.8 or later in your path; do
redis-server --versionto check):
If you created a new module (file) or package (directory) in the library, please see “Adding new modules and packages” in the next section.
Push your changes to a feature branch on GitLab/GitHub (e.g., called
nis the issue number and
issue-abbrevis a very short abbreviation of the issue title) and ensure that the build passes. The build is defined in
appveyor.ymlfor public builds); tox is configured in
tox.ini. The build includes unit tests, integration tests, and style checks and runs on Python 2.7 and 3.5 across multiple platforms; if it fails, please find the error in the build log, fix it, and try again.
Add a line to
CHANGELOGunder the current version-in-progress describing your changes simply and concisely. Add yourself to
AUTHORSif you are not already listed.
If you’ve made multiple commits, please squash them and
git push -fto the feature branch.
Create a merge/pull request for your feature branch into
master, referencing the GitLab/GitHub issue.
Adding new modules and packages¶
If a new module or package is created, either by hand or in the auto-generated code from Thrift, a small amount of additional configuration must be performed.
In either case, the name of the package (if it is a package and not a
module) should be added to the
packages parameter in
The name of the package or module should be added to the subpackage or
submodule list in
A new ReStructuredText file should also be created under
the package or module; follow the conventions set by the other packages
If the new module or package was written by hand, a guard should be
docs/conf.py so that that
module or package is not ignored by the documentation parser. If it is
a package, a guard should also be added to
generate.bash so that
the corresponding directory is not deleted when the auto-generated code
is copied into
concrete/ from the Thrift build directory.
If a new package was generated by Thrift, a corresponding exclude
should be added to the flake8 configuration in
setup.cfg and the
ttypes module should be added to the star imports
concrete/__init__.py. If a new module (not package) was
generated by thrift, no action is necessary.
Branches, versions, and releases¶
The master branch is kept stable at all times. Before a commit is pushed to master, it should be checked by CI on another branch. The recommended way of maintaining this is to do all work in feature branches that are kept up-to-date with master and pushed to GitLab, waiting for CI to finish before merging.
We use zest.releaser to manage versions, the
releases. (Making a new release is a many-step process that requires
great care; doing so by hand is strongly discouraged.)
Using zest.releaser, stable versions are released to PyPI
and master is kept on a development version number (so that a stable
version number never represents more than one snapshot of the code).
To make a new release install zest.releaser
pip install zest.releaser) and run
Testing PyPI releases¶
To test how changes to concrete-python will show up on PyPI (for
example, how the readme is rendered) you can use the PyPI testing
site. To do so, set the following in
repository = https://testpypi.python.org/pypi
You will also need to create a testpypi user account and you may need
to request access to the
concrete package on testpypi.
The automated build checks for syntax errors in the documentation. When a push is made to the GitHub repository the online documentation is automatically re-generated. You can run the automatic validation and generate the HTML documentation locally by doing:
tox -e docs
The generated HTML documentation is stored it in
(relative to the top of your repository). Open this file path in a
web browser to check how your changes will look when published online.
(Re)generating code from concrete¶
The Python code generated by the thrift compiler on the schema defined
in the concrete project is checked in to concrete-python manually after
applying necessary patches. For trivial modifications to the schema
this process is automated by
generate.bash, which assumes concrete
has been cloned alongside concrete-python (in the same parent
After this succeeds, tests should be run and the changes should be
manually inspected (
git diff) for sanity. Note that this will not
delete previously-generated files that are no longer produced by
thrift (whose entries were removed from the schema).
generate.bash is not sufficient: the patches (in
patches/) document where it (thrift) falls short on the
previously-compiled schema. Additionally, if new packages
(namespaces) are added to the schema, they must be added to
generate.bash throws an error, the
necessary changes should be performed manually and checked in to the
index, at which point the generated code should be removed from the
working tree, raw (unpatched) generated code should be generated, and
new patches should be produced and stored in
git diff. See the arguments to
generate.bash for generating
the unpatched code.