# Tuf > Contribute to python-tuf by submitting pull requests against the "develop" --- # Source: https://raw.githubusercontent.com/theupdateframework/python-tuf/develop/docs/CONTRIBUTING.rst Instructions for contributors ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Contribute to python-tuf by submitting pull requests against the "develop" branch of this repository. Detailed instructions are available in our `development guidelines `_. All submitted code should follow our `style guidelines `_ and must be `unit tested <#unit-tests>`_. .. note:: Also see `development installation instructions `_. DCO === Contributors must indicate acceptance of the `Developer Certificate of Origin `_ by appending a ``Signed-off-by: Your Name `` to each git commit message (see `git commit --signoff `_). Testing ======= With `tox `_ the whole test suite can be executed in a separate *virtual environment* for each supported Python version available on the system. ``tuf`` and its dependencies are installed automatically for each tox run. :: tox Below, you will see more details about each step managed by ``tox``, in case you need debug/run outside ``tox``. Unit tests ---------- test suite can be executed directly as well (in this case the environment managed by tox is not used): :: python3 -m unittest Individual tests can also be executed. Optional ``-v`` flags can be added to increase log level up to DEBUG (``-vvvv``). :: python3 tests/test_updater_ng.py -v Coverage -------- To run the tests and measure their code coverage, the aggregation script can be invoked with the ``coverage`` tool (requires installation of ``coverage``, e.g. via PyPI). :: coverage run -m unittest Auto-formatting --------------- The linter in CI/CD will check that new TUF code is formatted with `ruff `_. Auto-formatting can be done on the command line: :: tox -e fix --- # Source: https://raw.githubusercontent.com/theupdateframework/python-tuf/develop/docs/INSTALLATION.rst Installation ============ All versions of ``python-tuf`` can be installed from `PyPI `_ with `pip `_. :: python3 -m pip install tuf By default tuf is installed as pure python package with limited cryptographic abilities. See `Install with full cryptographic abilities`_ for more options. Install with full cryptographic abilities ----------------------------------------- Default installation supports signature verification only, using a pure Python *ed25519* implementation. While this allows to operate a *basic client* on almost any computing device, you will need additional cryptographic abilities for *repository* code, i.e. key and signature generation, additional algorithms, and more performant backends. Opt-in is available via ``securesystemslib``. .. note:: Please consult with underlying crypto backend installation docs. e.g. `cryptography `_ for possible system dependencies. :: python3 -m pip securesystemslib[crypto] tuf Install for development ----------------------- To install tuf in editable mode together with development dependencies, `clone `_ the `python-tuf repository `_ from GitHub, change into the project root directory, and install with pip (using `venv `_ is recommended). .. note:: Development installation will `Install with full cryptographic abilities`_. Please check above for possible system dependencies. :: python3 -m pip install -r requirements/dev.txt Verify release signatures ------------------------- Releases on PyPI are signed with a maintainer key using `gpg `_ (see `MAINTAINERS.txt `_ for key fingerprints). Signatures can be downloaded from the `GitHub release `_ page (look for *\*.asc* files in the *Assets* section). Below code shows how to verify the signature of a `built `_ distribution, signed by the maintainer *Lukas Pühringer*. It works alike for `source `_ distributions. :: # Get wheel from PyPI and signature from GitHub python3 -m pip download --no-deps tuf==0.20.0 wget https://github.com/theupdateframework/python-tuf/releases/download/v0.20.0/tuf-0.20.0-py3-none-any.whl.asc # Get public key, compare fingerprint in MAINTAINERS.txt, and verify with gpg gpg --recv-keys 89A2AD3C07D962E8 gpg --verify tuf-0.20.0-py3-none-any.whl.asc # Output: # gpg: assuming signed data in 'tuf-0.20.0-py3-none-any.whl' # gpg: Signature made Thu Dec 16 09:21:38 2021 CET # gpg: using RSA key 8BA69B87D43BE294F23E812089A2AD3C07D962E8 # gpg: Good signature from "Lukas Pühringer " [ultimate] --- # Source: https://raw.githubusercontent.com/theupdateframework/python-tuf/develop/docs/index.rst Python-TUF |version| Developer Documentation ======================================================================= This documentation provides essential information for those developing software with the `Python reference implementation of The Update Framework (TUF) `_. The reference implementation provides easy-to-use components for Python developers but also aims to be a readable guide and demonstration for those working on implementing TUF in their own languages, environments, or update systems. .. toctree:: :maxdepth: 1 :caption: Contents: api/api-reference INSTALLATION Usage examples Contribute --- # Source: https://raw.githubusercontent.com/theupdateframework/python-tuf/develop/docs/1.0.0-ANNOUNCEMENT.md # Announcing TUF 1.0.0 Python-TUF v1.0.0 is a rewritten stable reference implementation of the TUF specification, which *currently* includes: - a modern low-level [*metadata API*](https://theupdateframework.readthedocs.io/en/latest/api/tuf.api.html) - a fully specification-compliant [*updater client*](https://theupdateframework.readthedocs.io/en/latest/api/tuf.ngclient.html), serving as a more robust and yet more flexible stand-in replacement for the legacy client updater For the reasons outlined in [ADR 10](https://github.com/theupdateframework/python-tuf/blob/develop/docs/adr/0010-repository-library-design.md ), this release *does not yet* include *repository tool*-like functionality. However, the new *metadata API* makes it easy to replicate the desired functionality tailored to the specific needs of any given repository (see *Migration* for details). As discussed in [ADR 2](https://github.com/theupdateframework/python-tuf/blob/develop/docs/adr/0002-pre-1-0-deprecation-strategy.md), this release *does not* include any legacy code, as its maintenance has become infeasible for the python-tuf team. The pre-1.0.0 deprecation strategy from ADR 2 applies as follows: > *Bugs reported with tuf versions prior to 1.0.0 will likely not be addressed directly by tuf’s maintainers. Pull Requests to fix bugs in the last release prior to 1.0.0 will be considered, and merged (subject to normal review processes). Note that there may be delays due to the lack of developer resources for reviewing such pull requests.* ## Migration Given the clean cut with the legacy reference implementation, we provide the following migration support: - detailed code documentation on [https://theupdateframework.readthedocs.io](https://theupdateframework.readthedocs.io/) - verbose [code examples](https://github.com/theupdateframework/python-tuf/tree/develop/examples) for *client updater* usage, and repository-side operations based on the low-level *metadata API* - individual migration support upon [request](https://github.com/theupdateframework/python-tuf#contact) - targeted migration support initiative for known users --- # Source: https://raw.githubusercontent.com/theupdateframework/python-tuf/develop/docs/CODE-OF-CONDUCT.md ## The Update Framework Community Code of Conduct The Update Framework follows the [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md) --- # Source: https://raw.githubusercontent.com/theupdateframework/python-tuf/develop/docs/GOVERNANCE.md # TUF governance This document covers the project's governance and committer process. The project consists of the TUF [specification](https://github.com/theupdateframework/specification) and [reference implementation](https://github.com/theupdateframework/python-tuf). ## Maintainership and Consensus Builder The project is maintained by the people indicated in [MAINTAINERS](MAINTAINERS.txt). A maintainer is expected to (1) submit and review GitHub pull requests and (2) open issues or [submit vulnerability reports](https://github.com/theupdateframework/python-tuf#security-issues-and-bugs). A maintainer has the authority to approve or reject pull requests submitted by contributors. More significant changes in the project, such as those that require a TAP or changes in governance, are guided by a maintainer called the Consensus Builder (CB). The project's Consensus Builder (CB) is Justin Cappos , who has a lifetime appointment. ## Contributions [A contributor can submit GitHub pull requests](CONTRIBUTING.rst) to the project's repositories. They must follow the project's [code of conduct](CODE-OF-CONDUCT.md), the [developer certificate of origin](https://developercertificate.org/), the [code style guidelines](https://github.com/secure-systems-lab/code-style-guidelines), and must unit test any new software feature or change. Submitted pull requests undergo review and automated testing, including, but not limited to: * Unit and build testing via [GitHub Actions](https://github.com/theupdateframework/python-tuf/actions) and [Tox](https://tox.readthedocs.io/en/latest/). * Static code analysis via [Pylint](https://www.pylint.org/) and [Bandit](https://wiki.openstack.org/wiki/Security/Projects/Bandit). - Auto-formatting with [black](https://black.readthedocs.io/) and [isort](https://pycqa.github.io/isort/). * Checks for Signed-off-by commits via [Probot: DCO](https://github.com/probot/dco). * Review by one or more [maintainers](MAINTAINERS.txt). A contributor can propose changes to the specification with a [TUF Augmentation Proposal](https://github.com/theupdateframework/taps) (TAP). It is a design document providing information to the TUF community, or describing a new feature for TUF or its processes or environment. A [TAP](TAP.rst) can be approved or rejected by the CB after it has been reviewed and discussed. Discussions take place on the project's [mailing list](https://groups.google.com/forum/?fromgroups#!forum/theupdateframework) or the TAPs GitHub issue tracker. ## Changes in maintainership A contributor to the project must express interest in becoming a maintainer. The CB has the authority to add or remove maintainers. ## Changes in governance The CB supervises changes in governance, but a majority of maintainers must vote +1 on the PR. ## Changes in the consensus builder The consensus builder may be appointed for a fixed term or it may be a lifetime appointment. To initiate a change of consensus builder, or a change in the length of the appointment, a GitHub PR must be opened. If a fixed term is specified, the PR should be opened no earlier than 6 weeks before the end of the CB's term. If there is not a fixed term appointment, the PR may be opened at any time. In either case, the PR must be kept open for no less than 4 weeks. Additionally, the PR can only be merged with more +1 than -1 in the binding votes. Anyone from the community can vote on the PR with either +1 or -1. Only votes from maintainers that have been listed in the top-level [MAINTAINERS](MAINTAINERS.txt) file before the PR is opened are binding. When there are conflicting PRs about changes in the consensus builder, the PR with the most binding +1 votes is merged. The consensus builder can volunteer to step down. --- # Source: https://raw.githubusercontent.com/theupdateframework/python-tuf/develop/docs/RELEASE.md # Release process **Prerequisites (one-time setup)** 1. Enable "Trusted Publishing" in PyPI project settings * Publisher: GitHub * Owner: theupdateframework * Project: python-tuf * Workflow: cd.yml * Environment: release 1. Go to [GitHub settings](https://github.com/theupdateframework/python-tuf/settings/environments), create an [environment](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#creating-an-environment) called `release` and configure [review protection](https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment#required-reviewers). ## Release 1. Ensure `docs/CHANGELOG.md` contains a one-line summary of each [notable change](https://keepachangelog.com/) since the prior release 2. Update `tuf/__init__.py` to the new version number `A.B.C` 3. Create a PR with updated `CHANGELOG.md` and version bumps ➔ Review PR on GitHub 4. Once the PR is merged, pull the updated `develop` branch locally 5. Create a signed tag for the version number on the merge commit `git tag --sign vA.B.C -m "vA.B.C"` 6. Push the tag to GitHub `git push origin vA.B.C` *A tag push triggers the [CD workflow](https://github.com/theupdateframework/python-tuf/blob/develop/.github/workflows/cd.yml), which runs the tests, builds source dist and wheel, creates a preliminary GitHub release under `vA.B.C-rc`, and pauses for review.* 7. Run `verify_release --skip-pypi` locally to make sure a build on your machine matches the preliminary release artifacts published on GitHub. ➔ [Review *deployment*](https://docs.github.com/en/actions/managing-workflow-runs/reviewing-deployments) on GitHub *An approval resumes the CD workflow to publish the release on PyPI, and to finalize the GitHub release (removes `-rc` suffix and updates release notes).* 8. Run `verify_release` to make sure the PyPI release artifacts match the local build as well. When called as `verify_release --sign []` the script additionally creates gpg release signatures. When signed by maintainers with a corresponding GPG fingerprint in the MAINTAINERS.md file, these signature files should be made available on the GitHub release page under Assets. 9. Announce the release on [#tuf on CNCF Slack](https://cloud-native.slack.com/archives/C8NMD3QJ3) 10. Ensure [POUF 1](https://github.com/theupdateframework/taps/blob/master/POUFs/reference-POUF/pouf1.md), for the reference implementation, is up-to-date --- # Source: https://raw.githubusercontent.com/theupdateframework/python-tuf/develop/docs/SECURITY.md # Security Issues and Bugs Security issues can be reported to maintainers [privately via GitHub](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing/privately-reporting-a-security-vulnerability): - [**Report new vulnerability**](https://github.com/theupdateframework/python-tuf/security/advisories/new) Please do not use the GitHub issue tracker to submit vulnerability reports. The issue tracker is intended for bug reports and to make feature requests. Major feature requests, such as design changes to the specification, should be proposed via a [TUF Augmentation Proposal](https://theupdateframework.github.io/specification/latest/#tuf-augmentation-proposal-tap-support) (TAP). --- # Source: https://raw.githubusercontent.com/theupdateframework/python-tuf/develop/docs/index.md --- title: "Python-TUF development blog" --- This is the development blog for the [Python-TUF](https://github.com/theupdateframework/python-tuf) project, welcome! If you want to learn how to use Python-TUF, check out our [developer documentation](https://theupdateframework.readthedocs.io/en/latest/). --- # Source: https://raw.githubusercontent.com/theupdateframework/python-tuf/develop/docs/repository-library-design.md # Python-tuf repository API proposal: _minimal repository abstraction_ This is an attachment to ADR 10: _Repository library design built on top of Metadata API_, and documents the design proposal in Dec 2021. ## Design principles Primary goals of this repository library design are 1. Support full range of repository implementations: from command line “repository editing” tools to production repositories like PyPI 2. Provide canonical solutions for the difficult repository problems but avoid making implementation decisions 3. Keep python-tuf maintenance burden in mind: less is more Why does this design look so different from both legacy python-tuf code and other implementations? * Most existing implementations are focused on a specific use case (typically a command line application): this is a valid design choice but severely limits goal #1 * The problem space contains many application decisions. Many implementations solve this by creating functions with 15 arguments: this design tries to find another way (#2) * The Metadata API makes modifying individual pieces of metadata simpler. This, combined with good repository API design, should enable more variance in where things are implemented: The repository library does not have to implement every little detail as we can safely let specific implementations handle things, see goal #3 * This variance means we can start by implementing a minimal design: as experience from implementations is collected, we can then move implementation details into the library (goals #2, #3) ## Design ### Application and library components ![Design: Application and library components](repository-library-design-ownership.jpg) The design expects a fully functional repository application to contain code at three levels: * Repository library (abstract classes that are part of python-tuf) * The Repository abstract class provides an ergonomic abstract metadata editing API for all code levels to use. It also provides implementations for some core edit actions like _snapshot update_. * A small amount of related functionality is also provided (private key management API, maybe repository validation). * is a very small library: possibly a few hundred lines of code. * Concrete Repository implementation (typically part of application code, implements interfaces provided by the repository API in python-tuf) * Contains the “application level” decisions that the Repository abstraction requires to operate: examples of application decisions include * _When should “targets” metadata next expire when it is edited?_ * _What is the current “targets” metadata version? Where do we load it from?_ * _Where to store current “targets” after editing? Should the previous version be deleted from storage?_ * Actual application * Uses the Repository API to do the repository actions it needs to do For context here’s a trivial example showing what “ergonomic editing” means -- this key-adding code could be in the application (or later, if common patterns are found, in the python-tuf library): ```python with repository.edit(“targets”) as targets: # adds a key for role1 (as an example, arbitrary edits are allowed) targets.add_key(key, “role1”) ``` This code loads current targets metadata for editing, adds the key to a role, and handles version and expiry bumps before persisting the new targets version. The reason for the context manager style is that it manages two things simultaneously: * Hides the complexity of loading and persisting metadata, and updating expiry and versions from the editing code (by putting it in the repository implementation that is defined in python-tuf but implemented by the application) * Still allows completely arbitrary edits on the metadata in question: now the library does not need to anticipate what application wants to do and on the other hand library can still provide e.g. snapshot functionality without knowing about the application decisions mentioned in previous point. Other designs do not seem to manage both of these. ### How the components are used ![Design: How components are used](repository-library-design-usage.jpg) The core idea here is that because editing is ergonomic enough, when new functionality (like “developer uploads new targets”) is added, _it can be added at any level_: the application might add a `handle_new_target_files()` method that adds a bunch of targets into the metadata, but one of the previous layers could offer that as a helper function as well: code in both cases would look similar as it would use the common editing interface. The proposed design is purposefully spartan in that the library provides very few high-level actions (the prototype only provided _sign_ and _snapshot_): everything else is left to implementer at this point. As we gain experience of common usage patterns we can start providing other features as well. There are a few additional items worth mentioning: * Private key management: the Repository API should come with a “keyring abstraction” -- a way for the application to provide roles’ private keys for the Repository to use. Some implementations could be provided as well. * Validating repository state: the design is very much focused on enabling efficient editing of individual metadata. Implementations are also likely to be interested in validating (after some edits) that the repository is correct according to client workflow and that it contains the expected changes. The Repository API should provide some validation, but we should recognise that validation may be implementation specific. * Improved metadata editing: There are a small number of improvements that could be made to metadata editing. These do not necessarily need to be part of the repository API: they could be part of Metadata API as well It would make sense for python-tuf to ship with at least one concrete Repository implementation: possibly a repo.py look alike. This implementation should not be part of the library but an example. ## Details This section includes links to a Proof of Concept implementation in [repository-editor-for-tuf](https://github.com/vmware-labs/repository-editor-for-tuf/): it should not be seen as the exact proposed API but a prototype of the ideas. The ideas in this document map to POC components like this: | Concept | repository-editor-for-tuf implementation | |-|-| | Repository API | [librepo/repo.py](https://github.com/vmware-labs/repository-editor-for-tuf/blob/main/tufrepo/librepo/repo.py), [librepo/keys.py](https://github.com/vmware-labs/repository-editor-for-tuf/blob/main/tufrepo/librepo/repo.py) | | Example of repository implementation | [git_repo.py](https://github.com/vmware-labs/repository-editor-for-tuf/blob/main/tufrepo/git_repo.py) | |Application code | [cli.py (command line app)](https://github.com/vmware-labs/repository-editor-for-tuf/blob/main/tufrepo/cli.py), [keys_impl.py (keyring implementation)](https://github.com/vmware-labs/repository-editor-for-tuf/blob/main/tufrepo/keys_impl.py) | | Repository validation | [verifier.py (very rough, not intended for python-tuf)](https://github.com/vmware-labs/repository-editor-for-tuf/blob/main/tufrepo/verifier.py) | Improved Metadata editing | [helpers.py](https://github.com/vmware-labs/repository-editor-for-tuf/blob/main/tufrepo/helpers.py) ### Repository API Repository itself is a minimal abstract class: The value of this class is in defining the abstract method signatures (most importantly `_load`, `_save()`, `edit()`) that enable ergonomic metadata editing. The Repository class in this proposal includes concrete implementations only for the following: * `sign()` -- signing without editing metadata payload * `snapshot()` -- updates snapshot and timestamp metadata based on given input. Note that a concrete Repository implementation could provide an easier to use snapshot that does not require input (see example in git_repo.py) More concrete method implementations (see cli.py for examples) could be added to Repository itself but none seem essential at this point. The current prototype API defines five abstract methods that take care of access to metadata storage, expiry updates, version updates and signing. These must be implemented in the concrete implementation: * **keyring()**: A property that returns the private key mapping that should be used for signing. * **_load()**: Loads metadata from storage or cache. Is used by edit() and sign(). * **_save()**: Signs and persists metadata in cache/storage. Is used by edit() and sign(). * **edit()**: The ContextManager that enables ergonomic metadata editing by handling expiry and version number management. * **init_role()**: initializes new metadata handling expiry and version number. (_init_role is in a way a special case of edit and should potentially be integrated there_). The API requires a “Keyring” abstraction that the repository code can use to lookup a set of signers for a specific role. Specific implementations of Keyring could include a file-based keyring for testing, env-var keyring for CI use, etc. Some implementations should be provided in the python-tuf code base and more could be implemented in applications. _Prototype status: Prototype Repository and Keyring abstractions exist in librepo/repo.py._ ### Example concrete Repository implementation The design decisions that the included example `GitRepository` makes are not important but provide an example of what is possible: * Metadata versions are stored in files in git, with filenames that allow serving the metadata directory as is over HTTP * Version bumps are made based on git status (so edits in staging area only bump version once) * “Current version” when loading metadata is decided based on filenames on disk * Files are removed once they are no longer part of the snapshot (to keep directory uncluttered) * Expiry times are decided based on an application specific metadata field * Private keys can be stored in a file or in environment variables (for CI use) Note that GitRepository implementation is significantly larger than the Repository interface -- but all of the complexity in GitRepository is really related to the design decisions made there. _Prototype status: The GitRepository example exists in git_repo.py._ ### Validating repository state This is mostly undesigned but something built on top of TrustedMetadataSet (currently ngclient component) might work as a way to easily check specific aspects like: * Is top-level metadata valid according to client workflow * Is a role included in the snapshot and the delegation tree It’s likely that different implementations will have different needs though: a command line app for small repos might want to validate loading all metadata into memory, but a server application hosting tens of thousands of pieces of metadata is unlikely to do so. _Prototype status: A very rough implementation exists in verifier.py : this is unlikely to be very useful_ ### Improved metadata editing Currently the identified improvement areas are: * Metadata initialization: this could potentially be improved by adding default argument values to Metadata API constructors * Modifying and looking up data about roles in delegating metadata (root/targets): they do similar things but root and targets do not have identical API. This may be a very specific use case and not interesting for some applications _Prototype status: Some potential improvements have been collected in helpers.py_