Skip to content

Development

Layout

rostree/
├── docs/                 # Documentation (this folder)
├── src/rostree/          # Main package
│   ├── core/
│   │   ├── index.py      # One-pass cached package index
│   │   ├── finder.py     # Workspace scanning + name lookups
│   │   ├── parser.py     # package.xml parsing (memoized)
│   │   ├── tree.py       # DependencyGraph + DependencyNode
│   │   ├── graph.py      # DOT / Mermaid generation
│   │   ├── junit.py      # JUnit report for `rostree check --junit`
│   │   └── webview.py    # Interactive HTML graph (inlines web/)
│   ├── api.py            # Public API
│   ├── cli.py            # Command line interface
│   ├── web/              # graph.html/.css/.js — the viewer, shipped as data
│   └── tui/              # Textual TUI
├── examples/             # Tools built on rostree.api, embedded in the docs and run by the tests
├── scripts/release/      # The release script publish.yml runs on every merge
├── tests/                # pytest (incl. TUI pilot tests)
├── pyproject.toml
├── .pre-commit-config.yaml
├── mkdocs.yml            # Docs site config (published to GitHub Pages)
└── .github/workflows/    # CI, docs and publish

Install (dev)

# From repo root
pip install -e ".[dev]"
# or: uv pip install -e ".[dev]"

Dev extras: pytest, pytest-cov, ruff, black, bandit.

Pre-commit

Commit messages must follow Conventional Commits (e.g. fix:, feat:, chore:). Pre-commit runs ruff and black and enforces that.

pip install -e ".[dev]" pre-commit
pre-commit install
pre-commit install --hook-type commit-msg
  • Ruff — Lint and fix (src, tests, examples). The enabled rule set is pinned in [tool.ruff.lint] and the version is pinned in the dev extra, so a new ruff release cannot silently change what CI enforces.
  • Black — Format (line-length 100).
  • Bandit — The security scan Codacy runs (src, examples).
  • conventional-pre-commit — Commit message prefix check (commit-msg hook).

Ruff, black and bandit are local hooks: they run the versions installed from the dev extra, the same ones CI installs, rather than copies pinned separately in .pre-commit-config.yaml. Separate pins drifted once — pre-commit ran ruff 0.8 while CI ran 0.16.

How changes are proposed and reviewed is in CONTRIBUTING.md.

Static analysis

Pull requests are scanned by Codacy, which runs two tools whose findings look alike but are suppressed differently:

Tool Suppressed by Reported as
Bandit # nosec B123 Low / Medium / High
Semgrep # nosemgrep Critical

A # nosec does nothing to a Semgrep finding, and vice versa, so the few lines that trip both carry both. Three things about placement, each of which cost a CI round to learn:

  1. Bandit reads everything after nosec as a list of rule IDs, so the reasoning goes on the line above, not trailing it, and nosec comes last.
  2. Both comments must sit on the line the tool reports. Adding them can push a line past 100 characters, at which point black splits the call and strands the comment on the closing paren — where it silences nothing. Put it on the subprocess.run( line and let the arguments wrap.
  3. Semgrep does not accept a trailing # nosemgrep on an import; it has to go on the line before.

.codacy.yaml carries a fourth: an engines.semgrep.exclude_paths block is not honoured at all, though the identical engines.bandit one is. Semgrep is therefore only ever silenced in the source.

Bandit also runs in ci.yml and pre-commit, so a finding lands next to the lint failures instead of only in a dashboard:

bandit -c pyproject.toml -r src tests examples scripts

.codacy.yaml and [tool.bandit] are kept in step with each other; changing one without the other is how the two disagree.

One rule is switched off for tests/ only, B101 (assert used) — every pytest assertion trips it. The rule exists because python -O strips asserts, which is not how a test suite runs. There are no asserts under src/.

tests/test_cli_commands.py reads a JUnit report back to check it, and uses defusedxml to do it — not because a file the test wrote three lines earlier is a threat, but because it costs nothing: defusedxml is already a runtime dependency for core/parser.py.

Everything under src/ is still scanned, and the accepted findings there carry their reason next to them, so a new finding on one of those lines still has to be looked at rather than inheriting a blanket exemption. There are two:

  • The subprocess calls in cli.py — fixed argv, absolute path from shutil.which, no shell anywhere.
  • core/junit.py — the only module that writes XML, and it never reads any. defusedxml, the replacement both tools recommend, exports no Element/SubElement/ElementTree, so there is nothing to switch to on the writing side and nothing to defend against either: the only untrusted values are package names, which ElementTree escapes. Keeping it in its own small module means that argument is made once, at the top of a file you can read in a minute, rather than buried in a 1,300-line CLI.

Bandit logs a nosec encountered, but no failed test warning for a couple of them — it attributes multi-line statements to the wrong line — which is noise, not a stale suppression.

CI

  • ci.yml — lint (ruff, black, bandit) plus pytest with coverage on Python 3.10–3.12.
  • docs.yml — build the site with --strict, and deploy it to GitHub Pages from main.
  • pr-title.yml — check that a pull request's title is a Conventional Commit.
  • publish.yml — the release: on every merge to main, decide whether the merged pull requests call for one, then bump, tag, publish to PyPI (Trusted Publishing), create the GitHub release and redeploy the docs.

CI runs on every push/PR to main/master.

Releases

Versions are never bumped by hand, and there is no release pull request to merge. On every push to main, publish.yml:

  1. lists the commits since the last v* tag and asks the GitHub API which pull request each came from, so the pull request title decides — whatever the merge method, and whatever the individual commits were called;
  2. runs scripts/release/release.py plan, which applies Conventional Commits: ! or a BREAKING CHANGE: footer is major, feat minor, fix, perf, revert and deps patch, and anything else — or a merge that only contains docs:, chore:, ci:… — releases nothing and stops there;
  3. runs release.py apply, which bumps pyproject.toml, the version and date-released of CITATION.cff and the version of codemeta.json, and turns ## [Unreleased] into the new entry — the notes written there, followed by the pull requests that released them, or an entry generated from their titles when nobody wrote any;
  4. commits chore(release): vX.Y.Z to main, tags it, builds the wheel, checks rostree --version from it, publishes to PyPI, creates the GitHub release with the changelog entry as its notes, and redeploys these docs.

The commit and tag are pushed with the workflow's own token. That needs no setting, approval or personal token — main only has to accept pushes from Actions — and it starts no other workflow, which is why publish.yml builds and deploys everything itself rather than leaving it to a tag or release trigger. PyPI's trusted publisher names publish.yml, so the release has to stay in that file.

tests/test_release.py runs the script against copies of the real files, so a change to their layout that would break the release fails CI first.

Tests

pytest tests -v

TUI behaviour is covered by tests/test_tui_app.py, which drives the real app through Textual's pilot harness (key presses, background workers, lazy expansion) rather than testing helpers in isolation. tests/conftest.py clears the package index and parse caches between tests — they are process-wide by design, so tests must not share them.

The HTML graph viewer

src/rostree/web/ holds real .html, .css and .js files rather than Python strings, so they can be edited and diffed like the front-end code they are. core/webview.py inlines all three into one document at generation time.

There is no build step and no dependency, on purpose: the output has to work from a file:// URL with no network, which rules out a CDN, and vendoring a layout library to avoid one would be a worse trade than the ~200 lines of layered-DAG layout in graph.js.

To work on it, generate a page and open it:

rostree graph nav2_bringup -f html -o /tmp/g.html && xdg-open /tmp/g.html

Two things that are easy to get wrong and have already bitten:

  • Do not call setPointerCapture on the canvas. Chromium retargets the compatibility mouse events to the capturing element, so every click arrives on the <svg> and no node is ever clickable. Pan by listening on window.
  • [hidden] needs display: none !important, because several of the things it is applied to set an explicit display further down the stylesheet.

tests/test_webview.py covers the payload and the document — that it is self-contained, that no placeholder survives, and that a </script> in a package description cannot break out of the data block. Behaviour inside the page is not unit-tested; check it in a browser.

Docs

docs/ is a MkDocs site using Material, published to https://guilyx.github.io/rostree by docs.yml on every push to main.

pip install -e ".[docs]"
mkdocs serve          # live reload on http://127.0.0.1:8000
mkdocs build --strict # what CI runs
  • docs/index.md — Site home.
  • docs/overview.md — System overview and data flow.
  • docs/package-discovery.md — How packages are found (env vars, workspaces).
  • docs/dependency-trees.md — parsing, the package index, repeat collapsing, graphs.
  • docs/usage.md — CLI, TUI keys, Python API.
  • docs/development.md — This file.
  • docs/changelog.md — a one-line page that pulls in the root CHANGELOG.md, so the changelog is written in exactly one place.

Two things the build enforces, both worth knowing before a PR fails on them:

  • strict: true. A link that resolves to nothing is a build failure, not a warning. Anything the site links to has to exist inside docs/ — a repo-relative path like ../src/rostree/cli.py will not resolve, so link to the file on GitHub by full URL instead. That is why the three such links inside CHANGELOG.md are absolute: they have to work both on the site and in the file itself.
  • docs/review.md is excluded (exclude_docs in mkdocs.yml). It is the pre-0.3 code audit, kept as a record; most of what it reports is fixed, and publishing it as current documentation would mislead. roadmap.md therefore links to it on GitHub rather than as a page.

Keep the root README.md lean; link to these docs for details.