Development¶
Layout¶
rostree/
├── docs/ # Documentation (this folder)
├── src/rostree/ # Main package
│ ├── core/
│ │ ├── index.py # One-pass cached package index
│ │ ├── finder.py # Workspace scanning + name lookups
│ │ ├── parser.py # package.xml parsing (memoized)
│ │ ├── tree.py # DependencyGraph + DependencyNode
│ │ ├── graph.py # DOT / Mermaid generation
│ │ ├── junit.py # JUnit report for `rostree check --junit`
│ │ └── webview.py # Interactive HTML graph (inlines web/)
│ ├── api.py # Public API
│ ├── cli.py # Command line interface
│ ├── web/ # graph.html/.css/.js — the viewer, shipped as data
│ └── tui/ # Textual TUI
├── examples/ # Tools built on rostree.api, embedded in the docs and run by the tests
├── scripts/release/ # The release script publish.yml runs on every merge
├── tests/ # pytest (incl. TUI pilot tests)
├── pyproject.toml
├── .pre-commit-config.yaml
├── mkdocs.yml # Docs site config (published to GitHub Pages)
└── .github/workflows/ # CI, docs and publish
Install (dev)¶
Dev extras: pytest, pytest-cov, ruff, black, bandit.
Pre-commit¶
Commit messages must follow Conventional Commits (e.g. fix:, feat:, chore:). Pre-commit runs ruff and black and enforces that.
- Ruff — Lint and fix (src, tests, examples). The enabled rule set is pinned
in
[tool.ruff.lint]and the version is pinned in thedevextra, so a new ruff release cannot silently change what CI enforces. - Black — Format (line-length 100).
- Bandit — The security scan Codacy runs (src, examples).
- conventional-pre-commit — Commit message prefix check (commit-msg hook).
Ruff, black and bandit are local hooks: they run the versions installed from the
dev extra, the same ones CI installs, rather than copies pinned separately in
.pre-commit-config.yaml. Separate pins drifted once — pre-commit ran ruff 0.8
while CI ran 0.16.
How changes are proposed and reviewed is in CONTRIBUTING.md.
Static analysis¶
Pull requests are scanned by Codacy, which runs two tools whose findings look alike but are suppressed differently:
| Tool | Suppressed by | Reported as |
|---|---|---|
| Bandit | # nosec B123 |
Low / Medium / High |
| Semgrep | # nosemgrep |
Critical |
A # nosec does nothing to a Semgrep finding, and vice versa, so the few lines
that trip both carry both. Three things about placement, each of which cost a
CI round to learn:
- Bandit reads everything after
nosecas a list of rule IDs, so the reasoning goes on the line above, not trailing it, andnoseccomes last. - Both comments must sit on the line the tool reports. Adding them can push a
line past 100 characters, at which point black splits the call and strands the
comment on the closing paren — where it silences nothing. Put it on the
subprocess.run(line and let the arguments wrap. - Semgrep does not accept a trailing
# nosemgrepon animport; it has to go on the line before.
.codacy.yaml carries a fourth: an engines.semgrep.exclude_paths block is not
honoured at all, though the identical engines.bandit one is. Semgrep is
therefore only ever silenced in the source.
Bandit also runs in ci.yml and pre-commit, so a finding lands next to the lint
failures instead of only in a dashboard:
.codacy.yaml and [tool.bandit] are kept in step with each other; changing one
without the other is how the two disagree.
One rule is switched off for tests/ only, B101 (assert used) — every
pytest assertion trips it. The rule exists because python -O strips asserts,
which is not how a test suite runs. There are no asserts under src/.
tests/test_cli_commands.py reads a JUnit report back to check it, and uses
defusedxml to do it — not because a file the test wrote three lines earlier is a
threat, but because it costs nothing: defusedxml is already a runtime dependency
for core/parser.py.
Everything under src/ is still scanned, and the accepted findings there carry
their reason next to them, so a new finding on one of those lines still has to
be looked at rather than inheriting a blanket exemption. There are two:
- The
subprocesscalls incli.py— fixed argv, absolute path fromshutil.which, no shell anywhere. core/junit.py— the only module that writes XML, and it never reads any.defusedxml, the replacement both tools recommend, exports noElement/SubElement/ElementTree, so there is nothing to switch to on the writing side and nothing to defend against either: the only untrusted values are package names, which ElementTree escapes. Keeping it in its own small module means that argument is made once, at the top of a file you can read in a minute, rather than buried in a 1,300-line CLI.
Bandit logs a nosec encountered, but no failed test warning for a couple of
them — it attributes multi-line statements to the wrong line — which is noise,
not a stale suppression.
CI¶
- ci.yml — lint (ruff, black, bandit) plus pytest with coverage on Python 3.10–3.12.
- docs.yml — build the site with
--strict, and deploy it to GitHub Pages frommain. - pr-title.yml — check that a pull request's title is a Conventional Commit.
- publish.yml — the release: on every merge to
main, decide whether the merged pull requests call for one, then bump, tag, publish to PyPI (Trusted Publishing), create the GitHub release and redeploy the docs.
CI runs on every push/PR to main/master.
Releases¶
Versions are never bumped by hand, and there is no release pull request to merge.
On every push to main, publish.yml:
- lists the commits since the last
v*tag and asks the GitHub API which pull request each came from, so the pull request title decides — whatever the merge method, and whatever the individual commits were called; - runs
scripts/release/release.py plan, which applies Conventional Commits:!or aBREAKING CHANGE:footer is major,featminor,fix,perf,revertanddepspatch, and anything else — or a merge that only containsdocs:,chore:,ci:… — releases nothing and stops there; - runs
release.py apply, which bumpspyproject.toml, theversionanddate-releasedofCITATION.cffand theversionofcodemeta.json, and turns## [Unreleased]into the new entry — the notes written there, followed by the pull requests that released them, or an entry generated from their titles when nobody wrote any; - commits
chore(release): vX.Y.Ztomain, tags it, builds the wheel, checksrostree --versionfrom it, publishes to PyPI, creates the GitHub release with the changelog entry as its notes, and redeploys these docs.
The commit and tag are pushed with the workflow's own token. That needs no
setting, approval or personal token — main only has to accept pushes from
Actions — and it starts no other workflow, which is why publish.yml builds and
deploys everything itself rather than leaving it to a tag or release trigger.
PyPI's trusted publisher names publish.yml, so the release has to stay in that
file.
tests/test_release.py runs the script against copies of the real files, so a
change to their layout that would break the release fails CI first.
Tests¶
TUI behaviour is covered by tests/test_tui_app.py, which drives the real app
through Textual's pilot harness (key presses, background workers, lazy expansion)
rather than testing helpers in isolation. tests/conftest.py clears the package
index and parse caches between tests — they are process-wide by design, so tests
must not share them.
The HTML graph viewer¶
src/rostree/web/ holds real .html, .css and .js files rather than Python
strings, so they can be edited and diffed like the front-end code they are.
core/webview.py inlines all three into one document at generation time.
There is no build step and no dependency, on purpose: the output has to work
from a file:// URL with no network, which rules out a CDN, and vendoring a
layout library to avoid one would be a worse trade than the ~200 lines of
layered-DAG layout in graph.js.
To work on it, generate a page and open it:
Two things that are easy to get wrong and have already bitten:
- Do not call
setPointerCaptureon the canvas. Chromium retargets the compatibility mouse events to the capturing element, so everyclickarrives on the<svg>and no node is ever clickable. Pan by listening onwindow. [hidden]needsdisplay: none !important, because several of the things it is applied to set an explicitdisplayfurther down the stylesheet.
tests/test_webview.py covers the payload and the document — that it is
self-contained, that no placeholder survives, and that a </script> in a
package description cannot break out of the data block. Behaviour inside the
page is not unit-tested; check it in a browser.
Docs¶
docs/ is a MkDocs site using
Material, published to
https://guilyx.github.io/rostree by docs.yml on every push to main.
pip install -e ".[docs]"
mkdocs serve # live reload on http://127.0.0.1:8000
mkdocs build --strict # what CI runs
- docs/index.md — Site home.
- docs/overview.md — System overview and data flow.
- docs/package-discovery.md — How packages are found (env vars, workspaces).
- docs/dependency-trees.md — parsing, the package index, repeat collapsing, graphs.
- docs/usage.md — CLI, TUI keys, Python API.
- docs/development.md — This file.
- docs/changelog.md — a one-line page that pulls in the root
CHANGELOG.md, so the changelog is written in exactly one place.
Two things the build enforces, both worth knowing before a PR fails on them:
strict: true. A link that resolves to nothing is a build failure, not a warning. Anything the site links to has to exist insidedocs/— a repo-relative path like../src/rostree/cli.pywill not resolve, so link to the file on GitHub by full URL instead. That is why the three such links insideCHANGELOG.mdare absolute: they have to work both on the site and in the file itself.docs/review.mdis excluded (exclude_docsinmkdocs.yml). It is the pre-0.3 code audit, kept as a record; most of what it reports is fixed, and publishing it as current documentation would mislead.roadmap.mdtherefore links to it on GitHub rather than as a page.
Keep the root README.md lean; link to these docs for details.