Build on rostree¶
rostree is a library first. The CLI, the TUI and the HTML viewer are all built on
the same public API, so anything rostree can tell you on the command line, your
own tool can ask for directly — no parsing of terminal output.
This page shows how the pieces fit, then four complete programs built on them.
The programs live in
examples/ and the test
suite runs every one of them against a generated workspace, so the code below is
the code that is known to work.
How the pieces fit¶
graph LR
env[(AMENT_PREFIX_PATH,<br/>workspace src/)] --> index[PackageIndex<br/><small>core/index.py</small>]
index --> parser[parse_package_xml<br/><small>core/parser.py</small>]
parser --> graph[DependencyGraph<br/><small>build_graph</small>]
parser --> tree[DependencyNode<br/><small>build_tree</small>]
graph --> analysis[why · rdeps · diff<br/><small>core/analysis.py</small>]
graph --> render[DOT · Mermaid · HTML<br/><small>core/graph.py, core/webview.py</small>]
tree --> cli[CLI and TUI]
analysis --> cli
render --> cli
analysis --> you[your tool]
render --> you
graph --> you
| Layer | What it gives you | Entry points |
|---|---|---|
| Index | Every package visible to the environment, found once, looked up by name in constant time | get_index(), PackageIndex |
| Parser | One package.xml, with REP 149 conditions evaluated and the result cached |
get_package_info(), PackageInfo |
| Graph | Everything reachable from some roots: edges, versions, what is missing, what was past the depth limit | build_graph(), DependencyGraph |
| Tree | The same, as a printable tree with repeats collapsed | build_tree(), DependencyNode |
| Analysis | The questions: why, who depends on this, what changed | dependency_paths(), reverse_dependencies(), diff_dependencies() |
| Render | Text and pictures | to_dot(), to_mermaid(), to_html() |
Everything above is importable from rostree.api and covered by semantic
versioning. The usage guide lists every function and its
arguments; How the algorithms work explains what each one does
underneath.
Two habits make a tool built on rostree fast and correct:
- Build the index once and pass it around (
index=get_index()). Every function accepts one, and each lookup afterwards is a dictionary access instead of a filesystem scan. - Use the same scope arguments as the CLI —
dep_type,package_filter,report— so your tool androstreegive the same answer for the same question, and so a filter never shrinks a result without your code knowing.
Recipe: an architecture rule for CI¶
Some dependencies are design mistakes: a sensor driver should not reach into navigation, a message package should not depend on a node. This checks a rule across the whole workspace and, when it fails, prints the shortest chain that breaks it — so the failure says where to cut, not just that a rule was broken.
"""
Enforce an architecture rule in CI: some packages must never depend on others.
python forbidden_dependencies.py 'my_robot_driver_*' 'nav2_*'
Fails (exit 1) if any package matching the first glob depends, at any depth, on
a package matching the second — and prints the shortest chain that does it, so
the failure says *where* to cut, not just that a rule was broken.
"""
from __future__ import annotations
import fnmatch
import sys
from rostree.api import build_graph, get_index
from rostree.core.analysis import shortest_paths
def violations(source_glob: str, forbidden_glob: str) -> list[list[str]]:
index = get_index()
sources = [n for n in index.workspace_names() if fnmatch.fnmatchcase(n, source_glob)]
found = []
for source in sources:
graph = build_graph(source, index=index)
reached = sorted(n for n in graph.depths if fnmatch.fnmatchcase(n, forbidden_glob))
for target in reached:
if target != source:
found.append(shortest_paths(graph, source, target, limit=1)[0])
return found
def main(argv: list[str]) -> int:
source_glob, forbidden_glob = argv
bad = violations(source_glob, forbidden_glob)
for path in bad:
print("forbidden: " + " -> ".join(path))
if not bad:
print(f"ok: nothing matching {source_glob!r} depends on {forbidden_glob!r}")
return 1 if bad else 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
$ python examples/forbidden_dependencies.py 'my_robot_driver_*' 'nav2_*'
forbidden: my_robot_driver_cam -> nav2_core
Recipe: where does a change ripple furthest?¶
Rank your packages by how many others depend on them, directly and through any chain. The top of the list is where a breaking change costs the most.
"""
Which of your packages would a change ripple furthest from?
python ripple_report.py [N]
Ranks your workspace packages by how many other packages depend on them,
directly or transitively. The top of this list is where a breaking change costs
the most, and where tests and review matter most.
"""
from __future__ import annotations
import sys
from rostree.api import get_index, reverse_dependencies
def ripple(limit: int = 10) -> list[tuple[str, int, int]]:
index = get_index()
rows = []
for name in index.workspace_names():
direct = reverse_dependencies(name, index=index)
everything = reverse_dependencies(name, index=index, transitive=True)
rows.append((name, len(direct), len(everything)))
rows.sort(key=lambda row: (-row[2], -row[1], row[0]))
return rows[:limit]
def main(argv: list[str]) -> int:
limit = int(argv[0]) if argv else 10
print(f"{'package':32} {'direct':>7} {'transitive':>11}")
for name, direct, total in ripple(limit):
print(f"{name:32} {direct:>7} {total:>11}")
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
$ python examples/ripple_report.py 3
package direct transitive
my_robot_msgs 1 4
my_robot_utils 2 3
my_robot_driver_cam 1 1
The distro's own packages are never ranked: workspace_names() leaves them out,
wherever the distro is installed.
Recipe: make dependency changes visible in review¶
Commit a lock of a package's dependency set next to the code. A pull request that adds, drops or bumps a dependency then has to update the lock in the same diff, where a reviewer sees it.
"""
Fail CI when a package's dependency set changes without the snapshot changing too.
python drift_gate.py my_robot_bringup deps.lock.json # check
python drift_gate.py my_robot_bringup deps.lock.json --update # accept
Commit the snapshot next to the code. A pull request that pulls in, drops or
bumps a dependency then has to update it in the same diff — so the change is
something a reviewer sees, rather than something they find out about later.
"""
from __future__ import annotations
import json
import sys
from pathlib import Path
from rostree.api import build_graph, dependency_versions, diff_dependencies
def current(package: str) -> dict[str, str]:
return dependency_versions(build_graph(package), exclude=[package])
def main(argv: list[str]) -> int:
package, lock = argv[0], Path(argv[1])
now = current(package)
if "--update" in argv[2:]:
lock.write_text(json.dumps(now, indent=2, sort_keys=True) + "\n", encoding="utf-8")
print(f"wrote {len(now)} dependencies to {lock}")
return 0
diff = diff_dependencies(json.loads(lock.read_text(encoding="utf-8")), now)
if not diff:
print(f"ok: {package} matches {lock}")
return 0
for name in diff.added:
print(f"+ {name} {now[name]}".rstrip())
for name in diff.removed:
print(f"- {name}")
for name, (old, new) in diff.changed.items():
print(f"~ {name} {old} -> {new}")
print(f"dependencies of {package} changed; rerun with --update if that is intended")
return 1
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
$ python examples/drift_gate.py my_robot_bringup deps.lock.json
+ my_robot_extra 1.0.0
dependencies of my_robot_bringup changed; rerun with --update if that is intended
rostree diff --save / --against does the same from the command line, and also
records the scope the snapshot was taken under.
Recipe: a format rostree does not ship¶
build_graph returns plain data, so any output format is a loop away. This one
writes CSV; the same shape produces PlantUML, a database import, or the input of
another analysis.
"""
Write the dependency graph in a format rostree does not ship: here, CSV.
python export_edges.py my_robot_bringup > edges.csv
`build_graph` returns plain data — packages, edges, what is missing — so any
other format is a loop away. The same shape works for PlantUML, a database
import, or the input of another analysis tool.
"""
from __future__ import annotations
import csv
import sys
from rostree.api import build_graph, get_index
def rows(package: str):
index = get_index()
graph = build_graph(package, index=index)
for parent, children in sorted(graph.edges.items()):
source = index.get(parent)
for child in children:
target = index.get(child)
yield {
"from": parent,
"to": child,
"from_source": source.kind.value if source else "missing",
"to_source": target.kind.value if target else "missing",
"depth": graph.depths.get(parent, ""),
}
def main(argv: list[str]) -> int:
writer = csv.DictWriter(
sys.stdout, fieldnames=["from", "to", "from_source", "to_source", "depth"]
)
writer.writeheader()
writer.writerows(rows(argv[0]))
return 0
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
Adding to rostree itself¶
New ideas usually start as one of the scripts above. When one is useful enough to belong in the tool:
- Analysis goes in
core/analysis.pyas a pure function — no printing, nosys.exit, no environment reads — and is exported fromrostree.api. - A CLI command is then a thin
cmd_*function incli.pythat parses arguments, calls the analysis, and formats the result. It should accept the shared scope flags (_add_runtime_arg,_add_filter_args) and report what a filter removed. - Tests for the analysis use small generated graphs; for the command, drive
main([...])end to end, so the argument wiring is tested too.
See Development for setup, and the contributing guide in the repository for how changes are proposed and reviewed.