Add requirements-reference.txt#871
Conversation
Add loose version constraints to remaining tutorial requirements.txt files, generate requirements-reference.txt from PyPI, and check it in CI.
Remove extra trailing blank line in the Python dependencies section.
MakisH
left a comment
There was a problem hiding this comment.
Thank you for the contribution; this will be useful, and it comes at the right time.
My main suggestion, which will change a bit the code, is to split the requirements-reference.txt into one file per requirements.txt.
Other than that, most comments are regarding the version ranges. We do not need to document these in detail, but the comments here might be helpful in the future.
There was a problem hiding this comment.
Interesting approach, but this will not be very visible, will not be directly usable, and will quickly become outdated.
Why not just make a requirements-reference.txt next to each requirements.txt file? This would let one just do pip install -r requirements-reference.txt.
There was a problem hiding this comment.
Done, each requirements.txt now has a sibling requirements-reference.txt, so you can pip install -r requirements-reference.txt in that folder. The root file is removed.
| ## Python dependencies | ||
|
|
||
| Each Python participant lists dependencies in its local `requirements.txt` using loose version ranges (for example `numpy >1, <2`, `pyprecice~=3.0`). Tutorial `run.sh` scripts install from these files as before. | ||
|
|
||
| The repository root file `requirements-reference.txt` records the latest PyPI versions that satisfy those constraints at generation time. Regenerate it after changing tutorial requirements: | ||
|
|
||
| ```bash | ||
| python3 tools/report_tutorial_requirements.py | ||
| python3 tools/report_tutorial_requirements.py --check # CI-style check | ||
| ``` | ||
|
|
||
| This reference manifest is for documentation and reproducibility; it is not used by tutorial run scripts. | ||
|
|
||
| Legacy FEniCS packages (for example `fenics-dolfin`) are installed from the system or PPA in several tutorials and are listed in the reference file as non-PyPI dependencies. | ||
|
|
||
| When preparing a release pull request to `master`, regenerate and commit `requirements-reference.txt` so the pinned snapshot matches the current tutorial constraints. |
There was a problem hiding this comment.
This feels a bit disconnected here, but I cannot think of a better place for it right now. In the Python script, I suggested making a tools/releasing/ directory. It could be moved there.
There was a problem hiding this comment.
Docs moved to tools/releasing/README.md, with a short pointer from the root README.
| @@ -1,5 +1,5 @@ | |||
| setuptools # required by nutils | |||
| setuptools>=69.0.0 # required by nutils | |||
There was a problem hiding this comment.
Why specifically this version? No need to document, just for me to understand.
There was a problem hiding this comment.
Kept as a lower bound that works with current nutils; happy to loosen further if you prefer.
| numpy >1, <2 | ||
| pyprecice~=3.0 | ||
| setuptools | ||
| setuptools>=69.0.0 |
There was a problem hiding this comment.
Duplicate, can be removed (here and in more files)
There was a problem hiding this comment.
Yeah, I removed the duplicate lines.
| @@ -1,3 +1,3 @@ | |||
| matplotlib | |||
| matplotlib>=3.5,<4 | |||
There was a problem hiding this comment.
Aimed to stay on matplotlib 3.x. Unified to matplotlib>=3, <4 everywhere for consistency.
| matplotlib | ||
| treelog No newline at end of file | ||
| matplotlib>=3.5,<4 | ||
| treelog>=1,<2 No newline at end of file |
There was a problem hiding this comment.
Keep the treelog on 1.x; the exact pin is in the sibling reference file.
| # Not a pip requirements file (lists stdlib os, unpinned vtk); see develop naming quirk. | ||
| "flow-over-heated-plate/plot-final-interface-temperature-requirements.txt", |
There was a problem hiding this comment.
This should actually be a requirements.txt file. I wanted to tidy these plotting python scripts a bit.
Useful to have an exclusion list, I guess (mainly for unmaintained tutorials), but why does this need to be excluded?
There was a problem hiding this comment.
Done, moved to plot-final-interface-temperature/requirements.txt (dropped invalid OS), and removed the special exclusion.
There was a problem hiding this comment.
Let's move this into a new directory tools/releasing/. The documentation (from the root README.md) can also go there then.
There was a problem hiding this comment.
Done, script and docs are under tools/releasing/.
…ependency-versions-clean
Move the generator to tools/releasing/, emit sibling reference files instead of a root manifest, and clean up requirements from review feedback.
Summary
This PR implements #610 using the approach discussed after #749 and it is the rework of that:
requirements.txt fileskeep loose version ranges(for example numpy >1, <2, pyprecice~=3.0). This PR fills gaps where develop still had unpinned packages (~~39 files; ~ ~13 already had ranges).requirements-reference.txtrecords the exact PyPI versions that satisfy those ranges at generation time(Gemfile.lock style).tools/report_tutorial_requirements.pygenerates and checks that file from PyPI.check-requirements-reference.ymlruns--checkwhen requirements or the script change.Tutorial
run.sh scriptsare unchanged and still install from localrequirements.txt. The reference file is for reproducibility and releases, not for day-to-day installs.This is not tied to
reference_versions.yamland does not changetools/tests/requirements.txt(CI git refs are already handled by #857/#858).Test Plan
python3 tools/report_tutorial_requirements.py
python3 tools/report_tutorial_requirements.py --check
pre-commit run actionlint --files .github/workflows/check-requirements-reference.yml
pip install --dry-run -r elastic-tube-1d/fluid-python/requirements.txt
Closes #610