Skip to content

Add requirements-reference.txt#871

Open
PranjalManhgaye wants to merge 5 commits into
precice:developfrom
PranjalManhgaye:fix-610-python-dependency-versions-clean
Open

Add requirements-reference.txt#871
PranjalManhgaye wants to merge 5 commits into
precice:developfrom
PranjalManhgaye:fix-610-python-dependency-versions-clean

Conversation

@PranjalManhgaye

Copy link
Copy Markdown
Collaborator

Summary

This PR implements #610 using the approach discussed after #749 and it is the rework of that:

  • Tutorial requirements.txt files keep loose version ranges (for example numpy >1, <2, pyprecice~=3.0). This PR fills gaps where develop still had unpinned packages (~~39 files; ~ ~13 already had ranges).
  • A new root file requirements-reference.txt records the exact PyPI versions that satisfy those ranges at generation time (Gemfile.lock style).
  • tools/report_tutorial_requirements.py generates and checks that file from PyPI.
  • CI workflow check-requirements-reference.yml runs --check when requirements or the script change.
    Tutorial run.sh scripts are unchanged and still install from local requirements.txt. The reference file is for reproducibility and releases, not for day-to-day installs.

This is not tied to reference_versions.yaml and does not change tools/tests/requirements.txt (CI git refs are already handled by #857/#858).

Test Plan

  • python3 tools/report_tutorial_requirements.py

  • python3 tools/report_tutorial_requirements.py --check

  • pre-commit run actionlint --files .github/workflows/check-requirements-reference.yml

  • pip install --dry-run -r elastic-tube-1d/fluid-python/requirements.txt

Closes #610

Add loose version constraints to remaining tutorial requirements.txt
files, generate requirements-reference.txt from PyPI, and check it in CI.
Remove extra trailing blank line in the Python dependencies section.
@PranjalManhgaye PranjalManhgaye requested a review from MakisH July 9, 2026 05:36
@PranjalManhgaye PranjalManhgaye moved this from Planned next to Needs review in GSoC 2026: System tests improvements Jul 9, 2026
@PranjalManhgaye PranjalManhgaye changed the title Add tutorial Python dependency reference manifest for #610 Add tutorial Python dependency reference manifest (rework of #749, closes #610) Jul 9, 2026
@PranjalManhgaye PranjalManhgaye changed the title Add tutorial Python dependency reference manifest (rework of #749, closes #610) Add tutorial Python dependency reference manifest (rework of #749) Jul 9, 2026

@MakisH MakisH left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution; this will be useful, and it comes at the right time.

My main suggestion, which will change a bit the code, is to split the requirements-reference.txt into one file per requirements.txt.

Other than that, most comments are regarding the version ranges. We do not need to document these in detail, but the comments here might be helpful in the future.

Comment thread requirements-reference.txt Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting approach, but this will not be very visible, will not be directly usable, and will quickly become outdated.

Why not just make a requirements-reference.txt next to each requirements.txt file? This would let one just do pip install -r requirements-reference.txt.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, each requirements.txt now has a sibling requirements-reference.txt, so you can pip install -r requirements-reference.txt in that folder. The root file is removed.

Comment thread README.md Outdated
Comment on lines +17 to +32
## Python dependencies

Each Python participant lists dependencies in its local `requirements.txt` using loose version ranges (for example `numpy >1, <2`, `pyprecice~=3.0`). Tutorial `run.sh` scripts install from these files as before.

The repository root file `requirements-reference.txt` records the latest PyPI versions that satisfy those constraints at generation time. Regenerate it after changing tutorial requirements:

```bash
python3 tools/report_tutorial_requirements.py
python3 tools/report_tutorial_requirements.py --check # CI-style check
```

This reference manifest is for documentation and reproducibility; it is not used by tutorial run scripts.

Legacy FEniCS packages (for example `fenics-dolfin`) are installed from the system or PPA in several tutorials and are listed in the reference file as non-PyPI dependencies.

When preparing a release pull request to `master`, regenerate and commit `requirements-reference.txt` so the pinned snapshot matches the current tutorial constraints.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels a bit disconnected here, but I cannot think of a better place for it right now. In the Python script, I suggested making a tools/releasing/ directory. It could be moved there.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs moved to tools/releasing/README.md, with a short pointer from the root README.

@@ -1,5 +1,5 @@
setuptools # required by nutils
setuptools>=69.0.0 # required by nutils

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why specifically this version? No need to document, just for me to understand.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kept as a lower bound that works with current nutils; happy to loosen further if you prefer.

numpy >1, <2
pyprecice~=3.0
setuptools
setuptools>=69.0.0

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate, can be removed (here and in more files)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I removed the duplicate lines.

@@ -1,3 +1,3 @@
matplotlib
matplotlib>=3.5,<4

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this version range?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aimed to stay on matplotlib 3.x. Unified to matplotlib>=3, <4 everywhere for consistency.

Comment thread two-scale-heat-conduction/micro-nutils/requirements.txt Outdated
matplotlib
treelog No newline at end of file
matplotlib>=3.5,<4
treelog>=1,<2 No newline at end of file

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this version range?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep the treelog on 1.x; the exact pin is in the sibling reference file.

Comment thread tools/report_tutorial_requirements.py Outdated
Comment thread tools/report_tutorial_requirements.py Outdated
Comment on lines +33 to +34
# Not a pip requirements file (lists stdlib os, unpinned vtk); see develop naming quirk.
"flow-over-heated-plate/plot-final-interface-temperature-requirements.txt",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should actually be a requirements.txt file. I wanted to tidy these plotting python scripts a bit.

Useful to have an exclusion list, I guess (mainly for unmaintained tutorials), but why does this need to be excluded?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, moved to plot-final-interface-temperature/requirements.txt (dropped invalid OS), and removed the special exclusion.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move this into a new directory tools/releasing/. The documentation (from the root README.md) can also go there then.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, script and docs are under tools/releasing/.

@MakisH MakisH changed the title Add tutorial Python dependency reference manifest (rework of #749) Add requirements-reference.txt Jul 9, 2026
Move the generator to tools/releasing/, emit sibling reference files
instead of a root manifest, and clean up requirements from review feedback.
@PranjalManhgaye PranjalManhgaye requested a review from MakisH July 10, 2026 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Needs review

Development

Successfully merging this pull request may close these issues.

Fix versions of Python dependencies for each distribution

2 participants