Skip to content

Deduplicate metadata download boilerplate#1027

Merged
Zethson merged 1 commit into
mainfrom
refactor/metadata-download-helper
Jun 26, 2026
Merged

Deduplicate metadata download boilerplate#1027
Zethson merged 1 commit into
mainfrom
refactor/metadata-download-helper

Conversation

@Zethson

@Zethson Zethson commented Jun 26, 2026

Copy link
Copy Markdown
Member

Summary

The identical download-and-cache block (build the cache path, skip if the file is already present, _download from the pertpy example-data server) was repeated seven times in _cell_line.py and again in _moa.py and _drug.py.

This PR extracts it into a single MetaData._download_metadata staticmethod, sitting next to the existing _warn_unmatch helper on the shared superclass:

_BASE_URL = "https://exampledata.scverse.org/pertpy"

@staticmethod
def _download_metadata(file_name: str) -> Path:
    """Download a metadata file into scanpy's cache directory if absent and return its path."""
    path = Path(settings.cachedir) / file_name
    if not path.exists():
        _download(url=f"{MetaData._BASE_URL}/{file_name}", output_file_name=file_name,
                  output_path=settings.cachedir, block_size=4096)
    return path

Each _download_* method drops from ~12 lines to 2–3, and the base URL / cache-skip logic now lives in one place.

Details

  • The pharmgkb branch in _drug.py stays inline on purpose: it downloads a zip and renames the extracted file, with the existence check on a different file than the one downloaded, so folding it into the generic helper would change behavior.
  • Collapsed the threefold source dispatch in Drug.annotate / Drug.lookup now that the lazy-load is uniform.
  • Removed the now-unused Path / settings / _download imports from _cell_line.py and _moa.py.

Net change: +92 / −159 in the metadata package.

Tests

Added network-free unit tests (tests/metadata/test_metadata.py) for the new helper: the existing metadata tests cover downloads end-to-end but require the network, and the refactor moved URL construction into one spot that now affects every download. The new tests mock _download and assert (a) the URL is built from the base and (b) a cached file is not re-fetched.

The full tests/metadata/ suite (12 tests, real downloads) passes locally, confirming the refactor is behavior-preserving across all sources.

🤖 Generated with Claude Code

The same download-and-cache block (build cache path, skip if present,
`_download` from the pertpy example-data server) was repeated seven times
in `_cell_line.py` and again in `_moa.py` and `_drug.py`.

Extract it into a single `MetaData._download_metadata` staticmethod next to
`_warn_unmatch`, centralizing the base URL and cache-skip logic. The
pharmgkb branch stays inline since it downloads a zip and renames the
extracted file with the existence check on a different file.

Also collapse the threefold source dispatch in `Drug.annotate`/`Drug.lookup`
now that the lazy-load is uniform.

Add network-free unit tests for the new helper covering URL construction and
the cached-file skip.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codecov-commenter

codecov-commenter commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.78788% with 7 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.98%. Comparing base (473da92) to head (6512ca1).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
pertpy/metadata/_cell_line.py 60.00% 4 Missing ⚠️
pertpy/metadata/_drug.py 72.72% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1027      +/-   ##
==========================================
+ Coverage   77.95%   77.98%   +0.02%     
==========================================
  Files          50       50              
  Lines        6587     6554      -33     
==========================================
- Hits         5135     5111      -24     
+ Misses       1452     1443       -9     
Files with missing lines Coverage Δ
pertpy/metadata/_metadata.py 74.19% <100.00%> (+12.28%) ⬆️
pertpy/metadata/_moa.py 74.28% <100.00%> (-3.77%) ⬇️
pertpy/metadata/_drug.py 84.78% <72.72%> (+0.52%) ⬆️
pertpy/metadata/_cell_line.py 55.11% <60.00%> (-1.25%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Zethson Zethson merged commit eaea364 into main Jun 26, 2026
19 checks passed
@Zethson Zethson deleted the refactor/metadata-download-helper branch June 26, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants