chore: add created_at/updated_at to packages-db schemas for Tinybird sync

[joanagmaia]

Summary

• Adds a migration (V1780600000__add_created_updated_at.sql) to add created_at / updated_at audit columns to all packages-db tables that need them, enabling Tinybird watermark-based sync.
• Updates all INSERT/UPSERT queries across packages_worker and data-access-layer to populate the new columns.
• Excludes tables that already have equivalent timestamp columns (e.g. package_name_history, audit logs, state-machine tables).

Notes

• repos already has created_at (stores the GitHub repo creation date) — only updated_at is added there.
• Partitioned tables (versions, package_dependencies, downloads_daily, downloads_last_30d): PostgreSQL 12+ propagates new columns to all child partitions automatically.

🤖 Generated with Claude Code

---

Note

Medium Risk
Requires a coordinated DB migration and touches many core ingestion write paths; incorrect rollout or missed writers could skew Tinybird incremental sync watermarks.

Overview
Adds created_at / updated_at (or created_at only where last_synced_at, verified_at, etc. already act as change timestamps) across packages-db tables via migration V1780600000__add_created_updated_at.sql, aimed at Tinybird watermark-based sync. Audit/state tables and repos are intentionally left unchanged per the migration comments.

Ingestion and app writes are updated so new rows set these columns: deps.dev merge SQL (ingestPackages, ingestVersions, ingestDependencies, ingestAdvisories, ingestRepos for package_repos) and npm/OSV paths in data-access-layer (packages, versions, downloads, maintainers, funding links, advisories). Upsert paths that already update rows now refresh updated_at where that column exists (e.g. OSV advisories, downloads_last_30d, maintainers); many bulk merges still use ON CONFLICT DO NOTHING, so existing rows keep migration backfill timestamps until a real update path runs.

^{Reviewed by Cursor Bugbot for commit c4af1ac. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

• feat: add user authentication (CM-123)
• feat: add user authentication (IN-123)

Projects:

• CM: Community Data Platform
• IN: Insights

Please add a Jira issue key to your PR title.

[Copilot]

Pull request overview

This PR adds created_at / updated_at audit timestamps across the packages-db schema so Tinybird can use timestamp watermarks for incremental sync, and updates the main write paths (packages_worker + DAL) to populate/bump these columns.

Changes:

• Add Flyway migration V1780600000__add_created_updated_at.sql to introduce created_at / updated_at (and updated_at only for repos).
• Update DAL INSERT/UPSERT queries to write created_at / updated_at and bump updated_at on conflict updates.
• Update packages_worker deps.dev ingestion and repo enrichment SQL to set/bump updated_at.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
services/libs/data-access-layer/src/packages/versions.ts	Adds created_at/updated_at to npm versions upsert and bumps updated_at on conflict.
services/libs/data-access-layer/src/packages/repos.ts	Adds created_at/updated_at to package_repos upsert and bumps updated_at on conflict.
services/libs/data-access-layer/src/packages/packages.ts	Adds created_at/updated_at to npm package upsert and bumps updated_at on conflict.
services/libs/data-access-layer/src/packages/osv.ts	Adds created_at/updated_at to OSV advisory-related inserts/upserts and bumps updated_at on conflict.
services/libs/data-access-layer/src/packages/maintainers.ts	Adds created_at/updated_at to maintainers + package_maintainers writes and bumps updated_at on conflict.
services/libs/data-access-layer/src/packages/fundingLinks.ts	Adds created_at/updated_at to funding link insert path.
services/libs/data-access-layer/src/packages/downloadsLast30d.ts	Adds created_at/updated_at to last-30d downloads upsert and bumps updated_at on conflict.
services/libs/data-access-layer/src/packages/downloadsDaily.ts	Adds created_at/updated_at to daily downloads insert path.
services/apps/packages_worker/src/enricher/updateEnrichedRepos.ts	Bumps repos.updated_at when enrichment updates or when marking repos skipped.
services/apps/packages_worker/src/deps-dev/workflows/ingestVersions.ts	Adds created_at/updated_at to deps.dev versions merge inserts.
services/apps/packages_worker/src/deps-dev/workflows/ingestRepos.ts	Adds updated_at to deps.dev repos inserts; adds created_at/updated_at to package_repos inserts.
services/apps/packages_worker/src/deps-dev/workflows/ingestPackages.ts	Adds created_at/updated_at to deps.dev packages merge inserts.
services/apps/packages_worker/src/deps-dev/workflows/ingestDependencies.ts	Adds created_at/updated_at to deps.dev dependencies merge inserts.
services/apps/packages_worker/src/deps-dev/workflows/ingestAdvisories.ts	Adds created_at/updated_at to deps.dev advisories/advisory_packages/affected_ranges inserts.
services/apps/packages_worker/src/deps-dev/activities/rankPackagesUniverse.ts	Adds created_at/updated_at to packages_universe repopulation insert.
backend/src/osspckgs/migrations/V1780600000__add_created_updated_at.sql	Adds audit timestamps to packages-db tables for Tinybird watermarking.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit c4af1ac. Configure here.}

[cursor]

Missing updated_at on package_id resolve

Medium Severity

This PR adds updated_at on advisory_packages for Tinybird watermarks and bumps it in upsertAdvisoryPackage, but resolveMissingPackageIds still only sets package_id. Resolved links can stay at an old updated_at, so incremental sync may skip rows whose foreign key was backfilled.

 

^{Reviewed by Cursor Bugbot for commit c4af1ac. Configure here.}