Skip to content

[core] Introduce file resource management#8179

Open
gavin9402 wants to merge 3 commits into
apache:masterfrom
gavin9402:resource-management
Open

[core] Introduce file resource management#8179
gavin9402 wants to merge 3 commits into
apache:masterfrom
gavin9402:resource-management

Conversation

@gavin9402

@gavin9402 gavin9402 commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Purpose

Introduce resource management capabilities to the REST Catalog, providing a unified way to manage file resources (FILE, JAR, PY, ARCHIVE) associated with databases. This lays the foundation for upcoming ML model and function features, where users will need to reference and manage external file resources such as model artifacts, UDF JARs, and Python scripts.

Changes

Resource Model (paimon-api)

  • Resource interface and AbstractResource base class — define the resource abstraction with properties like name, type, description, URI, and custom properties
  • FileResource, JarResource, PyResource, ArchiveResource — concrete resource types for FILE/JAR/PY/ARCHIVE
  • ResourceType enum — four supported resource types
  • ResourceChange — change operations for altering resources (setProperty, removeProperty, setDescription, setUri)
  • ResourceDeserializer — Jackson deserializer for polymorphic resource deserialization

REST API (paimon-api)

  • ResourcePaths — URL path builders for resource endpoints (/resources, /resource-details, /resources/{name})
  • RESTApi — 8 new resource management API methods: listResources, listResourcesPaged, listResourceDetailsPaged, getResource, createResource, dropResource, alterResource, listResourcesPagedGlobally
  • Request/Response classes: CreateResourceRequest, AlterResourceRequest, GetResourceResponse, ListResourcesResponse, ListResourceDetailsResponse, ListResourcesGloballyResponse

Catalog Interface (paimon-core)

  • Catalog — 8 new interface methods for resource CRUD + ResourceAlreadyExistException and ResourceNotExistException inner exception classes
  • AbstractCatalog — default UnsupportedOperationException implementations
  • DelegateCatalog — delegation implementations
  • RESTCatalog — full REST-backed implementations

Tests (paimon-core)

  • RESTApiJsonTest — JSON serialization/deserialization tests for resource request/response classes
  • RESTCatalogTest — integration tests for resource CRUD operations
  • RESTCatalogServer — mock REST server with resource management route handlers
  • MockRESTMessage — test helper methods for constructing resource test data

API Summary

Operation Method Endpoint
List resources GET /v1/{prefix}/databases/{db}/resources
List resource details GET /v1/{prefix}/databases/{db}/resource-details
Get resource GET /v1/{prefix}/databases/{db}/resources/{name}
Create resource POST /v1/{prefix}/databases/{db}/resources
Drop resource DELETE /v1/{prefix}/databases/{db}/resources/{name}
Alter resource POST /v1/{prefix}/databases/{db}/resources/{name}
List resources globally GET /v1/{prefix}/resources

Tests

mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -Dtest="RESTApiJsonTest,RESTCatalogTest" test

@TheR1sing3un

Copy link
Copy Markdown
Member

It's a very surprising pr. Is there any relevant pip to provide more background information?

@gavin9402

Copy link
Copy Markdown
Contributor Author

It's a very surprising pr. Is there any relevant pip to provide more background information?

Thank you for your suggestion. I will submit the PIP as soon as possible.

@JingsongLi

Copy link
Copy Markdown
Contributor

First PR, I think you can focus on Resource introducing.

@gavin9402 gavin9402 force-pushed the resource-management branch from 57a260d to 81f5235 Compare June 10, 2026 02:03
@gavin9402

Copy link
Copy Markdown
Contributor Author

First PR, I think you can focus on Resource introducing.

Sure, let me revise it.

@gavin9402 gavin9402 force-pushed the resource-management branch from 81f5235 to 25c4603 Compare June 10, 2026 02:50
@gavin9402 gavin9402 changed the title [core] Introduce resource && ML model management [core] Introduce file resource management Jun 10, 2026
@gavin9402 gavin9402 requested a review from JingsongLi June 10, 2026 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants