[common] Support catalog context without Hadoop configuration#8193
[common] Support catalog context without Hadoop configuration#8193tchivs wants to merge 3 commits into
Conversation
|
Added the Trino 440 companion branch that consumes this API:
The Trino side uses Validation run locally:
|
|
@JingsongLi PTAL. This is a narrower follow-up to #6653 based on your previous feedback:
Thanks. |
|
Additional Trino 440 companion validation passed locally on branch https://github.com/tchivs/trino/tree/paimon/trino-440-paimon-1.5:
|
| hadoopConf == null ? getHadoopConfiguration(options) : hadoopConf); | ||
| hadoopConf == null && !loadHadoopConf | ||
| ? null | ||
| : new SerializableConfiguration( |
There was a problem hiding this comment.
Can you just create a method and do try catch? If there is no Hadoop class, set it directly to NULL.
Purpose
This is a narrower follow-up to #6653 for engines such as Trino that provide their own FileIO and should not require Hadoop configuration initialization.
Instead of refactoring the FileIO/CatalogContext hierarchy, this keeps all existing
CatalogContext.create(...)behavior unchanged and adds an explicitCatalogContext.createWithoutHadoop(...)factory for the no-Hadoop path.Changes
CatalogContext.create(...)overloads loading Hadoop configuration by default for compatibility.CatalogContext.createWithoutHadoop(...)for callers that provide their ownFileIOLoader.hadoopConf()fail with a clearIllegalStateExceptionwhen called on a Hadoop-free context.Tests
mvn -pl paimon-core -am -Pfast-build -DfailIfNoTests=false -Dtest=CatalogFactoryTest testmvn -pl paimon-common -am -Pfast-build -DfailIfNoTests=false -Dtest=FileIOTest,ResolvingFileIOTest testNotes
A companion Trino change is being prepared to consume this API by passing Trino's FileIO loader and disabling Hadoop default configuration loading.