Skip to content

fix: Avoid changing table configs when creating a table with an existing base path on Spark#14175

Merged
yihua merged 4 commits into
apache:masterfrom
yihua:fix-avoid-table-config-update-on-external-table
Oct 30, 2025
Merged

fix: Avoid changing table configs when creating a table with an existing base path on Spark#14175
yihua merged 4 commits into
apache:masterfrom
yihua:fix-avoid-table-config-update-on-external-table

Conversation

@yihua

@yihua yihua commented Oct 28, 2025

Copy link
Copy Markdown
Contributor

Describe the issue this Pull Request addresses

When creating an external table using the following CREATE TABLE SQL statement on Spark, the table config file .hoodie/hoodie.properties is updated with additional table configs, hoodie.table.create.schema and hoodie.database.name. However, these two table configs are not needed for an external table, especially for reading the table only. Updating the table config requires write access which is not needed for read-only case, causing confusion to users.

CREATE TABLE IF NOT EXISTS table_name USING hudi LOCATION '<table_base_path>'

Summary and Changelog

This PR fixes the issue above, to keep the table config the same during executing CREATE TABLE SQL statement if the table's base path exists, where the change to table configs is unnecessary.

A new test is added to validate the change.

Impact

Improves CREATE TABLE SQL experience.

Risk Level

low

Documentation Update

N/A

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

@github-actions github-actions Bot added the size:M PR with lines of changes in (100, 300] label Oct 28, 2025
@yihua yihua changed the title fix: Avoid changing table configs when creating an external table with an existing base path on Spark fix: Avoid changing table configs when creating a table with an existing base path on Spark Oct 29, 2025
@yihua yihua added this to the release-1.1.0 milestone Oct 29, 2025
@yihua yihua force-pushed the fix-avoid-table-config-update-on-external-table branch from adecff2 to 3bb2a60 Compare October 30, 2025 05:43
@hudi-bot

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands@hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@yihua yihua merged commit 2eddf41 into apache:master Oct 30, 2025
70 checks passed
yihua added a commit to yihua/hudi that referenced this pull request Nov 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release-1.1.0 size:M PR with lines of changes in (100, 300]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants