Skip to content

Replace time based logic with atomic rename for write#2897

Merged
jaydeluca merged 5 commits into
open-telemetry:mainfrom
mamazzol:disk-buffering-atomic-rename
Jul 8, 2026
Merged

Replace time based logic with atomic rename for write#2897
jaydeluca merged 5 commits into
open-telemetry:mainfrom
mamazzol:disk-buffering-atomic-rename

Conversation

@mamazzol

@mamazzol mamazzol commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Description:

Feature addition: This PR replaces the time-based separation with an atomic rename

  • The writer always writes to .tmp. On close the temporary file is atomically renamed to its final name via File.renameTo, which delegates to POSIX rename(2) for same-directory renames.
  • FolderManager now sweeps for orphan temporary files at construction and promotes them.
  • A read attempt that finds no ready file will force-close an expired writer instead of waiting for the next append.

As a consequence, minFileAgeForReadMillis is no longer required for correctness (default is now 0) and the build-time validation minFileAgeForRead > maxFileAgeForWrite has been dropped.

Existing Issue(s):

#2889

Testing:

./gradlew :disk-buffering:check is green.

  • New WritableFileTest case for the staging-then-rename behavior.
  • New FolderManagerTest cases for orphan recovery and force-close-on-read with an expired writer.

Documentation:

  • README.md: updated the writer/reader synchronization explanation and the minFileAgeForReadMillis description.
  • DESIGN.md: added a short section on the rename-on-close model.
  • FileStorageConfiguration#getMinFileAgeForReadMillis Javadoc updated to describe the new default and use case.

Copilot AI review requested due to automatic review settings June 2, 2026 12:23
@mamazzol
mamazzol requested a review from a team as a code owner June 2, 2026 12:23
@linux-foundation-easycla

linux-foundation-easycla Bot commented Jun 2, 2026

Copy link
Copy Markdown

CLA Signed
The committers listed above are authorized under a signed CLA.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR changes disk-buffering writer/reader coordination to use a “stage then rename-on-close” workflow, reducing reliance on time-based read delays and improving crash recovery for in-flight writes.

Changes:

  • Writer now appends to <timestamp>.tmp and promotes to the final numeric filename on close.
  • Folder startup recovers orphan *.tmp files by promoting them to final names when safe.
  • Configuration/docs updated to make minFileAgeForReadMillis optional (default 0) and update tests accordingly.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
disk-buffering/src/main/java/io/opentelemetry/contrib/disk/buffering/internal/storage/files/WritableFile.java Implements staging + rename-on-close promotion behavior.
disk-buffering/src/main/java/io/opentelemetry/contrib/disk/buffering/internal/storage/FolderManager.java Recovers orphan temp files and allows reader to close expired writer to surface data.
disk-buffering/src/main/java/io/opentelemetry/contrib/disk/buffering/storage/impl/FileStorageConfiguration.java Updates defaults/docs and removes min-read-age vs max-write-age validation.
disk-buffering/src/test/java/io/opentelemetry/contrib/disk/buffering/internal/storage/files/WritableFileTest.java Adds/updates tests for staged writes and promotion-on-close behavior.
disk-buffering/src/test/java/io/opentelemetry/contrib/disk/buffering/internal/storage/StorageTest.java Updates expectations to account for visible .tmp files while writing.
disk-buffering/src/test/java/io/opentelemetry/contrib/disk/buffering/internal/storage/FolderManagerTest.java Adds coverage for orphan *.tmp recovery and adjusts filename expectations.
disk-buffering/README.md Updates user-facing explanation for rename-on-close synchronization.
disk-buffering/DESIGN.md Documents the new synchronization approach and crash recovery.
CHANGELOG.md Notes behavioral/config changes for the release notes.

Comment thread disk-buffering/README.md Outdated

@LikeTheSalad LikeTheSalad left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, @mamazzol 🙏

Please take a look at Copilot's suggestions because I think those are valid points. But apart from those, I think it's good to go.

@mamazzol
mamazzol requested a review from LikeTheSalad June 3, 2026 15:27

@LikeTheSalad LikeTheSalad left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! 🙏

@breedx-splk breedx-splk changed the title Replace time based logic with atomci rename for write Replace time based logic with atomic rename for write Jun 11, 2026

@breedx-splk breedx-splk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't fully reviewed this yet, but I wanted to ask the same question I asked in the issue: What happens to the staging files when the process exits and restarts? Will the staged files be moved over or ?

@breedx-splk

Copy link
Copy Markdown
Contributor

I haven't fully reviewed this yet, but I wanted to ask the same question I asked in the issue: What happens to the staging files when the process exits and restarts? Will the staged files be moved over or ?

Ok, I see it now, it does look like there is explicit code to cover this case and recover the "orphaned" files. Killer! Thanks.

@mamazzol

Copy link
Copy Markdown
Contributor Author

I haven't fully reviewed this yet, but I wanted to ask the same question I asked in the issue: What happens to the staging files when the process exits and restarts? Will the staged files be moved over or ?

Ok, I see it now, it does look like there is explicit code to cover this case and recover the "orphaned" files. Killer! Thanks.

Apologies, I missed the original question! Happy to see you liked the approach :)

@mamazzol

Copy link
Copy Markdown
Contributor Author

Hello @breedx-splk is anything blocking this PR? Apologies for the ping, just checking!

@LikeTheSalad

Copy link
Copy Markdown
Contributor

I think @breedx-splk is OOO for a while. Can anyone else take a look? Or should we wait until @breedx-splk is back? @open-telemetry/java-contrib-approvers

@breedx-splk

Copy link
Copy Markdown
Contributor

I'm back and taking a look now, sorry for the lag. Were all of the copilot comments responded to with code? If so, can you please close those?

Comment thread disk-buffering/README.md
@breedx-splk

Copy link
Copy Markdown
Contributor

Looks good to me. I definitely favor this approach over the pervious timeout-based one. Not yet giving approval, because I want to ensure that the other comments are all addressed (including some one that @LikeTheSalad pointed out).

Thanks for this contribution! 🍻

@mamazzol

mamazzol commented Jul 7, 2026

Copy link
Copy Markdown
Contributor Author

Hi @breedx-splk thanks for the feedback!
I worked on the latest things highlighted and resolved all the comments. Everything is accounted for.
Let me know what you think, happy to do more improvements

@breedx-splk

Copy link
Copy Markdown
Contributor

@LikeTheSalad ready for merge? I'm into it.

Thanks again @mamazzol for the contribution!

@LikeTheSalad

Copy link
Copy Markdown
Contributor

@LikeTheSalad ready for merge? I'm into it.

Thanks again @mamazzol for the contribution!

I'm ready, thanks!

@jaydeluca
jaydeluca added this pull request to the merge queue Jul 8, 2026
Merged via the queue into open-telemetry:main with commit 99913b8 Jul 8, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants