fix(downloadFile): write directly to file using Guzzle's "sink" option#70
Open
oleksandr-nc wants to merge 1 commit intomainfrom
Open
fix(downloadFile): write directly to file using Guzzle's "sink" option#70oleksandr-nc wants to merge 1 commit intomainfrom
oleksandr-nc wants to merge 1 commit intomainfrom
Conversation
01c8830 to
a0a5430
Compare
Signed-off-by: Oleksander Piskun <oleksandr2088@icloud.com>
a0a5430 to
d301e69
Compare
Author
|
Unfortunetly, locally I can not reproduce the error from the CI, for me at my local machine this code works.. Edited: Looking at this issue this PR will make app incompataible(unless Nextcloud maybe added support for |
Member
|
mh, looks like we're stuck then, no? either we break sse or we cause OOM. 🤔 |
Member
|
Could we use the sink option to download to a temp file and then stream copy to the encrypted file? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR transitions our file download logic from using Guzzle's stream => true option to the more memory-efficient sink option, mirroring improvements made in other integrations (e.g., OneDrive Integration).
While
stream => trueprevents Guzzle from buffering the entire response body in memory upfront, it returns aPsr\Http\Message\StreamInterface. When we iteratively read from this stream (e.g., withfread()), each chunk is copied into a PHP string variable before being written to disk. This creates temporary data duplication in PHP's userland memory for each chunk.Furthermore, even with
stream => true, data passes through various buffering layers:These, combined with the chunk copied into our PHP variable, contribute to memory pressure during large file downloads, potentially leading to memory_limit issues.
The Guzzle sink option provides a more direct path. Guzzle streams the HTTP response body directly into the provided writable resource (e.g., a file handle) which significantly reduces memory overhead.