Zip metadata files

[GWMcElfresh]

Rationale

Currently, when saving the metadata to disk, the files aren't gzipped, although I believe this is the intention given the file name here:

DiscvrLabKeyModules/singlecell/resources/chunks/Functions.R

Line 84 in 35255c6

metaFile <- paste0(outputPrefix, '.', datasetIdForFile, '.seurat.meta.txt.gz')

Related Pull Requests

entertainingly, this is likely why the metadata files are probably large here: bimberlabinternal/Rdiscvr#68 but nothing direct.

Changes

single line change here to zip the file, which should automatically clean up the connection as well.

DiscvrLabKeyModules/singlecell/resources/chunks/Functions.R

Line 110 in 35255c6

write.table(metaDf, file = metaFile, quote = T, row.names = F, sep = ',', col.names = T)

[bbimber]

@GWMcElfresh, doesnt it already gzip them? not here, but downstream?

https://prime-seq.ohsu.edu/Labs/Bimber/1947/pipeline-browse.view?returnUrl=%2FLabs%2FBimber%2F1947%2Fpipeline-status-details.view%3FrowId%3D615309&path=sequenceOutputPipeline%2FSequenceOutput_2026-04-09_16-08-50

[bbimber]

it's sorta moot here, but are you sure we dont need to explicitly close the file connection? google suggests yes:

# 1. Create and open the connection
con <- gzfile("data.csv.gz", "w")

# 2. Write data to the connection
write.csv(mtcars, con)

# 3. Explicitly close the connection
close(con)

[GWMcElfresh]

My understanding is that because we're not explicitly opening the connection with conn <- gzfile(...), and instead call it on the fly during the write.table() call, R is smart about it and closes it automatically. Otherwise yeah, we'd need to close it.

[bbimber]

My understanding is that because we're not explicitly opening the connection with conn <- gzfile(...), and instead call it on the fly during the write.table() call, R is smart about it and closes it automatically. Otherwise yeah, we'd need to close it.

OK, I dont really know. It's possible you beat gemini on this one

[GWMcElfresh]

@GWMcElfresh, doesnt it already gzip them? not here, but downstream?

https://prime-seq.ohsu.edu/Labs/Bimber/1947/pipeline-browse.view?returnUrl=%2FLabs%2FBimber%2F1947%2Fpipeline-status-details.view%3FrowId%3D615309&path=sequenceOutputPipeline%2FSequenceOutput_2026-04-09_16-08-50

I don't think it gzips them - it's just named .gz. Or if it does gzip them at some point, the files are at least not gzipped when they're written.

See this:

$du -sh SingleCell.CustomUCell.EC_V4_Spleen_Myeloid.seurat.meta.txt.gz
404M    SingleCell.CustomUCell.EC_V4_Spleen_Myeloid.seurat.meta.txt.gz

$mv SingleCell.CustomUCell.EC_V4_Spleen_Myeloid.seurat.meta.txt.gz file_without_gz_suffix.txt
$gzip file_without_gz_suffix.txt

$du -sh file_without_gz_suffix.txt.gz
115M    file_without_gz_suffix.txt.gz

Further - nano (vim etc gunzip on the fly) on the original file:

nano on the gzipped file:

[bbimber]

OK, you are correct. I will fix that; however, I want to also make something to retroactively gzip existing files.

I googled the write.table(gzipfile()) pattern, and I cannot find anything that says this will automatically handle closing the connection. Do you see something different?

[GWMcElfresh]

This is apparently complicated (i.e. depends on the function used to write) - I'll test with delta-ing showConnections() calls in the morning.

Worst case scenario is explicitly define the connection and close()ing it.

[bbimber]

It's certainly worth knowing if there's something better (seems like switching from base R to tidyr or something would do that); however, this is a base R solution that should work:

793037a

[GWMcElfresh]

thanks! I'll see if I can reproduce that linked Rdiscvr issue whenever the retrospective gzip happens too. I think it was probably auto-detecting the file as text, and the httr parsing issue probably doesn't happen on non-text files.