When passing an `Entry` type to `File#get_output_stream` the entry is
used to create a `StreamableStream`, which preserves all the info in the
entry, such as timestamp, etc. But then in `put_next_entry` all that is
lost due to the test for `kind_of?(Entry)` which a `StreamableStream` is
not. See #503 for details.
This change tests for `StreamableStream`s in `put_next_entry` and uses
them directly. Some set-up within `Entry` needed to be made more robust
to cope with this, but otherwise it's a low impact change, which does
fix the problem.
The reason this case was being missed before is that the tests weren't
testing `get_output_stream` with an `Entry` object, so I have also added
that test too.
Fixes#503.
Fixed error:
ZipFileTest#test_open_buffer_no_op_does_not_change_file:
Errno::ENOTEMPTY: Directory not empty @ dir_s_rmdir - D:/a/_temp/d20210605-6612-1yi35sp
C:/hostedtoolcache/windows/Ruby/2.4.10/x64/lib/ruby/2.4.0/fileutils.rb:1335:in `rmdir'
C:/hostedtoolcache/windows/Ruby/2.4.10/x64/lib/ruby/2.4.0/fileutils.rb:1335:in `block in remove_dir1'
C:/hostedtoolcache/windows/Ruby/2.4.10/x64/lib/ruby/2.4.0/fileutils.rb:1349:in `platform_support'
C:/hostedtoolcache/windows/Ruby/2.4.10/x64/lib/ruby/2.4.0/fileutils.rb:1334:in `remove_dir1'
C:/hostedtoolcache/windows/Ruby/2.4.10/x64/lib/ruby/2.4.0/fileutils.rb:1327:in `remove'
C:/hostedtoolcache/windows/Ruby/2.4.10/x64/lib/ruby/2.4.0/fileutils.rb:689:in `block in remove_entry'
C:/hostedtoolcache/windows/Ruby/2.4.10/x64/lib/ruby/2.4.0/fileutils.rb:1384:in `ensure in postorder_traverse'
C:/hostedtoolcache/windows/Ruby/2.4.10/x64/lib/ruby/2.4.0/fileutils.rb:1384:in `postorder_traverse'
C:/hostedtoolcache/windows/Ruby/2.4.10/x64/lib/ruby/2.4.0/fileutils.rb:687:in `remove_entry'
C:/hostedtoolcache/windows/Ruby/2.4.10/x64/lib/ruby/2.4.0/tmpdir.rb:101:in `mktmpdir'
D:/a/rubyzip/rubyzip/test/file_test.rb:136:in `test_open_buffer_no_op_does_not_change_file'
Rationale:
File#dup does not behave like what you would expect from #dup on Ruby.
File#dup calls dup(2), which has OS dependant behavoir.
On Windows, calling File#dup seems to cause an extra reference
to an open file, which prevents deleting that file later.
With this commit, we leave out the call to File#dup on Windows.
It is not clear to me that removing this call has no undesired
consequences, but all other existing tests still succeed.
This greatly simplifies the creation of `Entry` objects when only a
couple of fields are not set to their defaults, while at the same time
allowing an `Entry` to be fully configured at creation time if
appropriate.
This fundamentally changes the `Entry` API and means that some
convenience methods in `OutputStream` and `File` have needed to be
refactored.
There was some fairly odd stuff going on in `put_next_entry` that
allowed for data within an `Entry` to be overridden and prevented an
`Entry` from being a single point of truth. Fixing this also simplifies
the code within `File` and still passes all tests.
Also, fixing the above means we can stop passing the compression level
around as a parameter and use the value stored in each `Entry` directly.
Let's keep `compression_level` out of the `Entry` public API though as
it only makes sense when writing an `Entry`: there doesn't seem to be an
obvious way to read what level of compression was used when reading an
`Entry` from a zip file.
Allow an Entry to specify a compression level and pass this down to the
underlying OutputStream infrastructure. OutputStream has been able to
specify a compression level for a while but this has, up until now, only
ever been set to the default.
This fundamentally changes the API so will need a major version bump.
When support for ZipCrypto was added, an internal StringIO buffer was
added to Deflater, in order to fix a decryption bug. While this worked,
it caused unlimited memory growth when compressing large files. The
proper fix is to writer the encryption header in init_next_entry instead
of finalize_current_entry, so the headers are written before any
encrypted data. Because of this fix we can remove the buffering in
Deflater, which keeps memory usage low and allows to stream compressed
data while it is written.
This should fix issue #233.
Adding the io parameter to OutputStream::write_buffer breaks backward
compatibility with v1.1.0. Adding a default value reinstates backward
compatibility.
The local header size computed from the central directory entry
is incorrect due to the Zip64Placeholder in the local entry.
This caused us to seek to the wrong location when copying an
unchanged compressed data stream.
(The same problem could occur when modifying any zip file where
the local header and central directory header contain different
variable-sized fields, so it's a good idea not to trust the CD
to tell us the local header size in any case.)
This commit adds the capability of creating archives larger than
4GB via zip64 extensions. It also fixes bugs reading archives of
this size (specifically, the 64-bit offset of the local file
header was not being read from the central directory entry).
To maximize compatibility, zip64 extensions are used only when
required. Unfortunately, at the time we write a local file header,
we don't know the size of the file and thus whether a Zip64
Extended Information Extra Field will be required. Therefore
this commit writes a 'placeholder' extra field to reserve space
for the zip64 entry, which will be written if necessary when
we update the local entry with the final sizes and CRC. I use
the signature "\x99\x99" for this field, following the example
of DotNetZip which does the same.
This commit also adds a rake task, zip64_full_test, which
fully tests zip64 by actually creating and verifying a 4GB zip
file. Please note, however, that this test requires UnZip
version 6.00 or newer, which may not be supplied by your OS.
This test doesn't run along with the main unit tests because
it takes a few minutes to complete.