In gh-38154, we started handling ClosedByInterruptException. The
FileChannel was repaired by recreating it and then the exception was
rethrown. This allowed other threads to use the channel that had been
read by an interrupted thread while allowing that interruption to
continue.
This approach has proven to be insufficient as there are scenarios
where the read needs to succeed on the interrupted thread. This
commit updates the handling of ClosedByInterruptException so that
this is the case. The FileChannel is recreated as before but the
thread's interrupted flag is now cleared before retrying the read.
The flag is then reinstated so that any subsequent actions that
should fail due to the interruption will do so.
We could clear and reinstate the interrupted flag before the first
read, rather than catching ClosedByInterruptException. This approach
was rejected as it will have an impact on the performance of the
happy path where the thread hasn't been interrupted.
Fixes gh-38611
Relax the constraint that a `NestedLocation` must have a nested entry
name specified so that URLs can be split and rebuilt.
Prior to this commit, given a URL of the following form:
jar:nested:/myjar.jar!/nested.jar!/my/file
It was possible to create a FileSystem from
"jar:nested:/myjar.jar!/nested.jar" and from that create a path to
"my/file".
However, it wasn't possible to create a FileSystem from
"jar:nested:/myjar.jar", then create another file system from the path
"nested.jar" and then finally create a path to "/nested.jar".
This was because `nested:/myjar.jar` was not considered a value URL
because it didn't include a nested entry name.
Projects such as `JobRunr` were relying on the ability to compose file
systems, so it makes sense to remove our somewhat artificial
restriction.
Fixes gh-38592
Update `NestedByteChannel.read` so that it loops until all
remaining data has been read into the buffer. Prior to this
commit, it was possible for to read only some bytes into the
buffer. Although it looks like this should be OK according to
the API documentation, the `ZipFileSystem` relies on all
remaining bytes being returned.
Fixes gh-38595
Update jar `Handler` code so that the `parseUrl` method can accept an
empty `spec`. Prior to this commit, a `classLoader.getResource("")`
call would result in a `null` result. This breaks a number of things
including `ClassPathResource` and `PathMatchingResourcePatternResolver`.
Fixes gh-38524
Update `JarUrlConnection` and `NestedUrlConnection` so that calls
to `getLastModified()` and `getHeaderFieldDate("last-modified", 0)`
always return a result.
Fixes gh-38204
When an interrupted that calls FileChannel.read, the channel is
closed and the read fails with a ClosedByInterruptException. The
closure of the channel makes it unusable by other threads. To
allow other threads to read from the data block, this commit
recreates the FileChannel when a read fails on an interrupted
thread with a ClosedByInterruptException. The exception is then
rethrown to continue the thread's interruption.
Closes gh-38154
Fix issues with `DataBlockInputStream` including the fact that remain
bytes were not tracked correctly. Also add some tests and fix a few
other unusual details with the implementation.
Closes gh-38066
Update JarUrlConnection so that the full raw zip data is returned from
nested jars when no entry name is specified. This update allows
Tomcat's `WarURLConnection` to work with our nested connections since
they can parse the returned raw zip data.
Fixes gh-38047
The zip specification states that when 'bit 3' of the general purpose
flags is set then a data descriptor record must be present. Prior to
this commit, our `VirtualZipDataBlock` ignored such records and would
create invalid data.
Although the generated data would work for zip parsers that read the
central directory records, it causes problems with streaming reader
implementations such as `JarInputStream`.
This commit updates the code so that it now copies the data descriptor
records. It support both blocks that have a signature and those that
don't. It also updates the generation logic to correctly deal with
any extra data bytes present after the local file header record.
Fixes gh-38063
Add a `NestedFileSystemProvider` implementation so that the JDK's
`ZipFileSystem` can load content from nested jars and nested
directory entries.
Creating a `ZipFileSystem` may be a relatively expensive operation as
zip structures need to be parsed and in the case of directory entries
a virtual datablock nees to be generated on the fly. As such, we
install the `ZipFileSystem` as late as possible since in a typical
application it may never be needed.
This commit also tweaks Gradle and Maven plugins to ensure that the
service loader file is written to repackaged jars.
Closes gh-7161
Update `DefaultCleanerTracking` and `@AssertFileChannelDataBlocksClosed`
to capture and store the source object if it is a `Cleanable` so that
it can be released later.
Although the real cleaner cannot keep a reference to `obj`, it is safe
for us to do so in tests since we are in control of the object lifecycle
and we don't need it to be garbage collected.
This commit also updates the `UrlJarFile` to call the cleaner so that
it can be tracked.
See gh-37668
Rewrite nested jar code to better align with the implementations
provided in Java 17. This update makes two fundamental changes to
the previous implementation:
- Resource cleanup is now handled using the `java.lang.ref.Cleaner`
- Jar URLs now use the form `jar:nested:/my.jar/!nested.jar!/entry`
Unlike the previous `jar🫙/my,jar!/nested.jar!/entry` URL format,
the new format is compatible with Java's default Jar URL handler.
Specifically, it now only uses a single `jar:` prefix and it no longer
includes multiple `!/` separators.
In addition to the changes above, many of the ancillary classes have
also been refactored and updated to create cleaner APIs.
Closes gh-37668
Update `JarFile` and related classes so that `close()` is not longer
called early.
Prior to this commit, we would always immediately close the underlying
jar file to prevent file locking issues with our build. This causes
issues on certain JVMs when they attempt to verify a signed jar.
The file lock issues have now been solved by returning a custom input
stream from `JarUrlConnection` which captures and delegates the close
method.
Fixes gh-29356
Update the Maven and Gradle packaging for war files so that a
`classpath.idx` file is written into the archive that provides the
original order of the classpath, as was previously done for jar files.
The `WarLauncher` class will use this file when running as an exploded
archive to ensure that the classpath order is the same as when running
from the far war.
Fixes gh-19875
Previously, a Zip64 jar file was identified by the number of entries
in the central directory being 0xFFFF. This value indicates that
there the number of entries is too big for the 2-byte field. However,
a jar may be in Zip64 format due to it exceeding the Zip format's
maximum size rather than its maximum number of entries so this field
cannot be used as a reliable indicator. The Zip specification doesn't
require any of the fields of the end of central directory record to
have a value of 0xFFFF (2-byte fields) or 0xFFFFFFFF (4-byte fields)
when using Zip64 format so we need to take a different approach.
Additionally, a number of places in the code assumed that an entry's
offset would always be available from the central directory file
header directly. This assumption did not hold true when the jar was
a Zip64 archive due to its size as the offset's value would be
0xFFFFFFF indicating that it should be read from the Zip64 extended
information field within the header's extra field instead.
This commit updates the Zip64 detection to look for the Zip64 end of
central directory locator instead. If present, it begins 20 bytes
before the beginning of the end of central directory record. Its
first four bytes are always 0x07064b50. The code that reads the
local header offset has also been updated to refer to the Zip64
extended information field when the offset is too large to fit in
the 4-byte field in the central directory file header. To allow
greater-than-4-byte offsets to be handled, a number of fields,
method parameters, and local variables have had their type changed
from an int to a long.
Fixes gh-27822
Update the jar `Handler` class to support a non-reflective fallback
mechanism when possible. The updated code attempts to capture a regular
jar URL before our handler is installed. It can then use that URL as
context when creating the a fallback URL. The JDK jar `Handler` will
be copied from the context URL to the fallback URL.
Without this commit, resolving new Tomcat URLs of the form
`jar:war:file:...` would result in an ugly "Illegal reflective access"
warning.
Fixes gh-18631
Update the performance improvements to push certificate loading
and storage into the `JarFileEntries` class. This allows us to
keep certificates without needing to cache all entry data. We
now also keep certificates and code signers in a dedicated class
which is set whenever the full jar stream as been read, even if
the contained values are `null`. The logic that assumes META-INF
entries are not signed has been removed in favor of delegating to
the streamed entry results.
See gh-19041
Update Spring Boot nested JarFile support to improve the performance of
signed jars. Prior to this commit, `certificates` and `codeSigners`
were read by streaming the entire jar whenever the existing values
were `null`. Unfortunately, the contract for `getCertificates` and
get `getCodeSigners` states that `null` is a valid return value. This
meant that full jar streaming would occur whenever either method was
called on an entry that had no result. The problem was further
exacerbated by the fact that entries might not be cached.
See gh-19041
Previously, PropertiesLauncher would close each archive that it
iterated over when creating its ClassLoader. This was not aligned
with JarLauncher's behaviour and left the ClassLoader with closed
archives. The close was introduced in [1] and became more apparent
following the change to fail operations on closed archives [2].
This commit updates Launcher to remove the close() that was added in
[1]. This aligns the behavior of PropertiesLauncher with JarLauncher
and ensures that the ClassLoader does not have entries backed by
closed archives on its classpath.
Fixes gh-23165
[1] ad72f86bdb
[2] ed7a5db174
Moritz Halbritter
Yanming Zhou
Andy Wilkinson
Phillip Webb
Phillip Webb
Andy Wilkinson
Moritz Halbritter
Krzysztof Krason
Scott Frederick
dreis2211
Stephane Nicoll
weixsun
Stephane Nicoll
mathieufortin01
Andy Wilkinson