7 Replies Latest reply: Apr 5, 2008 8:40 PM by 807591 RSS

    Bug in java.util.zip?

    807591
      Hi, I have some code that uses java.util.zip to put "MyFile" into "My.zip" file (using ZipOutputStream, etc). When I use java.util.zip to reopen the zip file, I run into a problem.

      I open a ZipInputStream and call getNextEntry(), and get a ZipEntry for "MyFile" file I zipped. I then call ZipEntry.getSize() but it returns -1 (size unknown). But if I create a ZipFile object for the zip and call getEntry("MyFile"), when I call ZipEntry.getSize() it returns the correct size.

      I thought it may be reasonable that ZipInputStream does not know the size of a zipped entry in a file (until it had read through/unpacked the object). However if I create the "My.zip" file with either Winzip or Winrar, when I open it with ZipInputStream, call getNextEntry() then ZipEntry.getSize(), it DOES return the correct size.

      So why can I get the size of a ZipEntry (via ZipInputStream) from a zip created with Winzip, but not a zip created with java.util.zip?

      *******

      I tried using the ZipFile class (which gives me the entry size correctly) together with ZipInputStream to read "MyFile", so this seemed like a workaround. I have used this technique for one part of the project.
      However, the above example is a simplification of what I'm actually trying to do in another part of the project. I am unzipping an entire jar file (11MB), replacing 3 class files and zipping it up again (a type of incremental patch/upgrade system). After the jar is recreated, it no longer works (an 'unable to load main class' exception occurs). If I open the jar with winzip, the class is there, has the right size and package path, the manifest is there, etc.
      If I open the rebuilt jar with my sample program, using ZipInputStream, getNextEntry() then ZipEntry.getSize(), it returns -1 (as expected). I'm wondering - is this what the JVM/Classloader is attempting to do also, which is why it's failing to find (or load) the main class??
      As a test, I took the rebuilt jar, extracted all with winzip then built it all into a new zip (and renamed to a jar), then it once again worked. Also to note - I've tried changing the java.util.zip api calls to JarEntry, JarOutputStream, etc, but this has the same problem.

      So can anyone help? Is the java.util.zip api not producing zips quite right? (Winzip seems to work better.) Or did I miss something?
      I have a couple of small test cases if anyone is interested - I can post here if anyone wants them.

      Thanks in advance,
      Phil
        • 1. Re: Bug in java.util.zip?
          807591
          The standard format for a zip file requires that the length of an entry be on a header preceding the entry itself. When you use a tool like ZipOutputStream which writes to a stream it can't know that length when writing the header and, because it's a stream, can't random write to fill in the length after the entry has been compressed.

          To allow for this, there's an option in the zip file specification which allows you to deffer the length fields of an entry to a footer, setting the lengths in the header to a -1 null value. Most zip utilities put the lengths in the header (and, indeed, I've had versions of WInZip which wouldn't read the files produced by ZipOutputStream for this reason).
          • 2. Re: Bug in java.util.zip?
            807591
            ... but when I construct my zip file and use putNextEntry(), for the ZipEntry I use not only setName() but also setSize(), setTime(), setMethod(). And this is before the data is written. Is this not the correct way to do it?
            • 3. Re: Bug in java.util.zip?
              807591
              TimeLord2004 wrote:
              ... but when I construct my zip file and use putNextEntry(), for the ZipEntry I use not only setName() but also setSize(), setTime(), setMethod(). And this is before the data is written. Is this not the correct way to do it?
              There is no "correct way" in the way you mean, ZipOutputStream simply can't determine the entry size (especially the compresed size) early enough to put the data into a header. So it puts it in a footer instead. The data is also in the directory at the end of the zip file, of course. It can't make use of the entry size you supply since the null entry size tells the reader that the data is in a footer instead.

              In fact I wound up writing my own zip file writer class (which is limited to actual files) to get arround the limitations of earlier versions of WinZip by doing random writes to fill in the header feilds after the entry was written. Later versions of WinZip seem to cope.
              • 4. Re: Bug in java.util.zip?
                807591
                Thanks malcolmmc for the comments but I'd just like to clarify some details incase there's some confusion.

                I've only mentioned Winzip to indicate that it appears to create zips 'correctly'... the main problem I have is with creating a jar with java.util.jar.JarOutputStream or java.util.zip.ZipOutputStream. A jar created with these api's does not work correctly.

                Phil
                • 5. Re: Bug in java.util.zip?
                  807591
                  TimeLord2004 wrote:
                  Thanks malcolmmc for the comments but I'd just like to clarify some details incase there's some confusion.

                  I've only mentioned Winzip to indicate that it appears to create zips 'correctly'... the main problem I have is with creating a jar with java.util.jar.JarOutputStream or java.util.zip.ZipOutputStream. A jar created with these api's does not work correctly.
                  It sounds like this could actually be a bug. The ZipOutputStream is correct, according to the official zip file specification, but maybe the classloader assumes the footer-free format. OTOH I'd expect the classloader to use the directory, which is the same as you've noted when using ZipFile or JarFile classes. The problem would normally apear only when reading the archive sequentially.

                  Try Sun's bug database.
                  • 6. Re: Bug in java.util.zip?
                    791266
                    TimeLord2004 wrote:
                    Thanks malcolmmc for the comments but I'd just like to clarify some details incase there's some confusion.

                    I've only mentioned Winzip to indicate that it appears to create zips 'correctly'... the main problem I have is with creating a jar with java.util.jar.JarOutputStream or java.util.zip.ZipOutputStream. A jar created with these api's does not work correctly.

                    Phil
                    I can't remember now what I did how I did it, but I used JarOutputStream a couple of years ago to produce jar files at runtime and I could use those jars on the classpath after I had saved them to the file system. Those jars did however not contain a manifest file (since I didn't need that)
                    • 7. Re: Bug in java.util.zip?
                      807591
                      Hi, I appear to have figured the problem out, after playing with a test program for a while. I created a small 'Hello' test program and then a second one to use JarOurputStream to create an executable jar (with Main-Class: Hello), using the Manifest class.

                      I found:
                      a) The jar initially worked correctly, but if I extracted the contents and rezipped (with winzip), it no longer worked.
                      b) If I took the working jar and removed the manifest then reinserted it (with winzip), it no longer worked.
                      c) If I took the working jar and removed the Hello.class then reinserted it (with winzip), it still worked.
                      d) If I put 'Hello' into a package (eg 'com') then created the jar with my program, it did not work (main class not found).

                      The answer? I'm doing all this in Windows. If you remove the meta-inf/Manifest.mf file then re-add (all with winzip) then it must be being stored as meta-inf\Manifest.mf - and that's right, that means the Manifest no longer works (or rather, is no longer found). A big thing to note though - winzip always shows it (before and after) as meta-inf\Manifest.mf ... so it is actually possible to have both meta-inf\Manifest.mf and meta-inf/Manifest.mf in a single zip file, but winzip displays them both as meta-inf\Manifest.mf !! Try and delete one and they both get removed (although this may be fixed in a newer version of winzip).
                      Similarly, the problem with 'Hello.class' - once in a package, the path was com/Hello, but I was adding it as com\Hello .... enough to stop this from working.

                      The thing was, I did not make a conscious decision to use \ in place of / or anything like that. I have a generic method which takes all files in a directory (and sub-dirs) and create a zip (jar) from them. Each ZipEntry is created according to its relative path on disk, hence I was creating a ZipEntry as com\mycompany\Abc.class. To fix, before creating the ZipEntry:
                      entryName = entryName.replaceAll("\\"+System.getProperty("file.separator"), "/");
                      I dunno, maybe this whole problem would be obvious to some ppl out there, but thought it worth sharing incase anyone was curious, or has had similar trouble. So seems the issue with getSize() returning -1 is maybe a symptom of this problem but a bit of a red herring.
                      I spent many many hours on this! (I did encounter this problem with repackaging jars once before, some time ago, but never got to the bottom of it and moved onto other issues.) I don't think the solution above was something that anyone suggested (therefore worth sharing), and I cannot see mention of it in the sparse javadocs. Imho, worth being in the javadocs!

                      Phil