Closed Bug 140301 Opened 23 years ago Closed 22 years ago

tgz files get silently unzipped

Categories

(Core Graveyard :: File Handling, defect)

Product:
Core Graveyard
Component:
File Handling
Platform:
x86
Windows 2000
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
VERIFIED FIXED

People

(Reporter: jamesrome, Assigned: law)

References

(
URL
)

Details

Attachments

(3 files)

nsHttp:5 log from James (gzipped)
(deleted), application/x-gzip
Details
log file asked for by zbarsky
(deleted), text/plain
Details
Patch to add "tgz" to nonDecodableExtensions[] in nsExternalHelperAppService.cpp
(deleted), patch
Biesinger
: review+
bzbarsky
: superreview+
asa
: approval1.4+
Details | Diff | Splinter Review 
When one downloads files with a tgz extension (and I think gz also), they get
silently unzipped when they are stored (even if one right-clicks and does "save
as"). But the extension is not fixed up to reflect this. So, when one tries to
open the files with, say, WinZip, it says the file is invalid.

gz and tgz files that are being saved (and not displayed) should not be unzipped
without 1) informing the user and 2) changing the extension to reflect it.
Which build are you using?  This should be fixed for tgz files being sent as
application/tar or application/x-tar and content-encoding: gzip (which is what
this site does)...  More to the point, the files are _not_ unzipped for me in
Linux trunk build 2002-04-25-07.
Assignee: blaker → law
Component: Download Manager → File Handling
I have the 4/16 build.
It does work on today's build. But it appends .tar to the file name instead of
replacing the .tgz. I would still like the choice of being able to keep the file
in its compressed form.
That's exactly what should be happening (and what I see) -- the file getting
saved as a .tgz file, compressed.

Are you saving with shift-click or by clicking on the link?  Are you spoofing
your useragent?  Could you set NSPR_LOG_MODULES to nsHttp:5, set NSPR_LOG_FILE
to some filename, and do a minimal run: start up mozilla, point it to that page,
save the file, exit.  Then attach the log file to this bug.
Attached file nsHttp:5 log from James (gzipped) (deleted) — 

  

  



    
    

    
  
That log shows what happens in http-land when doing right-click-and-save-link. 
In the process, the file gets a ".gz" extension in the filepicker and then gets
saved  unzipped.  (The second case, when just clicking on the file, the file
gets saved with a .tar extension, still unzipped).

Here are the relevant parts of the log, with my annotations:

We ask the server what sort of data this is:

0[234400]: http request [
0[234400]:   HEAD
http://www.laurentconstantin.com/common/lcrzo/download/v4/lcrzo-4.08-src.tgz
HTTP/1.1
0[234400]:   Host: www.laurentconstantin.com
0[234400]:   User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US;
rv:0.9.9+) Gecko/20020426
0[234400]:   Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/j
peg,image/gif;q=0.2,text/css,*/*;q=0.1
0[234400]:   Accept-Language: en-us, en;q=0.50
0[234400]:   Accept-Encoding: gzip, deflate, compress;q=0.9
0[234400]:   Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66
0[234400]:   Keep-Alive: 300
0[234400]:   Proxy-Connection: keep-alive
0[234400]:   Pragma: no-cache
0[234400]:   Cache-Control: no-cache
0[234400]: ]

It responds that it's application/x-tar, encoding gzip:

2740[114fe50]: http response [
2740[114fe50]:   HTTP/1.0 200 OK
2740[114fe50]:   X-Cache: MISS from cache.nxs.net
2740[114fe50]:   Content-Type: application/x-tar
2740[114fe50]:   Content-Encoding: gzip
2740[114fe50]:   Accept-Ranges: bytes
2740[114fe50]:   Proxy-Connection: close
2740[114fe50]:   Content-Length: 660205
2740[114fe50]:   Server: Apache
2740[114fe50]:   Last-Modified: Mon, 08 Apr 2002 05:28:53 GMT
2740[114fe50]:   Etag: "2c32f-a12ed-3cb12a95"
2740[114fe50]:   Date: Fri, 26 Apr 2002 18:37:44 GMT
2740[114fe50]:   Connection: close
2740[114fe50]: ]

We put up the file save dialog and append ".gz" to the filename, since
apparently the system does not have ".tgz" set as a valid extension for
gzipped files.  We also set a "don't decompress this data" flag.
User picks a filename, we contact the server to get the data:

0[234400]: http request [
0[234400]:   GET
http://www.laurentconstantin.com/common/lcrzo/download/v4/lcrzo-4.08-src.tgz
HTTP/1.1
0[234400]:   Host: www.laurentconstantin.com
0[234400]:   User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US;
rv:0.9.9+) Gecko/20020426
0[234400]:   Accept:
text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,video/x-mng,image/png,image/j
peg,image/gif;q=0.2,text/css,*/*;q=0.1
0[234400]:   Accept-Language: en-us, en;q=0.50
0[234400]:   Accept-Encoding: gzip, deflate, compress;q=0.9
0[234400]:   Accept-Charset: ISO-8859-1, utf-8;q=0.66, *;q=0.66
0[234400]:   Keep-Alive: 300
0[234400]:   Proxy-Connection: keep-alive
0[234400]:   Pragma: no-cache
0[234400]:   Cache-Control: no-cache
0[234400]: ]

And it responds:

2740[114fe50]: http response [
2740[114fe50]:   HTTP/1.0 200 OK
2740[114fe50]:   X-Cache: MISS from cache.nxs.net
2740[114fe50]:   Content-Type: application/x-tar
2740[114fe50]:   Accept-Ranges: bytes
2740[114fe50]:   Proxy-Connection: close
2740[114fe50]:   Server: Apache
2740[114fe50]:   Last-Modified: Mon, 08 Apr 2002 05:28:53 GMT
2740[114fe50]:   Etag: "2c32f-a12ed-3cb12a95"
2740[114fe50]:   Date: Fri, 26 Apr 2002 18:37:59 GMT
2740[114fe50]:   Connection: close
2740[114fe50]: ]

Notice that the type is still application/x-tar, but there is no
content-length and there is no content-encoding!  So it looks like at
this point the server is sending un-encoded data back...

-------------------------------------

I have asked James to try this also without the proxy he normally uses...

  

  



    
    

    
  
so, in other words, this server is not implementing HEAD correctly :-(

  

  



    
    

    
  
Not quite so simple... On Linux the GET has a Content-Encoding and all that.  So
this is either Windows-specific, or a proxy problem, or _something_ weird.... No
sane server should actually _decompress_ .tgz files and serve them unzipped....

  

  



    
    

    
  
More comments from James:

With the proxy off, I get the same 2 file names, but the tgz.tar version is 
definitely unzipped (4.630 KB) while the .tgz.gz is 645 KB.

--------------------------------------------------

Well, that's definitely correct behavior... it seems like the server is
sometimes sending the data unzipped (and the proxy somehow screws us up
there)...  In that test, did the .gz version come first?  Or the .tar version?

  

  



    
    

    
  
I first did the straight download via clicking the link and then right-clicking
with save as

  

  


QA Contact: sairuh → petersen

    
    

    
  
James, is this still a problem?  I'm in the middle of doing stuff to this code,
and it'd be good to know so I can fix it if so.  ;)

  

  



    
    

    
  
The above URL now asks what I want to do. But it offers to use
.tgz.tar for the name rather than the correct I think) .tgz.

I used the .tgz extension and winzip then says it contains a tar file and
offers to expand it.

  

  



    
    

    
  
So this is now basically bug 144334?

  

  



    
    

    
  
It is 144334 for sure. But originally, This bug saved the file unzipped with a
.gz extension.

  

  



    
    

    
  
Right.  I realize that.  :)  I made a bunch of changes to this code designed to
prevent that... sounds like one of them did the trick.  Mind if I mark this
worksforme, since neither of us can reproduce it anymore?

  

  



    
    

    
  
Fine by me!

  

  



    
    

    
  
okee. ;)

  

  


Status: UNCONFIRMED → RESOLVED
Closed: 22 years ago
Resolution: --- → WORKSFORME

    
    

    
  
This has raised its head again (11/5 win2k build). I downloas a .zip file. It
asks me if I want to open it with winzip or save it. I select save. It downloads
it and stores it with a .zip extension. However, the file has already been
unzipped and should have a .exe extension.

  

  


Status: RESOLVED → UNCONFIRMED
Resolution: WORKSFORME → ---

    
    

    
  
Which zip files does that happen for?  URL please?

  

  



    
    

    
  
This is getting worse and worse.(11/19 build, win2k)
I just received e-mail with .png attachments. When I  try to open them, it
claims they are zip files with extensions .png.zip. They are not.and winzip
comfirms this!

Here is the header:
------_=_NextPart_000_01C290E3.DD9D1370
Content-Type: application/octet-stream;
	name="SFO_ITPUT_vis_box_trellis.PNG"
Content-Transfer-Encoding: base64
Content-Disposition: attachment;
	filename="SFO_ITPUT_vis_box_trellis.PNG"

  

  



    
    

    
  
Sounds like you have a mapping from .zip to application/octet-stream in your
helper app preferences.  Which you should really not do unless you know exactly
what you're doing.

How about answering my questions from comment 19?

  

  



    
    

    
  
http://download.com.com/3000-2141-10154259.html
Click the top-left download item. 
I put this up one time before. I have no idea where it went.

You were right about the helper app. I did not put this in. But there was just a
new version of winzip. Maybe it did it!

  

  



    
    

    
  
Unlikely, since winzip does not edit the mozilla prefs.

Did you ever have a .zip file that was served as application/octet-stream and
you chose to open in winzip and said remember this decision?  That would have
done it... (yes there is a bug on the fact that the UI makes this so easy to do).

  

  



    
    

    
  
Not that I recall. Does the dialog tell you what mime type it is serving?

  

  



    
    

    
  
Yes, it does.  In square brackets.

  

  


Depends on: 86640

    
    

    
  
I suspect the problem that caused this bug can never happen again now that bug
86640 is fixed.

  

  



    
    

    
  
This is broken again. (build 2003020208 win2k)
Clicking on a link such as file.tar.gz now saves the file as file.tar.gz.tar.
And the download manager always shows the file as being 1 kb of 1 kb, even
though it was a lot bigger as the file was transferred.
For an example, click the top tar link on
http://www.cs.dartmouth.edu/~beej/bgp/java/

The action on .exe files is even worse. They get a .jpg extension and get opened
in my photo application. See bug 189552

  

  


Status: UNCONFIRMED → NEW
Ever confirmed: true

    
    

    
  
I know what it _NAMES_ it.  But is it actually gunzipped?  Or just named
idiotically?

  

  



    
    

    
  
they both have the same size as the right-clicked save-as version, so the name
is probably wrong. Of course, it is hard to tell what is in the file on a PC
without trying it.
I did. it appears to have been unzipped in both versions.
file.tar.gz (right-clicked save as)
file.tar.gz.tar (direct save)

It should nbot unzip them without my telling it to.

  

  



    
    

    
  
Hmm... how big are your saved versions of
http://www.cs.dartmouth.edu/~beej/bgp/java/BGP4_1.4.15.tar.gz (the "tar" link
off http://www.cs.dartmouth.edu/~beej/bgp/java/)?

(The reason I'm asking all these questions is that I cannot reproduce the
problem on Linux, and the same code runs on both platforms....)

  

  



    
    

    
  
1380kb. Download manager says 1 kb

  

  



    
    

    
  
Hmmm... That _is_ unzipped, then.  When I download here I get 293kb.  Not good.  ;)

What is your "network.http.accept-encoding" preference set to?  (Use
about:config to see.)


  

  


Keywords: helpwanted

    
    

    
  
how do I access about:config? It iusn't in the help menu or edit preferences
that I can see....

  

  



    
    

    
  
You type "about:config" in the URL bar, then hit enter.

  

  



    
    

    
  
On two different win2k machines, it says
default     string    gzip,deflate,compress,q=0.9

It says that on my linux machine also

  

  



    
    

    
  
Ok, that's what I have too...

See, here's the problem.  That site sends the file as application/x-tar,
encoding gzip.  We have code to explicitly check for application/x-tar and _NOT_
decompress it.  In my testing, this code is called properly...  it sounds like
it's not being called in your build.  Which is odd, since there is no codepath
that allows saving without calling that code.

Do you have any extensions (multizilla or the like) installed?  Or a plain
vanilla Mozilla build from Mozilla.org?

  

  



    
    

    
  
I have the tool bar and spell checker

  

  



    
    

    
  
Which tool bar? And more to the point, does the problem appear _without_ the
tool bar?

  

  



    
    

    
  
http://www.xulplanet.com/downloads/prefbar/
Not sure how to remove it.

Interesting. On my laptop it worked (which doesn't have the prefbar). There it
popped up a box saying application/x-tar one time (but not again). The action is
identical on my desktop and laptop for the prefs zip-file, save to disk).

  

  



    
    

    
  
OK... that would not affect the direct-click version of saving.

What else is different between the laptop and desktop?


  

  



    
    

    
  
Hard to say since my desktop has 400 GB of disks filled with stuff!

And the download manager is also saying all files are 1 Kb except for .exe files.



  

  



    
    

    
  
Let's focus on the problem at hand, ok?  The UI behavior is not relevant; what's
relevant is the unzipping.

Does this happen if you create a new Mozilla profile?  I'm assuming that this
was a clean Mozilla install following the release notes, bot just unzipping on
top of a previous build?

  

  



    
    

    
  
1) I did a clean install last time because I was unable to do anything. So I
wiped the directory after an uninstall and then did a reinstall.

2) Using a new profile worked! 

  

  



    
    

    
  
Ok.... could you possibly mail me your mimeTypes.rdf and prefs.js files?

  

  



    
    

    
  
James, does the account having the problem include a helper app mapping for
application/x-tar files?  What about the one that is _not_ having the problem?

  

  



    
    

    
  
WinZip is registered for application/x-tar.

When I go to this site, and when I do the download, the saved file is listed as
lczroex-4.17.0-src.tgz.tar. However, the helper application settings for this
have save to disk checked, NOT "open it with the default application."



  

  



    
    

    
  
James, which profile is that in?  The "broken" one or the "non-broken" one?

  

  



    
    

    
  
The only one I use, which is I guess broken in the sense that it does not
properly fix up the file names so that you know what is happening.

  

  



    
    

    
  
Wait.  Is this bug about files getting UNZIPPED?  Or about them being MISNAMED?
 See comment 42 again.

  

  



    
    

    
  
They are getting unzipped and misnamed!

  

  



    
    

    
  
OK.  Could you create a new profile as per comment 43?  If that does not show
the problem (again per comment 43), could you add an entry for application/x-tar
to that profile that is identical to the one in the "broken" profile?  Then test
with that new profile again?

  

  



    
    

    
  
With a profile with NO helper apps defined, I get the identical behavior. The
file gets automatically unzipped and saved with the name
lcrzoex-4.17.0-src.tgz.tar. 

  

  



    
    

    
  
That's not the behavior you described in comment 43....

Are you using a proxy?  Are the steps to reproduce still the same as in comment 0?

  

  



    
    

    
  
No proxy. But I have a much more recent Mozilla build 2003041704.
What happens when you try it with the URL?


  

  



    
    

    
  
No  proxy. But I am using a much later build 2003041704. I had been just
clicking on the file link in the last 2 posts..

I went back to the profile with no helper apps and right-clicked and did save
target as. It gave me the original name src.tgz, but the saved file was saved
unzipped. And since the extension is wrong, it is hard to know what form it is
in (It was .tar).

  

  



    
    

    
  
When I save, no matter how I do it, the file is not unzipped.

biesi, can you reproduce this?  I suppose it could be Windows-only....

  

  



    
    

    
  
Ahhh. Yes, it works fine on Linux!
I am installing a more recent build to see if it still does. I will post if it
stops working on Linux.

  

  



    
    

    
  
wfm with win2k build 20040319..


  

  



    
    

    
  
Yup, it works just fine on Linux with the 040105 build.

On windows, I also note that the download manager lists the file size as 1KB of
1KB rather than 329KB of 329KB.

(Actually, there is another bug here because it should be a lowercase k in kB).

  

  



    
    

    
  
Hmmm.... It worked fine on my laptop (win2k). It also has WinZip registered to
handle these files with "Save to Disk" checked.

  

  



    
    

    
  
> On windows, I also note that the download manager lists the file size as 1KB of
> 1KB rather than 329KB of 329KB.

Is the server returning the same headers on Windows and Linux?  See comment 6
again; though there I think it was a proxy bug and you say you are not using a
proxy now...

(And no, "KB" is in fact correct; a "kB" is 1000 bytes, while a "KB" is 1024 bytes.)

  

  



    
    

    
  
win2k build 2003041609: downloading
http://www.laurentconstantin.com/common/lcrzoex/download/v4/lcrzoex-4.17.0-src.tgz
offers a filename of "lcrzoex-4.17.0-src.tgz.tar" but does not uncompress the
file (ie. it is really a tgz file)

  

  



    
    

    
  
I don't remember where to put the commands to make the file you requested.
I put
user_pref("NSPR_LOG_MODULES", "nsHttp:5");
user_pref("NSPR_LOG_FILE", "tgz.log");
in prefs.js, but that did not seem to make a file

  

  



    
    

    
  
You have to do it at the prompt (using "set"):

set NSPR_LOG_MODULES=nsHttp:5
set NSPR_LOG_FILE=c:\log.txt
mozilla

  

  



    
    

    
  

      
      
      
      

        Attached file
          
        log file asked for by zbarsky (deleted)
        — 
      

    


  
This is zipped text

  

  



    
    

    
  
That log shows exactly what the first log shows -- the server responds to the
HEAD with a reasonable response, then responds to the GET with a response that
has no content-length, no content-encoding, and type application/x-tar.

Darin, do you have any idea what's up with that?

  

  


Blocks: 205904

    
    

    
  
I have seen some servers use "Content-Type: x-tar-gz" and "Content-Encoding:
x-gzip" for .tgz files.  For example,
http://heanet.dl.sourceforge.net/sourceforge/rox/archive-1.9.1.tgz gives the
following headers at the moment...

  Date: Mon, 02 Jun 2003 01:34:18 GMT
  Server: Apache/2.0.46 (Unix)
  Last-Modified: Wed, 05 Feb 2003 13:04:21 GMT
  ETag: "75003b3e-425d-6cadf40"
  Accept-Ranges: bytes
  Content-Length: 16989
  Connection: close
  Content-Type: application/x-tar-gz
  Content-Encoding: x-gzip

Mozilla has no clue about "application/x-tar-gz" or the ".tgz" extension, so it
uncompresses the file and saves it with the native .tgz extension, creating a
filesystem lie (a .tgz file that's really uncompressed).

This is arguably a broken server and should be fixed (we've notified the
SourceForge admins), but I believe that ".tgz" should be added to the list of
nonDecodableExtensions in nsExternalHelperAppService.cpp; such a change will
almost certainly break nothing (since we try to avoid uncompressing tarballs in
general) and fix this problem on a number of servers of questionable configuration.

It might also make sense to add "application/x-tar-gz" to the list of
nonDecodableTypes, but I'm not sure I want to give that MIME "type" so much
credibility...

  

  



    
    

    
  
Dan, you're right.  We should just add .tgz to that extension list.  There was
another bug that I recall where that came up...

Could you post a patch?  (I have no tree right now to diff against.)  Don't add
that application/x-tar-gz type to the type list, though...

  

  



    
    

    
  


  
Here's the patch.

  

  



    
    

    
  
Comment on attachment 124709 [details] [diff] [review]
Patch to add "tgz" to nonDecodableExtensions[] in nsExternalHelperAppService.cpp

sr=me.	biesi, would you review (and check in, if you could?)

  

  



        Attachment #124709 -
        Flags: superreview+

        Attachment #124709 -
        Flags: review?(cbiesinger)

        Attachment #124709 -
        Flags: review?(cbiesinger) → review+

    
    

    
  
sure, no problem.

checked in:
Checking in uriloader/exthandler/nsExternalHelperAppService.cpp;
/cvsroot/mozilla/uriloader/exthandler/nsExternalHelperAppService.cpp,v  <-- 
nsExternalHelperAppService.cpp
new revision: 1.187; previous revision: 1.186
done



  

  


Status: NEW → RESOLVED
Closed: 22 years ago → 22 years ago
Keywords: helpwanted
Resolution: --- → FIXED

    
    

    
  
Comment on attachment 124709 [details] [diff] [review]
Patch to add "tgz" to nonDecodableExtensions[] in nsExternalHelperAppService.cpp

a=asa (on behalf of drivers) for checkin to the 1.4 branch.
Please add the "fixed1.4" keyword when this has landed on the branch. Thanks.

  

  



        Attachment #124709 -
        Flags: approval1.4? → approval1.4+

    
    

    
  
fixed on 1.4 branch too:

Checking in uriloader/exthandler/nsExternalHelperAppService.cpp;
/cvsroot/mozilla/uriloader/exthandler/nsExternalHelperAppService.cpp,v  <-- 
nsExternalHelperAppService.cpp
new revision: 1.186.4.1; previous revision: 1.186
done


  

  


Keywords: fixed1.4

    
    

    
  
Verified on win32 (2003-06-06-08) trunk and (2003-06-06-05) branch

  

  


Status: RESOLVED → VERIFIED
Keywords: fixed1.4 → verified1.4

    
    

    
  
*** Bug 205904 has been marked as a duplicate of this bug. ***

  

  


Product: Core → Core Graveyard

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: