User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322) Build Identifier: Problem uploading Firefox extension. Will not allow over 3.7 megs to upload the software. I found this problem 4 months ago and it had to do with the php script for the upload. Seems it has not been addressed. I found that you can issue bugs so here it is. Reproducible: Always

Any progress on this?

Mass change - bugs to be read / (re)confirmed.

Seems a reasonable request to me

We will solve this in this next version of AMO by allowing users to upload from a URL.

The max filesize when uploading is 32M. I think this has more to do with running out of memory parsing the add-ons. Still an issue as of today though with a 9.2M file.

continuing from Bug 545374: I made some other tests: 1) an upload of a very simple xpi (25k) filled with bitmap images (for a total of more than 5Mb of xpi) successfully passed the test this would contrast with previous findings and tell me that it does not depend on size of the whole xpi but on problematic content of the java jar bundle 2) my extension xpi, with the big 1.5Mb java bundle with extension name changed to xcd (unknown extension): "NO RESPONSE" PROBLEM, mmmm.. weird, so, either the parser recognizes that it is a java jar by its content (i.e. not relying on the extension) and parsing it caused the error or...well, I do not know. 3) the very simple xpi of above, stuffed with: a) one copy of the problematic bundle: OK b) two copies of the problematic bundle: OK c) three copies of the problematic bundle: SAME "NO RESPONSE" PROBLEM as for 2) So, it could definitively be an out-of-memory problem, because it seems to not depend on rigid size of the xpi (test 1) nor is the "problematic" bundle deterministically causing problems to any xpi which contains it (both tests 3.a and 3.b passed) though it "contributes" to let the test fail (see test 3.c failing, while test 1 passes with a larger in size xpi). However, Jorge Villalobos told that, from the third of February 2010, .class files are ignored, and the bundle is a java bundle so it should not cause any problem at all...strange. I can only add the information that it is not a simple jar lib, but an OSGi bunlde, so it is a jar containing just few three .class files and then two other jars...may it be of any help? I think that these last tests are the best that I can do from here (user's side), then the sole thing which can be done for this is to see the logs of the upload procedure. Could I send you the xpi (or even the sole jar bundle) and let you test? Let me know if I can provide any further information or help in some way. P.S: in the meanwhile, if the bug is not fixed, could I get the xpi uploaded to AMO in some other way? :-)

Please attach the file here and provide clear steps to reproduce the problem.

The file is more than 2b so I cannot attach it here. Anyway, here's the link to it. http://semanticturkey.googlecode.com/files/SemanticTurkey-0.7.0-2010-02-8_b81.xpi and here are steps to reproduce the problem: Reproducible: Always Steps to Reproduce: 1. build an extension of 3Mb (fill an available extension with dumb files) 2. try to upload it (can be reproduced both by submitting an update, by submittig a new extension or by testing it) 3. if you fail to reproduce this, we can send you our specific extension Actual Results: Nothing happens on the upload site after the file has been uploaded Expected Results: I tested with smaller-than-3Mb files, and saw the result of the upload (reported error if some problems are present in the install.rdf, or continue with the upload process if everything is fine).

(In reply to comment #11) > > http://semanticturkey.googlecode.com/files/SemanticTurkey-0.7.0-2010-02-8_b81.xpi > Got it. So, a few things here (none of which are related to this bug, sigh). This extension fails to be added to AMO because the install.rdf has an updateURL, updateKey, and 3.5.0 isn't a valid minimum version. With those fixed, I can upload it and hit the validator. The validator tries to return a few strings it finds, and in doing so iconv() chokes on U+FFFD. It's just a warning, but appears to break badly enough that there is no output from that PHP thread. The patch I'm attaching fixes this problem, and once the patch is applied I can successfully upload/validate this add-on with half the memory production has. The add-on has 3296 warnings, but that's another issue. Sooo...wenzel. This patch is at the heart of AMO and I'm nervous about applying it and I'm also not sure why we don't run into it more often. It looks like the ord() check right above it is a similar situation and we could easily add a substitution there for this char which is chr(239). Since I have this patch written, I'll r? you, but I'd be fine doing the chr(239) substitute too. (I'm leaning towards the substitution since it's late in the game for remora, but I'm also sleepy and might not be making sound decisions. :)

thanks a lot Wil, I did not pay much attention to install.rdf for doing the tests because it was just to get the server return something, anyway I corrected it but sent you the wrong version, I was sleepy too :-) P.S: By the way, updateURL, updateKey etc... are always attached to the RDF by an automatic tool which i use: http://cms.xulbooster.org/ Since they are empty, they should be ignored by the parser so I would post it as an improvement request here rather than a bug in xulbooster. Do you agree on that?

Comment on attachment 426459 [details] [diff] [review] UTF-8 That looks good, though I am slightly unhappy that iconv already has IGNORE in its settings and still throws "fatal warnings". :(

Yeah, wth. Anyway, this is in r62152.

I CC'd QA so they can keep an eye out for anything strange with string output on the site, although I expect it to be fine. We ran off topic on this, but the original bug might have been hitting that problem and not a size limit. I successfully uploaded the 4.5M with no troubles. Are there other add-ons that are still hitting a limit?

None that I know of. It's very rare for add-ons to be this big.

Alright, let's call this fixed.

sorry, maybe I did not catch the ending. Has the patch be applied? because comment 16 tells that the 4.5Mb file can now be uploaded without problems, while I tried to upload it now and I find the same situation as before. (I tried both new version with fixed install.rdf as well as the original one the link of which I posted here in comment 11)

It's been applied on preview.addons.mozilla.org. It should be on the live site in a couple of weeks.

(In reply to comment #20) > It's been applied on preview.addons.mozilla.org. It should be on the live site > in a couple of weeks. one motnh has passed (so live site should have been updated with the committed changes on preview), but it seems the bug is still here. Though Wil's patch seems to have worked: in fact, from the preview site, I'm able to upload a new version, but not from the live site. With the live site, I tried: https://addons.mozilla.org/it/developers/versions/add/8880 and got no answer, the file is updated, I wait...then nothing, as on the bug report. I then tried: https://preview.addons.mozilla.org/it/developers/versions/add/8880 and, yes, it goes right through the validation process. So, again, the preview site works (and started to work after you applied to patch), while the live one does not. It seems the patch has not been applied. If you want to replicate the problem, I can give you the last version of the xpi: http://semanticturkey.googlecode.com/files/SemanticTurkey-0.7.1-2010-03-10_b87.xpi just tried 10 minutes ago and I can confirm the problem is still there P.S. from a personal mail, Wil asked me to reopen the bug if i still get problems; sorry it seems I'm not able to do it (is reopening allowed to users?) and I would avoid opening a new one since the problem is the same.

Reopening.

I can confirm that the fix is in production and this still isn't working there but is on preview.

I have the same error today (20100409) when I try to upload a new version of the openinfocard addon. The xpi file can be found here: http://openinfocard.googlecode.com/files/xmldap-0.9.9.201004090914.xpi

The author of the Life Science Dictionary Tool has reported us the same issue. https://addons.mozilla.org/firefox/addon/12157 It contains a large dictionary SQLite DB (53.9 MB), and the compressed XPI size is 16 MB. Here's a copy of the latest version of the add-on he tried to upload: http://getfirefox.jp/temp/065742f2f68c52680fe8a6043f7e5d22/lifedictoolbar_2010042800_release.xpi Wil, any ETA for this?

This will happen with offline processing which is getting spec'd out now. ETA is Q3/Q4.

OK. So, can we manually upload such large files via the amo-editors list or Bugzilla?

I don't have a way right now

We used the following escamotage. A fake extension is hosted on our FF site https://addons.mozilla.org/it/firefox/addon/8880 The extension than asks users to download the real xpi from our site. It also gives a link to this bugzilla page to assure users that there is currently a bug in AMO preventing us to upload the XPI and that the new download is not a malware. The other solution would have been to host the main xpi ourself, but I prefer to keep the original download counter and statistics active, which would have been deleted in case of a move to self-hosted extension. A bit unconfortable, but at the end is the best solution (given the above constraints) I found.

Thanks for the guidance. I'll tell the author there are some workarounds.

The official workaround from a few years ago is here: https://wiki.mozilla.org/AMO:Admins/Large_Files It should still work.

FYI, we don't allow add-ons to download or install code or other add-ons. The workaround in comment 29 will most likely result in the add-on being disabled.

Can you give us a link to the latest version of this?

Hi, do you mean our extension? The latest official release is at: http://code.google.com/p/semanticturkey/downloads/detail?name=SemanticTurkey-0.7.1r.xpi&can=2&q=

Thanks, I was looking for a large add-on to try with the new tools. Semantic Turkey is 4.5M and works fine (although fails validation due to an element in the install.rdf). The life science dictionary from comment 25 validates in about 5-8 seconds after uploading and passes with a mere 4 warnings. So...things are looking up. You can expect the new dev tools to land on Thursday.

Hi, We've also a new release (still in development, but it is working and anyway, it's just to test the upload) which is bigger (mostly due to new java bundles, which however seemed to be the problem with the upload service). If you want to give it a try, I uploaded it on: http://semanticturkey.googlecode.com/files/SemanticTurkey-0.7.2-SNAPSHOT-2011-01-12_b120.1.xpi

Yep, that one works fine too. thanks.

I haven't tested the new dev tools with large files, but marking as fixed per Wil's comments. Good job!

Hi again, tried today to upload a new version of my extension: https://addons.mozilla.org/addon/semantic-turkey in this case, I upload the new version, it gets checked with no errors and a few warnings (mostly due to the presence of java classes, telling that .class is not an accepted extension, though my extension is based on the java-firefox bridge and contains classes bundled inside jars). Unfortunately, the "add version" button (sorry not sure about the exact label of the button, my one is in italian) is disabled: if I click on it nothing happens, while the "cancel" link is working correctly. The link to my extension is: http://semanticturkey.googlecode.com/files/SemanticTurkey-0.7.2-2011-01-20_b128.3.xpi

sorry for the posting on the duplicate bug, just posted comment on the wrong window (I was looking at the duplicate bug, where I started to report my problems, and then by mistake posted there). However, I just now cross-checked that I can upload extensions (tried with a smaller one), though my one did fail ("add version" button not working), as I reported on comment 39 in *this* bug.

That sounds like a different bug. Can you make sure you're giving the JS time to load and making sure you have updated JS (hard refresh the page). We're pushing an update tonight that might help but I think it's unrelated. If you still have troubles please file a separate bug, as this no longer pertains to file size. Thanks.