Closed
Bug 642395
Opened 14 years ago
Closed 14 years ago
Deal with bogus certs issued by Comodo partner
Categories
(Core :: Security, defect)
Tracking
()
VERIFIED
FIXED
People
(Reporter: dveditz, Assigned: KaiE)
References
Details
(Whiteboard: [sg:high])
Attachments
(10 files, 3 obsolete files)
(deleted),
application/zip
|
Details | |
(deleted),
application/zip
|
Details | |
(deleted),
text/plain
|
Details | |
(deleted),
patch
|
rrelyea
:
review+
briansmith
:
feedback+
bzbarsky
:
feedback+
jst
:
feedback+
beltzner
:
approval2.0+
dveditz
:
approval1.9.2.16+
dveditz
:
approval1.9.1.18+
|
Details | Diff | Splinter Review |
(deleted),
patch
|
KaiE
:
review+
beltzner
:
superreview+
beltzner
:
approval2.0+
dveditz
:
approval1.9.2.16+
dveditz
:
approval1.9.1.18+
|
Details | Diff | Splinter Review |
(deleted),
application/zip
|
Details | |
(deleted),
patch
|
briansmith
:
review-
|
Details | Diff | Splinter Review |
(deleted),
patch
|
Details | Diff | Splinter Review | |
(deleted),
text/plain
|
Details | |
(deleted),
text/plain
|
Details |
According to mail from Comodo to the security alias "A long-term trusted partner of Comodo which has an RA function suffered an internal security breach and the attacker caused 7 certificates to be issued." The bogus certs were revoked within hours.
Unfortunately there are scenarios where the MITM can prevent the victim from contacting the OSCP responder and the certs will continue to be trusted. Given the high profile of the sites involved (including our own addons.mozilla.org) we should explicitly disable these certs if we can.
The CNs involved are
addons.mozilla.org
login.live.com
mail.google.com
www.google.com
login.yahoo.com
login.skype.com
global trustee
Can they be added as untrusted to the built-in certificate store, or does that only work for signing certificates? Can we ship a CRL containing these certs?
Reporter | ||
Updated•14 years ago
|
blocking1.9.1: --- → ?
blocking1.9.2: --- → ?
blocking2.0: --- → ?
status1.9.1:
--- → wanted
status1.9.2:
--- → wanted
Whiteboard: [sg:high]
Assignee | ||
Updated•14 years ago
|
Attachment #519860 -
Attachment description: Bad certs (PEM) zipped → Bad certs (text dumps) zipped
Comment 1•14 years ago
|
||
Comment 2•14 years ago
|
||
Revocation is in this CRL:
http://crl.comodoca.com/UTN-USERFirst-Hardware.crl
Here is the relevant section of the CRL:
openssl crl -text -inform DER -in UTN-USERFirst-Hardware.crl
Serial Number: 047ECBE9FCA55F7BD09EAE36E10CAE1E (mail.google.com)
Revocation Date: Mar 15 19:04:03 2011 GMT
Serial Number: F5C86AF36162F13A64F54F6DC9587C06 (www.google.com)
Revocation Date: Mar 15 19:04:24 2011 GMT
Serial Number: E9028B9578E415DC1A710A2B88154447 (login.skype.com)
Revocation Date: Mar 15 19:05:26 2011 GMT
Serial Number: 9239D5348F40D1695A745470E1F23F43 (addons.mozilla.org)
Revocation Date: Mar 15 20:15:20 2011 GMT
Serial Number: D7558FDAF5F1105BB213282B707729A3 (login.yahoo.com)
Revocation Date: Mar 15 20:15:26 2011 GMT
Serial Number: 392A434F0E07DF1F8AA305DE34E0C229
Revocation Date: Mar 15 20:15:38 2011 GMT
Serial Number: 3E75CED46B693021218830AE86A82A71
Revocation Date: Mar 15 20:15:47 2011 GMT
Serial Number: B0B7133ED096F9B56FAE91C874BD3AC0 (login.live.com)
Revocation Date: Mar 15 20:16:03 2011 GMT
Serial Number: D8F35F4EB7872B2DAB0692E315382FB0 (global trustee)
Revocation Date: Mar 15 20:19:04 2011 GMT
This OCSP responder, which is embedded in the certs, should also return "revoked":
http://ocsp.comodoca.com
Gerv
Assignee | ||
Comment 3•14 years ago
|
||
The idea to explicitly add the bad certs to the roots module is a good idea, but I don't know how to make this work. I don't see a way to make a cert as "explicitly not trusted". All I know of is "no explicit trust", but this is not sufficient, as the cert will still be trusted, if it can be chained to a trusted root.
The idea to use a CRL is problematic, because:
- I don't know an easy way to preship a CRL with binary NSS, because NSS reads CRLs from the database
- the approach might require that the applications ships the CRL and imports it at runtime. I think this approach is more work than the following one.
I'd like to propose the following approach (which, in a discussion with Gerv and Dan, we're currently favoring):
- block the certs at the application level
(PSM = all mozilla apps = all SSL sockets)
- embed the certs into PSM
- extend PSM's existing code for SSL_AuthCertificateHook
http://www.mozilla.org/projects/security/pki/nss/ref/ssl/sslfnc.html#1088805
- compare the server cert with our embedded blacklist
- in order to embed the blacklist, use
"issuer name" and "serial number" as the primary key to
decide about rejection
Implementation details:
I already have a tool that dumps this ID information for a given cert.
(I'm using this tool to extract the information that we need to grant the EV status to a root).
At runtime, when we arrive at the function, we have available:
- the full binary blob of the cert
- the ascii (utf8?) presentation of the issuer name
- the binary encoding of the serial number
I think it's tricky to store binary encoding into sourcecode.
It's better to use base64 encoding.
This has a minimal performance disadvante, we must convert the site's serial number from binary to base64. (But really, this little cpu overhead shouldn't be worrisome, there are many more expensive operations involved in processing certificates than this, anyway)
In theory, in order to be 100% sure we're checking the correct certificates, we ought to compare the binary encoding of the issuer name.
However, in this particular scenario, we are able to simplify, IMHO. The certificates to be blocked all use an issuer name that consists of pure ASCII characters. I propose we save CPU cycles, and do a standard string comparison for the issuer name.
When we find a match, we'll set error code SEC_ERROR_REVOKED_CERTIFICATE and abort the verification.
The user will see the existing error page "site of cert is revoked".
The user will NOT be able to override.
Assignee | ||
Comment 4•14 years ago
|
||
Demonstration patch.
This patch blocks the certificate currently being used at https://kuix.de
With this patch applied, I can't connect to the server anymore.
I will soon attach a patch that blocks all 7 problematic certs from Comodo.
Assignee | ||
Comment 5•14 years ago
|
||
I have received a test certificate from comodo for the host kuix.de
I have installed the test certificate on a SECONDARY port at my server,
https://kuix.de:9449/
Gerv has suggested, we could avoid the runtime conversion from binary-to-base64, by encoding the binary encoding of the serial numbers using hexadecimal characters \x00. Thanks to Gerv for that good idea.
I've enhanced the patch in bug 421989, and I now have a tool that prints such information, that can be easily copied to C source code.
Assignee | ||
Comment 6•14 years ago
|
||
This text file contains a dump of the identifying information of 8 certs:
- the 7 affected certs
- the test certificate from the same issuer
This information could be used for review purposes.
Assignee | ||
Comment 7•14 years ago
|
||
Attachment #519872 -
Attachment is obsolete: true
Assignee | ||
Comment 8•14 years ago
|
||
Test instructions:
(a)
Go to a site that uses a cert from the blacklist:
https://kuix.de:9449/
Expected:
ERROR, revoked
(b)
Go to a site that uses a good cert:
https://aws.bayerischer-golfverband.de/
Expected:
Site should display fine
(c)
Go to a site that was issued from a different CA:
https://www.mozilla.org
Expected:
Site should display fine
FYI, (a) and (b) use certs that were issued from the same root CA.
If (a) fails and (b) works, this proves that we correctly distinguish between separate serial numbers of the same issuer.
If (c) works, this proves that our blacklist doesn't hurt other CAs.
Assignee | ||
Comment 9•14 years ago
|
||
I would like to note, I'm confident in the encoding of this patch, because I didn't do it manually.
The sourcecode of that static list was automatically produced from the DER encoded certificates, and the code it produced for the test certificate at https://kuix.de:9449/ correctly caused it to be blocked. I think it's reasonable to conclude the code works for all certs correctly (after code review).
Assignee | ||
Comment 10•14 years ago
|
||
I've tested the patch with 4.0 (2.0), 3.6 (1.9.2) and 3.5 (1.9.1), and it worked in all versions.
1.9.1 branch requires a slightly merged patch, but only because a neighbourhood function is different.
Assignee | ||
Comment 11•14 years ago
|
||
Gerv noted that my code dumps some certificates with a leading zero, and produces thereby 17 bytes long serial numbers.
I believe this is correct, I think the binary encoding of those serial numbers is indeed 17 bytes.
When dumping such certs with NSS pp tool, it prints a leading 00:
I imported such a cert, using Firefox and cert manager, to a server tab, and then clicked "view", and it displays a 17 bytes serial number with leading 00:
Assignee | ||
Comment 12•14 years ago
|
||
And the reason why I wrote comment 11, because Gerv noticed that the text dumps we have received, all serial numbers were 16 bytes long.
I believe those text dumps were produced using openssl. It appears that openssl strips initial 00: in the human readable dump.
Comment 13•14 years ago
|
||
> The idea to use a CRL is problematic, because:
> - I don't know an easy way to preship a CRL with binary NSS, because NSS reads
CRLs from the database
NSS reads CRLs from tokens. adding a CRL to the builtins should work.
> - the approach might require that the applications ships the CRL and imports it
> - block the certs at the application level
> (PSM = all mozilla apps = all SSL sockets)
> - embed the certs into PSM
do you think this is the last time we are going to have this problem? It seems
to me if we can't block the certs in builtins, we should modify NSS so that we
can.
bob
Comment 14•14 years ago
|
||
Bob: should we ask Comodo to produce for us a special CRL with the seven certs in it? Their current CRL is 113k, even compressed.
Gerv
Assignee | ||
Comment 15•14 years ago
|
||
Attachment #519897 -
Attachment is obsolete: true
Comment 16•14 years ago
|
||
On the NSS call, we decided speed was more important than pretty. That we'd keep this bug open for the 'pretty' fix, but the fastest way to solve this is kai's patch.
NOTE: kai's patch only turns off SSL. Since these certs are only valid for SSL, that should be sufficient for these certs.
bob
Comment 17•14 years ago
|
||
Comment on attachment 519939 [details] [diff] [review]
Patch v4
r+ rrelyea
Attachment #519939 -
Flags: review+
Assignee | ||
Comment 18•14 years ago
|
||
Adding Rob @ Comodo to CC list.
Assignee | ||
Updated•14 years ago
|
Assignee: nobody → kaie
Reporter | ||
Comment 19•14 years ago
|
||
Comment on attachment 519939 [details] [diff] [review]
Patch v4
I want to chemspill this
Attachment #519939 -
Flags: approval2.0?
Attachment #519939 -
Flags: approval1.9.2.16?
Attachment #519939 -
Flags: approval1.9.1.18?
Comment 20•14 years ago
|
||
Robin Alden at Comodo (now CCed) tells me:
- Microsoft are aware of the problem.
- Opera are looking at options; they may add the roots to their "untrusted" store. Or, because they always check revocation status, they may rely on OCSP.
- Chrome are writing a patch; there is a stable version due to ship next week, and they could add the patch to that, but they may do an even earlier release for this problem.
Comodo are also discussing the matter with law enforcement.
Gerv
Comment 21•14 years ago
|
||
Can I get a summary of the risk inherent in this code, please, and what QA would need to be revalidated if we took it in an RC2?
Assignee | ||
Comment 22•14 years ago
|
||
Mike,
this code path is executed each time we do a handshake with any SSL server.
However, the test starts with a simple string comparison.
If the issuer name is not matching, then all remaining code is skipped.
This alone eliminates the risk for all CAs that are not connected to this issue.
If the issuer name matches, we will compare certificate serial numbers, of the server cert, against our blacklist.
The rest of the code is skipping initial zeros, and looping through our blacklist, then a final memory comparison.
This isn't risky code, no memory allocation etc., but simple iteration, looping and comparison, and code review shows we carefully deal with boundaries. I don't see risks for side effects.
I made it easy for testing.
We received a test certificate, that I installed on a server, and that we will keep around for testing the QA candidate, and longer.
The test instructions are described in comment 8.
In my opinion, the only work that needs to be repeated, if you want to: Test that you can still connect to a variety of https/SSL sites.
If you still can connect, and are not being blocked, all is fine.
Updated•14 years ago
|
tracking-fennec: --- → ?
OS: Mac OS X → All
Updated•14 years ago
|
Attachment #519939 -
Flags: feedback?(bsmith)
Updated•14 years ago
|
Attachment #519939 -
Flags: feedback?(benjamin)
Updated•14 years ago
|
Attachment #519939 -
Flags: feedback?(bzbarsky)
Comment 23•14 years ago
|
||
(In reply to comment #22)
> I made it easy for testing.
> We received a test certificate, that I installed on a server, and that we will
> keep around for testing the QA candidate, and longer.
That is to say: we asked Comodo to issue us another certificate from the same root the bogus ones were issued from, but for a domain Kai controls. We then added that cert's serial number to the blacklist (which is why it's 8 certificates long rather than 7). This means we don't need to set up a server using one of the compromised certs (which is impossible, as we don't have the private keys) in order to test the blocking. Instead, we can use https://kuix.de:9449/ , which is using the cert Kai obtained and we have now blacklisted.
Gerv
Updated•14 years ago
|
Attachment #519939 -
Flags: feedback?(benjamin)
Comment 24•14 years ago
|
||
Comment on attachment 519939 [details] [diff] [review]
Patch v4
FWIW, the patch looks good to me.
Attachment #519939 -
Flags: feedback+
Reporter | ||
Comment 25•14 years ago
|
||
Filed bug 642503 on moving a more generic version of this blacklist functionality into NSS proper for a future version.
Comment 26•14 years ago
|
||
Hello, RC2.
Updated•14 years ago
|
blocking2.0: ? → final+
Updated•14 years ago
|
tracking-fennec: ? → 2.0+
I'm not familiar with this area, and don't understand the leading zeroes thing (or why the leading zeros are kept in the literal table only to be stripped out with an extra loop). But it looks okay, and low risk.
Comment 28•14 years ago
|
||
the tests from comment 8 all work fine in Fennec with this patch
Assignee | ||
Comment 29•14 years ago
|
||
(In reply to comment #27)
> I'm not familiar with this area, and don't understand the leading zeroes thing
> (or why the leading zeros are kept in the literal table only to be stripped out
> with an extra loop). But it looks okay, and low risk.
Sorry, the code in the static list was automatically created using a tool. I wanted to avoid the risk of manually tweaking it, because manual tweaking has the risk of human error. Also, if I tweaked it now, I might forget about that later, if there's ever a need to recreate or extend the list.
This filtering was a late addition, suggested during code review, to make sure we always make a correct match, regardless of leading zeros.
Updated•14 years ago
|
tracking-fennec: ? → ---
Assignee | ||
Comment 31•14 years ago
|
||
The tool I'm refering to is NSS "pp" tool, plus the patch attached to bug 421989.
It produced attachment 519892 [details] (which contains the leading zeros).
Comment 32•14 years ago
|
||
Comment on attachment 519939 [details] [diff] [review]
Patch v4
This seems reasonable to me, provided the leading 0s can be ignored (that is, that there are never serial numbers that are the same except one has more leading zeros than the other).
Is that the case?
Attachment #519939 -
Flags: feedback?(bzbarsky) → feedback+
Updated•14 years ago
|
tracking-fennec: --- → 2.0+
Comment 33•14 years ago
|
||
I'm also not quite sure why the auto-generated serial numbers have leading zeros for 6 of the certs, but not the other two...
Assignee | ||
Comment 34•14 years ago
|
||
Bob would be able to explain the issue with the leading zeros better, I'll try to repeat what he said.
It depends on whether using a signed vs. an unsigned variable to the decoding of the serial number. The serial number is DER encoded. The raw encoding sometimes takes up 17 bytes, when having a 16 bytes serial number with the highest bit set.
(Don't ask me why that is, it appears to be an implementation detail/behaviour of the encoders/decoders used.)
We see a leading zero for all serial numbers that start with hex 8 or higher. No leading zero for 7 or lower.
While discussing/reviewing the patch, Bob suggested, stripping the initial zeros will always make it work.
Since Bob is the expert with properties of certificates and serial numbers, I also conclude he is sure that stripping leading zeros is fine. The answer to comment 32 is yes. Serial numbers are really numbers, not byte arrays.
Assignee | ||
Comment 35•14 years ago
|
||
Another supporting point, when the "openssl x509" tool dumps one of those certificates, and prints the serial numbers in human readable form, it strips away leading zeros, too.
Comment 36•14 years ago
|
||
The main concern now is with the logic in nsNSSBadCertHandler, namely the hitting of the following assertion:
NS_NOTREACHED("why did NSS call our bad cert handler if all looks good? Let's cancel the connection");
and the fact that nsHandleSSLError and cancel_and_failure() are not called before returning SECFailure. Kai and I are discussing it on IRC.
Assignee | ||
Comment 37•14 years ago
|
||
I believe comment 36 isn't a new scenario.
I guess when we hit a revoked cert in the wild, we'll probably get exactly the same code path as now.
The bad-cert-handler is designed to ignore any other errors that it doesn't deal with explicitly - including the "revoked" scenario. Obviously that failure reason was "forgotton" when writing the bad cert handler.
I don't see a strong reason to fix this now. But it's a very good opportunity to fix it. The only thing that should be done in addition, is ignoring and a little bit of cleanup.
So, I'm proposing this tiny addon patch. The error code, that we set in the other code, is available in this bad-cert-handler, if fetched as the first operation in the function. This is documented at http://www.mozilla.org/projects/security/pki/nss/ref/ssl/sslfnc.html#1088928
The patch fetches this status, and only if we see the "revoked" status, we'll give special treatment: Ignore and cleanup. The cleanup code "cancel_and_failure" is the same as we do in all other exit paths (except in the not-reached-assertion path that Brian discovered).
Thanks a lot for discovering this.
Assignee | ||
Comment 38•14 years ago
|
||
This patch builds. I'm testing it in a moment.
Assignee | ||
Comment 39•14 years ago
|
||
No, this incremental patch does not work. we no longer get the error page.
Assignee | ||
Comment 40•14 years ago
|
||
Ok, the patch is not necessary. With this patch we prevent the "default catch all" that will display an error.
The only thing that is necessary, and should be done in a follow-up patch: Remove the "not-reached" assertion when a cert is revoked.
Assignee | ||
Updated•14 years ago
|
Attachment #520017 -
Attachment is obsolete: true
Comment 41•14 years ago
|
||
> Bob would be able to explain the issue with the leading zeros better,
> I'll try to repeat what he said.
Serial numbers are unsigned integers. DER INTS are all signed, so when you encode an unsigned integer with DER you need to make sure the leading bit is not zero. This CA issues serial numbers which are 16 bytes long. If the lead byte starts with a zero (0x00-0x7f) then the serial number is 16 bytes long, If the lead starts with a 1 (0x80-0xff) then a zero is added as padding.
Different uses of these values tend to decode the resulting bytes differently, so it's usually best to canonicalize the bytes before comparing them. the easiest way of doing that is to strip the leading bytes.
bob
Comment 42•14 years ago
|
||
Comment on attachment 519939 [details] [diff] [review]
Patch v4
The patch looks sufficient to me. Like Kai said, the unusual code path with the assertion should also be tripped by a normally-revoked cert. I will file a separate bug to deal with that.
Attachment #519939 -
Flags: feedback?(bsmith) → feedback+
Comment 43•14 years ago
|
||
Rob: can we get 'er landed? a=beltzner for branches.
tracking-fennec: 2.0+ → ---
Updated•14 years ago
|
Attachment #519939 -
Flags: approval2.0?
Attachment #519939 -
Flags: approval2.0+
Attachment #519939 -
Flags: approval1.9.2.16?
Attachment #519939 -
Flags: approval1.9.2.16+
Attachment #519939 -
Flags: approval1.9.1.18?
Attachment #519939 -
Flags: approval1.9.1.18+
Comment 44•14 years ago
|
||
Comodo say:
Gerv,
We have analysed our OCSP responder logs and we see no evidence of
any of these certificates having seen traffic in the wild other than
2 hits from the attacker himself (from the already mentioned
212.95.136.18 and the almost adjacent 212.95.136.20) and also hits
from Comodo staff testing and from Microsoft, Google, and (we think)
Mozilla staff testing.
The earliest OCSP hit we see is at 15/Mar/2011:21:39:43 UTC, which
is after the latest revocation - which was at 20:19 UTC.
Regards
Robin
Comment 45•14 years ago
|
||
mozilla-central:
http://hg.mozilla.org/mozilla-central/rev/f6215eef2276
mozilla-2.1:
http://hg.mozilla.org/releases/mozilla-2.1/rev/76bd4daa2d95
mozilla-2.0:
http://hg.mozilla.org/releases/mozilla-2.0/rev/3d4c3670c0bd
Waiting for builds before landing on relbranches for mozilla-2.0 and mozilla-2.1, currently testing 1.9.2 and 1.9.1.
Three out of seven builds done.
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → FIXED
Updated•14 years ago
|
tracking-fennec: --- → 2.0+
Comment 46•14 years ago
|
||
Robert, thanks for explaining the leading 0 thing!
Reporter | ||
Comment 47•14 years ago
|
||
(In reply to comment #44)
> Comodo say:
> We have analysed our OCSP responder logs and we see no evidence of
> any of these certificates having seen traffic in the wild other than
> 2 hits from the attacker himself (from the already mentioned
> 212.95.136.18 and the almost adjacent 212.95.136.20)
Does Comodo keep baseline traffic stats such that you would detect a drop in normal OCSP pings coming from the region of that IP (Tehran), or other regions in case that address is misdirection? Lack of normal OCSP traffic might be evidence of an attempt to use those certs by blackholing the OCSP responder.
But as long as you're otherwise seeing normal OCSP traffic the lack of pings for those specific certs is comforting.
Comment 48•14 years ago
|
||
I've verified the bug on Linux with the steps on comment #8 and the examples in comment #0 with the exception of "global trustee." In the RC1 I can load the page https://kuix.de:9449/ but in the RC I get a page with this error:
Secure Connection Failed
An error occurred during a connection to kuix.de:9449.
Peer's Certificate has been revoked.
(Error code: sec_error_revoked_certificate)
I've also tested around functionality in AMO, like installing and updating extensions, as well as functionality in the sites mentioned in comment #0. I've seen no problems with those.
Comment 49•14 years ago
|
||
UPDATE: The attacker also caused the bad certificate for login.yahoo.com to be reissued twice. The attached patch will blacklist these 2 additional certs.
These 2 certs were revoked at the same time the other 7 certs were revoked, but somehow we omitted to communicate the details to you. Sincere apologies.
Attachment #520160 -
Flags: superreview?
Attachment #520160 -
Flags: review?
Attachment #520160 -
Flags: feedback?
Attachment #520160 -
Flags: approval2.0?
Attachment #520160 -
Flags: approval1.9.2.16?
Attachment #520160 -
Flags: approval1.9.1.18?
Comment 50•14 years ago
|
||
Comment 51•14 years ago
|
||
Rob: Can you totally utterly definitely confirm there are no more?
I note that these two certs are the two revocations in the middle of the set from the CRL, as quoted in comment 2. As that CRL is used regularly (multiple times per day), I asked Robin Alden of Comodo on the phone specifically about those two revocations - whether it was coincidence (i.e. they were unrelated), or whether there were more mis-issued certificates. He told me it was coincidence. :-|
Gerv
Comment 52•14 years ago
|
||
> He told me it was coincidence. :-|
I was wrong, and apologize.
I'll let Rob give his confirmation separately.
Regards
Robin
Assignee | ||
Comment 53•14 years ago
|
||
Comment on attachment 520160 [details] [diff] [review]
Additional Patch - 2 more bad certs
I confirm this patch is correct.
My tool produced exactly the same code when dumping the additional new attached certs.
Attachment #520160 -
Flags: review? → review+
Comment 54•14 years ago
|
||
> Rob: Can you totally utterly definitely confirm there are no more?
Gerv,
I confirm that the Comodo Partner Account that suffered the security breach has (totally, utterly and definitely) *not* mis-issued any further certificates, and we are not aware that any other Accounts have been breached.
The "2 more bad certs" in comment 50 need to be blacklisted. After that, there are no more.
Comment 55•14 years ago
|
||
Comment on attachment 520160 [details] [diff] [review]
Additional Patch - 2 more bad certs
First and foremost: ugh.
Second, we will take this patch on:
mozilla-central
releases/mozilla-2.0 (default and GECKO20_2011031715_RELBRANCH)
releases/mozilla-2.1 (default and GECKO21_20110317_RELBRANCH)
After we get the confirmed changeset landings, we will:
respin mozilla-central nightlies
make a Firefox 4 RC2 build3 for Windows, OSX and Linux
make a Firefox 4 RC1 build2 for Android
Attachment #520160 -
Flags: superreview?
Attachment #520160 -
Flags: superreview+
Attachment #520160 -
Flags: feedback?
Attachment #520160 -
Flags: approval2.0?
Attachment #520160 -
Flags: approval2.0+
Comment 56•14 years ago
|
||
That's actually Firefox 4 RC1 build3 for Android
Comment 57•14 years ago
|
||
And actually Android and Maemo, but who's counting?
Comment 58•14 years ago
|
||
Changeset numbers:
mozilla-2.1 default: 63440:f4cceb1f9b59
mozilla-2.1 GECKO21_20110317_RELBRANCH: 63441:0f7aaaf20cb3
mozilla-2.0 default: 63341:afbc0b4fd618
mozilla-2.0 GECKO20_2011031715_RELBRANCH: 63342:6be9e31d01b4
mozilla-central default: 63437:55f344578932
Gerv
Comment 59•14 years ago
|
||
2.0 tip: http://hg.mozilla.org/releases/mozilla-2.0/rev/afbc0b4fd618
2.0 branch: http://hg.mozilla.org/releases/mozilla-2.0/rev/6be9e31d01b4
2.1 tip: http://hg.mozilla.org/releases/mozilla-2.1/rev/f4cceb1f9b59
2.1 branch: http://hg.mozilla.org/releases/mozilla-2.1/rev/0f7aaaf20cb3
m-c tip: http://hg.mozilla.org/mozilla-central/rev/55f344578932
Gerv
Reporter | ||
Comment 60•14 years ago
|
||
Comment on attachment 520160 [details] [diff] [review]
Additional Patch - 2 more bad certs
Approved for 1.9.2.16 and 1.9.1.18, a=dveditz for release-drivers
Attachment #520160 -
Flags: approval1.9.2.16?
Attachment #520160 -
Flags: approval1.9.2.16+
Attachment #520160 -
Flags: approval1.9.1.18?
Attachment #520160 -
Flags: approval1.9.1.18+
Reporter | ||
Updated•14 years ago
|
blocking1.9.1: ? → .18+
blocking1.9.2: ? → .16+
Comment 61•14 years ago
|
||
I've reverified the fix in build 3 of RC2 of Firefox 4 now.
Comment 62•14 years ago
|
||
verified FIXED in build 3 of RC1 for Fennec 4 based on the test instructions listed in comment #8:
Mozilla/5.0 (Android; Linux armv7l; rv:2.1) Gecko/20110318 Firefox/4.0b13pre Fennec/4.0 ID:20110318114419
Comment 63•14 years ago
|
||
(In reply to comment #62)
> verified FIXED in build 3 of RC1 for Fennec 4 based on the test instructions
> listed in comment #8:
>
> Mozilla/5.0 (Android; Linux armv7l; rv:2.1) Gecko/20110318 Firefox/4.0b13pre
> Fennec/4.0 ID:20110318114419
also double checked ssl sites for comment 0 on fennec. all good.
Reporter | ||
Comment 64•14 years ago
|
||
(In reply to comment #20)
> Robin Alden at Comodo (now CCed) tells me:
> - Microsoft are [...]
> - Opera are [...]
> - Chrome are [...]
Robin: what about Apple/Safari? Have they been informed?
Assignee | ||
Comment 65•14 years ago
|
||
Let's not forget to land the additional patch into 1.9.1 and 1.9.2
Comment 66•14 years ago
|
||
(In reply to comment #64)
> (In reply to comment #20)
> > Robin Alden at Comodo (now CCed) tells me:
> > - Microsoft are [...]
> > - Opera are [...]
> > - Chrome are [...]
>
> Robin: what about Apple/Safari? Have they been informed?
Yes. I had a positive response and a follow up request for more information from product-security@apple.com.
Comment 67•14 years ago
|
||
mozilla-1.9.1
-------------
GECKO19117_2011012114_RELBRANCH:
http://hg.mozilla.org/releases/mozilla-1.9.1/rev/8da078c0caf8
http://hg.mozilla.org/releases/mozilla-1.9.1/rev/3ab1ade7d8c6
default:
http://hg.mozilla.org/releases/mozilla-1.9.1/rev/e3479ffc1e3a
http://hg.mozilla.org/releases/mozilla-1.9.1/rev/679ce0aa830c
mozilla-1.9.2
-------------
GECKO19214_2011012112_RELBRANCH:
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/7027802cdb66
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/ab4f7fd95593
default:
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/0c824be2d87d
http://hg.mozilla.org/releases/mozilla-1.9.2/rev/ce2e6288fb66
Reporter | ||
Updated•14 years ago
|
Reporter | ||
Updated•14 years ago
|
Attachment #519939 -
Flags: approval1.9.2.17+
Attachment #519939 -
Flags: approval1.9.2.16+
Attachment #519939 -
Flags: approval1.9.1.19+
Attachment #519939 -
Flags: approval1.9.1.18+
Reporter | ||
Updated•14 years ago
|
Attachment #520160 -
Flags: approval1.9.2.17+
Attachment #520160 -
Flags: approval1.9.2.16+
Attachment #520160 -
Flags: approval1.9.1.19+
Attachment #520160 -
Flags: approval1.9.1.18+
Comment 68•14 years ago
|
||
Verified for 1.9.2.16 and 1.9.1.18 using repro steps in comment 8.
blocking1.9.1: .18+ → ?
blocking1.9.2: .16+ → ?
Reporter | ||
Updated•14 years ago
|
blocking1.9.1: ? → .18+
blocking1.9.2: ? → .16+
Assignee | ||
Comment 69•14 years ago
|
||
The patch introduced a small leak. When we detect a blacklisted cert, we exit the function early and don't clean up the cert.
The fix is to move the C++ cleanup variable further up, which I have missed when moving up the line to obtain the cert.
I filed bug 643694 to fix it. I propose to do it in the nex regular update cycle. I'm attaching the patch here, as this issue is not yet public.
Assignee | ||
Updated•14 years ago
|
Attachment #520887 -
Flags: review?(bsmith)
Comment 70•14 years ago
|
||
Comment on attachment 520887 [details] [diff] [review]
fix a leak
As mentioned on IRC, this:
> + CERTCertificateCleaner serverCertCleaner(serverCert);
> CERTCertificate *serverCert = SSL_PeerCertificate(fd);
should be:
> CERTCertificate *serverCert = SSL_PeerCertificate(fd);
> + CERTCertificateCleaner serverCertCleaner(serverCert);
Attachment #520887 -
Flags: review?(bsmith) → review-
Comment 71•14 years ago
|
||
The updates (not the blog post yet) should be live for 3.6 and 3.5 users.
Comment 72•14 years ago
|
||
Comment 73•14 years ago
|
||
Reporter | ||
Updated•14 years ago
|
Attachment #519860 -
Attachment is private: true
Reporter | ||
Updated•14 years ago
|
Attachment #519863 -
Attachment is private: true
Reporter | ||
Updated•14 years ago
|
Attachment #519892 -
Attachment is private: true
Reporter | ||
Updated•14 years ago
|
Attachment #520163 -
Attachment is private: true
Reporter | ||
Updated•14 years ago
|
Group: core-security
Reporter | ||
Updated•14 years ago
|
Attachment #519860 -
Attachment is private: false
Reporter | ||
Updated•14 years ago
|
Attachment #519863 -
Attachment is private: false
Reporter | ||
Updated•14 years ago
|
Attachment #519892 -
Attachment is private: false
Reporter | ||
Updated•14 years ago
|
Attachment #520163 -
Attachment is private: false
Comment 74•14 years ago
|
||
I reiterate my objection to Mozilla allowing the included certification authorities to outsource to third-party registration authorities.
Comment 75•14 years ago
|
||
May I ask why this bug (and the whole issue) has been kept secret for over a week? To spare my time, let me quote The Register [1] since it exactly reflects my thoughts:
<snip>
The decision by Google, Microsoft, Mozilla and Comodo to keep the world in the dark for eight days comes as a slap in the face to their users.
“The attackers had all they needed,” said Marsh Ray, a researcher and software developer at two-factor authentication service PhoneFactor. “Knowing which certificates have been compromised gives an immediate step people can take to secure their systems.”
None of the companies would explain why they waited so long to disclose the attack.
</snip>
If there is some policy due to which this was kept under the hood, then it's completely flawed and needs to be rethinked ASAP.
[1] http://www.theregister.co.uk/2011/03/23/gmail_microsoft_web_credential_forgeries/page2.html
Comment 76•14 years ago
|
||
Mozilla has made a more detailed statement about the Comodo misissuance incident:
http://blog.mozilla.com/security/2011/03/25/comodo-certificate-issue-follow-up/
Please continue discussion in the mozilla.dev.security.policy discussion forum:
https://www.mozilla.org/about/forums/#dev-security-policy
Gerv
Comment 77•14 years ago
|
||
backport to 1.8.1.24.
If anyone has time to take a look at it, I'd be very grateful
Assignee | ||
Comment 78•14 years ago
|
||
Comment on attachment 522155 [details] [diff] [review]
fix for 1.8.1(.24)
Looks OK to me.
Reporter | ||
Comment 79•14 years ago
|
||
Found a hacker's claim of responsibility at http://pastebin.com/74KXCaEZ
Could be BS, but there are some testable claims
- the name of the RA
- the ceo's account
- the "comodo username"
- did the RA really have a trustdll.dll
- was it C#
- did it really hardcode in their password/username?
This is all depressingly plausible. Is trustdll.dll something Comodo distributes, or was that winning idea solely the RAs? Does it really take only a name and password, and do RAs typically leave those hardcoded into internet-connected systems?
Found a similar (unverified) claim in response to a Heise article on the subject, guy claiming to be a reseller (presumably restricted by DV checks at the RA or Comodo level?) who could get around that by calling the APIs directly and bypassing the app they were given.
http://www.heise.de/security/news/foren/S-Kenne-ich-von-Comodo-nicht-anders-ich-kann-selber-solche-Zertifikate-ausstellen/forum-196553/msg-20015933/read/
Comment 80•14 years ago
|
||
(In reply to comment #79)
> Created attachment 522228 [details]
> --> https://bugzilla.mozilla.org/attachment.cgi?id=522228
> Hacker claim
>
> Found a hacker's claim of responsibility at http://pastebin.com/74KXCaEZ
>
> Could be BS, but there are some testable claims
> - the name of the RA
> - the ceo's account
> - the "comodo username"
> - did the RA really have a trustdll.dll
> - was it C#
> - did it really hardcode in their password/username?
There's also http://pastebin.com/DBDqm6Km, which apparently contains the decompiled source code to trustdll.dll. This can also be tested.
Comment 81•14 years ago
|
||
(In reply to comment #80)
> There's also http://pastebin.com/DBDqm6Km, which apparently contains the
> decompiled source code to trustdll.dll. This can also be tested.
Attached (pastebins suck].
Comment 82•14 years ago
|
||
> guy claiming to be a reseller (presumably restricted by DV checks at
> the RA or Comodo level?) who could get around that by calling the APIs
> directly and bypassing the app they were given.
That claim should be easy to verify, somebody just needs to enroll as reseller and try it.
Comment 83•14 years ago
|
||
I've heard the same claims from intermediate CAs using the API.
See Also: → https://launchpad.net/bugs/741528
Comment 84•14 years ago
|
||
> Hacker claim
One wonders why he didn't just post some data signed with one of the private keys he generated. He seems particularly anxious to prove he was the one who pulled this off.
bob
Comment 85•14 years ago
|
||
Does that help? http://pastebin.com/X8znzPWH
Comment 86•14 years ago
|
||
(In reply to comment #85)
> Does that help? http://pastebin.com/X8znzPWH
Since pastebins suck, here's the content:
For some real dumbs, I bet they don't have IQ above 75, WHO STILL thinks I'm not the hacker, here is mozilla addon's certificate, check it's serial with one published on all the internet:
http://www.multiupload.com/J9I8NFWPT0
I really worry about you guys (people who still have doubts) even for surfing in internet, have you ever visited a doctor?
Private key for above certificate:
http://www.multiupload.com/SI4FKWJ5KY
@ioerror, when I say you have relations with intelligence agencies and you pass traffic, I have my reasons: http://bit.ly/dK0oB5 #comodogate
Thanks to Robert Graham for pointing out that private key is encrypted with a passphrase, here is private key without passphrase, I don't want to give away my passphare:
-----BEGIN RSA PRIVATE KEY-----
MIIEowIBAAKCAQEAq8ZtNvMVc3iDc850hdWu7LLw4CQfE4O4IKy7mv6Iu6uhHQsf
RQCqSbc1Nwxq70dMudG+41cSBI2Sx7bsAby22seBOCCtcoXmDvyBbAetaHY4xUTX
zMZKxZc+ZPRR5vB+suxW9yWCTUmYyxaY3SPxiZHRF5dAmSbW4qIrXt+9ifIbGlMt
zFBBetA9KgxVcBQB6VhJEHoLk4KL4R7tOoAQgs6WijTwzNfTubRQh1VUCbidQihV
AOWMNVS/3SWRRrcN5V2DqOWL+4TkPK522sRDK1t0C/i+XWjxeFu1zn3xXZlA2sru
OIFQvpihbLgkrfOvjA/XESgshBhMfbXZjzC1GwIDAQABAoIBAQCJoijaEXWLmvFA
thiZL7jEATCNd4PK4AyFacG8E9w8+uzR15qLcFgBTqF95R49cNSiQtP/VkGikkkc
ao25aprcu2PnNA+lpnHKajnM9G3WOHuOXHXIps08es3MmBKTxvjNph6cUlqQULrz
Zry+29DpmIN/snpY/EzLNIMptn4o6xnsjAIgJDpQfFKQztxdmZU6S6eVVn0mJ5cx
q+8TTjStaMbh+Yy73s+rcaCXzL7yqWDb1l5oQJ/DMYNfufY6lcLgZUMwFxYKjCFN
ScAPCiXFUKTzY3Hy1Z4tLndFxipyEPywDep1TB2nMb+F3OOXUs3z+kKVjGFaGnLZ
591n3x3hAoGBAOOgsb4QybjHh9+CxhUkfsqcztGGdaiI3U5R1qefXL7R47qCWfGc
FKdoJh3JwJzHEDX68ZmHz9dPhSXw6YrlLblCi6U/3g7BOMme5KRZKBTjHFo7O9II
B0laE5ISRH4OccsOC3XUf9XBkm8szzEBj95DgzB0QydPL4jp7NY0h0QrAoGBAMEv
jEFkr/JCRe2RWUSx/a1WT/DHnVLMnDb/FryN2M1fAerpMYNUc2rnndjp2cYbsGLs
cSF6Xecm3mUGqn8Y5r8QqBwxCp5OunCFCXEJvkiU3NSs8oskCsB8QJ6vk3qmauUK
jClX91heSCigwhC2t+1txnF290m/y0T46EfqOSrRAoGAUlyVk4D9jEdeCWiHBaVj
3ynnx3ZQYj/LW4hPE+2coErPjG+X3c0sx/nuOL8EW3XHjtCS1IuIj45tTfIifqg3
6B2E67D1Rv9w7br5XeIIl64pVxixp2hSQp8+D49eiwHs+JzHVsYhzxUwR9u9yCyZ
gsGI2WJn3fRP7ck+ca8l9msCgYB4B2Hec3+6RqEKBSfwvaI+44TRtkSyYDyjEwT+
bCeLGn+ng/Hmhj8b6gKx9kH/i86g+AUmZtAXQZgmLukaBM/BYMkCkxnk2EeQh6gh
Goumrw8x+K7N8rvXcpv3vGEmcGW0H0SMn4In3pR44cER/2Tx2SXV87Obl9Xk6b3w
iL+yMQKBgFjXcmiBW8lw3l2CaVckd/1SzrT80AfRpMT9vafurxe+iAhl9SDAdoZe
3RlshoItDQLW1ROlkLhM7Pdq/XZvLRm128hiIGKTDBnxtfN8TKAg+V7V+/TTfdqv
8jq7epvZsq5vjOC1FZh2gOhf50QwpqDJktjdyka1sPiBKQSoxfbZ
-----END RSA PRIVATE KEY-----
Comment 87•14 years ago
|
||
And the private key has been verified as matching the public key attached to this bug:
http://erratasec.blogspot.com/2011/03/verifying-comodo-hackers-key.html
So this guy either did it, or is part of the group that did it.
Gerv
Comment 88•14 years ago
|
||
(In reply to comment #69)
> Created attachment 520887 [details] [diff] [review]
> fix a leak
This was landed as part of bug 644012.
Comment 89•14 years ago
|
||
Presumably comment 8 can be morphed into a test, either automated or Litmus, or both. Setting flags.
Flags: in-testsuite?
Flags: in-litmus?
Comment 90•14 years ago
|
||
(In reply to comment #89)
> Presumably comment 8 can be morphed into a test, either automated or Litmus, or
> both. Setting flags.
Litmus testcase has been added -- please review:
https://litmus.mozilla.org/show_test.cgi?id=15365
Flags: in-litmus? → in-litmus+
Comment 91•14 years ago
|
||
(In reply to comment #90)
> (In reply to comment #89)
> > Presumably comment 8 can be morphed into a test, either automated or Litmus, or
> > both. Setting flags.
>
> Litmus testcase has been added -- please review:
> https://litmus.mozilla.org/show_test.cgi?id=15365
This test presupposes that https://kuix.de:9449/ will be up forever. Since the bogus test cert is keyed to that site, we cannot move the testcase to our QA server.
Comment 92•14 years ago
|
||
(In reply to comment #91)
> > Litmus testcase has been added -- please review:
> > https://litmus.mozilla.org/show_test.cgi?id=15365
>
> This test presupposes that https://kuix.de:9449/ will be up forever. Since the
> bogus test cert is keyed to that site, we cannot move the testcase to our QA
> server.
Correct. As far as I understand it, the purpose is to test a certificate which has been revoked by the CA. Is there a way that we can spoof a revoked certificate? If we can, then we could create a test on the QA server...
Assignee | ||
Comment 93•14 years ago
|
||
Testing is tricky.
Do you want to test the blacklist mechanism?
The test cert on kuix.de:9449 will expire soon. I'm not sure what will happen then. You might get EITHER error expired OR error revoked. Only if the error REVOKED then you can continue to use this blacklist test. If EXPIRED has higher precendece and is the error that will be reported, this test ability will be gone. You must test this probably around April 20.
If you want to test revocation mechanisms in general, that's another story, and I agree, that could be automated, but it would also require to have someone permanently operate a server with a revoked cert.
Comment 94•14 years ago
|
||
(In reply to comment #93)
> If you want to test revocation mechanisms in general, that's another story, and
> I agree, that could be automated, but it would also require to have someone
> permanently operate a server with a revoked cert.
QA has a public facing server we can use for SSL work now.
Why does this need to be a litmus test?
Comment 96•14 years ago
|
||
Kyle, it doesn't. If someone wanted to craft an automated test, that would be great in my opinion.
Yeah, we should be able to test this through automation.
Flags: in-litmus+ → in-litmus-
Comment 98•14 years ago
|
||
(In reply to comment #97)
> Yeah, we should be able to test this through automation.
This should bear bug 617414 in mind. I don't know how SSL tests are currently performed and whether you'd need a couple more certificates from Comodo (one to blacklist and one to not) as this code is only run if the issuer name matches.
Comment 99•14 years ago
|
||
Resetting the Litmus flag to indicate that the test still exists in Litmus (and I think it should until it is replaced, or until it is no longer a valid test as per comment 93). Given that this is a security issue, surely a Litmus test is better in the interim? Let me know if I'm wrong.
Flags: in-litmus- → in-litmus+
Comment 100•14 years ago
|
||
ah... we have the private key for the revoked addons certificate!
bob
Comment 101•14 years ago
|
||
the litmus test should be reworked to use the addons cert for which we have the private key. in order to do this, the testcase will need to pin <'now'> to be a fixed date (e.g. 2011-03-28), on the client system, and the host will need to be happy to serve an expired certificate (possibly by pinning the date to the same).
Reporter | ||
Updated•13 years ago
|
status-firefox5:
--- → unaffected
You need to log in
before you can comment on or make changes to this bug.
Description
•