Closed Bug 875390 Opened 11 years ago Closed 11 years ago

[MMS] MMSC routed through the wrong network interface

Categories

(Core :: DOM: Device Interfaces, defect)

ARM
Gonk (Firefox OS)
defect
Not set
normal

Tracking

()

RESOLVED FIXED
mozilla24
blocking-b2g leo+
Tracking Status
firefox22 --- wontfix
firefox23 --- wontfix
firefox24 --- fixed
b2g18 --- fixed
b2g18-v1.0.0 --- wontfix
b2g18-v1.0.1 --- wontfix
b2g-v1.1hd --- fixed

People

(Reporter: dietrich, Assigned: gerard-majax)

References

Details

Attachments

(1 file, 4 obsolete files)

STR
1. compose message
2. add single contact
3. attach image
4. hit send

Actual: message sits forever with spinner next to it, until i blow away gaia profile.

adb logcat error when i hit send

E/GeckoConsole(  108): [JavaScript Error: "lastSegment is undefined" {file: "jar:file:///system/b2g/omni.ja!/components/RadioInterfaceLayer.js" line: 2544}]
E/GeckoConsole(  108): [JavaScript Error: "lastSegment is undefined" {file: "jar:file:///system/b2g/omni.ja!/components/RadioInterfaceLayer.js" line: 2544}]
blocking-b2g: --- → leo+
Whiteboard: [NO_UPLIFT]
Summary: send mms with image fails, spins forever → [MMS] send mms with image fails, spins forever
Whiteboard: [NO_UPLIFT]
This might be related to bad APN? bug 875491 ?
OS: Mac OS X → Gonk (Firefox OS)
Hardware: x86 → ARM
(In reply to Dietrich Ayala (:dietrich) from comment #0)
> STR
> 1. compose message
> 2. add single contact
> 3. attach image
> 4. hit send
> 
> Actual: message sits forever with spinner next to it, until i blow away gaia
> profile.
> 
> adb logcat error when i hit send
> 
> E/GeckoConsole(  108): [JavaScript Error: "lastSegment is undefined" {file:
> "jar:file:///system/b2g/omni.ja!/components/RadioInterfaceLayer.js" line:
> 2544}]
> E/GeckoConsole(  108): [JavaScript Error: "lastSegment is undefined" {file:
> "jar:file:///system/b2g/omni.ja!/components/RadioInterfaceLayer.js" line:
> 2544}]

I don't think it's related. Actually, I'm also having this issue on my Nexus S, and I've noticed reports of the radio/ril code trying to setup a rmnet1 interface and failing due to missing rights.

When I compare to my HTC Desire Z under Android 2.3.3 trying to send MMS, the rmnet1 does not popup, and instead rmnet0 is reconfigured for the mms APN.

I'll look into it today, since I think I reproduce it easily.
Assignee: nobody → lissyx+mozillians
(In reply to Corey Frang [:gnarf] [:gnarf37] from comment #1)
> This might be related to bad APN? bug 875491 ?

For sure in my case the APN is good, since I'm able to *receive* and *download* MMS. But sending just fails.
the "lastSegment" problem exists for some time now, this is definitely not related.
Also, I see this too.

Since we have this after we relaunch the Sms app, it means we get the message in a "sending" state from gecko, indicating a gecko problem. Changing the component to reflect this.
Component: Gaia::SMS → DOM: Device Interfaces
Product: Boot2Gecko → Core
Note that it also works sometimes. The first step could be to have a reproducible test case ?
Just did some tests, it works on my Keon, and I see a rmnet1 interface being brought up and correctly configured.
It seems to correlate what I was suspecting earlier:

See the commit at, https://www.codeaurora.org/gitweb/quic/qrd-android/?p=platform/system/core.git;a=commitdiff;h=74510e75;hp=4b132448eed28fd42f1118ac9efb6a8121e67d8b it changes permissions of "net.rmnet0." devices to "net.rmnet", i.e., if the RIL tries to configure a rmnet1, in the first case it would not have the necessary permissions, and in the other case, it will.
On Nexus S, and presumably the reporter's device, there is this code: https://android.googlesource.com/platform/system/core/+/android-4.0.4_r2.1/init/property_service.c
Flags: needinfo?(dietrich)
(In reply to Alexandre LISSY :gerard-majax from comment #9)
> On Nexus S, and presumably the reporter's device, there is this code:
> https://android.googlesource.com/platform/system/core/+/android-4.0.4_r2.1/
> init/property_service.c

Dietrich, what's your device ?
What I've been able to understand for now, once this "net.rmnet0." thing is addressed:
 - the rmnet1 interfaces comes up correctly
 - rmnet0 is used to transport the mms payload

tcpdump ran on my Nexus S shows that routing is somehow wrong and thus the packets that should get through rmnet1 gets via rmnet0.
FYI, I've tried a spurious hack, forcing a route to the MMSC on the rmnet1 interface, and that results in the MmsService's XHR send to get a HTTP 500 error instead of the HTTP 403 error I was previously having (which is symptomatic of not being on the correct network).
Looks like the HTTP 500 errors are due to the missing patch that has been landed on master in bug 873145
Depends on: 873145
Summary: [MMS] send mms with image fails, spins forever → [MMS] MMSC routed through the wrong network interface
Flags: needinfo?(dietrich)
my device was Unagi + AT&T.
I'm wondering exactly what happens, I've just been able to send a MMS with patch from bug 873145 without having to hack the network routes.
Alex, maybe you didn't have 3G ?
Maybe, but I think I saw the rmnet1 interface being used. One difference, though, is that I was roaming on Orange's network while during my tests in the office I was on Free's network.
Same routing error when no 3G is enabled, and WiFi is enabled: MMS is being sent through WiFi, which obviously fails.
(In reply to Alexandre LISSY :gerard-majax from comment #18)
> Same routing error when no 3G is enabled, and WiFi is enabled: MMS is being
> sent through WiFi, which obviously fails.

and FYI it happens both on Nexus S (gecko b12g18) and Keon (gecko master).
After testing different things since a couple of hours, I only see two solutions here:
 - when sending a MMS, change default route to use the APN for MMS
 - when sending a MMS, add a route for the MMSC through the APN for MMS

The second case means that we need to perform DNS resolution by ourselves before asking libnetutils through network worker to add a route. The first case will obviously work, but seems way too much invasive for me.
Flags: needinfo?(vyang)
Attached patch Adding a route for MMSC through APN interface (obsolete) (deleted) — Splinter Review
Please find attached a patch that adds a route for the MMSC through the APN interface. This allows to make MMS sendable.

Since I'm not sure it's the correct solution, this is not a final patch, so don't worry for the lack of bug number etc. However, if you judge this is a correct solution, then I'll commit this properly.
Attachment #755070 - Flags: review?(vyang)
Flags: needinfo?(vyang)
Well I should have seen this coming, but of course, synchronous call to resolve the name will have some issues when we have not yet any data connectivity ...
(In reply to Alexandre LISSY :gerard-majax from comment #2)
> (In reply to Dietrich Ayala (:dietrich) from comment #0)
> > STR
> > 1. compose message
> > 2. add single contact
> > 3. attach image
> > 4. hit send
> > 
> > Actual: message sits forever with spinner next to it, until i blow away gaia
> > profile.
> > 
> > adb logcat error when i hit send
> > 
> > E/GeckoConsole(  108): [JavaScript Error: "lastSegment is undefined" {file:
> > "jar:file:///system/b2g/omni.ja!/components/RadioInterfaceLayer.js" line:
> > 2544}]
> > E/GeckoConsole(  108): [JavaScript Error: "lastSegment is undefined" {file:
> > "jar:file:///system/b2g/omni.ja!/components/RadioInterfaceLayer.js" line:
> > 2544}]
> 
> I don't think it's related. Actually, I'm also having this issue on my Nexus
> S, and I've noticed reports of the radio/ril code trying to setup a rmnet1
> interface and failing due to missing rights.

That's bug 872219.  If you consider that's necessary for v1.1, please go nominating it as leo+.

> When I compare to my HTC Desire Z under Android 2.3.3 trying to send MMS,
> the rmnet1 does not popup, and instead rmnet0 is reconfigured for the mms
> APN.
> 
> I'll look into it today, since I think I reproduce it easily.

I think I'll need more information for this bug.  The reason we have only rmnet0 might come from shared APN of data/mms connections.  What's the APN config you have now?

(In reply to Alexandre LISSY :gerard-majax from comment #15)
> I'm wondering exactly what happens, I've just been able to send a MMS with
> patch from bug 873145 without having to hack the network routes.

Bug 873145 is to remove a redundant header field in send request.  This extra field is parsed by some carriers and somehow results in format error and fails the transaction.  So it doesn't necessarily affect you.
(In reply to Vicamo Yang [:vicamo][:vyang] from comment #23)

> > When I compare to my HTC Desire Z under Android 2.3.3 trying to send MMS,
> > the rmnet1 does not popup, and instead rmnet0 is reconfigured for the mms
> > APN.
> > 
> > I'll look into it today, since I think I reproduce it easily.
> 
> I think I'll need more information for this bug.  The reason we have only
> rmnet0 might come from shared APN of data/mms connections.  What's the APN
> config you have now?

Sorry if I was not clear, on Firefox OS, we have both interfaces that are up at the same time, while on my Desire Z, I have only one.

> 
> (In reply to Alexandre LISSY :gerard-majax from comment #15)
> > I'm wondering exactly what happens, I've just been able to send a MMS with
> > patch from bug 873145 without having to hack the network routes.
> 
> Bug 873145 is to remove a redundant header field in send request.  This
> extra field is parsed by some carriers and somehow results in format error
> and fails the transaction.  So it doesn't necessarily affect you.

Actually, it does :)
Shared interface(one rmnet0) or independant interface(rmnet0 & rmnet1) is operator dependant.

Please see APN database in https://github.com/mozilla-b2g/gaia/blob/master/shared/resources/apn.json to find the APN setting for the operator been tested.

Is there MMS proxy setting for the operator in APN database?

When establishing data call for MMS APN, it adds MMS proxy as host route for MMS interface, and all MMS messages go through MMS proxy.  Could the problem be missing MMS proxy setting?
Second version, which as far as I've tested works well when:
 - no data (no wifi nor 3G enabled)
 - WiFi enabled, or 3G enabled

Vicamo, if I was not clear enough previously, shortly the issue is that when we try sending a MMS, network stack might already have default route configured, thus we end up using them to contact the MMSC. If we contact the MMSC through WiFi or Data connection instead of the network interface that is linked to the MMS APN, then it will fail (my MMSC replies HTTP/403 in this case). The purpose of the present patch is to ensure that, in case we can do DNS successful resolution against the MMSC, then we will add a more specific route for the MMSc through the network interface of the MMS APN.

If we don't have successfull DNS resolution, then it means that either there is not yet any network route available (i.e. we have no WiFi nor 3G enabled) in which case the default route that will be setup will be through the MMS APN. Or it means network is broken :), in which case the message should at some time be marked as error.
Attachment #755070 - Attachment is obsolete: true
Attachment #755070 - Flags: review?(vyang)
Attachment #755261 - Flags: review?(vyang)
(In reply to Shian-Yow Wu from comment #25)
> Shared interface(one rmnet0) or independant interface(rmnet0 & rmnet1) is
> operator dependant.
> 
> Please see APN database in
> https://github.com/mozilla-b2g/gaia/blob/master/shared/resources/apn.json to
> find the APN setting for the operator been tested.
> 
> Is there MMS proxy setting for the operator in APN database?
> 
> When establishing data call for MMS APN, it adds MMS proxy as host route for
> MMS interface, and all MMS messages go through MMS proxy.  Could the problem
> be missing MMS proxy setting?

Yes, I saw this part, adding mmsproxy route. The network I'm using for testing is defined as this:
{"carrier":"Free","apn":"free","type":["default","supl"]},
{"carrier":"Free MMS","mmsc":"http://mms.free.fr","apn":"mmsfree","type":["mms"]}

Which seems consistent with what I have on my Android device. No proxy in both case.

Carrier's configuration is also provided here https://mobile.free.fr/assistance/250.html, it's in french but it should not be an issue:

    Données cellulaires Internet (DATA) 
    - Nom du point d'accès (APN) : free
    - MCC : 208
    - MNC : 15
    - MMSC (si disponible) : http://mms.free.fr 
    MMS
    - Nom du point d'accès (APN) : mmsfree
    - MMSC : http://mms.free.fr
    - MCC : 208
    - MNC : 15
    - Type d'APN : mms
    Activer l'itinérance des données (si option disponible) -> Enable roaming
Dietrich, can you try with my patch ?
Flags: needinfo?(dietrich)
(In reply to Alexandre LISSY :gerard-majax from comment #27)
> Yes, I saw this part, adding mmsproxy route. The network I'm using for
> testing is defined as this:
> {"carrier":"Free","apn":"free","type":["default","supl"]},
> {"carrier":"Free
> MMS","mmsc":"http://mms.free.fr","apn":"mmsfree","type":["mms"]}
> 
> Which seems consistent with what I have on my Android device. No proxy in
> both case.
> 

Yes, there are a few MMS APNs in the database missing MMS proxy setting.  When MMS proxy missing, resolving MMSC's IP address and adding it to routing table of MMS interface seems a nice solution!

IIRC, the Android 2.3 phone supports only one simultaneous APN.  So it will deactivate previous established data connection when sending MMS, which is similar to solution 1 you proposed in comment 20.
(In reply to Shian-Yow Wu from comment #29)
> (In reply to Alexandre LISSY :gerard-majax from comment #27)
> > Yes, I saw this part, adding mmsproxy route. The network I'm using for
> > testing is defined as this:
> > {"carrier":"Free","apn":"free","type":["default","supl"]},
> > {"carrier":"Free
> > MMS","mmsc":"http://mms.free.fr","apn":"mmsfree","type":["mms"]}
> > 
> > Which seems consistent with what I have on my Android device. No proxy in
> > both case.
> > 
> 
> Yes, there are a few MMS APNs in the database missing MMS proxy setting. 
> When MMS proxy missing, resolving MMSC's IP address and adding it to routing
> table of MMS interface seems a nice solution!

In my case, this is not a missing proxy, the carrier has no proxy for the MMSC/MMS APN :)
Comment on attachment 755261 [details] [diff] [review]
Adding a route for MMSC through APN interface v2

Review of attachment 755261 [details] [diff] [review]:
-----------------------------------------------------------------

The thing you're going to do here is to resolve mmsc address and add a host route for it.  However, at the time "network-interface-registered" is emitted, the default route and default DNS may and may not set to the ones come from rmnet1, they're always set to active ones.  So actually there is still risk when mmsc is only resolvable through DNS servers come from ril data connection and we're connected to WiFi.  Well, this could be a corner case and is beyond the scope of this patch.  We're planning some architecture adjustments to NetworkManager and related components.  Hope we can clean all these TODOs by the time.

::: dom/system/gonk/NetworkManager.js
@@ +133,5 @@
>  function NetworkManager() {
>    this.networkInterfaces = {};
>    Services.obs.addObserver(this, TOPIC_INTERFACE_STATE_CHANGED, true);
> +  Services.obs.addObserver(this, TOPIC_INTERFACE_REGISTERED, true);
> +  Services.obs.addObserver(this, TOPIC_INTERFACE_UNREGISTERED, true);

It's also a pity Android libnetutils::ifc_act_on_route() calls getaddrinfo() with AI_NUMERICHOST flag, which follows we can't pass string based destination to ifc_add_route() to create host routes.  However, we have already {add,remove}HostRoute() for manipulating host routes.  Please turn the two functions to accept (ifname, gateway, hosts[]) instead.

@@ +548,5 @@
> +    let retval = [];
> +
> +    if (type != Ci.nsINetworkInterface.NETWORK_TYPE_MOBILE_MMS) {
> +      return retval;
> +    }

That's already checked above.

@@ +560,5 @@
> +    if (!mmscHost) {
> +      return retval;
> +    }
> +
> +    let mmscIps = gDNSService.resolve(mmscHost, 0);

That's a blocking call, and I can't come out a plan to use asyncResolve() after staring into the code for two hours :( Please file a bug for replacing it and place a TODO comment here.

@@ +569,5 @@
> +
> +    return retval;
> +  },
> +
> +  addMMSCRoute: function addMMSCRoute(network) {

Actually I'd rather name it "addHostRouteWithResolve" and move mmsc address stuff out of this function.

@@ +572,5 @@
> +
> +  addMMSCRoute: function addMMSCRoute(network) {
> +    debug("Going to add MMSC route on " + network.name);
> +    let options = {
> +      cmd: "addMMSCRoute",

Please accommodate addHostRoute to accept arbitrary number of destination hosts instead.

@@ +580,5 @@
> +    };
> +    this.worker.postMessage(options);
> +  },
> +
> +  removeMMSCRoute: function removeMMSCRoute(network) {

same here.

@@ +583,5 @@
> +
> +  removeMMSCRoute: function removeMMSCRoute(network) {
> +    debug("Going to remove MMSC route on " + network.name);
> +    let options = {
> +      cmd: "removeMMSCRoute",

ditto

::: dom/system/gonk/net_worker.js
@@ +207,5 @@
>    libnetutils.ifc_add_route(options.ifname, options.httpproxy, 32, options.gateway);
>    libnetutils.ifc_add_route(options.ifname, options.mmsproxy, 32, options.gateway);
>  }
>  
> +function addMMSCRoute(options) {

ditto

@@ +225,5 @@
>    libnetutils.ifc_remove_route(options.ifname, options.httpproxy, 32, options.gateway);
>    libnetutils.ifc_remove_route(options.ifname, options.mmsproxy, 32, options.gateway);
>  }
>  
> +function removeMMSCRoute(options) {

ditto
Attachment #755261 - Flags: review?(vyang)
There are also some carriers in current APN database have MMS proxy using hostname.
For example:

  "20": [
    {"carrier":"3","apn":"three.co.uk","mmsc":"http://mms.um.three.co.uk:10021/mmsc","mmsproxy":"mms.three.co.uk","mmsport":"8799","type":["default","supl","mms"]}
  ],

If we do DNS resolve for MMSC, the MMS proxy should be considered as well.
(In reply to Vicamo Yang [:vicamo][:vyang] from comment #31)
> Comment on attachment 755261 [details] [diff] [review]
> Adding a route for MMSC through APN interface v2
> 
> Review of attachment 755261 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> The thing you're going to do here is to resolve mmsc address and add a host
> route for it.  However, at the time "network-interface-registered" is
> emitted, the default route and default DNS may and may not set to the ones
> come from rmnet1, they're always set to active ones.  So actually there is
> still risk when mmsc is only resolvable through DNS servers come from ril
> data connection and we're connected to WiFi.  Well, this could be a corner
> case and is beyond the scope of this patch.  We're planning some
> architecture adjustments to NetworkManager and related components.  Hope we
> can clean all these TODOs by the time.
> 
> ::: dom/system/gonk/NetworkManager.js
> @@ +133,5 @@
> >  function NetworkManager() {
> >    this.networkInterfaces = {};
> >    Services.obs.addObserver(this, TOPIC_INTERFACE_STATE_CHANGED, true);
> > +  Services.obs.addObserver(this, TOPIC_INTERFACE_REGISTERED, true);
> > +  Services.obs.addObserver(this, TOPIC_INTERFACE_UNREGISTERED, true);
> 
> It's also a pity Android libnetutils::ifc_act_on_route() calls getaddrinfo()
> with AI_NUMERICHOST flag, which follows we can't pass string based
> destination to ifc_add_route() to create host routes.  However, we have
> already {add,remove}HostRoute() for manipulating host routes.  Please turn
> the two functions to accept (ifname, gateway, hosts[]) instead.
> 
> @@ +548,5 @@
> > +    let retval = [];
> > +
> > +    if (type != Ci.nsINetworkInterface.NETWORK_TYPE_MOBILE_MMS) {
> > +      return retval;
> > +    }
> 
> That's already checked above.
> 
> @@ +560,5 @@
> > +    if (!mmscHost) {
> > +      return retval;
> > +    }
> > +
> > +    let mmscIps = gDNSService.resolve(mmscHost, 0);
> 
> That's a blocking call, and I can't come out a plan to use asyncResolve()
> after staring into the code for two hours :( Please file a bug for replacing
> it and place a TODO comment here.
> 
> @@ +569,5 @@
> > +
> > +    return retval;
> > +  },
> > +
> > +  addMMSCRoute: function addMMSCRoute(network) {
> 
> Actually I'd rather name it "addHostRouteWithResolve" and move mmsc address
> stuff out of this function.
> 
> @@ +572,5 @@
> > +
> > +  addMMSCRoute: function addMMSCRoute(network) {
> > +    debug("Going to add MMSC route on " + network.name);
> > +    let options = {
> > +      cmd: "addMMSCRoute",
> 
> Please accommodate addHostRoute to accept arbitrary number of destination
> hosts instead.
> 
> @@ +580,5 @@
> > +    };
> > +    this.worker.postMessage(options);
> > +  },
> > +
> > +  removeMMSCRoute: function removeMMSCRoute(network) {
> 
> same here.
> 
> @@ +583,5 @@
> > +
> > +  removeMMSCRoute: function removeMMSCRoute(network) {
> > +    debug("Going to remove MMSC route on " + network.name);
> > +    let options = {
> > +      cmd: "removeMMSCRoute",
> 
> ditto
> 
> ::: dom/system/gonk/net_worker.js
> @@ +207,5 @@
> >    libnetutils.ifc_add_route(options.ifname, options.httpproxy, 32, options.gateway);
> >    libnetutils.ifc_add_route(options.ifname, options.mmsproxy, 32, options.gateway);
> >  }
> >  
> > +function addMMSCRoute(options) {
> 
> ditto
> 
> @@ +225,5 @@
> >    libnetutils.ifc_remove_route(options.ifname, options.httpproxy, 32, options.gateway);
> >    libnetutils.ifc_remove_route(options.ifname, options.mmsproxy, 32, options.gateway);
> >  }
> >  
> > +function removeMMSCRoute(options) {
> 
> ditto

Thanks for your inputs.

However, in my first version, I was doing exactly what you say: setting route for MMSC in the addHostRoute. And I experienced the corner case you fear about "network-interface-registered": since there might not have been any DNS server available, and due to the blocking call, it was not working at all. Hence justifying the addMMSCRoute() stuff.

I don't know the implications however of the blocking call in the network worker.
So I will rename addMMSCRoute to AddHostRouteWithResolve, which will call AddHostRoute() after having resolved both mmsc and/or mmsproxy addresses.
Please find a new version of the patch.
Attachment #755261 - Attachment is obsolete: true
Attachment #755836 - Flags: review?(vyang)
Comment on attachment 755836 [details] [diff] [review]
Adding a route for MMSC/MMS Proxy through APN interface v3

Review of attachment 755836 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/system/gonk/NetworkManager.js
@@ +549,5 @@
> +      let hostname = hosts[i].split('/')[2];
> +      if (!hostname) {
> +        return retval;
> +      }
> +  

nit: trailing ws.

@@ +568,5 @@
> +      ifname: network.name,
> +      gateway: network.gateway,
> +      hostnames: this.hostToIP(
> +        [ Services.prefs.getCharPref("ril.mms.mmsproxy"),
> +        Services.prefs.getCharPref("ril.mms.mmsc") ])

Don't hard code MMS specific stuff in these functions.

@@ +573,5 @@
> +    };
> +    this.worker.postMessage(options);
> +  },
> +
> +  removeHostRouteWithResolve: function removeMMSCRoute(network) {

nit: s/removeMMSCRoute/removeHostRouteWithResolve/

::: dom/system/gonk/net_worker.js
@@ +203,5 @@
>  function addHostRoute(options) {
> +  debug("addHostRoute() " + JSON.stringify(options));
> +  if (options.dns1) {
> +    libnetutils.ifc_add_route(options.ifname, options.dns1, 32, options.gateway);
> +  }

Please make {add,remove}HostRoute{,WithRoute} accept (ifname, gw, dest[]) instead.
Attachment #755836 - Flags: review?(vyang)
Should address the previous comments
Attachment #755836 - Attachment is obsolete: true
Attachment #755886 - Flags: review?(vyang)
Comment on attachment 755886 [details] [diff] [review]
Adding a route for MMSC/MMS Proxy through APN interface v4

Review of attachment 755886 [details] [diff] [review]:
-----------------------------------------------------------------

Thank you :)
Attachment #755886 - Flags: review?(vyang) → review+
I am not going to be able to apply and test this patch. Here's some more information though:

* looks like when wifi is on, send/receive doesn't work. not 100% sure about this though.

* when wifi is off, and Edge on, send/receive works.

* when wifi is off, and GPRS on, send/receive fails.

* AT&T US default APN is incorrect for MMS, but works for data. once i manually change to both data and message APN settings to new correct APN, can send MMS over Edge.
Flags: needinfo?(dietrich)
Ready to be merged patch, no change apart from debug message fixes
Attachment #755886 - Attachment is obsolete: true
Attachment #756438 - Flags: review+
https://hg.mozilla.org/mozilla-central/rev/df933307c25d
Status: NEW → RESOLVED
Closed: 11 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla24
Blocks: 879225
No longer depends on: 879225
Flags: in-moztrap?
Flags: in-moztrap?
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: