Phone: Samsung Galaxy S II Build: 6 April SMS'd someone using 10 digit phone number. Their reply came through with "+1" prepended. Treated as a different thread. That's confusing.

We haven't decided yet where to tackle phone number normalization. Probably in the UI (Gaia), in which case this bug would be WONTFIX or something.

I think we should do this in the platform. It's going to be extremely difficult to get this right, and the amount of information we need to expose to allow content normalize correctly will likely be high. If there's any use case for exposing the non-normalized number I would recommend we do that separately, and default to normalized. I guess that's a good discussion for dev-webapi.

To normalize a phone number, it may involve telephone numbering plans[1]. The most direct requirement for phone number normalization might be county code detection/retrieval. Android[2] supports several ways: GSM network, GPS location, SIM, and locale settings. For GSM network, we can get country code from MCC, which is already available in current RIL imp. There might be also another problem in matching contact phone numbers with the stored, normalized one. Android matches each message, sent or received or ..., to store with a contact before storing it in database. The stored number in Android is the phone number of the contact, so it's free from this problem. However, if you change the number of a contact from a local dailling plan to a normalized international one, your app still have to match each message with existing contacts for the stored number is different now. The MMS application in Android maintains its own contact cache for these queries. BTW, Android also creates/matches thread id for all messages, and B2G has only per message id. [1]: http://en.wikipedia.org/wiki/Telephone_numbering_plan [2]: frameworks/base/services/java/com/android/server/location/ComprehensiveCountryDetector.java

Nice to have. We wouldn't hold V1 for this, so blocking-.

(In reply to Dietrich Ayala (:dietrich) from comment #4) > Nice to have. We wouldn't hold V1 for this, so blocking-. I disagree, I think this would be a really big problem in day-to-day use for anyone using a B2G phone.

I'm still not convinced we need to do this at the Gecko level, though. With enough SIM, network provider, GPS, and contacts info exposed to content, apps can easily do this on their own. In fact, I would say we've exposed pretty much all the necessary bits already or are about to. The best part about shipping this in Gaia is that we can have much tighter bugfix/update cycles and can bring up new locales or whatever much more quickly.

I think it would be pretty hard to do this in gaia, and you're pushing the burden onto every app that has to deal with phone numbers. This feels like something we should do in gecko, to me.

(In reply to ben turner [:bent] from comment #5) > (In reply to Dietrich Ayala (:dietrich) from comment #4) > > Nice to have. We wouldn't hold V1 for this, so blocking-. > > I disagree, I think this would be a really big problem in day-to-day use for > anyone using a B2G phone. I highly agree with Ben, this is a serious issue and we shouldn't ask apps to solve that themselves. I would be actually very worried if my SMS app was requesting access to SIM info, network provider and GPS...

(In reply to Mounir Lamouri (:volkmar) (:mounir) from comment #8) > (In reply to ben turner [:bent] from comment #5) > > (In reply to Dietrich Ayala (:dietrich) from comment #4) > > I disagree, I think this would be a really big problem in day-to-day use for > > anyone using a B2G phone. > > I highly agree with Ben, this is a serious issue and we shouldn't ask apps > to solve that themselves. I would be actually very worried if my SMS app was > requesting access to SIM info, network provider and GPS... Ok. If we're going to mark this issue as to-be-solved-in-gecko, then we must have some solution first. Let's talk about solution. My major problem is the following use cases: step 1.) store received message with a normalized/unnormalized number. step 2.) change the corresponding contact's number to a totally different one step 3.) open messaging app. What should happen? step 4.) another message received with previous/current number of the contact. What should happen? step 5.) just change the number from a local numbering plan to an international one. step 6.) Same question with step 3. step 7.) Same question with step 4. For step 3, Android still relates those previously related messages, but can't recognize the sender. By changing the number back, those messages are still threaded and with sender contact icon shown. This means an additional `threadId` attribute for each message and in-app contact resolution are required. For step 4, Android can't relate messages from new/old numbers. Newly received messages are in a different thread. Step 6, 7, Android can relate previously received messages as well as newly received ones. This means in-app contact resolution is required. There is no doubt that `threadId` can be done in gecko. The problem is, how to move Android's in-app contact resolution into gecko as well? Should SmsDatabaseService listen changed event of contact service and modify fields accordingly? Should the `threadId` attribute have some kind of relation to some unique id of each contact? What happens when a user swaps the numbers of his two contacts?

FYI: Android uses this library to deal with the phone numbers http://code.google.com/p/libphonenumber/

Here is a phone number analysis tool: https://www.numberingplans.com/?page=analysis&sub=phonenr You can submit your mobile phone number and see if it can successfully decrypt it. I got "This telephone number is most likely an India number" with a Taiwan mobile phone number 0910xxxxxx.

(In reply to Dietrich Ayala (:dietrich) from comment #4) > Nice to have. We wouldn't hold V1 for this, so blocking-. Based on subsequent comments, would you mind reconsidering this decision, Dietrich?

Re-requesting, based on previous comments.

In the absence of a proper fix, is there something we can do that would help with issues specific to Brazil?

When does this happen? Does it only happen when receiving text messages from people out-of-country? If this only happens in rare cases, then I would say it sucks, but wouldn't stop us from shipping. Also, I do *not* think we should use heuristics to guess where a specific unprefixed number is coming from. We need something more reliable than that. Possibly we could match after stipping country code.

(In reply to Jonas Sicking (:sicking) from comment #15) > When does this happen? Does it only happen when receiving text messages from > people out-of-country? > No. I send a message to my friend, 555-5555. My SMS program saves an outgoing message to 555-5555, and shows it in a conversation view for 555-5555. This number isn't in canonical form but the network knows how to deliver it. My friend replies. The network sends the number in canonical form, +15555555. My SMS program records an incoming message from +15555555. This opens a conversation view with +15555555. Now I have two different conversations that are actually with the same person. It's certainly possible to hard-code the canonical form for $TARGET_MARKET. But then you've hard coded that logic in every app that has to deal with numbers (dialer, sms, contacts, ...). And forget handling international numbers.

I think not solving this has serious usability issues

Dietrich, can we get this blocking-basecamp+'d?

I'll make sure we discuss this in the triage meeting.

Ok, if this affects all messages then I agree it's a problem. I'd really like to avoid using heuristics though. Can we have a setting (using the setting API) for which country code is the "default" country code. That way the SMS and Telephony APIs can strip that code from any incoming messages that have that country code. Or add that country code to all messages which doesn't have a country code, and then strip the code in the UI of the SMS/telephony apps? Can we even figure out which country code we are in using the RIL?

(In reply to Jonas Sicking (:sicking) from comment #20) > That way the SMS and Telephony APIs can strip that code from any incoming > messages that have that country code. Or add that country code to all > messages which doesn't have a country code, and then strip the code in the > UI of the SMS/telephony apps? That sounds good to me. This means normalizing in the RIL / SMS DB when we process an incoming or outgoing message by adding the country prefix. Don't even have to strip in the UI IMHO. > Can we even figure out which country code we are in using the RIL? Yup, we expose the MCC (mobile country code) via an XPCOM API and even to content via navigator.mozMobileConnection.voice.operator.mcc (IIRC).

(In reply to Philipp von Weitershausen [:philikon] from comment #21) > (In reply to Jonas Sicking (:sicking) from comment #20) > > That way the SMS and Telephony APIs can strip that code from any incoming > > messages that have that country code. Or add that country code to all > > messages which doesn't have a country code, and then strip the code in the > > UI of the SMS/telephony apps? > > That sounds good to me. This means normalizing in the RIL / SMS DB when we > process an incoming or outgoing message by adding the country prefix. Don't > even have to strip in the UI IMHO. A simple usecase: Mary, came to Taiwan the day before, wants to text one of her friends in US. The number saved in contacts database doesn't contains country code. Then, what country code should be used in saving those sent message? The one from MCC? Of course not. A "default" country code for US? What if she want to text her Taiwanese friends? > > Can we even figure out which country code we are in using the RIL? > > Yup, we expose the MCC (mobile country code) via an XPCOM API and even to > content via navigator.mozMobileConnection.voice.operator.mcc (IIRC).

(In reply to Vicamo Yang [:vicamo] from comment #22) > (In reply to Philipp von Weitershausen [:philikon] from comment #21) > > (In reply to Jonas Sicking (:sicking) from comment #20) > > > That way the SMS and Telephony APIs can strip that code from any incoming > > > messages that have that country code. Or add that country code to all > > > messages which doesn't have a country code, and then strip the code in the > > > UI of the SMS/telephony apps? > > > > That sounds good to me. This means normalizing in the RIL / SMS DB when we > > process an incoming or outgoing message by adding the country prefix. Don't > > even have to strip in the UI IMHO. > > A simple usecase: Mary, came to Taiwan the day before, wants to text one of > her friends in US. The number saved in contacts database doesn't contains > country code. That's a common issue that I doubt can be solved easily. We can use heuristic to fix that but as a user, I wouldn't except the phone to fix that for me, I will just add the country code to the number. > Then, what country code should be used in saving those sent message? If the number in contact list is 555-555-5555, the message will be sent to 555-555-5555 and if the sent is successful, we will prefix the number with the country code. So if the sending happens to be successful in Taiwan, it will be sent to a Taiwanese number. However, I see two issues with prefixing the number with the country code when saving the sent message: - some messages aren't prefixed with the country numbers. I don't know if that's common everywhere but there are some short numbers in France that are used for various stuff like operators information. If I text 1234, I might not want this to be saved as +33 1234. I have no idea if that would work actually. - some countries might have different ways to prefix numbers. IIRC, the US only add +1 in front of the number. In France, you have to remove the leading 0 and add +33.

(In reply to Mounir Lamouri (:mounir) from comment #23) > (In reply to Vicamo Yang [:vicamo] from comment #22) > > (In reply to Philipp von Weitershausen [:philikon] from comment #21) > > > (In reply to Jonas Sicking (:sicking) from comment #20) > > > > That way the SMS and Telephony APIs can strip that code from any incoming > > > > messages that have that country code. Or add that country code to all > > > > messages which doesn't have a country code, and then strip the code in the > > > > UI of the SMS/telephony apps? > > > > > > That sounds good to me. This means normalizing in the RIL / SMS DB when we > > > process an incoming or outgoing message by adding the country prefix. Don't > > > even have to strip in the UI IMHO. > > > > A simple usecase: Mary, came to Taiwan the day before, wants to text one of > > her friends in US. The number saved in contacts database doesn't contains > > country code. > > That's a common issue that I doubt can be solved easily. We can use > heuristic to fix that but as a user, I wouldn't except the phone to fix that > for me, I will just add the country code to the number. True. Simply feel saving some kind of default country code along with the outgoing number doesn't help in some cases. > > Then, what country code should be used in saving those sent message? > > If the number in contact list is 555-555-5555, the message will be sent to > 555-555-5555 and if the sent is successful, we will prefix the number with > the country code. "successful" is a ambiguous term in networking. For SMS, an immediate "successful" response from rild means the message is accepted by SMSC. It doesn't guarantee a successful delivery. The number might be a valid number form, but still results in destination unreachable error in the end. Then saving it with the country code just doesn't help. > So if the sending happens to be successful in Taiwan, it will be sent to a Taiwanese > number. > > However, I see two issues with prefixing the number with the country code > when saving the sent message: > - some messages aren't prefixed with the country numbers. I don't know if > that's common everywhere but there are some short numbers in France that are > used for various stuff like operators information. If I text 1234, I might > not want this to be saved as +33 1234. I have no idea if that would work > actually. Same in Taiwan. When you're going to call for Taxi, you may dial 55688. And it fails if prefixed with +886. > - some countries might have different ways to prefix numbers. IIRC, the US > only add +1 in front of the number. In France, you have to remove the > leading 0 and add +33. Same in Taiwan for mobile numbers. Anyway, the Android solve the problem (for SMS) with following steps: [Sending] 1. send message with exactly the number typed or that from contacts database. 2. store sent message without change, normalization, country code prefix, etc. [Receiving] 1. try to match the number carried by the incoming message with all contacts. If matched, store the message with the number from contacts database. Or, store it as original one. [Application] 1. try matching all contacts yet again on loading messages from database. No country code is stored in SMS database. In this way, the numbers stored almost always match one contact at most, no matter it's prefixed with country code or not.

How are we doing here?

Renom if you think we can't ship a v1 without this.

Per IRC conversations with a few other folks, I think the best course of action if there is disagreement on whether this blocks or not is to do the following: - Move the blocking-basecamp flag to ? for re-evaluation - Indicate a rationale for why you disagree

(In reply to Andreas Gal :gal from comment #26) > How are we doing here? I'm still working on bug 780558 and bug 778093 Cell Broadcast. No much progress by now.

Philikon suggests we can fix this at the Gaia level if necessary. Strong nice to have, but not a blocker.

Filed Gaia issue for this, since no platform fix for V1: https://github.com/mozilla-b2g/gaia/issues/4528

Gecko had finally solved this in bug 833291.