We'll need to find a set of good search engines for India, in particular Hindi. Same for RSS readers, and do we have a good news feed for India? Is http://www.bbc.co.uk/hindi/ any good? Rajesh, any suggestions? Mic, I guess there should be more people on CC here, can you find out who?

As far as I know google gives best result for all langs and particularly for Hindi. You are right :) http://www.bbc.co.uk/hindi/ is ths best news feed for India in Hindi.

hi working on cc list for others to comment, will post something shortly mic

Seth, this is the bug for the search and feeds changes for the Hindi localization. Could you CC the people you wanted to comment here, too?

Axel, the people who are doing research on this are management school students. Once we get deeper into the project, which should be sometime in late August '07, I will introduce them to the bug tracking system.

Putting this bug on Mic's radar a bit more prominently.

I am not able to attach the report as it is 522 KB and the maximum attachment allowed is 300 KB. I have mailed the report to Seth.The references of page numbers given below are from that report. Request Seth to please upload it. The document attached gives the top of the mind recall and the most used web portals by the Indian user 1)For the news feed, as mentioned on Page 11 , Times of India is the favorite news paper for the internet users , followed by Hindu. The top news websites is indiatimes, ndtv and rediff( Page 10) . The feeds of these can be added 2)India Today is the most preferred magazine of the internet users ( Page 11) 3)Preferred websites for top financial Info are Money Control &Yahoo ( Page 10) 4)Google leaves everyone far behind in information search (Page 9) 5)Top online shopping websites are Ebay, Rediff and Indiatimes(Page 3) Thanks Vijayendra Haryal Team Lead,Team 21 IIM Ahmedabad

Things mentioned in comment #6 are mostly of English language,but this bug wants it for Hindi. I also want to read the full report.

There are no clear winners in Hindi as of now. In a survey that was conducted, 18% of the net users highlight Hindi as their preferred language of reading, 17% also indicate that they visit Hindi websites already.This phenomena over language preference is beginning to emerge. We would do more research and let you know but as of now. Chances are bleak that we will find any reliable source of information which will give us clear preferences for Hindi websites. The only way to do this is to conduct some primary research .

Some facts about regional portals- 1)It is still a market in its infancy - industry estimates that out of 20 billion pages globally on the Net, there are only one million Hindi pages (Tamil would have half-a-million). 2)Over 90 per cent of Indian portals are in English. A recent study shows that the top search engines together (includes Google and AOL, MSN and Yahoo) were able to identify over two lakh Hindi website links on the portal bbc.co.uk/Hindi - again reflecting the limited nature of Hindi content available on the Net. 3)Delhi based Indicus Analytics found that the biggest problem for users in Hindi was how to access pages in Hindi.You could not use existing popular search engines, as you could not query in Hindi. The company launched Raftaar.com - a search engine with a soft key, so that you can type in Hindi alphabets (which are on the screen) with a click of the PC button. 4)Microsoft as part of its special initiative launched "Bhasa India" a community portal in various Indian languages which provides users a platform to express their views on regional language computing, attract developers to post queries and even use the knowledge of users to create technical glossary of software and hardware technology. The result: it is getting over 300,000 hits a month and has over 20,000 odd registered users. 5)"Only five per cent of internet advertising goes to regional languages - the rest is all with English portals. Source: Rediff.com

Here is a report on the various search engines in India. Again , there are no clear winners as of now.

Thanks Vijay! I have some different views on the stats shown. Things started changing very fast in last few years and particulary last two years. Earlier there were several technical problems that were barrier in the path of language preference. You just remember the television early days when dominance of english channel existed for few years. But when vernacular language television revolution started, market share for English became very thin. So its hard to believe the statistics. Anyway there is also a saying about stats...lies, damn lies and statistics. What happened in television field already will be replicated on Internet also. Among 10 most circulated newspaper probably one one english newspaper is coming. Please go through the following sites. You can see the changes and inclination of the major player of web. http://in.jagran.yahoo.com/ http://in.hindi.yahoo.com/ http://content.msn.co.in/hindi/Default http://navbharattimes.indiatimes.com/ http://news.google.co.in/news?ned=hi_in You correctly mentioned that the biggest problem for users in Hindi was how to access pages in Hindi. That was a problem of past. So I am telling in past few years things changed entirely after unicode came into the scene. So its was a problem of past that we were unable to query in Hindi in search engine. So we have to think according to new outlook. And again google seems ahead of all. The seach engine you mentioned are of very limited users though these are made for India search.

As far as the statistics go, I already mentioned that secondary sources would not be able to give a clear picture. Things are changing agreed, but the market is still in the infant stage. Both MSN and Yahoo have launched their regional portals in 2006. The results have not been very clear about the success. However, one interesting statistic is that out of the 18% of the net users highlight Hindi as their preferred language of reading, 17% also indicate that they visit Hindi websites already. The trend is good as u rightly mentioned and the potential is there no doubt. Also as far as the search engines are concerned, we mentioned before that there are no clear winners and google is the right choice. The optimism with the regional languages is good, but we strongly recommend a reality check. We did conduct some indepth interviews and found out that an interface in Hindi is important. We would be sharing more insights tomorrow in the meeting with Seth

Online Trends in India, created by student team at Indian Institute of Management, Ahmedabad

(part 2) Online Trends in India, created by student team at Indian Institute of Management, Ahmedabad

Hey everyone reading the posts above and the work Team 21 I'd like some reactions to a proposed list of search engines: Google Yahoo eBay Rediff (?? should we use this, I'm not sure if they have their own search engine built for Hindi, if not Google might be just as good?) Wikipedia Others?? e.g., is there a better local Amazon type provider? I'm not sure about Amazon as it doesn't seem to even register as a site well known or used by users - having said that I'm not sure if this data is based only on english users vs hindi Also what about Answers? I don't see that ranked highly either I look forward to your thoughts

on RSS feeds is BBC sufficient or should it be a local provider that specializes in Hindi language news? thanks

List of search engines are good enough. The reputation of BBC is very good in the Indian audience and generally people of India think that service of BBC is unbiased. But BBC is anable to provide all the remote news that all are coming from villages and small towns of India. So for RSS feeds we should take some sites that have good reach all over Hindi belt. For example, http://in.jagran.yahoo.com/news/

Rajesh/Vijay on RSS Feeds - earlier in the bug Comment #1 it seemed BBC was good but in comment #17 you're suggesting in.jargran.yahoo.com/news. Please advise which is your preference. on Search engine's thanks for your comments Rajesh. Could you offer more specific comments on my questions about Rediff and a local version of Amazon see comment #15 Vijay - any thoughts in reaction to comments #15 or #17? thanks mic

For search Engines, Google should do the needful. For newspapers , here is the trend. I will cover more in a short report later. Top ten list (present readership and decline/growth figures): 1. Dainik Jagran (Hindi) – readership of 1.65 crore (1 crore = 10 million). Decline by 3.5 per cent. 2. Dainik Bhaskar (Hindi)- readership of 1.28 crore. A growth of 2.4 per cent. 3. Malayala Manorama (Malyalam) - readership of 86.62 lakh (one lakh is one hundred thousand) A decline of almost 2 lakhs. 4. Hindustan Times (English) - readership of 85.51 lakh. A decline of 5.5 per cent. 5. Amar Ujala (Hindi) - readership of 80.75 lakh. A decline of 2.1 per cent. 6..Daily Thanthi (Tamil) - readership of 79.1 lakh. A decline of 5.2 per cent. 7. Rajasthan Patrika (Hindi) – readerships of 74.02 lakh. And increase of 6.5 per cent, highest increase amongst the top 10 dailies. 8. Eenadu (Telugu) - readership of 69.19 lakh. A decline 4.34 per cent. 9. Ananda Bazar Patrika (Bengali) - readership of 68.55 lakh. An increase of 4.84 per cent. 10. Times of India (English): readership of 68.28 lakh. An increase of 0.69 per cent. ( 2007 survey: Source Rediff.com) One more thing which we need to look into very seriously is compatibility of fonts . For example , please have a a look at http://www.amarujala.com/today/default.asp it's not able to show the Hindi Font. This is very disturbing. While the same page is opening up in Internet Explorer very well in Hindi. I would request someone from the technical team to please look into it and offer us their comments Based on this our approach will be determined as to which provider to suggest

The trend is that the print readership is declining in India which is similar to the Global Trend.Some points to note are 1)Only ET ( Economic Times), an English Business Newspaper, could maintain it's readership and even grown in the Delhi and Mumbai. 2)Among the top ten listed, six out of ten show declining trend ( including English).Times of India and Anand Bazar Patrika ( No 10 and 9 respectively) were not there in the top 10 last year. They have replaced Lokmat, which is the largest read Marathi daily 3)Amongst all the southern newspapers, Malayalam is doing the best If we take the success of print readership of a news paper as a proxy for the online readership as well, Dainik Jagran and Dainik Bhaskar seem to be the obvious choices

in.jargran.yahoo.com/news is still in Beta...So BBC we can choose. Rediff is okay. I can not say more about local amazon type provider.

We think that, we can rely on print readership as a proxy for online and therefore Dainik Jagran and Dainik Bhaskar seem to be appropriate choices. As Dainik Jagran is still in Beta we may like to go with http://www.bhaskar.com/ Apart from being the second most read daily, it has "localized news" for every major town in the Hindi speaking states and secondly as the Hindi script is compatible with Firefox.

if we decide going by this logic better go in.jargran.yahoo.com/news Jagran has base in two most Hindi speaking populated states ie Uttar Pradesh and Bihar where Dianik Bhaskar has no base.

If we are fine with the Beta version, then in.jargran.yahoo.com/news is the obvious choice as I mentioned before. As far as Dainik Bhaskar is concerned, it is the second most read daily in India.Dainik Bhaskar is a Hindi-language daily newspaper of India. It first started in the state of Madhya Pradesh. Currently it is published from many cities of North India. The Dainik Bhaskar Group has soared to the top of the print media industry with its flagship Hindi daily, Dainik Bhaskar and the Gujarati frontrunner, Divya Bhaskar. Dainik Bhaskar has a large presence in Madhya Pradesh, Rajasthan, Punjab, Haryana, Himachal Pradesh, Chhatisgarh and Uttar Pradesh; while Divya Bhaskaris the largest circulated regional daily in Gujarat.

Hi actually I talked with Chofmann about this to get more clarification. We default ship with BBC as its the best coverage and our rationale was to include it such that the user might go "what's this" and realize that they can add additional RSS feeds into the browser. If BBC provides sufficient coverage in Hindi than we should keep it as is. If BBC does NOT provide sufficient coverage then lets look at other options. Please LMK your thoughts on which we should ship with in light of this further clarification. Thanks mic

BBC won't give the local flavor ( Rajesh, please correct me if I am wrong). By this I mean the localized news of the city won't be there. This is there in case of The hindi dailies which we have mentioned above. The local news of the city is an important differentiator.

I think Dainik Bhaskar is not coming from Uttar Pradesh and also bihar jharkhand etc. So population and area wise jagran is leading Hindi Daily. Yes certainly it is more localized. If for feed we need all major news and views then its BBC that we should choose. If we need more local flavour then jagran is okay. I have a question, whether we are going to adopt this choice forever or we can change from time to time? I am continous travelling these days. So delay in replying. Sorry for this problem.

Back to search for a moment, looking at the stats on wikipedia, the Hindi section there seems to be growing nicely, I'd think we should add it. I guess that rediff is on our agenda, too? I'd like to see a patch adding the two, you can take a sneak peek at what runa did in bug 398992, and the files are at http://mxr.mozilla.org/l10n/source/bn-IN/browser/searchplugins/ (or in cvs). The change from bn to hi in wikipedia should be fairly trivial. As for the feed: We can change that, but I'd rather not do that often. We're essentially changing other peoples bookmarks each time we do that, as the bookmark goes to a redirect on our server, and we just change the redirect. And messing with other peoples profile isn't nice :-) I think that having the jagran feed would give a more 'Indian' touch to the Hindi localization. Given how many folks used the English builds at foss.in, I'm not sure if that's a pro or a contra, though. It might be that the BBC feed for India is actually a good thing for en-IN, and the jagran feed is a good thing for hi-IN. Just guessing, Mic had more discussions on the choice of feeds since her last comment, maybe there's newer insight from her now.

comment #28 thanks Axel - yes there is new guidelines: http://wiki.mozilla.org/Firefox_web_services_guidelines comment #27 the choice can change from time to time for RSS feed best choice is in language and popular for users. from comments #23 and #24 it seems that in.jargran.yahoo.com/news. So let's go with jargran for hi-IN for search please confirm the list is: google.in yahoo.in amazon.in ebay.in wikipedia.in Rediff is only a suggestion IF it provides good search in Hindi language. If not then we can leave it with Google and Yahoo. These two are supposed to provide good Hindi language search for Global results and local results. For RSS Readers: currently the default is Bloglines, My Yahoo, Google Bloglines is NOT available in Hindi so please recommend whether you think it is useful to include (if users can't read english and it's not popular than I would not recommend including it). I'm not sure about My Yahoo or Google. If they are also not available in Hindi let's see if we can get a Hindi reader. Any ideas?

Thank you so much. Its all okay! For RSS Readers: We can skip Bloglines because its not popular here. So better to keep yahoo and google.

Hrm. I don't find no amazon.in, and wikipedia.in should be hi.wikipedia.org, right? google.in should just be google.com, AFAICT.

apologies Axel, there is not Amazon India - right Runa? so it would have to be amazon.com and for wikipedia the url you have is correct. google is also correct mic

Mic, I am Rajesh...not Runa... Seems Bengali work is also in progress. :-) http://india.amazon.com/ is available.

Rajesh :) I'm sorry about that. (yes, I am working on the Bengali build as well) India.Amazon.com - I had seen that site, it seems to be more of a more of a corporate information (for developers) site than a commerce site, so, shall we go back to using the default en-US search plug in for amazon?

For Amazon using default en-US is better. you are right site is having only corporate info only.

Rajesh, can you create a patch to implement the remaining changes for search? Create a cvs diff -uN and attach the output for review before committing to CVS, please? Changes should include: cvs removal of google.xml, that's not needed, it's picked up from en-US at build time. addition of search plugin for hi.wikipedia.org, likely wikipedia-hi.xml. I prefer http://mxr.mozilla.org/l10n/source/lv/browser/searchplugins/wikipedia-en.xml as example. Changes to list.txt, notably adding wikipedia-hi and amazondotcom, so that the build actually picks up the wikipedia plugin that you created, and the amazondotcom one from en-US. The latter works like google.xml, thus there are no changes required for the google in list.txt.

ping?

pike, can you please review it and let me know if something more is needed from my side.

Comment on attachment 296489 [details] [diff] [review] patch for searchplugin Sorry for the lag. r-, two things: You don't need amazon.xml, that's picked up from mozilla/browser/locales/en-US/searchplugins by the build process. The changed to list.txt is right, though. You need to cvs add wikipedia-hi.xml You might want to look at http://mxr.mozilla.org/l10n/source/it/browser/searchplugins/wikipedia-it.xml again, we improved the current ones a bit just in the last few days, most of that is copy and paste. To guide you before, the <ShortName> should be "विकिपीडिया (हिन्दी)", that's what hi.wikipedia.org uses. The template should be either http://hi.wikipedia.org/wiki/Special:Search or the translated one, though I don't even find the translated name mediawiki uses for Search. Looking at other languages, there might not be one, so I'd just use Special:Search, that works. Anyway, then do a cvs -z3 diff -uN The 'N' makes new and removed files show up in the diff, so that I can actually review those.

was not well last week, so will do this week.

Hi Pike, I just commited the changes... Can you please review...

(In reply to comment #41) > Hi Pike, > > I just commited the changes... Can you please review... Hi Rajesh, It's generally best to generate a diff, using |cvs diff -up8N|, and attach that here *before* committing to CVS. Much easier to review that way. I will post comments on the file you've committed to CVS, though.

><SearchPlugin xmlns="http://www.mozilla.org/2006/browser/search/" xmlns:os="http://a9.com/-/spec/opensearch/1.1/"> The "xmlns:os" part isn't needed, just remove it. ><ShortName>Wikipedia (Hindi)</ShortName> ><Description>Hindi Wikipedia</Description> You should use the values from the Wikipedia plugin for these, unless there is some reason not to. This will ensure that users don't get prompted to install the engine again when they visit Wikipedia. <ShortName>विकिपीडिया (हिन्दी)</ShortName> <Description>विकिपीडिया (हिन्दी)</Description> Could you make these changes, then generate a diff using |cvs diff -up8N|, and attach the output here?

Hi Gavin, Can you please review the latest patch ?

Comment on attachment 302989 [details] [diff] [review] searchplugin patch Looks good, thanks!

are all changes complete then? can we close this bug?

Attachment 302989 [details] [diff] didn't land yet. Rajesh, would you land that change, referencing this bug, describing the change, and mentioning gavin's review? Something like "bug 384408, updating wikipedia plugin to match wikipedia's site, r=gavin" would work in this case.

Comment on attachment 302989 [details] [diff] [review] searchplugin patch bug 384408, updating wikipedia plugin to match wikipedia's site, r=gavin

Comment on attachment 302989 [details] [diff] [review] searchplugin patch I'm assuming this was just a mistake...

sorry. I did mistake. I landed the change. please chk if everything is right :)

(In reply to comment #50) > I landed the change. please chk if everything is right :) Yeah, looks good. Thank you!

The title in the search link on hi.wikipedia.org changed, let's match up with that. I switched the suggest Url over to use Params, too, as I was already on it. Rajesh, can you review the patch? And land it, if you like it. Again with a comment referencing this bug, something like "bug 384408, update wikipedia plugin to match site, patch by l10n@mozilla.com, r=me" or so ("me" in this case would mean you, as you land).

Comment on attachment 309075 [details] [diff] [review] update wikipedia plugin to match hi.wikipedia.org Rajesh, you landed the first chunk of this as part of bug 425688, could we still get the suggestions part fixed?

Please chk it for the suggestion part.

Comment on attachment 314311 [details] [diff] [review] searchplugin_suggestionpartdiff r=me, please check in with a good comment again.

committed the same.

http://in.jagran.yahoo.com/news/ doesn't seem to have a feed, AFAICT.

At this page http://in.jagran.yahoo.com/news/national.html we can find feed for different catagories like http://in.jagran.yahoo.com/news/rss/national/politics.xml http://in.jagran.yahoo.com/news/rss/national/crime.xml and similarly for other catagories also. But it seems there is not a single place to get all. better we should go according to the description and comment #1 of this bug :-).

Mic, http://www.bbc.co.uk/hindi/index.xml good for you, too?

per comment #59 and comment #1 - find change. thanks

can we move ahead to make it final and resolve?

i'm ok with that. axel did you already create a it server side bug for this?

Not yet. I'll need to do some source verification before closing this bug, too.

here's what needs to happen to close this bug which Axel will do bloglines needs to be removed from list per comment #30 amazon.com needs to be added again per comment #39 server side change for http://www.bbc.co.uk/hindi/index.xml has happened bug 440802

Here's the patch to fix the remaining issues, gonna land that in a minute.

Checking in searchplugins/list.txt; /l10n/l10n/hi-IN/browser/searchplugins/list.txt,v <-- list.txt new revision: 1.5; previous revision: 1.4 done Checking in chrome/browser-region/region.properties; /l10n/l10n/hi-IN/browser/chrome/browser-region/region.properties,v <-- region.properties new revision: 1.4; previous revision: 1.3 done Marking FIXED with reference http://hg.mozilla.org/users/axel_mozilla.com/l10n-src-verification/index.cgi/file/ccc442125f2c/reference/MOZILLA_1_8_BRANCH/hi-IN/.