User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040614 Firefox/0.9 Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7) Gecko/20040614 Firefox/0.9 Anyone can rate many times the same extension, so it is possible to put any extension in TOP 5 rated extension if someone vote 5 stars many times or it is possible to go down the rating of any extension if someone vote 0 stars some times for this extension. Reproducible: Always Steps to Reproduce: 1. go to http://update.mozilla.org/ 2. choice any extension 3. rate many times

Confirming.. this is a real bug..

Ok, i'm going to broaden the scope of this bug... "Make Comments not Suck" means.. -- Need Pages for the Comments more-info tab. Those lists are getting long quickly. -- Need checks for duplicate commenting/rating. -- Need Moderation ability. It's been proposed that logins should be required to comment, I'm not sure i'm really in favor of this... :-/ Comments?

People won't moderate their own projects' comments (see mozdev) so we'll need assigned moderators. That's going to get ugly fast. If we don't have login-we could do an image identification thing like Yahoo Mail and Hotmail does. And let's not reinvent the wheel with something inferior to what's already out there, there is problably plenty of open-source comment code on sourceforge, and kerz also has his that maybe he'll lend.

Not kerz's server, just his comments code. (Don't want any confusion) ;-) We just need to look around at what other people use. Slashcode is obviously out of the question, but there are lesser comment systems available.

More comment/rating suckage: - Newly added extensions (with 0 ratings) shows up as 5-star and is therefore put on the top of the "top rated" list. They should be probably rated at 2.5 stars by default. - The 5-star system is still a little ambiguous. What does 5 stars mean? It worked perfectly with no flaws seen? What does 2 stars mean? It didn't work except on Tuesday? Perhaps instead of the user picking stars, there sould be some questions that the server then turns into a 1-5 star rating. - The last comment made by an anonymous user, whether a good comment or porn spam, is put on the main information window for the extension. Perhaps some other method should be used to determine which comment to place here.

I noticed that a new extension which is only rated once with 5 stars immediately shows up as the best rated extension. You probably want to build in some threshold of 10 ratings or so

We should convert line breaks in the submitted comments into BR tags. I'm sure I've seen some comments where the user thought the line breaks would show but didn't. Just a nice thing to have.

My own thoughts.... 1. A login system required before rating would be, frankly, a major incentive for me to try out the BugMeNot extension... In order to abuse commenting, with the big extensions having been rated many times, you'd need a script, plenty of free time, and some very bad manners. So how about checking IP addresses? Maybe some basic threshhold- every 5th comment can come from the same IP, no more? 2. Actually, if comments do stay with the extension across versions (not sure), the comment should explicitly include what version the person was trying. Then new releases wouldn't be tarnished with old bug complaints. 3. I second the idea of making rating meanings more consistent. If you wanted to give the database a stretch, you could always break it up into more fields- "install", "functionality", "completeness" (does it do everything it promised?), "overall", etc.

Just commenting on another requested feature, so it doesn't get lost, Word Filter.

My $0.02: Do not allow the "No Comment?" option. This is the easiest way to sabotage another person’s extension rating. Many times I've seen my extensions go from 5 stars to 3, without any comments left. Was somebody really unhappy or did they just want to lower my ranking under their extension? I believe this to be a very important bug to fix. Thanks, Jeremy

It'd be beneficial for somebody to see what version a comment was written about.

Example: Since it was listed on UMO, the Optimoz Mouse Gestures extension's rating always moved around 4.5. In the last few days, it dropped all of a sudden to 3.2. Possible explanations: - our 1.0 release is worse than all previous releases - it works only for 10% of the users that install it - the other 90% of them rate it 0 stars - but don't leave a comment So unless all of the above is true, the conclusion is really obvious: someone thought it's funny to do repeated ratings just to *pull down* MozGest's overall rating. This is a severe issue that makes the whole ratings system *useless*. The danger that it's abused as a "weapon" against extensions you don't like and/or regard as inferior to your own pet extension is huge, and the above example surely isn't the first, and won't be the last one. Therefore, something must be done as soon as possible! I *strongly* suggest that... 1) To make ratings more transparent, the ability to rate without leaving a comment is *removed*. Right now, it makes cheating way too easy, and too tempting. (Short-term) 2) Existing non-comment ratings are deleted from the database. (Short-term) 3) For each comment, the client IP, the user agent and perhaps some other characteristics of the connection are stored. Along with posting time comparisons, they could be used to have at least an indication whether comments could be duplicates. (Long-term)

3 can't prevent against intentional abuse if the user is smart because someone could hop around computers on campus and vote 100 times against your extension, or use dialup to change their IP. There is no method that guarantee no intentional abuse, but at least their could be a deterrent. Knowing the email of who voted on what could at least allow people to see patterns in vote changing (i.e. the same 5 people ALWAYS voting the same way on every extension would help). We could require a valid email in order to make a post, and that email would be stored but not displayed on the site. A confirmation email would also be sent out to make sure you submitted the comment using a valid email address. An LDAP server would probably be better, but barring that, in the day there is an LDAP server, it wouldn't be too hard to integrate a user name into the email address and know which comments they have made because there email address has been recorded. Then the name field would simply be replaced by their username if they created one. Example form: Name [ ] (Required) Email [ ] (Required, but will not be displayed) URL [ ] (Optional) Comment: +-------------------------------+ | | | | | | +-------------------------------+ Notes: Your IPv4 is logged as xxx.xxx.xxx.xxx Your email will be recorded, and a confirmation email sent to that address, but it will not be displayed.

(In reply to comment #13) > 3 can't prevent against intentional abuse if the user is smart because someone > could hop around computers on campus and vote 100 times against your extension, > or use dialup to change their IP. You'll admit that this would at least be an improvement compared to the current situation! (Besides, item 3 is a more long-term issue and should be discussed thoroughly in a separate bug IMO.) I know you can't prevent people from abusing the system, but you can at least make it a bit more cumbersome. I think we should focus now on items 1 and 2, as they should be easy to implement, and will have a big impact: Requiring a comment increases the effort for each rating so much that rating multiple times takes much longer than today: for example, "abusers" have to get creative in the comments so they all don't look the same (and if they do, it's easily recognizable by staff/author/visitors). In contrast, legitimate (single) rating is not made more difficult, or even discouraged: After all, If I rate something, I surely have to say some words about what exactly is great/bad/missing/not working...

When fixed, I think this bug should have the security flag set. It might be good to record hostmask too, since IPs can be much different and mask to a similiar hostname. Perhaps a server-side script could throw up a warning flag to the extension manager when either IPs or hostmask only differ by a certain amount between comments. I still think we should do this right in this bug, and implement it fully, since we will likely have all the ratings reset after this. To not do it right this time might mean having the ratings reset all over again. The email identification method would be a good addition to 1 and 2. Email accounts as an identification method is quite viable, as shown with Bugzilla, as long as we set it up in a way that emails can later be replaced by a username and LDAP (perhaps by having an email address field, and username field both set to the email address). The amount of effort it takes to create separate email accounts, and vote with comments, on different IP addresses and netmasks will make it too much work for all but the most determined people, since we'll see a pattern if they use the same email addresses to vote down/up multiple extensions. I think that it's best to do it right the first time (save for LDAP which would hold this bug back too long), with it planned such that LDAP can be integrated in the future, and then when this bug is fixed, to mark it flagged as a sensitive bug so it's not so blatantly obvious how to cheat the system (create different emails, go to different IPs on different ISPs and post votes with comments without a noticeable pattern between extensions.

CCing Ben C from bug 34118 he likely knows what would be necessary to make this non-LDAP fix compatible with a future integration with LDAP.

(In reply to comment #12) 1) Its likely that the rating-only mode will be removed, not because of the abuse aspects, but the less usefulness of the information it provides. > > 2) Existing non-comment ratings are deleted from the database. > (Short-term) This isn't going to happen :-) Asking to have valid data manipulated is inappropriate. :-) > > 3) For each comment, the client IP, the user agent and perhaps some > other characteristics of the connection are stored. Along with > posting time comparisons, they could be used to have at least an > indication whether comments could be duplicates. > (Long-term) Any detailed information the system already collects is not exposed to the authors or users to see. Duplicate comment prevention w/ or w/o null comments being allowed, is a major issue the system has. Limiting it and saying that only null comments are a problem is just plain *wrong*. One of the issues, is that it's really easy to hit F5/ctrl+r on the returned page, and repost the data. For the record, the user who abused the Mouse Gestures extension negatively, also abused the All-on-One Gestures extension postively. :-) So yeah, its abuse, but it's not likely to be the kind of abuse that's preventable, if you block one method, another will appear. Till you end up requireing registration for commenting, which isn't going to happen. Requiring an e-mail and spamming that e-mail with a confirmation message that serves no useful purpose is also not likely to happen here. :-) An LDAP server doesn't particularly serve any needs here. Don't make this a bigger issue than it is, with over-complex solutions that require more time to implement than they're worth.

(In reply to comment #17) > Its likely that the rating-only mode will be removed, not because of the > abuse aspects, but the less usefulness of the information it provides. > > > 2) Existing non-comment ratings are deleted from the database. > > (Short-term) > This isn't going to happen :-) Asking to have valid data manipulated is > inappropriate. :-) I don't understand this - you will disallow future non-comment ratings, but keep the existing ones? Then say goodbye to the transparency of the UMO system, if it ever had such a thing. This way, neither visitors nor authors can retrace an extension's rating: New extensions, added after the change, will only have visible ratings, but existing ones will have both visible and invisible, and only UMO staff knows how their rating was calculated... > > 3) For each comment, the client IP, the user agent and perhaps some > > other characteristics of the connection are stored. Along with > > posting time comparisons, they could be used to have at least an > > indication whether comments could be duplicates. > > (Long-term) > Any detailed information the system already collects is not exposed to the > authors or users to see. I never suggested exposing this information to authors or visitors. I was merely suggesting just recording this information, as I was under the impression there would be no precautions like this. > Duplicate comment prevention w/ or w/o null comments > being allowed, is a major issue the system has. Limiting it and saying that only > null comments are a problem is just plain *wrong*. One of the issues, is that > it's really easy to hit F5/ctrl+r on the returned page, and repost the data. Well, a possible fix for this might be to include a "form submission ID" (random number) in a hidden field when the rating form is displayed, and never allow two successive ratings with the same ID. > For the record, the user who abused the Mouse Gestures extension negatively, > also abused the All-on-One Gestures extension postively. :-) May I then politely suggest to remove this particular user's ratings from the database? I hope you don't regard this as an example of "Asking to have valid data manipulated"... > So yeah, its abuse, but it's not likely to be the kind of abuse that's > preventable, if you block one method, another will appear. True, you can never prevent things like this 100%. But as stated above, some changes could discourage users a bit from abusing the system, as it would be too cumbersome. Disallowing non-comment ratings, though you have other reasons to it, is one such change. > Till you end up requireing registration for commenting, which isn't > going to happen. Right, I would even say it is counter-productive as it discourages both abusive *and* legal use of ratings.

(In reply to comment #18) > I don't understand this - you will disallow future non-comment ratings, but keep > the existing ones? Then say goodbye to the transparency of the UMO system, if it > ever had such a thing. No, I said I wasn't going to just delete the current ratings-only submissions like they never mattered. :-) Most likely, I'll go through the DB and eliminate the dupes and mark the non-comment ratings that are _valid_ (read as: non-duplicate, perfectly useful rating from a user) with a comment that it was a rating-only post and it'll be exposed to both authors and visitors. That'll take place once rating-only commenting is suspended. Pardon the vagueness there. > I never suggested exposing this information to authors or visitors. I was merely > suggesting just recording this information, as I was under the impression there > would be no precautions like this. Yeah, comments/ratings already monitors date/time and user IP collection. :-) It doesn't collect a user_agent as that info is not particularly reasonable to collect, IMO > Well, a possible fix for this might be to include a "form submission ID" (random > number) in a hidden field when the rating form is displayed, and never allow two > successive ratings with the same ID. > I like that idea. :-) could have it as part of an md5 hash of a random # or something along those lines for the random id. > May I then politely suggest to remove this particular user's ratings from the > database? I hope you don't regard this as an example of "Asking to have valid > data manipulated"... Already done, both for your extension and AiO. Though the cumulative rating might not update itself till somebody comments/rates again. Because of the nature of the cached rating value. There were close to 1200 bogus ratings records deleted. :-) the IP involved only touched those 2 extensions as well.

I realized a comment policy will just lead to spam comments, such as "Nothing cause the idiots required something here". Personally, I think all ratings should be removed since we don't know which are valid and which are not. Then enact an email policy. The email address is a widely accepted identifier. Passport and Bugzilla use it, for instance. We don't need to register them, we can simply check for another rating by the same email when they submit, and if so, ask them are they sure they want to change it. Because of IP logging, and the hidden form entry being mentioned, they'd have to get their friends involved (who probably wouldn't be willing to put the time into dealing with the email), or use multiple ISPs, and that would limit the affects of abuse. That's if they even knew how this system worked, which they wouldn't when this bug is closed. Personally, if they aren't willing to exert the effort to click a link in an email, I don't trust the rating they give. > Requiring an e-mail and spamming that e-mail with a confirmation message that > serves no useful purpose is also not likely to happen here. No useful purpose? It is the only good deterrent. It's not SPAM since they requested it (and we said they'd recieve it). How hard is it to click a link in the email? If you want me to go around from computer to computer in a college computer lab to prove my point, I can. Or even if you can somehow protect against something like that, I can still IM all my friends to tell them to rate down some extension (I could even give them a link with CGI headers). Seriously, they wouldn't bother if they had to deal with some email, but if it's just a click, they would. > Till you end up requireing registration for commenting, which isn't going to > happen. Registration is unecessary. They supply their email address and rate an extension. The email gets sent to the email address and the rating is not registered until they click the link on the email. No registration required, only a list of ratings waiting to be confirmed, which get dropped after so many days.

Maybe to solve this problem, it would be best to look at a system larger than update.mozilla.org that already works well? download.com (ie: cnet) has had a rating system in operation for years, with many more downloads than update.mozilla and many many more users and raters. And given that they host quite a bit of commercial software, they are probably dealing with a much larger spam problem And yet their system works quite a bit better than the one here, it seems to very accurately describe the good and bad qualities of each piece of software. Which is really the most useful part of the rating system, allowing users to evaluate the software before they download and try it out. If there is anything about download.com that is significantly different from here, it is the more involved process of actually making a rating. You comment and you rate the software on a number of different factors. It forces the rater to really think about their rating, and dissuades people from making "casual ratings." I have seen people use the update.mozilla rating system on my extension to ask a question, leaving the rating at 0 stars simply because they did not want to bother rating. I have also been rated down because people couldn't install because of mirroring problems with update.mozilla, which is just lame. I agree with the guy who is saying we should just delete the old data. As much as I like all the glowing reviews already there, the data in general is just bad. Once the system is fixed, the old data should probably just be removed. Particularly if you switch to a multiple-factor rating system, the old rating data would make no sense in this new context. The only thing of value that might be left is the occasional insightful comment which could be imported somehow into the new system if it isn't too much work. Something else that I think would be useful is something like the "downloads in the last week" thing. Rating data becomes less relevant over time, since old ratings apply to old versions. Maybe something like "average rating over the last month" or something would be more useful. And it also means that you could justify deleting the old ratings, since they will all be a month old in short order. Also, I think forming an average from say 3 ratings is pointless. The average should be N/A, you can't make any judgement from that few ratings. And it is just an invitation for unscrupulous people to rate that project down to get them off the top 5 list. Blocking duplicates from the same IP is a no-brainer. And banning/deleting/low-weighting ratings that don't have comments is also a really good idea. That data just pollutes the average. A rating with an insightful justification obviously means a lot more than an anonymous rating with no justification. Another thought is to throw out outliers. Take the top 5% of ratings and the bottom 5% and throw them out. This will get rid of the effect of nutbars trying to slam certain projects like mozgestures, and also the opposite effect of people trying to hype a particular project (ie: all-in-one) And one more idea: do the amazon thing and have a meta-rating system where people can say "did you find this review useful?" At the very least, that could push useful reviews up to the top. As it is now, sorting by date is a crapshoot that can result in all sorts of uninformative comments appearing at the top.

Yes, this is quite annoying. I think logins should be optional, but not required. I think the best way to do it is allow one rating per I.P. per extension/theme, and set a cookie as well. This isn't fool proof (i.e. one can use proxys, and clear his cookies) but I think it would help.

This is more than annoying! I have calculated the average scoring for a few themes: #votes #total avg actual in the 'Top Rated' Walnut 15 58 3.9 3.5 (9% too low!) Nautipolis 28 124 4.4 2.2 (50% too low!) GrayModern 72 302 4.2 4.9 (17% too high!) Curacao 71 322 4.5 4.4 (3% too low!) Actual placement in list is: 1. GrayModern 2. Curacao 14. Walnut 30. Nautipolis Nautipolis is clearly placed 'too low', and Curacao and Nautipolis should actually be above GrayModern. So the rating is not only faulty because of the not so secure voting, it is also really doing bad maths... Greetings, a frustrated themer...

(In reply to comment #23) > I have calculated the average scoring for a few themes: > #votes #total avg actual in the 'Top Rated' > Walnut 15 58 3.9 3.5 (9% too low!) > Nautipolis 28 124 4.4 2.2 (50% too low!) > GrayModern 72 302 4.2 4.9 (17% too high!) > Curacao 71 322 4.5 4.4 (3% too low!) The overall rating includes non-comment-ratings, which are not visible to visitors. Wolf wrote he will disallow such ratings sometime, as they are hardly useful. From this point on, the existing non-comment-ratings will be displayed, so one can retrace the rating calculation.

Still, an hopeless rating system, better completely disabled until someone really fixes it.

Should that be put in an asterisk on the page?

My 2 cents: - The rating system is often wrongly used as a forum without the essential features of a forum that allow the participants to communicate with each other. Maybe add a small forum to each extension and let the user decide whichever is more appropriate for his situation – posting a review or a forum topic. Sourceforge are doing exactly that with their projects. If not, allow the author to optionally provide a link to the extension forum and display this along with the link to the homepage. IMHO, the link text should express the action that the user wants to take. "Support" or "Forum" expresses such action. "Homepage" often does not. - A closely related issue: right now, the main extension page says: "Having a problem with this Extension? For Help and Technical Support, visit the Extension's Homepage." This is nice, but apparently doesn't do the trick – many people still use the rating system to report bugs (which are sometimes not-really-bugs), or to ask questions and give the author no way of contacting and helping them solve the problem. Maybe have the "Having a problem..." text appear on the rating submission form. Also, maybe have a checkbox on the rating submission form saying something like "I'm having a problem and have contacted the author". The value doesn't have to be inserted into the database, but it will make the user think about contacting the author. In any case, I believe that there should be additional optional –Support- URL that the author can provide and the link to that URL should be prominently displayed on the extension/theme details page and on the rating submission page. This is again straight out of Sourceforge. - Allow users to edit their rating/comments. There are sometimes two ratings by the same user which look something like this: 1.Doesn't work. Tried to download it and got an error. 0 stars. 2.Ooops. Tried again after a minute and it's Ok. Really great extension! 5 stars. This user is really satisfied, but his average rating is 2.5! We need a way for the user to edit their posts. I believe that a complex register/login system isn't necessary for this, a simple cookie that expires after a week can be used to identify the user if the user isn't registered. With that said – it would be nice for the author to be able to contact the user. >> The last comment made by an anonymous user, whether a good comment or porn spam, is put on the main information window for the extension. Perhaps some other method should be used to determine which comment to place here. Totally agree – maybe show a random review out of the last 10?

I like the idea of banning ratings-only comments, but I'd love to see the reverse -- comment only, no rating. In particular (and this goes back to an earlier comment about people not finding the Tech Support / Extension home page link), I periodically drop into the comments to say "yes, that's fixed... no, that's a bug, please file it at... no, it's not *supposed* to work on Macs right now..." etc. And I have to include a rating. I'd rather not be given my own ext. 5 stars, but OTOH I certainly don't want to *reduce* the rating.

I agree with a default "No Rating" option

First of all, congratulations on the Firefox 1.0 release! Like we all know, its huge success has been overwhelming the UMO server lately. This, along with the fact that there is no multiple submission (IP) check, caused many comments to be submitted multiple times. I guess the user hits "Submit", the server updates the DB but the user doesn't get any confirmation (server timeout), so he/she keeps on hitting "Submit". I believe that Bugzilla does have such protection. I really shouldn't be complaining - FoxyTunes received multiple –positive- feedbacks because of this problem ;), but I believe that this is a problem with both positive and negative comments.

Sorry if this error is posted in the wrong place but... When i try to add a comment/rating to extensions or themes I get an SQL error that looks like this: Warning: mysql_fetch_array(): supplied argument is not a valid MySQL result resource in /opt/update/core/postfeedback.php on line 69 Warning: Cannot modify header information - headers already sent by (output started at /opt/update/core/postfeedback.php:69) in /opt/update/core/postfeedback.php on line 89 Even though I get the error, it still takes my feedback and comments and adds them to the database. In the process of trying to get it to work, comments are added multiple times. Hence, I have rated some extensions 5 times without realizing it. Many of the duplicate entries may be attributed to this. Aside from fixing the error from showing up, there should be a way to remove duplicate entries.

*** Bug 269749 has been marked as a duplicate of this bug. ***

patrick: Please file a new bug on that issue, as this is an enhancement META bug.

I have an idea that will limit some of the damage done by problems with the current rating system: Eliminate the "top 5 rated" sites from the front page, and add an extra 5 onto most popular to make it "top 10 popular". Here is my reasoning: - the top 5 rated are essentially meaningless given the problems with the rating system, and ultimately they just encourage people to abuse the system so as to get their extension on the main page, or get someone else's extension off that page. I think there would be a lot less abuse of the rating system, and the ratings and comments would be a lot more meaningful, if the top 5 was eliminated. - the top 5 rated wastes space by showing extensions that are already listed in most popular - the top 5 rated is actually only the top 4, since it shows foxytunes twice. ie: it is broken - top 10 most popular is more meaningful since it shows extensions people actually use. top 5 rated often just shows new extensions and punishes "contentious" extensions, neither of which are an indication of quality or usefulness Having clicked on the links at the bottom left corner of that page "popular" and "top rated", and going through them all, I can say with certainty that the popular list is much more useful, and the more "popular" extensions that can be listed on the main page, the better. If there is room, I think the front page should have the top 20 most popular. Given that new firefox users will give up easily when browsing UMO, the extensions they are most likely to install ( by definition, the most popular ) should all be 1 click away.

Summary of Comments/Ratings Not Suck Changes to Be Implemented: -- Split Comments into Pages for Ease of Browsing (10, 25, 50) -- Sort Comments by Date, Descending Order (Newest First) (Bug 255721) -- Disable No Comment Mode, require a comment. -- Allow Null ratings from Authors via DevCP only. (Bug 247413) -- Collect e-mail (optionally) for authors to respond. (Bug 253089) -- Add a formkey to prevent (or reduce) multiple postings. -- turn linebreaks into <br>'s -- Record what version # a comment was written about. -- meta-ratings (was this useful?) info. (Used for frontpage sorting top 5 most useful will be shown at random most likely) -- Comment Moderation from DevCP for Admins/Editors. -- Average Rating for Comments/Ratings for Previous 30 days. -- A Max Life for an Comment/Rating before its deleted entirely? (120days?)

A very good list! I would like to add one item: do the maths correctly on the averaging of the ratings (see comment #23).

(In reply to comment #36) > A very good list! > I would like to add one item: do the maths correctly on the averaging of the > ratings (see comment #23). Comment #24 has already addressed the inaccuracies of your math. You're not shown all the ratings in the list to verify the average, you're only shown comment-including ratings values. (List Item #3 removes the ratings-only feature and therefore all ratings/comments will be shown.)

So to get from a score of 4.4 to 2.2, it means that for all of the shown comments, at least the same amount is not shown and has a score of 0 or 1. I'm guessing that a lot of 0 entries were added (from the same poster?) This still means that the system is very unfair, and a strange way of rating the work that people put it for nothing... Also showing results from different data then is showing, shows an incomplete and inconsistent viewpoint, added by miscounted downloads, (see the different download numbers on the frontpage and on the items themselves), increases the unfairness of the whole 'update.mozilla.org' site. So, please fix the issues on your list, and keep the good work of rethinking/re-implementing the rating system.

The bug in Comment #31 has been fixed. Thanks Patrick.

Hi Wolf. I don't know if this is the right place, but can you collapse to 1 occurrence the following opinion on FlashGot (id=220), which is duplicated 140 (one hundred forty) times consecutively? I won't comment on the mind status of the poster (and that opionion list is about 1/2 megabyte long). Please please please. Thank you very much --------------------- Posted by Jason LaRue Never Again (0 stars) Now my computer crashes all the time and I can't get rid of the spyware junk that keeps reappearing. I have to do a complete reinstall. This piece of crap extension sucks and you suck.

(In reply to comment #40) Done.. He probably didn't think it worked, because of the bug reported in comment 31 where it returned an error but worked anyway, there's a *lot* of duplicate items now. heh. I've been thinking and I agree with what somebody mentioned here, most likely when update-beta goes live with the improved comments/ratings system, it'll launch with none of the current rating data. I'll just empty the table.

(In reply to comment #41) > (In reply to comment #40) > Done.. Thank you very much, I was sucking a bit too much ;-) >He probably didn't think it worked Many other people tought the same, but he was specially bright and quick to understand: average duplicates number seems 3.5, he's 40 times smarter :-) > I've been thinking and I agree with what somebody mentioned here, most likely > when update-beta goes live with the improved comments/ratings system, it'll > launch with none of the current rating data. I'll just empty the table. It seems quite fair... Wolf, if you need any help in implementing changes, I'm here.

Is there a way to keep the valid comments? (i.e. the entries with a text comment, and the duplicated entries removed)

(In reply to comment #43) > Is there a way to keep the valid comments? > (i.e. the entries with a text comment, and the duplicated entries removed) This is what wolf did for FlashGot (there were 140 "valid" duplicate text comments). But I guess it is superseded by the "fresh restart" policy he suggested.

Wolf, the Mouse Gestures extension has once again received a bunch of invalid ratings, starting on 2004-11-15. The ones with comments are positive, but as the extension dropped to 4.4 after being 4.5 for months, I suppose someone made several negative non-comment ratings... Could you look into the DB and do a little cleanup?

Just a curiosity: how exactly did FoxyTunes to replicate itself? Looking at the DB structure it seems quite a miracle! Both entries have the same ID=219, and t_main.ID is a primary key...

Correction: the multi-ratings thins for MozGest seems to have started on 2004-11-09 (at least the visible ones).

Wolf, I believe that the valid text comments should be retained. I understand that from the developer point of view it's much easier not to support previous versions and DB schemas. The problem is that some of the users really take some time to write an informative and constructive comment. Removing such posts would IMHO be disrespectful to them – like their opinion doesn't matter and can be easily discarded. Most forums, blogs and other "community" sites have strict policies of not removing user comments (only by moderation, when the comment is abusive or obscene). I would personally think twice about posting a comment or a review on a site knowing that my posting might be removed after a few weeks/days, and if that would happen after I've already posted, I would definitely not post anything on such a site again. I believe that the existing comments should be moderated, removing duplicate and abusive ones. Please let me know if I can help with this in any way (not with my own extensions of course :) )

I agree with Alex, a script can find repeat-comments. Ratings without comments can be dropped. This could all be automated.

Just to confirm and stress: the rating-calculation is not right: If you add up all the 'stars' of the comments page of GrayModern, divided by the number of visible comments (including a stars rating) you get: 1539/359 = 4.3 But doing this for Littlefox, you get: 832/182 = 4.6. But on the frontpage of themes for firefox, Graymodern gets a 4.9 and Littlefox gets a 4.5.

(In reply to comment #50) > Just to confirm and stress: the rating-calculation is not right: > If you add up all the 'stars' of the comments page of GrayModern, divided > by the number of visible comments (including a stars rating) you get: > 1539/359 = 4.3 > > But doing this for Littlefox, you get: 832/182 = 4.6. > > But on the frontpage of themes for firefox, Graymodern gets a 4.9 and Littlefox > gets a 4.5. GrayModern (ID # 103) has 3074 ratings. (not 359.) LittleFox (ID # 304) has 254. (not 182.) Your calculations cannot include and verify all the records because they're simply not being exposed to you. Ratings w/o a comment are not shown and therefore cannot be properly validated. but they *do* exist in the DB and are used in the averages. Though, GrayModern seems to have way too many hidden records IMO. Which upon further investigation is now down to 523 total records. The other 2551 were abusive hits from serveral IPs one ISP. "TELUS Communications Inc.". This didn't just single out GrayModern though, but several extensions/themes (and of course, all rated 5. heh). This reenforces why it's likely that update-beta will likely be a feedback start-over.

Another suggestion (I believe something like this has already been mentioned) – the rating calculation should weigh in the total number of comments. Also, IMHO, there is a need for a minimal number of comments for an average to be calculated. For example, suppose there is an extension that is written especially for paleontologists (who are about 0.05% of all Firefox users). There are 3 paleontologists who love it and gave it 5 stars. So, we have a top rated extension at the top of the list that doesn’t interest 99.95% of Firefox users. Also, Wolf – can you please post somewhere the number of invisible ratings vs. the number of visible ones for all the themes and extensions? I think most of us will find this information interesting. Thanks :)

Googlebar also has a lot of repetitive 0 and 5 star ratings, including such treasured comments as "nothing works in 1.0, why oh why did I ever upgrade firefox?". (...So, mark us down for that cleanup script when it gets going). Could we please add a disclaimer on the comment page? "Please report problems to the theme authors directly rather than in feedback. Comments should pertain to the specific extension you are reviewing, and help us keep them useful by limiting youself to one comment". People still just aren't thinking and still have way too much free time- maybe the "where to get help" notice on the main download page is helping (GB is getting more support requests at our homepage lately, certainly), but some people are still slipping through the cracks and bombarding the page with rants about that which we can't do anything about. I also like the idea of breaking ratings up into sections ("appearance", "features", "stability", "install")- anything that makes people think before posting a review by requiring all those comments should make the system more meaningful, so long as all those sections are required.

Bulk Moving Web Site bugs to new component. (Filter: massumowebsitespam)

(In reply to comment #35) Summary of Comments/Ratings Not Suck Changes to Be Implemented: Done -- Split Comments into Pages for Ease of Browsing (10, 25, 50) Done -- Sort Comments by Date, Descending Order (Newest First) (Bug 255721) Done -- Disable No Comment Mode, require a comment. Done -- Allow Null ratings from Authors via DevCP only. (Bug 247413) Done -- Collect e-mail (optionally) for authors to respond. (Bug 253089) Done -- Add a formkey to prevent (or reduce) multiple postings. Done -- turn linebreaks into <br>'s Done -- Record what version # a comment was written about. -- meta-ratings (was this useful?) info. (Used for frontpage sorting top 5 most useful will be shown at random most likely) Done -- Comment Moderation from DevCP for Admins/Editors. Done -- Average Rating for Comments/Ratings for Previous 30 days. -- A Max Life for an Comment/Rating before its deleted entirely? (120days?) The ones marked done are now checked in and live update-beta on the website. The remaining two, meta-ratings has a experimental implementation now, but I'm not confident in it's usefulness in a production enviroment, as it lacks protection from repeat hits. So It'll likely be disabled before update-beta goes live. but it's up now, for those who want to see the concept. A seperate bug will be filed on the finalizing and beefing up to production quality of that feature if it's determined to be beneficial to have. The other the maximum lifetime, hasn't been given much discussion, I don't personally feel that the UMO comments system is like bugzilla in that everything that's submitted is somehow, historical record. I'd like to see old comments degrade and leave the system. I propose a 4 month lifespan, which seems adequate, comments on this?

>Average Rating for Comments/Ratings for Previous 30 days Do you mean that we show comments older than 30 days but we average only those made since last month? I don't believe this would play in favour of clarity, unless we show proper user-oriented legends (it's going to be the same style of https://bugzilla.mozilla.org/show_bug.cgi?id=257229, "Number of downloads confusing" - BTW, any news about that?) > I'd like to see old comments > degrade and leave the system. I propose a 4 month lifespan, which seems > adequate, comments on this? It seems wise to me, considering that we have queries on the involved tables from the public web site: an huge number of (unuseful) old records can impact the overall performance. Anyway, if it is not overcomplicated, I'd suggest to shorten the lifespan to 1 month (coherently with proposed average rating), unless the extension has not been updated yet: in other words, since we save version info now, we should keep comments that refer to latest version, even if they are 1 months old or older. Average would be computed on live comments, without time constraints. This is an unbiased opinion, since FlashGot is update almost every week :-)

May be instead of a time-based threshold, why not only keep the last 50 or so comments? When they paged into 10, 25 and 50, keeping more than 50 seems unneccessary. This will at least keep a reasonable amount of comments online even if versions don't change much (or too often), and even when only few users are commenting, and thus only a few per month trickle in...

(In reply to comment #57) > May be instead of a time-based threshold, why not only keep the last > 50 or so comments? When they paged into 10, 25 and 50, keeping more than 50 > seems unneccessary. I'd also suggest doing this - it seems the most generic (and fair) solution. Additionally, I'd like to have the number of visible ratings equal to the number of ratings going into the average. Then we can avoid lengthy explanations...

There's a fairly simple reason, IMO, why time works much better than a # of comments flat limit, and that is that all themes/extensions are not equal in popularity. If you set it to be only 50, then a popular extension is going to run through those 50 and being rotating much faster (perhaps for a TTL of less than a day) than another extension which gets a comment every other week. (fairly extreme example). Downloads is set to be in the last week to keep Most Popular competitive. As it stood before, the longer posted popular extensions were sure to stay on top. In Bugzilla, bugs as they age, become less likely to be worthwhile, given the speed of product development. I feel the same thing applies here with comment/ratings, in general, it keeps the newest comments counting the most, fading out issues that appear, (such as a few hundred bogus ratings if such abuse evolves into the new system via a new method). If you set it to a 50-limit, a malcious user could spam you 50 times and drop you to 0, which isn't at all desireable. All in all, it keeps the top rated lists more fluid in nature, which I realize, may or may not be, in some authors viewpoints (particularly top rated extension authors, heh) a good thing. A maximum lifetime for comments before they're deleted, which is greater than the used 30 days for ratings is a bit vague. But, while a comment may have age degraded to be less useful for top rated/summary rating, that only applies to it's rating, not it's content. So, since the comment half seems to have a longer lifespan, we don't delete it until it has outlived its usefulness. Which I'd say would be about 90-120 days depending on the level of incoming comments and the speed of development. (if it's comment #1000 in 120 days, then it's never going to be seen) Comment #56 questions the clarity of having the rating average over 30 days and not total. It also points out the lack of clarity on download counts. The upcoming major design change gives space in the relavent places to give a brief description about how both are measured (the "legend" the comment requests). Not a lengthy description by any means. "Ratings are based on feedback over the last month from people who use these extensions." "The most popular downloads over the last week."

(In reply to comment #59) > All in all, it keeps the top rated lists more fluid in > nature, which I realize, may or may not be, in some authors viewpoints > (particularly top rated extension authors, heh) a good thing. I completely agree with your statement, even if it implies I should not ;-) In fact, I definitely prefer a fluid system that promotes good competition (improving quickly software quality) over a conservative one. > "Ratings are based on feedback over the last month from people who use these > extensions." > "The most popular downloads over the last week." Good work Wolf, it seems clear enough now :-)

Mass-resolving bugs that have been fixed on trunk.

Sorry for the bugspam, reopening bugs wrongly marked as resolved.

After having just gone through and cleaned up the feedback data for all 55 Firefox 1.0 compliant themes, I'll add my two cents: - God help us if we try to do the same for the rest of the themes/extensions. That took me 6 hours, and whilst I'm pretty sure I got 99.9% of the duplicates/garbage out, I probably didn't do a perfect job. - I'm all in favour of dumping the existing feedback data. Do we really need 6 month old comments? Would you want to try sorting out what should and shouldn't be there?

I am all for dumping the existing comment data. There is so much garbage in them. My extension, Weatherfox, has probably over a 100 text comments, but on Saturday night it dropped from 4.8 to 4.0 with one text post from this guy who said "This extension didn't deserve number one. 0 rating". I'm sure a lot of the other authors would agree that a fresh start is needed.

(In reply to comment #64) > I am all for dumping the existing comment data. There is so much garbage in > them. My extension, Weatherfox, has probably over a 100 text comments, but on > Saturday night it dropped from 4.8 to 4.0 with one text post from this guy who > said "This extension didn't deserve number one. 0 rating". I'm sure a lot of the > other authors would agree that a fresh start is needed. Jon, I believe your rating has dropped not due to the visible text comments, but due to the invilisble ones, and those are going to be dropped anyway in the new system. I think that even when the non-text invisible comments are a thing of the past, we'll still need some mechanism for human moderation. Otherwise, the same user that obviously submitted many invisible comments can submit many visible zero-star comments in the new system - and with no moderators these comments won't be removed and will lower the ratings. As to removing the existing -visible- comments - like I said in comment #48 - going over the WeatherFox comments I see that there are many grateful users that wrote a real mini review of the extension. Dumping these comments is IMHO not fair to these users.

Wolf, one more question/comment - in the old system, the user with a problem often posts a comment with 0/1 stars and describes the problem. If I manage to get my reply through to this user and solve the problem, the user often posts a follow up comment with 4/5 stars saying the problem has been solved. How is this going to work in the new system when the user won't be able to post multiple comments?

> Do we really need 6 month old comments? Yes. It's not fair to people if they are removed. Dump any ratings without comments. Then just choose a team of moderators and assign a set of extensions to each one of them. They can mark any that look suspicous and then you can take a close look at them. I do like the idea of older ratings decreasing in how much they are weighted.

*** Bug 272192 has been marked as a duplicate of this bug. ***

Wolf, I'm looking forward for the new rating system. And yet, I don't know if it would be enough to prevent the vandalism that FlashGot seems to unavoidably attract: have you seen last comments? As some users of mine said, this rating system has turned into a "battlefield" or a "kindergarden". I guess Weatherfox author changed extension name into ForecastFox to get (temporarily) rid of the same kind of spam ;) Sorry I registered flashgot.net one week ago... Thumbs up for fresh start and moderation. When will the new system go live?

I think we should give this a try, and if it doens't work, knock out ratings altogether.

Disable Meta Ratings Reimplement rating average stars. Linkify Commenters names for Authors/Admins submittin null-rating comments in DevCP.

Final patch checked in. Bug 275371 is for the rest of the meta-ratings work.