The New Rating System

At first thought, Leif, my reaction was, "Yes!" Make them justify their vote. But that certainly would discourage people from voting, particularly if an article was so off-base, so poorly constructed—or even contained bad/detrimental advice— that it should never have been posted. You would have only positive comments and positive ratings. I think we need some sort of checks and balances here.

I guess I'm thinking of some of the recent tips, which weren't even tips at all, just a 1-2 sentence "philosophy of life" statement, and others that were blatant sales messages. Not sure everyone would be comfortable stating that in a comment, but we need a system that will give people the message when their tip/article is totally not helpful, or even detrimental.

This is such a complicated issue.

Okay, I'll chime in again with the risk of my comment being subconsciously rated...

My thoughts are pretty straight forward, even after all this discussion.

I say do away with the rating. It's a privilege enough to get the article posted and to share your expertise with everyone. And comments to me are much more valuable than a number rating.

And the one thing, though this may have been commented on, is all articles are subjective. For myself, one can be very useful, another not so useful.

But all of us should go by the content and not a number posted by the article. Unfortunately for some articles, a low number may discourage the reader from even making an attempt to read the article.

There are a lot of venues available for articles. I think it's in this communities best interest to get as many high quality articles as possible. This brings more traffic to the site via google. That is a contribution also.

Alright, I'm gonna follow John's lead and jump in this REALLY late!

I see a lot of suggestions for improvements. And I really like the labels that could be added (entertaining, paradigm-shifting, life-changing, informative, inspiring, helpful, etc.)

But since no one's playing devil's advocate, let me try.

Why are we so afraid of ratings? Why NOT have a system that rewards what the community deems excellent, and a system that lets others know when to "try harder"?

If I'm looking for help with marketing, I read the highest-rated articles first. (They ARE more helpful than the lowest rated articles, in terms of concepts, originality, helpfulness and readability.)

Ratings provide a helpful, quick clue to me as to what I might find valuable. I have gone through and read many of the highest-rated articles and have learned a LOT from them. (They are more helpful than almost any articles I have found in magazines.)

I have also read some of the lower-rated articles, to see if indeed they are not as good. The truth is, they're not nearly as helpful and they're not as well written.

I belong to an organization that lets us submit our original songs for critiques. I spent many hundreds of dollars recording my first demos and was very proud of them.

The message I got from the industry experts? A whole lotta "5's" sprinkled with some 6's and occasional 7's in the various categories (out of 10.) I got a big, huge "TRY HARDER."

And do you know what it made me do? It made me hire a songwriting coach. It made me try harder. It made me get feedback from every songwriter I respected. It made me re-write what I thoght were my best songs, and work with co-writers who knew things I didn't know.

Three years later, I'm a great F#@%*ing songwriter. And I still have room for improvement, and I still ask for feedback and re-write my songs.

The problem I see with a "liked it"! button or any other thumbs up approach is that is becomes a popularity contest. The more people you know, the more likely you are to have more readers and get more "points." I actually think the current rating system by numbers is MORE fair.

Hate the idea of having to comment to vote... if I don't like an article, I don't trash it (I have never rated an article below 5, and if I really hate it I just don't rate it) but I really don't want to have to justify why I'm not ranking something higher to the author. If people are afraid of low ratings, they'll be terrified by the critiques that might come with them.

Personally, I think it is helpful to have ratings, as both a reader and a writer. I'm fascinated with our indie-community rebellion on the matter. I do "get" it. I went to Evergreen State for awhile, the college with no grades. (And so maybe we allow people to submit articles without requiring ratings, if they don't want them.)

But just like our member "ranking" (how high up you are in the member roster based on your activity of various kinds), maybe the actual "number" the articles is ranked is a composite of many factors, including the readership, how many comments, how helpful members "rated" it (so the rating is only one piece of the "ranking"), and perhaps also if readers found it "inspiring" or "entertaining" or "paradigm shifting."

Thanks for all the thoughts... I love the rich feedback this question has brought up!

I have a bit of trouble with any kind of rating system. I like to encourage people to be creative and offer something of value. Sometimes I have written something that I am excited about and think- wow people will like this- and then I am surprised when it falls flat. Other times I have received many, many positive engaging comments below and yet the rating is low which has made no sense at all.

My preference is to attach any kind of ratings to the comment section. Maybe put forth a request " if you received some valuable input from this article please take a moment and mention it below with a thumbs up. I want to hear the comment with it.

If I didn't receive any comments at all that would be helpful information to me that i have missed the mark.

In other words, I would like the ratings and comments to be there for my benefit and learning as "learning" is what its all about.

One more thing- an article with specific "how to" information can be poorly written but people like it because of the information that is shared. However, I see just as much value in personal development articles that are meant to inspire or motivate and it is not about having new information.

Great article today, Kate. And thanks everyone for continuing the conversation about this. The truth is we are so busy working on much bigger things at the moment - Groups - that updating the rating system is a very low priority in our current workload. Maybe by the time we get ready to focus on reworking this we'll have a consensus that it doesn't need changing? Or rather, a lack of consensus on how to change it?

Yes - I definitely see a lack of consensus in how to change it! I don't disagree that it could be better, but making it better and still meaningful could get complicated. (Not that you shouldn't go there.)

I would miss the "expert badges," which I think are a great way for the community to give a big "thumbs up" for someone's expertise. I find those helpful.

And with Biznik growing like crazy (and the article volume ramping up with it), I think the rating system will soon become more valuable than ever for "sorting" through what could be an unlimited sea of articles.

At the same time, hopefully there's a way to get all articles at least viewed and read and rated numerous times so that one grumpy "competitor" (who doesn't know that competition is "out") can't sabotage an article.

I do think the "thumbs up" only option actually only makes the potential "popularity contest" problem WORSE, because people who know more people will of course get lots of thumbs ups, and a new member from, say, Alabama is going to feel very neglected.

Removing all rating mechanisms entirely would be a great equalizer, which does have it's good points. But my vote is still to keep the ratings. I believe there is value in community members being able to rank the helpfulness of an article. I do like the suggestion to expand the ratings to more options. (But then again, simple can be good.)

Thanks for publishing it, Dan. It struck me as no small irony that the guy spearheading the "kill the ratings" movement had to approve my article. Then again, maybe you weren't "spearheading" at all, but simply responding to feedback. I enjoyed playing devils' advocate. And I think there is value in this conversation; we are all learning from each other.

I would like to see a profile rating system for comments. Categories could include:

1. Most comments by an individual in a single thread
2. Least defensive responses
3. Highest volume of inapplicable remarks
4. Most humorous miss of the point
5. Most contentious response set
6. Most repetitive response items
7. Clearest lack of understanding
8. Cleverest misuse of grammar
9. Best use of personal plugging disguised as comments
10. Most likely to be ignored