The survival of the fittest proposal
I've made a SurvivalOfTheFittestDiscussion page for discussing this proposal. RjMunro
The concept
Let's consider a piece of metadata, such as a specific album title, call it "Alpha" as entered by Alice. Let's imagine that Bob decides to change the title to "Bravo". Currently, Bob enters a moderation into the queue, and people vote on whether the change in name from Alpha to Bravo makes the data better or worse. This is fine in simple cases, but when Colin comes along and changes the name of "Alpha" to "Charlie", while the voting is still open, a complex situation arises. Under SurvivalOfTheFittest, Instead of voting on Bob's moderation as a yes/no to make a change, Bob's moderation becomes a "vote" for "Bravo", Alice's moderation becomes a vote for "Alpha", and Colin's moderation becomes a vote for "Charlie". Voting is left open for subsequent visitors to this album to decide between the three, or add their own if they don't agree with any of them.
A person who has made a proposal will automatically have their vote allocated to that proposal, but they can change them to other proposals later if they realise they were originally incorrect, and the new proposal is better. Other people will hopefully be able to pick from one of the several proposals that are left. Previous proposers and/or voters should be able to subscribe to be emailed whenever a new proposal is made against their item.
SurvivalOfTheFittest is a bit of a paradigm shift from the current voting system. "Votes" and "moderations" might be a bit of a semantic trap. What is meant is "State that they agree with". At any time we should be able to shift our allegencies, or switch to abstain on a particular piece of data. Much of the earlier discussion on improving the voting / moderation system seems to center broadly on /. style voting on proposed changes to the DB as if they were articles/comments. However, from a user perspective, the important thing is whether the data is *correct* - it doesn't matter how the data get's there.
Of course, there is more detail to be worked out than what is outlined above. Merge type operations can be handled by having "merge artist A into artist B" as just a shortcut way to change the value of "artist" on all artist A's tracks to artist B. Other types of operation may need some more consideration, particularly as AdvancedRelationships and other proposals get implimented and the data structure gets more complicated. One Advanced Relationship may not be to the exclusion of another one, so these may end up as binary yes/no votes, but the application of other features from this proposal are still relevant.
Editor Rating and Removal of data
The system works best if combined with the EditorRating. Votes should carry a weight equivalent to an editor's rating at the time of voting. If they subsequently aquire a higher rating, then they could re-affirm their votes, but until they do, the vote should not gain in value. This could be achieved with a vote weight field in the table of votes. This could also possibly allow people to vote with less than their full strength if they are not sure, but I don't think it is neccesary to build an interface to allow this.
Non-leading data would have it's vote-weights reduced gradually over time. Once it's weight fell below a certain threshold (equivalent to approximately 1/2 a single newbie vote), the data would be expunged from the database. At some point shortly before this, people who have voted on the data due to be expunged could be encouraged to either reaffirm that vote, or cast their weight behind one of the alternatives.
In the SurvivalOfTheFittest context, an editor's rating should be calculated only on moderators proposals, not on their votes. So you get it by having things that you proposed voted on. You must not be able to gain an improved editor rating by simply agreeing with all existing front runners.
BTW: Manipulation using multiple accounts is always going to be an issue. The only solution would be an out of band verification, such as taking users social security number (but that is a /really/ stupid idea). We could attempt to detect it if it was suspected by analysing logs to see if two people usually log in from the same place and vote on each others suggestions.
Subjective Data
The system may work really well for subjective metadata, where there isn't a clearly 'correct' way of doing it. Two artist reviews bubbling to the top on SurvivalOfTheFittest sounds quite nice, as well as where an artist spans genres, it might be nice to generate statisics like this artist is 40% jazz and 60% RnB or something. The rules on non-leading votes decaying may need to be different for this type of data.