NeedsIntertwingling and probably NeedsUpdating

Are We Getting Rid of TRM?

There was recently (September 2005) a discussion on the mailing list about the long term future of TRM, starting with this post, and then continuing here, here, and in several other threads as well.

Are we getting rid of TRM? The short answer is no, not in the short term. However, there are big problems with TRMs, and in the long term these problems might well outweigh the benefits.

What TRM does

TRM is used by the MusicBrainzTagger as a way to identify music. A fingerprint is calculated by examining the music data itself, ignoring the name of the file, the ID3v2Tags, or any other metadata. The fingerprint is then sent to the MusicBrainz TRM server, which calculates an actual TRM, and returns the corresponding MusicBrainz ID and all the useful data like title, ArtistName and so on.

If the TRM hasn't been associated with a song yet, the user has a chance to figure out which song it should be associated with, and tell that to the tagger. The tagger then tells the database about the new association. Future users trying to identify the same track will then have their file identified correctly straight away.

Problems with TRM

The biggest problem with the TRM system as it stands is simply the load on the server. To keep things running smoothly, the entire database needs to be kept in RAM simultaneously. The TRM tables are huge, and once they grow past a certain point, they can't be kept in RAM. At this point the server starts swapping pages out to disk, which dramatically slows things down.

To keep the servers working, the administrators are forced to simply delete some proportion of the TRMs whenever the system gets too slow. Clearly, there's no point having users entering data if it's just going to get deleted, so this isn't really an acceptable situation.

A further problem with TRM is that, as the database gets bigger, the chances of a particular song sharing a TRM with another song increase. This is called a TrmCollision. There are several cases where dozens of songs all point to the same TRM. This makes the tagger less useful, as it gets the right answer less and less often as the database grows bigger.

One final problem with the TRM system is that, although the algorithm for calculating the audio fingerprint is public, the TRM server software is closed source. This goes against the open source philosophy of MusicBrainz. Furthermore, the software is nominally written and maintained by Relatable, but it does not seem to be actively maintained, and employees of Relatable have recently (Oct 2005) been uncontactable.

Lucene

To escape from these problems, the newer versions of the PicardTagger are going to use a text searching system called Lucene. Instead of calculating TRMs from the music data and looking it up on the server, Lucene will take whatever text information it can find (including the name of the file, the directory it lives in, and any tags in the file) and look it up in a text database. Lucene uses intelligent heuristics to figure out which results are most relevant. So far, results indicate that Lucene significantly outperforms TRM in even the most challenging cases.

However, there are a couple of problems with Lucene.

Firstly, Lucene requires a text index to work. The text index is likely to be very large; of the order of 250MB. This index could be stored locally on the user's disk, but they'd have to download it all, and that's probably not realistic for casual users. Another possibility would be to allow remote queries, but that would require a server with very high bandwidth capabilities. Some reasonable technical solution can probably be figured out for this.

The other problem is that Lucene requires at least some textual hints before it can do anything. If you have a directory with untagged MP3s called "smells_like_teen_spirit.mp3", "in_bloom.mp3", "come_as_you_are.mp3" and so on, but the fourth file is just called "04.mp3", this might be enough for Lucene to figure out that it's the fourth track on "Nevermind", and it should be called "Breed". But if you give it just a single file named "04.mp3", it's probably not going to be able to figure out what it is. TRM at least has a chance. Similarly, if a file has completely the wrong information in its tag, Lucene will never notice.

Hybrid systems

The ideal tagger would have the best of both worlds: it would use textual lookup initially, but also try to confirm the decisions by doing a TRM lookup. However, the benefit of performing the extra TRM lookup has to be weighed against the costs of the TRM system. In the early days of MP3s, it was very common for there to be no metadata at all, but nowadays that situation is increasingly rare. Is it really worth overloading the MusicBrainz servers just to support this small case?

Alternatives to TRM and Relatable

Relatable aren't the only people in the world working on audio fingerprinting: it's a very active field. It's possible that MusicBrainz might be able to work with an equivalent system instead. However, as of right now, there doesn't seem to be any obvious solution out there.

The open source audio fingerprinting technology FDMF has been tested by Ben, who has come to the conclusion that it would generally be able to replace TRM, although there are some severe scalability problems (which TRM has, too but in a different way). See this IRC chatlog and the post of his test reults on the DevelopersMailingList for details. (2005-10-10)

There is a project called freetantrum on Sourceforge. It seems to be a dead project (it's home page was replaced with an advert for unrelated things in 2001), but it may be worth investigating and resurecting the code they produced.

As reported in this blog post, Rob has been discussing replacing TRM with a product from another company. There is some concern about the openness or closedness of the system. (2005-12-27)

It's been suggested that MusicBrainz developers could write our own audio fingerprinting system. The size of this undertaking shouldn't be underestimated. The code itself might not be overwhelmingly complicated to write, but designing an algorithm that actually works (that is, generates the same code for music that's essentially the same but has been compressed or edited differently, but which doesn't generate the same code for different pieces of music) is very difficult. It requires a great deal of insight into psychoacoustics, fuzzy matching, Bayesian networks, and other arcane topics. It's unlikely that any MusicBrainzContributor has the very specialised expertise required.

Also, since there are quite a few companies actively researching the problem, as well as commercially exploiting their results, there are likely to be intellectual property issues involved. If MusicBrainz develops and rolls out our own system, and then gets taken to court by a company alleging that the work infringes their patent, MusicBrainz could find itself suddenly having to switch off the entire system.

Nevertheless, Geoff Schmidt has proposed a FutureProofFingerPrint system that sounds promising.

The current plan

For the moment, the MusicBrainz server continues to support entry of new TRMs. Admins continue to periodically delete TRMs when the server gets overloaded. The new PicardTagger will show how well Lucene works, and hopefully lots of people will use that and provide feedback about how much they like it. Eventually there will probably come a time when TRMs will have outlived their usefulness.