m3lib File Format

This page documents the m3lib file format, the cache file format MusicIP uses.

The following format description has been used to decode a 71 MB cache file containing 127,924 file entries generated with MusicIP's genpuid program (version 1.2 for Linux). However, it should be understood that there are parts of this file which are potentially not defined here.

File Format

Basic Description

The m3lib file is laid out with three major sections. The first three sections are string tables containing in order the genre, artist and album strings. The forth section gives the file specific information including all PUID, filename/path, all primary tag information, and the matrix results of MusicIP's AudioFingerprinting.

Note: When the format below indicates that the file contains zero, this has been verified in the test file. This may not be the case in cache files which have been generated by other programs. For example it is likely that ratings are stored in the m3lib cache files, but there are none in the test file.

Data Types

The numbers stored in an m3lib file are in big-endian format. This is the endian format which is the standard for most Internet protocols. However, it is not the endian format used by Intel and AMD processors. So when decoding any number shown below as an int or short on a little-endian machine such as a Intel Pentium processor be sure to make the necessary conversion. This can be done with the ntohl and ntohs routines.

The date/time stored in an m3lib file are in time_t format. This format represents the number seconds since midnight on January 1, 1970. This number is also stored in big-endian format, and should be converted to little-endian on little-endian processors.

Format

18 bytes -- Unknown. In the test file the first byte was 0x20 and the remaining 17 bytes were zeros.

4 bytes -- time_t -- Date the m3lib file was last updated

28 bytes -- Unknown. In the test file the first 10 bytes were zeros, followed by 0x0101, followed by 16 bytes of zeros.

4 bytes -- unsigned int -- Number of Genre strings to follow

For each Genre string:
14 bytes -- All zeros
4 bytes -- unsigned int -- Genre string length
variable length string -- Genre string

4 bytes -- unsigned int -- Number of Artist strings to follow

For each Artist string:
2 byte -- unsigned short -- Artist string length
variable length string -- Artist string
17 bytes -- All zeros

4 bytes -- unsigned int -- Number of Album strings to follow

For each Album string:
3 bytes -- All zeros
4 bytes -- unsigned int -- Album string length
variable length string -- Album string

4 bytes -- unsigned int -- Number of sets of file information to follow

For each set of file information:

16 bytes -- The file's PUID. The first byte in the PUID is the first byte printed in the PUID. No endian conversion is necessary.

4 bytes -- Unknown

1 byte -- unsigned char -- Number of tags to follow. This is the variable content section for the file information.

For each Tag:
1 byte -- unsigned char -- Tag Identifier. The following are the only known m3lib tag identifiers (based on the test file):

Tag ID 0x01:
4 bytes -- unsigned int -- Unknown

Tag ID 0x03:
4 bytes -- unsigned int -- Unknown

Tag ID 0x0A (Publisher):
4 bytes -- unsigned int -- Publisher string length
variable length string -- Publisher string

Tag ID 0x0B (Track String):
Note: The Track Number is located further down in the file information. This tag has only been seen
when the MP3 file's ID3v2 tag contains both the track number and the total number of tracks (e.g. "4/12").
4 bytes -- unsigned int -- Track String string length
variable length string -- Track String string

Tag ID 0xC9:
20 bytes -- SHA? (unverified, SHA of what?)

Tag ID 0xCA:
20 bytes -- SHA? (unverified, SHA of what?)

4 bytes -- All zeros

4 bytes -- time_t -- Date the file information entry was added to the m3lib file

2 bytes -- unsigned short -- Song Title string length

variable length string -- Song Title string

2 bytes -- unsigned short -- Genre index. Lookup the Genre string from earlier in the file. This index is zero-based.

4 bytes -- unsigned int -- Artist index. Lookup the Artist string from earlier in the file. This index is zero-based.

4 bytes -- unsigned int -- Filename index.
The Filename index is an increasing number starting with one. If the Filename index has been used
previously in the file, use the string previously associated with that Filename index.
If and only if the Filename index is a new number, the Filename string will follow:
2 bytes -- unsigned short -- Filename string length
variable length string -- Filename string

4 bytes -- unsigned int -- File Path index.
The File Path index is an increasing number starting with one. If the File Path index has been used
previously in the file, use the string previously associated with that File Path index.
If and only if the File Path index is a new number, the File Path string will follow:
2 bytes -- unsigned short -- File Path string length
variable length string -- File Path string

4 bytes -- unsigned int -- Album index. Lookup the Album string from earlier in the file. This index is zero-based.

2 bytes -- unsigned short -- Release Year

2 bytes -- unsigned short -- Track Number

2 bytes -- unsigned short -- Track Length in seconds

2 bytes -- unsigned short -- Rate (the CBR rate or an approximation for VBR)

4 bytes -- All zeros

4 bytes -- unsigned int -- File size

4 bytes -- All zeros

4 bytes -- time_t -- File modification date

4 bytes -- unsigned int -- Flags. The flags are all unknown except 0x00000008 which conveys the remaining data block size.

If the Flags have bit 0x00000008 set:
564 bytes -- MusicMagic Fingerprint

70 bytes -- MusicMagic Data. If the Flags do not have bit 0x00000008 set, this block will be all zeros.

1 byte -- Zero

Internal Links

External Links

MusicIP Main Page

MusicDNS Web Page

Wikipedia Big/Little-Endian Reference

CategoryDevelopment CategoryExternal - Originally documented by Chase547 on 01/11/2008.