MusicBrainz Server / Setup

Contents

MusicBrainz Server virtual machine

Running a MusicBrainz Server as a virtual machine requires some Linux knowledge, but it is vastly simpler than installing the server from scratch. The pre-built virtual image can be imported into either VirtualBox or VMware. If you are using Amazon EC2 you can not use this virtual image and will instead have to follow the steps outlined in the source code INSTALL.md. To set up a virtual machine instance, download the torrent file and follow these steps:

MusicBrainz Server Virtual Machine (Virtual Box)
BitTorrent Download: musicbrainz-server-2013-10-14.ova.torrent
Size: 106Kb
Version: 2013-10-14

MusicBrainz Server Virtual Machine (Virtual Box)
FTP Download: musicbrainz-server-2013-10-14.ova
Size: 10GB Open Virtualization Archive (OVA)
(the extracted virtual drive is then 27 GB and can extend up to 40 GB)
Version: 2013-10-14
MD5: 191871ea1ebb50284cb3b14061ebd064

MusicBrainz Server Virtual Machine (VMWare)
BitTorrent Download: musicbrainz-server-vmware-2013-10-14.ova.torrent
Size: 87Kb
Version: 2013-10-14

MusicBrainz Server Virtual Machine (VMWare)
FTP Download: musicbrainz-server-vmware-2013-10-14.ova
Size: 8.7GB Open Virtualization Archive (OVA)
(the extracted virtual drive is then 27 GB and can extend up to 40 GB)
Version: 2013-10-14
MD5: d6b158ce3bef40e13943f23d4cc28a9b

Running with VMware

  1. Start VMWare Player/VMWare Fusion and choose Import... from the File menu. Select the downloaded file.
  2. Once VMWare has imported the file, select the imported virtual machine from the list of virtual machines and click on Start/Play.

Running with VirtualBox

  1. Start VirtualBox and choose Import Appliance from the File menu. Select the downloaded file.</strike>
  2. Once VirtualBox has imported the appliance, select the imported virtual machine from the list of virtual machines and click on Start.

Running with QEMU/KVM

  1. Extract the disk image:
    tar -xf musicbrainz-server-2013-10-14.ova
  2. Covert it from VMWare format:
    qemu-img convert -O qcow2 musicbrainz-server-2013-10-14-disk1.vmdk /var/lib/libvirt/images/musicbrainz-server-2013-10-14-disk1.qcow2
  3. Create and start the new VM:
    virt-install --ram 1024 --os-type=linux --os-variant=ubuntuprecise --name musicbrainz-server-2013-10-14 --import --disk /var/lib/libvirt/images/musicbrainz-server-2013-10-14-disk1.qcow2

Using the VM

  1. Once the instance has started up, log in on the console using the username vm and password musicbrainz. This account has sudo privileges -- if you would like to set a root password, you can do that via sudo.
  2. You can SSH into the machine via the port 2222 on localhost of the host OS. (e.g. ssh -p 2222 localhost )
  3. To access the MusicBrainz web site and web service, go to: http://localhost:5000
  4. To access the Search Server web service, go to: http://localhost:8080

Tuning your VM

We recommend that you give your VM 2GB of ram, if that is possible. The more RAM you give to the VM, the faster it is going to run. To change the memory settings, you will need to shut down the VM, change the settings and then re-start the VM. The Postgres database will be automatically tuned based on the available RAM when the VM starts up.

Running Replication

This VM comes "replication ready". To enable replication, and have the database catch up with the latest replication packets, do this:

bin/replicate now

This will load all of the changes to the database since the VM update. NOTE: Loading replication changes might take a long time. If the VM is more than a couple of weeks old, it might be better for you to import a fresh data set. Drop pre-filled database using command

dropdb musicbrainz_db
and check the INSTALL.md file for how to import new data.

Automating Replication

To turn on background replication, run:

bin/replicate start

to turn it off:

bin/replicate stop

We recommend leaving replication off for the time being, until you've built search indexes for the VM.

Building search indexes

The VM comes with support to build search indexes. In order to build the indexes, log in to the account and then:

bin/reindex

Depending on your machine, this may take quite a long time. We recommend that you leave this running overnight. After the indexes are complete, you should be able to carry out indexed searches in your VM.

Accessing the database

To access the main postgres database, you can do this:

sudo su - musicbrainz
cd musicbrainz-server/admin
./psql READWRITE

If you would like to access the DB from outside the virtual box, take a look at how to change postgres connection settings.

Turning the VM into development box

If you would like to use the VM to do development instead of using it as a simple database slave, you'll need to edit lib/DBDefs.pm and set REPLICATION_TYPE to RT_STANDALONE and run admin/psql READWRITE and execute the following queries:

DELETE FROM annotation WHERE editor > (SELECT max(id) FROM editor);
DELETE FROM release_annotation WHERE NOT EXISTS (SELECT 1 FROM annotation WHERE annotation.id = release_annotation.annotation);

then from the command line execute:

admin/psql READWRITE < admin/sql/CreateFKConstraints.sql
admin/psql READWRITE < admin/sql/CreateFunctions.sql

TODO: The server will probably run out of disk space during this process. We need to add instructions on how to move the DB to a new partition.

Migrating the MusicBrainz Server from a Virtual Machine to a Physical Machine

Taken from AskUbuntu.

It is possible to migrate the MusicBrainz Server image (or any virtual image) from a virtual machine onto bare metal, should you desire to.

Requirements

Step 1: Converting the Disk to a Raw format

This step is included mostly for the sake of convenience. By doing some prep work on the image before writing, standard Linux tools can be used to write to and manipulate the disk afterwards. This can be done from your current host environment without booting to your LiveCD.

cd /path/to/image/ 
# for VMWare
sudo apt-get install qemu-kvm
qemu-img convert your-vmware-disk.vmdk -O raw disk.img
# For VirtualBox
VBoxManage internalcommands converttoraw your-virtualbox-disk.vdi disk.img

Option 1: Write the Image to it's Own Disk

This assumes that you're going to overwrite an entire disk. This is the most ideal situation since the server will have an entire disk to work with. If you're looking to have the image installed along side an existing operating system, skip to Option 2.

sudo dd if=/media/external/disk.img of=/dev/sdX

Option 2: Installing the Image Alongside Another OS

This is potentially safer than the above method and is very similar. Your LiveCD must be the same Ubuntu version as the virtual image.

sudo mkdir /media/oldinstall
sudo mount -o loop /media/external/disk.img /media/oldinstall
sudo rsync /media/oldinstall /media/newinstall

Changing Kernel Versions

Parts taken from the Ubuntu community documentation.

For images from 2013-08-01 and older, you will need to change the kernel flavour from -virtual to -generic. The -virtual kernel flavour is lighter, but only supports hardware typically seen in virtual environments while -generic provides support for a wider range of hardware, such as what you would see in a physical server. Both of these versions are provided in the default Ubuntu repositories.

sudo mount /dev/sdXY /mnt
sudo mount /dev/sdXZ /mnt/boot
for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt$i; done
sudo chroot /mnt
sudo apt-get install linux-image-generic linux-header-generic
sudo apt-get purge --auto-remove linux-image-virtual linux-header-virtual

Setup MusicBrainz Server from source code

This can potentially be a very laborious and time consuming method of getting a functioning MusicBrainz server. Using the virtual machine is recommended.

Get a copy of musicbrainz-server from git:

git clone --recursive https://github.com/metabrainz/musicbrainz-server.git musicbrainz-server
cd musicbrainz-server

And follow the instructions in the INSTALL file.

Support

The setup process may look daunting, but please don't let this discourage you; the INSTALL is thorough and contains a lot of information, and we are willing to provide assistance. If you have questions about installing, join us in the #musicbrainz-devel IRC channel or post a question on the developers mailing list and we will attempt to help you out.

We recommend that you dive in and give it a try - who knows how far you'll get and what you might learn along the way!

Requirements

In order to set up a running MusicBrainz server with the full database you will need:

As a developer the following knowledge/skills are beneficial:

Note: The server has never been ported to Windows, and we suspect that it would be a fair amount of work to make that happen.

External Links