SongKong 1.8 with new performance status bar and improved Virtual Machine performance

Whilst SongKong works quicker when using a MusicBrainz virtual machine I confess it was not working quite as quick as I hoped, and I didn't know where the problem lay so I added some performance statistics to the new SongKong taskbar in SongKong 1.8.

MusicBrainz Delay

The MusicBrainz Delay records the additional delays added to SongKong to prevent more than one query per second being sent to MusicBrainz inline with its terms of service. When using a virtual machine the delay should have been zero, but this was not the case. This uncovered an issue that rate limiting was only disabled when doing MusicBrainz lookups, but not for searches, fixing this problem boosted performance

MusicBrainz / AcoustId / Discogs Queue

SongKong uses a number of online databases for matching songs, not just MusicBrainz. All these databases have terms and conditions that restrict usage to one query per second so if SongKong is ready to send queries more quickly than that we have to queue the tasks. This status bar shows the queue for MusicBrainz, AcoustID and Discogs databases and is updated every thirty seconds. The most important queue is the MusicBrainz queue and this can be eliminated completely by using the local MusicBrainz server.

Memory Usage

When SongKong is started the maximum memory it can ever use is specified, this is the second value displayed and SongKong can never use more memory than this regardless of how much memory your computer actually has. Whereas the first value is how much of that memory SongKong is actually using. Note that SongKong uses a garbage collector to release memory that is no longer required and this is called periodically, if SongKong is nowhere near max memory usage it tries to recover memory less often than it could, and so the current memory usage may be higher than SongKong is actually using.

On Windows you can modify the maximum memory allocated by editing SongKong.ini (32-bit) and SongKong64.ini (64-bit)

On OSX you can modify the maximum memory allocated by editing /Applications/SongKong.app/Info.plist

On Linux you can modify the maximum memory allocated by editing songkong.sh

Other Fixes in SongKong 1.8

If you had the Update Genres option enabled it could cause Fix Songs to cancel prematurely, this is noe resolved.

If you removed all countries from Preferred Countries so that it was an empty list this will prevent Fix Songs from running

Faster SongKong matching with the new Musicbrainz VM

To speed up matching it is possible to configure your own local MusicBrainz servers instead of using the official MusicBrainz servers and use this with SongKong. This avoids the requirement to only send one query per second to the MusicBrainz server so speeds up the matching that can be achieved by SongKong,

Additionally this resolves an issue when trying to use multiple copies of SongKong at the same time in the same home network. Usually with a home network external services such as MusicBrainz see all your computers via the same ipaddress, and therefore the rate limiter is applied to all computers as if they were one computer, this effectively prevents you using multiple copies of SongKong on a home network in parallel. But if the server being used does not impose rate limiting then there is no longer a problem.

But the problem has always been the difficulty of setting up a MusicBrainz server in the first place !

Now at long last MusicBrainz have now released a new Musicbrainz VM containing both Musicbrainz Server AND Search Server I've been trying it out and it works great !

The latest official MusicBrainz instructions are available from here. but in this blog we document the instructions so that they are more relevant to using the Virtual Machine with SongKong as follows. Warning some of the steps can take a few hours to run but the procedure is very simple.

1. Ensure you have a BitTorrent client such as Vuze installed, this is required to download the Virtual Machine, and is a much more reliable way to download large files.
2. Download the 9GB MuscbrainzVM via your BitTorrent client
3. Windows/Linux user can download and install from VmPlayer this is free and more reliable then VirtualBox
4. Mac users can use Vm Fusion, although unfortunately this is not free, or try the less reliable VirtualBox
5. The remainder of these instructions assume you are using VMPlayer, but the steps for VMFusion and VirtualBox are much the same.
6. Start VMplayer and choose Import Virtual Machine from the File menu. Select the downloaded file.
7. Select Options and increase Virtual Memory to at least 2GB if your machine is powerful enough
8. Select Play Virtual Machine
9. Log in on the console using the username vm and password musicbrainz
10. Update the database with the latest data added since the VM was created using bin/replicate now this may take a while depending on how old the VM is.
11. Build search indexes, this will take a few hours bin/reindex
12. Get the IP address of your virtual machine.using command:ifconfig | grep eth0 -A 1
13. You can check it is working in browser by pointing to port 5000 of the ipaddress found previously e.g. http://10.1.1.104:5000
14. Configure SongKong, set Server:MusicBrainzServer and Server:MusicBrainz Search Server to the same address e.g. http://10.1.1.104:5000
15. Now SongKong will use the local MusicBrainz server rather than the offical MusicBrainz server. This should be faster and more reliable.

What is the point of the iTunes Copy files to iTunes Media folder when adding to library ?

SongKong 1.7 now has support for updating for iTunes on OSX, so it has feature parity with SongKong on Windows, and playing with this got me thinking about the point of certain iTunes options.

Copy files to iTunes Media folder when adding to library

This option is enabled by default and if this option is enabled whenever adding songs to the iTunes library it will make a copy of every file and add it to iTunes media folder. So you end up with two copies of every file, one in its original location and one stored in the iTunes media folder.

Why would you want two versions of each file ?

 

Managing multiple copies of files is an unnecessary headache,  maybe there is some concern that iTunes will modify the file in some negative way but it seems unlikely.

If you have this option enabled subsequent modifications to the original file within SongKong will have no effect on the file used by iTunes because iTunes will now be using the copied file in the media folder UNLESS you rename or move the file then iTunes will see it as a different file and add it again !

Keep iTunes Media folder organized

Also enabled by default, this renames files based on their metadata once they are added to iTunes if the copy files options is enabled.  But this only renames the copy of the file, it doesn't help you with the original file if it is misnamed. And more importantly you have no say on how your file is renamed, iTunes renaming mask is rather simple and probably not what you really want to use.
 If you are using SongKong flexible rename mechanism why would you use iTunes limited filename and you probably don't want to iTunes to rename them differently as they are copied to the iTunes media folder anyway.

And if you are using SongKong to modify files already within the iTunes media folder you certainly dont want iTunes moving and renaming files as they are getting matched ?

Although you can use both options with SongKong I would recommend disabling both.

Better track matching when submitting new releases to Musicbrainz

For some time Jaikoz has allowed you to submit new releases to Musicbrainz, the selected songs in Jaikoz are used to open the MusicBrainz Release Editor with all the details filled in, so that you just need to check the details and then submit the form.

But in MusicBrainz recordings can be linked to multiple releases so Jaikoz tries to identify when a recording already exists instead of creating a new recording. Doing this makes Musicbrainz more useful so that in an ideal world every recording listed in MusicBrainz is actually a unique recording, and we can see all the releases that the recording is based on.

Jaikoz did this by comparing track titles and durations but sometimes recordings were incorrectly linked when in fact they were different versions of same song.

A more accurate solution

Now with Jaikoz 5.5.0 we have replaced this by matching using Acoustids, only Musicbrainz recordings with the same acoustId as your track are potential candidates, and only then do we do further checks on title and duration to ensure a good match.

Solving a bug in iTunes that prevents artwork being displayed

Occasionally I've seen artwork that displays okay in most applications failing to display in iTunes or the Finder within OSX, recently I had a nice example of this issue and looked into it further.

How do mp3s store text ?

Now mp3s store data in fields called frames, and many such frames allow the text encoding to be chosen. The default is ISO-8859-1 character set but this isn't very good for storing non-latin text so you can also store the text using UTF-16, however the downside of this is that two bytes are used instead of one for each character so it is usual to use this unless you need to. Mp3s use the APIC frame to store artwork, and this includes the text encoding field

Why do you need a text encoding field for storing artwork ?

The APIC frame has a description field allowing to describe the artwork, this is rarely used but you can use it. In Jaikoz if you click on the edit artwork dialog you can add a description or change the type of the artwork.

What has this got to do with iTunes ?

Even if the text encoding field is set to UTF-16 the artwork will not display unless the description text is actually encoded as ISO-8859-1, this is a bug in Tunes

How does this manifest itself in Jaikoz ?

If you change the default text encoding to UTF-16 then Jaikoz will use this for every frame in the file whether you need it or not causing a problem for your artwork

So what can I do?

In Jaikoz 5.5.0 we use text encoding of ISO-8859-1 for APIC frames regardless of the value set for the text encoding, no other frames are affected. The only downside to this is that if you want to add a description for your image, and you add one that cannot be encoded in ISO8859-1 the description will not be saved.

How do I fix existing files ?

Just open them in Jaikoz 5.5.0 and use the File/Force Save option, this rewrites all the fields in the file even if they haven't changed so they are rewritten with the current preferences.

Updates iTunes from SongKong 1.6 for Windows

Despite its shortcomings iTunes is a massively popular music player, not least of all because its the official way to move music from your computer to your iPod. Windows users of SongKong can now get SongKong to update iTunes with any changes to your music library made by SongKong, this includes the following situations:

When the metadata of songs already in iTunes are modified by SongKong.
When songs already in iTunes are renamed or moved by SongKong.
When SongKong matches a file not in iTunes , the file is automatically added to iTunes.
When duplicates are deleted by SongKong, the deleted files are removed from iTunes.

Jaikoz has had this support for some time, but because of various bugs in iTunes it does not always work 100%. So for SongKong I rewrite it from scratch and have created a more robust solution, SongKong completely ignores the iTunes.xml file because this can get out of date or corrupt and instead talks exclusively to iTunes itself, this is much more reliable.

Why Windows only ?

ITunes integration is one of the few area of SongKong that we have to use different technologies for different platforms. For Windows we use COM, but for OSX we will use Applescript, it made sense to get it fully working in one technology first and then move to the other, support for OSX will come in SongKong 1.7.