Music tagging with iTunes

Music tagging with iTunes

Even after all these years iTunes is still an essential application for purchasing and playing music for many. Because of that SongKong and Jaikoz have always supported automatic notification of your iTunes library  as you make changes to your music collection. This is necessary because unlike other software iTunes maintains its own model of everything known to iTunes but does not actively monitor your file system for changes made to your songs outside of iTunes.

For example if you rename a song that is already in the iTunes library without informing iTunes then it will not be able to find that song next time you try and play it.

If you delete a file because it is a duplicate without informing iTunes then it will remain in the iTunes library as an orphaned file.

Unless you add new files to iTunes it will be unaware of them, but if you add a file already known to iTunes as if it is a new file you can get duplicate files within iTunes.

Improvements to iTunes in SongKong 3.1

 We have now made some important improvements to SongKongs iTunes integration.

iTunes notified as songs as saved

Previously iTunes was notified once after all songs had been saved, if a large number of songs were modified this could cause a delay at the end of Fix Songs as SongKong created an iTunes playlist of all the songs modified.

Now we add songs to the iTunes playlist as they are saved, this avoids the delay at the end and also means that if SongKong encounters a problem before finishing details of all songs so far modified are sent to iTunes.

Progress indicator for creating the iTunes model and iTunes with changes 

In order to update iTunes SongKong has to create a presentation of the iTunes model so that it knows the names and locations of all the songs under iTunes control, this is done once every time you start SongKong ( if you have iTunes integration enabled). If you have a very large iTunes collection this can take a while, and if you are only matching one album in SongKong then building of the iTunes model can take longer than matching the album but there is no indicator in the progress bar to indicate this so it can look like SongKong has stopped working.

Updating also used to happen at the end and this could an additional delay, once again there was no progress indicator to make this clear.

But now we have added an indicator for the iTunes model creation and an indicator to show that iTunes has been updated with changes

 

Full SongKong 3.1 Changelist

SongKong 3.1 make some other improvements and fixes as well, these are all listed here

Rules Based Music Tagging

With conventional music tagging there is lots of manual editing or semi-automatic tagging but SongKong is a rules based tagger. The basic idea is that rules define how you want your music collection to be organized, then the rules can be applied to your whole music collection without any manual editing required giving a totally consistent music collection with the minimum effort on your part.  You can change the rules and then reapply to the whole collection to maintain a consistent music collection.

There are three main aspects to incorporating rules based tagging, lets summarize the perfect system:

1. It would be possible to correctly identify every file in your music
2. Once a song is identified the database it has been matched to would contain every desired attribute of the song that you are interested.
3. It would be possible to define rules to extract and apply the data in any way required.

And this is SongKongs implementation:

1. SongKong generates Acoustic fingerprints for each song and this can be looked up in the Acoustid database to identify the song, working in a similar way to Shazam. Acoustid currently contains fingerprints for 16 million songs allowing the majority of your songs to be identified, but Acoustid  can only identify the song not necessarily the album. However by combining this with comparing existing meta-data in your songs we can match to the album as well in the MusicBrainz database. MusicBrainz provides high quality detailed data onmore than 1 million albums.

You can influence the importance of metadata such as by specifying a preference of matching albums form certain countries of a preferred format such as CD or Vinyl.
 
So MusicBrainz/Acoustid provides good coverage for most music collections. But we also use the Discogs database as an alternative source.

Even so, we cannot guarantee 100% matching, but testing shows that in the majority of cases approximately 90% of a collection can be matched to MusicBrainz or Discogs.

It has to be said both of these databases do have better coverage for Pop/Rock/Electronic western music than Classical and World Music. But there are many projects ongoing to fill these gaps, for example the Music Technology Group at the Universistat Pompea Fabra is working with MusicBrainz to add Indian Raag Music.

The good news is that MusicBrainz open approach is fast becoming the de-facto standard database of music information.

2.  Some Music databases contain only the most basic information such as artist, album, title and year of release. But the MusicBrainz database consists of an incredibly rich semantic model, and allows new relationships to be defined between entities.

When SongKong matches to a song in MusicBrainz it is guaranteed to find at least 19 fields (such as artist, album ecetera) and often an additional 35 fields. This includes high quality artwork, usually at a resolution of at least 600 x 600 pixels.

These fields also includes some MusicBrainz Ids, this means that  your songs are compatible with other MusicBrainz enabled applications. It also means you can always look up the original source of the data at any time in the future.

3. SongKong is intended to be easy to use, but there is no one right way to organize your data. Everybody has different requirements based on their personal preferences and how and where they are going to play their music.

Here are a few scenarios handled by SongKong:

You are a DJ only interested in individual songs and has no interest in the album information. SongKong lets you specify exactly how files are named and stored, you can use any piece of metadata for deriving your filename ,and can use powerful Javascript expressions to manipulate the names.

Your music is already stored in iTunes. SongKong can work with iTunes automatically, updating the information in iTunes for songs that are already under iTunes control and adding songs that are not iTunes control to iTunes. 

You like to process songs in a pipeline moving them from unmatched to matched location. SongKong allows you to move files as they are matched to a new location, it also allows files that it failed to match to moved as well.

I hope this post helps you understand the basics of rule based tagging. Jaikoz also incorporates rule  based tagging , but also provides manual editing and semi-automated tagging.
 

New Song Changes Report in Songkong 1.9

With SongKong 1.9 we completely overhauled the song changes report, here is an example of such a report Song Changes Report

The main aims were to increase its responsiveness for big reports, and avoiding information overload by breaking down the information into smaller segments. To achieve this pages are split down into smaller parts and Javascript is liberally sprinkled over.

But the report is still quite complex so this post we explain the different sections in a bit more detail.

Summary

 

This summarizes how many songs have been loaded and checked, how long it took and how many and what percentage have been matched to MusicBrainz, Discogs and if we could find artwork for them.

One thing to consider when matching is whether to rematch songs that have already been matched or whether to ignore songs that are already matched. By default SongKong rematches all your songs but if you disable Basic:Match all songs, even if already matched then all songs that already have a Musicbrainz Id are ignored and not considered in the following sections.

MusicBrainz Summary

 

MusicBrainz is the primary data source used by SongKong.

This section shows what MusicBrainz releases your songs have been matched to. In the left handside we group the matches by the first letter of the album artist, if the first letter is not a letter they are put into the * grouping. Expanding on one of the letter groupings, lists the artists starting with that letter in alphabetical order. Selecting an artist from the list shows how your songs have been matched to their releases on the right hand side of the report.

By default only details of the first matching releases are shown, you can open close details by clicking on the small arrow in the release header information. The header tells you the name of the release together with its MusicBrainz Id, Record Label and Catalog No if known. It all shows you how many of the total songs on the release have found corresponding matches in your collection.

The details provide a hyperlink to open the MusicBrainz page for the release. Below this we list all the songs that were matched together with the song artist, Disc No and Track No.

If you have multiple copies of a song then the same release may be listed more than once, we always match the most complete match first.

Note this section looks at the details of your songs and their mapping to MusicBrainz, if the were already matched to MusicBrainz but not matched during this run of Fix Songs they will still show in this section.

Discogs Summary

This works in the same way as MusicBrainz. Songs can be matched to Discogs either by a direct search to Discogs or by a relationship existing from a MusicBrainz Release to a Discogs release. But Discogs is a secondary data source and Discogs matching and searching can be disabled, if both options are disabled in the Match tab then this section will not be shown.

Song Changes

 

This section shows what changes have been made to your files and is a listed as a file hierarchy. Each leaf folder can be selected and then the changes for each file in the folder are shown on the right handside of the report.

It is important to realize there is not a tight correspondence between songs matched in MusicBrainz/Discogs and Song Changes for the following reasons.

1. If the song was already matched to MusicBrainz and then rematched again to the same release they may not be any change required because the song already contains the fields matched.
2. Most songs can have Acoustids created, these are stored in the files. So even if no match was made to MusicBrainz or Discogs the song may still require saving
3. If the fiename mask has changed then the file will probably need to be renamed even if no metadata added to the files themselves.
4. If Fix Songs is canceled before its has completed songs may have been matched to MusicBrainz but interrupted before the changes could actually be written to file.

The list of changes made to a file is split into

1. Fields Added
2. Fields Modified
3. Fields Deleted

Songs Not Matched

Selecting this list all the songs that are not matched to either MusicBrainz or Discogs in the right hand side using a file hierarchy view.

Songs Not Matched to MuscBrainz

Selecting this list all the songs that are not matched to MusicBrainz in the right hand side using a file hierarchy view.

Songs Not Matched to Discogs

Selecting this list all the songs that are not matched to Discogs in the right hand side using a file hierarchy view.