Jaikoz 7.0 was a major release that completely rewrote the Discogs matching to use jthinks own version of the Discogs server. We got it mostly right, but didn't get it quite right so today we release Jaikoz 7.1 to rectify any regressions.
There are also a few other bug fixes thrown in for good measure, full details here
Meanwhile progress on adding MusicBrainz to the JthinkServer is progressing very well, its going to be really fast compared with the current SongKong and Jaikoz releases.
Wednesday, 20 August 2014
Monday, 4 August 2014
Jaikoz 7 introduces self hosted Discogs server
Quicker than planned I'm pleased to announce Jaikoz 7 has been released, it only has one new feature but its an important one, all of the Discogs matching of Jaikoz is now done via the same self-hosted version of Discogs that is already used by SongKong
Originally we looked at this because of the instability of the Discogs API in recent months, the latest forthcoming change would have required all Jaikoz customers to have had a Discogs account and seperately authorise Jaikoz to access their account. Having seen how badly Discogs implemented this for changes to their image archive I wasn't keen to expose Jaikoz customers to this again.
But there are may other advantages of self-hosting Discogs:
We can customize the search to work exactly how we require it for Jaikoz rather than working within the constraints of the Discogs provided search, this means we can get better search results and dramatically reduce the number of requests required, so you get better matches faster. We can control rate-limits based on usage by our customers only to increase throughput, and we can also move some processing out of the Jaikoz itself on the server simplifying Jaikoz code development and reducing client CPU usage, giving additional speedups.
Our Discogs server is updated monthly from the latest data provided by Discogs.
Additionally I do plan to support a way to add new releases to the Jthink database as they are added to the Discogs database for those of you who add new releases to Discogs and want to tag them from Jaikoz immediately.
In the next few months we intend to self-host MusicBrainz data as well, and because MusicBrainz is the primary database used by Jaikoz this will provide additional improvements in speed and accuracy over and above the Discogs improvements.
Full set of changes in this release can be found here
Saturday, 2 August 2014
SongKong 2.1.2 with some final fixes for the new features introduced in SongKong 2
I am now back from holiday and have a new release of SongKong that solves issues with the new Discogs Matching, Genre grey lists and Featured Artists option introduced in SongKong 2.0 - full details here
Now the rush is on to move Jaikoz to use the Jthink Discogs server before Discogs implement their api changes on August 15th, time is a little tight.
Now the rush is on to move Jaikoz to use the Jthink Discogs server before Discogs implement their api changes on August 15th, time is a little tight.
Friday, 18 July 2014
Finding more artwork by looking for similar releases
Hot on the heels of SongKong 2.1 we have SongKong 2.1.1 and this includes an important fix to the artwork finder.
Within MusicBrainz different versions of the same release can all be grouped together under a Release Group. For example a CD version and an LP version of the same release would be part of the same release group. Artwork is added at the release level but usually the artwork for all releases within a release group is the same or very similar. Because of this if no artwork can be found for the release that SongKong matched to then SongKong will check releases that are part of the release group and if any artwork found use that instead.
Discogs has a similar concept to Release Groups called Masters. Masters also nominate one release as the Main Release which is often the chronologically earliest. But currently most releases in Dicogs are not linked to Masters so it isn't terribly useful to SongKong at the moment.
Instead we search for simply search for releases with the same artist and release name as our matched release and check the artwork of these releases. At least that is what it was meant do to but it wasn't working, now fixed in SongKong 2.1.1 so the artwork count should continue to rise.
Some Background
Within MusicBrainz different versions of the same release can all be grouped together under a Release Group. For example a CD version and an LP version of the same release would be part of the same release group. Artwork is added at the release level but usually the artwork for all releases within a release group is the same or very similar. Because of this if no artwork can be found for the release that SongKong matched to then SongKong will check releases that are part of the release group and if any artwork found use that instead.
Discogs
Discogs has a similar concept to Release Groups called Masters. Masters also nominate one release as the Main Release which is often the chronologically earliest. But currently most releases in Dicogs are not linked to Masters so it isn't terribly useful to SongKong at the moment.
Instead we search for simply search for releases with the same artist and release name as our matched release and check the artwork of these releases. At least that is what it was meant do to but it wasn't working, now fixed in SongKong 2.1.1 so the artwork count should continue to rise.
Thursday, 17 July 2014
Featured Artists option now working in SongKong 2.1 release
Last week we released SongKong 2.0 with masses of improvements, unfortunately as is de riguer we also introduced a few bugs and the new When Tracks contain Featured Artists option didn't actually work.
There are all now fixed in SongKong 2.1 now available from the Download page
I thought it worth revisiting the featured artists option with some examples.
With the default Add all contributing artists to the artist field option enabled all artists credited for that track are added to the artist field and all artists credited to the release as a whole are added to the album artist field. So if the songs were matched to this release then we would have:
Album Artist : Mark Ronson
Album : Valerie
Track 1 Artist:Mark Ronson feat, Amy Winehouse
Track 1 Title :Valerie
If we used Only use main artist in the artist field and discard other artists option then we would get
Album Artist : Mark Ronson
Album : Valerie
Track 1 Artist:Mark Ronson
Track 1 Title :Valerie
or if we used Only use main artist in the artist field and add others to the title field then we would get
Album Artist : Mark Ronson
Album : Valerie
Track 1 Artist:Mark Ronson
Track 1 Title :Valerie (ft. Amy Winehouse)
If a song is equally credited to multiple artists then the multiple artists will be added to the artist field whatever option is selected, for example in this release David Bowie and Mick Jagger are equally credited at track and release level so both will always be added to the artist and album artist fields with their join phrase to give us
Album Artist : David Bowie & Mick Jagger
Album : Dancing in the Street
Track 1 Artist:David Bowie & Mick Jagger
Track 1 Title :Dancing in the Street
There are all now fixed in SongKong 2.1 now available from the Download page
I thought it worth revisiting the featured artists option with some examples.
With the default Add all contributing artists to the artist field option enabled all artists credited for that track are added to the artist field and all artists credited to the release as a whole are added to the album artist field. So if the songs were matched to this release then we would have:
Album Artist : Mark Ronson
Album : Valerie
Track 1 Artist:Mark Ronson feat, Amy Winehouse
Track 1 Title :Valerie
If we used Only use main artist in the artist field and discard other artists option then we would get
Album Artist : Mark Ronson
Album : Valerie
Track 1 Artist:Mark Ronson
Track 1 Title :Valerie
or if we used Only use main artist in the artist field and add others to the title field then we would get
Album Artist : Mark Ronson
Album : Valerie
Track 1 Artist:Mark Ronson
Track 1 Title :Valerie (ft. Amy Winehouse)
If a song is equally credited to multiple artists then the multiple artists will be added to the artist field whatever option is selected, for example in this release David Bowie and Mick Jagger are equally credited at track and release level so both will always be added to the artist and album artist fields with their join phrase to give us
Album Artist : David Bowie & Mick Jagger
Album : Dancing in the Street
Track 1 Artist:David Bowie & Mick Jagger
Track 1 Title :Dancing in the Street
How do we decide if they are credited equally ?
We look at the join phrase, currently ' & ', ' / ' and ' , ' are treated as equal weighting, in all other cases we consider the first artist to be the main artist. This will probably be expanded for other casesFriday, 11 July 2014
SongKong 2.0 is here
Self hosted Discogs Server
Pleased to say that SongKong 2.0 is now available. There are a host of improvements in this release but the most significant is that Discogs requests are now served by our own server rather than Discogs itself. The great thing about this is we can customize the search to work exactly how we require it for SongKong rather than working within the constraints of the Discogs provided search, this means we can get better search results and dramatically reduce the number of requests required, so you get better matches faster. We can also move some processing out of the SongKong itself on the server simplifying SongKong code development and reducing CPU usage, giving additional speedups. Our Discogs server will be updated monthly from the latest data provided by Discogs.Over the next month Jaikoz will be updated to use the self hosted Discogs Server as well.
In the next few months we intend to self-host MusicBrainz data as well, and because MusicBrainz is the primary database used by SongKong this will provide additional improvements in speed and accuracy over and above the Discogs improvements.
Genre Grey Lists
Now SongKong comes with a genre list Grey List that allows you to control what genres are used. Because genres are subjective there is no correct genre for a release but grey lists let you configurehow genres should be processed.
When songs are matched to a Discogs release we check the genres provided by Discogs for that release against the genres listed below, if the Discogs release has multiple genres in this list then we use the genre that comes earliest in this list, so the order of the list is important - if you want to keep them you should put more specialist genres such as 'Power pop' before more general categories such as 'Pop'. If no match is found no genres will be added to songs matched to that release. Having found a match SongKong uses that value if that is the only value on the line, but if there is a ';' and then a second value it will use that second value, this is useful for mapping multiple similar genres to a single genre.
Example:Map three genres to one
All three genres are mapped to a more generic genre when such specific genres are not required
Black Metal;Metal
Goth Metal;Metal
Doom Metal;Metal
Example:Dealing with spelling variations
Maps Bossa nova to Bossanova, Bossanova is allowed through untouched
Bossanova
Bossa nova;Bossanova
The default list is a list of all the styles and genres currently in the Discogs database with no additional mapping. But this is just the start, more improvements to come with later releases of SongKong.
New Update from Metadata Option
Previously SongKong could either rematch all songs or ignore songs already matched but now there is a new option Update Metadata and Filename Only that can be selected from For songs already matched to MusicBrainz on the first tab.With the default Update Metadata and Filename Only option SongKong will only try to match songs that have not already been matched to MusicBrainz previously, but it will update existing matches with the latest data or make adjustments based on customer preferences. For example if Update Artwork was disabled when originally matched and it is now enabled SongKong will check for artwork without modifying the releases that the songs have already been matched to. Or if you have changed your file rename mask then the new mask will be applied to these already matched songs in additionally to newly matched songs.
If the option is Rematch then SongKong disregards any matching previously done by SongKong or any other MusicBrainz enabled tagger such as Jaikoz and rematches from scratch.
If the option is Ignore then SongKong ignores any songs already matched to MusicBrainz releases and does not process them any further. Note that songs matched to a MusicBrainz recording only or a Discogs release are not ignored.
When Tracks contain Featured Artists Option
This new option is available on the Fix Songs Format tab.Sometimes tracks are credited to multiple artists, and with the default Add all contributing artists to the artist field option set all these artists will be used in the artist field for this song. But sometimes this can make it difficult to organize your songs because songs by the same main artist can be split up by your music player, if the artist worked with a number of contributing artists. To make things simpler we can ignore these additional artists by selecting the Only use main artist in the artist field and discard other artists option
Alternatively if you don't want the featured artists information to be completely lost they can be added to the title field instead with the Only use main artist in the artist field and add others to the title field. Note if a song is equally credited to multiple artists then the multiple artists will be added to the artist field whatever option is selected, the options only have an effect when there is a main artist and featured or guest artists.
Full details of all changes can be found in here
Thursday, 12 June 2014
MusicBrainz releases new Virtual machines of Search and Database
For faster matching with SongKong and Jaikoz you can run it against a local copy of the MusicBrainz database, these are provided as virtual machines that can be run on most hardware. MusicBrainz have just released a new version of these virtual machine images that contain the latest data and the latest code including the changes to the database design made in May 2014.
I strongly recommend using the VMWare image rather than the VirtualBox image if you want to take advantage of this, additional instructions for SongKong can be found here and for Jaikoz here
I strongly recommend using the VMWare image rather than the VirtualBox image if you want to take advantage of this, additional instructions for SongKong can be found here and for Jaikoz here
Friday, 16 May 2014
Improvements to Match to Recording Only section of the report in SongKong 1.26
SongKong 1.26
Today we release SongKong 1.26 this has a number of refinements to the matching process, testing has shown some very nice improvements to matching accuracy and coverage.Full details here
Match to Recording Only
There are also some nice improvements to Match to Recording Only section of the report, and I thought I would use this as an opportunity to explain this section further.Songs are usually distributed as a release, be it an album, a compilation , an e.p. or a single the songs is part of a larger release. This also applies to digital downloads, although it is easy to purchase tracks they are still distributed as part of a release - even if the release is just a one song digital single.
With SongKong ideally we want to match group of songs to a release, but here are many reason why it might not be possible to do this all of the following problems could prevent a match:
- The release does not exist in MusicBrainz or Discogs
- You do not have all the songs of the release and you have Only allow match if all songs in folder match to one album enabled
- Your songs for that release are distributed over multiple folders and you have Only allow match if all songs in folder match to one album enabled.
- The songs metadata is completely incorrect
- The songs have no metadata and are not known to AcoustId
- The songs match to multiple recordings by different artists via Acoustid
- The songs length is incorrect because it has been badly ripped.
Only allow match if all songs in folder match to one album are enabled we don't do this.
But even though we cannot identify the album this doesn't prevent SongKong matching the song. SongKong can use fingerprints to match to Acoustid, then use Acoustid to match to a suitable song (or recording in MusicBrainz parlance). When Acoustid matches to multiple recordings we take into account the existing song title (if any). Then we look at how many sources the choices have, Acoustid - Recording pairs that have been matched form multiple sources (i.e users) are more likely to be correct. Finally if we cannot make a safe decision about the name of the song we don't match to the recording. If we do make a match the only fields that are updated are:
- MusicBrainz Recording Id
- Song Title
- Song Artist
The Recording Only section of the report shows these matches. Originally SongKong simply listed the songs that were matched per artist in alphabetical order. But this was not very easy to decipher as a SongKong user myself I found I wanted the songs grouped by existing album name, this then helped me pinpoint problem recordings that were not matched often explaining the reason why the group of songs not matched to an album in the first place.
So here it is:
The MusicBrainz Recording shows the song titles that the songs have been matched to, clicking on a title will take you to the MusicBrainz page for that recording.
The Existing Info (Discno, TrackNo, Filename) column shows just that if they exist. Remember Match to Recording Only doesn't add release specific information like Disc No or Track No so these values are only shown if the song already contained them in its metadata. In this screenshot we see that songs did have Track No set but not Disc No set, so only track no and filename is shown.
Songs are grouped by existing release title of it exists, not the folder. So if two songs from different folders with the same value for artist and release are matched this means they will be listed in the same section. Whereas two songs in the same folder with different values for release would be listed in different sections.
Any songs matched to recording will always have an Artist field because this is updated when matched but they may not have a release field, in this case they are just grouped by the artist name
Both these points can be seen in these screenshot:
Although both songs are in the C:\MusicUnmatched\The Tea Party\Splendor Solis folder they are split into two sections because only one has the release field set as we can see with the help of Jaikoz
Thursday, 15 May 2014
MusicBrainz adds Series support
MusicBrainz have just done their bi-yearly schema update release,
although MusicBrainz do releases every two weeks throughout the year
they can only make changes that modify the database schema (such as
adding new tables) in schema update releases.
Highlight of this release is support for Series as suggested by long time Jaikoz user Elliot Chance http://tickets.musicbrainz.org/browse/MBS-2714. A series can link together releases or release groups such as this Bravo - German Pop Hits compilation
and you can also browse all the Series added so far.
This is interesting for SongKong and Jaikoz, I feel I should be making use of this, but not clear how yet - does anyone have any good ideas ?
Full release details can be found here ,and there are changes in the corresponding search done by yours truly.
There should be also new MusicBrainz virtual machine released with the latest database within the next few days.
Highlight of this release is support for Series as suggested by long time Jaikoz user Elliot Chance http://tickets.musicbrainz.org/browse/MBS-2714. A series can link together releases or release groups such as this Bravo - German Pop Hits compilation
and you can also browse all the Series added so far.
This is interesting for SongKong and Jaikoz, I feel I should be making use of this, but not clear how yet - does anyone have any good ideas ?
Full release details can be found here ,and there are changes in the corresponding search done by yours truly.
There should be also new MusicBrainz virtual machine released with the latest database within the next few days.
Wednesday, 7 May 2014
A new Discogs Image Archive
New Discogs Image Archive with SongKong 1.25
SongKong and Jaikoz both use the Cover Art Archive as their primary resource for artwork but a very important secondary resource has always been the cover art provided by Discogs.In March 2014 Discogs changed their terms and conditions so that from now any user of a third party application could access a maximum of 100 images in a 24 hour period, and was required to authenticate their access using OAuth. The reasoning behind this was to prevent websites hot limiting to Discogs images, I can certainly understand that issue but think that giving 3rd party applications access via an application key would have been a more sensible option. Forcing users to explicitly authorize their access was onerous but I went along with it and released SongKong 1.23 with the required authentication.
It then became apparent the 1000 images limit was also per application. So any application, regardless of how many
users it has, can only access 1000 images per day. This made it pointless for each customer to individually authenticate their access so I released SongKong 1.24 and this removed the requirement for customer authentication. But we still had a significant problem - a limitation of 1000 images per users was not too bad, but a limitation over the complete customer base of SongKong made use of Discogs cover art unsustainable. And of course the problems are even worse for Jaikoz with its larger customer base.
The solution was to cache the images provided by Discogs so any image only has to be looked up once from Discogs, then additional customers requesting the same image can use the cached image. Of course this requires the cached images to be available from a central server and I considered setting up my own Discogs Image Archive but this takes some effort and some time to build up the size of the archive. Luckily such a service has just been launched courtesy of the One Music Api, this archive doesn't yet contain all the images provided by Discogs but is growing every day and already provides good coverage.
Today I release SongKong 1.25 with support for the One Music Api Image Endpoint. Within a week I expect to have a new release of Jaikoz with the same support
Saturday, 26 April 2014
Sharing filename masks between SongKong and Jaikoz
A longtime customer of Jaikoz and SongKong asked on the forums whether you can share filename masks between SongKong and Jaikoz,well yes you can but with a couple of adjustments.
Both applications use Javascript expressions to rename files, although this powerful syntax takes a few minutes to understand, but there are three important differences.
An Example
SongKong mask
(Note if notempty() function is predefined function)
Both applications use Javascript expressions to rename files, although this powerful syntax takes a few minutes to understand, but there are three important differences.
Two masks versus one
In both applications we have the concept of a base folder, subfolder and filename which is described in detail here. With SongKong there is a single mask to modify the subfolder and filename in one go, in contrast Jaikoz has one mask for modifying the subfolder and one for modifying the filename.Field names
Fields names for both applications are the same, except Jaikoz uses trackno to represent a trackno and SongKong uses track.Functions
In Jaikoz Javascript functions needed to be included as part of the rename mask, but in SongKong they can be stored separately and referenced by masks, SongKong also allows rename masks to be named and stored so you can easily switch between different masks.An Example
SongKong mask
(Note if notempty() function is predefined function)
ifnotempty(albumartist,'/') + ifnotempty(album,' ') + (year.length>0 ? '(' + year + ') ' : '') + (mbreleasecountry.length>0 ? '(' + mbreleasecountry + ') ' : '') +'/' + (disctotal>1 ? discno + '-' : '') + track + ' - ' + title
Jaikoz SubFolder mask
function ifnotempty(value,sep){ return value.length > 0 ? value + sep : ''; } ifnotempty(albumartist,'/') + ifnotempty(album,' ') + (year.length>0 ? '(' + year + ') ' : '') + (mbreleasecountry.length>0 ? '(' + mbreleasecountry + ') ' : '')Jaikoz Filename Mask
(disctotal>1 ? discno + '-' : '') + trackno + ' - ' + title
Wednesday, 23 April 2014
SongKong 1.24 appears - Discogs Authorisation disappears
Discogs Authorisation Disappears
This new release of Songkong removes the short lived requirement to create a Discogs
account and authorise SongKong to use that account. However, unfortunately Discogs have recently implemented some rather
misleading and draconian rules regarding access to their image archive that doesn't
allow us to make full use of it - so this release is just an interim step.
But I have been searching for a better solution and think I have found it, this should be ready within the next month and will provide some serious improvements in cover art
But I have been searching for a better solution and think I have found it, this should be ready within the next month and will provide some serious improvements in cover art
Improved Multi-Disc Matching
As SongKong loads folders of songs for matching it looks out for multi-disc releases that have been stored as one folder per disc, it does some further checks and if the folders pass the checks then SongKong attempts to match the multi-folder set as one release and only resorting to checking each folder individually if a match could not be found.
But before this version the subfolders had to have a particular naming convention, the subfolders either had to start with the word Disc or just be a number such as Disc 01 , disc 2 or 03. Now with SongKong 1.24 other naming conventions such as CD01, Vol 1 or Volume 3 are supported, but this is still English centric so let me know if you use another naming convention that needs adding.
Its worth noting that if you just Only allow match if all songs in folder were matched enabled then there is still a good chance that a multi disc release is properly matched even if the release was not recognised as a multi-disc release at the matching stage. Each folder can get correctly matched to a disc of the release.
However if you have Only allow match if all songs in album were matched then matching at the subfolder stage only would not work because each subfolder only matches one disc of the release rather than the whole release.
Error Reporting
If SongKong is configured to update iTunes it will now complain and halt processing if it is unable to create a model of the iTunes library. It will also halt processing if it hits memory limits, although in normal circumstances this should never happen. Previously these errors were not correctly reported.
General Matching
A number of improvements have been made to improve the accuracy of release and recording only matches
Full change log is available here
Thursday, 17 April 2014
My first Sonos system
Last week I purchased a Sonos Connect to connect Sonos to my separates system and to find out what Sonos requires for the perfect metadata experience for users.
Here are some details about the installation experience and my first (very initial) impressions. Everything about the packaging and product itself is so Apple like I wonder if a former Apple designer designed it for them, but whoever did the design they did a good job.
Now my Hi-Fi is not in the same room as my router and I know at least one Sonos component should be connected directly to your router, but I have used Solwise home plugs in the past to spread my network and I couldn't see how Sonos could differentiate between a direct ethernet connection and an ethernet connection via a Solwise plug. I have router in one room with ethernet connection from router to Solwise Homeplug Av500. Then I have my hifi in next door room with sonos connect connected by ethernet cable to another solwise plug. Both plus are on the same electrical circuit.
I download sonos controller for my PC and the software tells me to hold down the two buttons and release, as soon as I do this the PC software goes to the next stage but after a while it tells me it could not find any Sonos devices connected.
If I repeat the process but turn of the Solwise plug then pressing the two buttons has no effect. So this told me that Sonos software is detecting that I've clicked on the Sonos Connect via my Solwise home connection yet it still isn't happy about something. My first thought was that I needed the bridge so I disconnected the Connect from my hi-fi and connected it directly to the router and it still doesn't work.
I then tried downloading the Sonos Controller for my Mac and this worked, reconnected the connect to hifi and solwise and it still worked, so the problem was actually the PC (and I havent resolved this issue yet)
As yet I have no have no connection issues at all and have ordered two Sonos Play 1s to test out the Sonos network over a larger area, the Sonos network is a Mesh network, every single Sonos component acts like a mini router allowing it to send a signal to any other Sonos component, so fingers crossed that this will work as expected.
My only disappointment so far is that Sonos supports Spotify Premium but not the free Spotify account I have had since the early days of Spotify.
Here are some details about the installation experience and my first (very initial) impressions. Everything about the packaging and product itself is so Apple like I wonder if a former Apple designer designed it for them, but whoever did the design they did a good job.
Now my Hi-Fi is not in the same room as my router and I know at least one Sonos component should be connected directly to your router, but I have used Solwise home plugs in the past to spread my network and I couldn't see how Sonos could differentiate between a direct ethernet connection and an ethernet connection via a Solwise plug. I have router in one room with ethernet connection from router to Solwise Homeplug Av500. Then I have my hifi in next door room with sonos connect connected by ethernet cable to another solwise plug. Both plus are on the same electrical circuit.
I download sonos controller for my PC and the software tells me to hold down the two buttons and release, as soon as I do this the PC software goes to the next stage but after a while it tells me it could not find any Sonos devices connected.
If I repeat the process but turn of the Solwise plug then pressing the two buttons has no effect. So this told me that Sonos software is detecting that I've clicked on the Sonos Connect via my Solwise home connection yet it still isn't happy about something. My first thought was that I needed the bridge so I disconnected the Connect from my hi-fi and connected it directly to the router and it still doesn't work.
I then tried downloading the Sonos Controller for my Mac and this worked, reconnected the connect to hifi and solwise and it still worked, so the problem was actually the PC (and I havent resolved this issue yet)
I tried playing Internet radio through Sonos and it worked perfectly time.
I then tried connecting to my music library on my NAS and it took a while, but once completed that also worked perfectly. When I view my music on my Mac or Phone controller not seeing any issues with metadata so far, but I noticed I have no artist images - something that I will investigate for Songkong and Jaikoz.
As yet I have no have no connection issues at all and have ordered two Sonos Play 1s to test out the Sonos network over a larger area, the Sonos network is a Mesh network, every single Sonos component acts like a mini router allowing it to send a signal to any other Sonos component, so fingers crossed that this will work as expected.
My only disappointment so far is that Sonos supports Spotify Premium but not the free Spotify account I have had since the early days of Spotify.
Wednesday, 16 April 2014
Getting Discogs Cover Art with SongKong
Discogs Image Authorization
SongKong uses various sources for finding cover art. Its primary source is the cover art provided by MusicBrainz via the Cover Art Archive, but the second most important source is the artwork provided by Discogs. However unfortunately recently Discogs changed the conditions of their API so that it no longer allows access to these images unless you have a Discogs account and authorise SongKong to use that account.
The good news is that Discogs accounts are completely free and straight-forward with no requirement to enter bank card details or sign up for any service so creating a Discogs account if you don't already have one is not such a hardship.The bad news is that Discogs is currently getting overloaded regularly and when this happens access to the API is curtailed, this includes the one-off image authorisation detailed below. So if possible don't try at internet peak times (evening time US timezone) , if it does fail at the first stage just try again.
With SongKong 1.23 released today it is recommended that you create a Discogs account if you do not have one already and then authorise SongKong for the best SongKong results as follows
1.Open your webbrowser and go to discogs.com
2. Login to Discogs or signup for an account if you do not already have one
3. Select File:Discogs Image Authorization within SongKong, and if you are logged into your Discogs account your webrowser should open a page similar to:
If instead SongKong complains that Image Authorization failed , please try again at a quieter time.
4. Select the Authorise button and it should take you to a page like
5. Go back to SongKong, there should be a user dialog open, enter the code into it as follows:
6. Select OK and the following window should display
7. And that's it, now if SongKong is unable to get cover art from a MusicBrainz release it can probably get one from a linked Discogs release.
Jaikoz customers, the same Discogs image authorization will be added to Jaikoz within two weeks.
SongKong 1.23
In this new release there also many improvements to matching and to the generated reports, I will explain these in more details in another post but for now all the fixes are listed athttp://jthink.net/songkong/news_build1030.jsp
Saturday, 5 April 2014
SongKong 1.22 with new matching improvements
Today we release a new version of SongKong as well as a number of bug fixes it includes some important improvements.
But in MusicBrainz songs are known as recordings and the same recording can be found on multiple releases whereby each release/recording combination is known as a track. If a songs Acoustid matches to just one MusicBrainz recording then we have can have almost absolute certainty that we have matched the correct song.
So in this new release if the match grouping to folder stages fails to match but we have acoustids for the songs that can be matched to individual MusicBrainz recordings then we can update the song artist and title without making only any assumptions about the release they belong to. This matching occurs regardless of any other options enabled or disabled.
This is incredibly useful for songs that have no metadata at all, or songs with incorrect titles, even though the albums are not known the songs are now correctly identified. Because the matching is done at the match to MusicBrainz stage the new information can be used to help match the songs to a Discogs release instead, so we are now making uses of Acoustids with Discogs even though Discogs doesn't directly support Acoustid !
These matches have their own section in the Fix Songs report. Because the matches are only partial they are included in the MusicBrainz match count but the songs are not moved to the matched folder if a Matched folder is enabled, and are not renamed if Rename filename from Metadata is enabled.
If the Only allow match if all songs in one folder match to one album is enabled then all songs in the grouping have to be matched to one folder. But to account for you having incomplete albums the album they are matched to can contain more songs than are matched.
But if you have complete albums you can get more accurate matching by enabling the new Only allow match if all songs in album were matched option as well then all the songs in the folder must match to same album and all songs on the album must be matched, so if there are 10 songs in folder it can only match a 10 song album
If Only allow match if all songs in one folder match to one album is enabled and Only allow match if all songs in album were matched is not enabled then any album matched must be completely matched but this doesn't have to be on a folder by folder basis, so for example if you have a folder containing 10 songs that failed to match any album but metadata in the allowed grouping of the songs into two groupings of 5 each, and one of those grouping could be matched to a 5 track release that would be okay.
Match song to recording only
In previous versions of SongKong we try to matching groupings (usually folders) of songs to a MusicBrainz release (usually an album) and optionally a Discogs release. But if we are unable to match all songs in the grouping to a release then as a customer you only have the option of not matching the songs at all or matching song by song and possibly matching to the incorrect release.
But in MusicBrainz songs are known as recordings and the same recording can be found on multiple releases whereby each release/recording combination is known as a track. If a songs Acoustid matches to just one MusicBrainz recording then we have can have almost absolute certainty that we have matched the correct song.
So in this new release if the match grouping to folder stages fails to match but we have acoustids for the songs that can be matched to individual MusicBrainz recordings then we can update the song artist and title without making only any assumptions about the release they belong to. This matching occurs regardless of any other options enabled or disabled.
This is incredibly useful for songs that have no metadata at all, or songs with incorrect titles, even though the albums are not known the songs are now correctly identified. Because the matching is done at the match to MusicBrainz stage the new information can be used to help match the songs to a Discogs release instead, so we are now making uses of Acoustids with Discogs even though Discogs doesn't directly support Acoustid !
These matches have their own section in the Fix Songs report. Because the matches are only partial they are included in the MusicBrainz match count but the songs are not moved to the matched folder if a Matched folder is enabled, and are not renamed if Rename filename from Metadata is enabled.
Only allow match if all songs in album were matched
If the Only allow match if all songs in one folder match to one album is enabled then all songs in the grouping have to be matched to one folder. But to account for you having incomplete albums the album they are matched to can contain more songs than are matched.
But if you have complete albums you can get more accurate matching by enabling the new Only allow match if all songs in album were matched option as well then all the songs in the folder must match to same album and all songs on the album must be matched, so if there are 10 songs in folder it can only match a 10 song album
If Only allow match if all songs in one folder match to one album is enabled and Only allow match if all songs in album were matched is not enabled then any album matched must be completely matched but this doesn't have to be on a folder by folder basis, so for example if you have a folder containing 10 songs that failed to match any album but metadata in the allowed grouping of the songs into two groupings of 5 each, and one of those grouping could be matched to a 5 track release that would be okay.
Undo Changes can now select the original location of files to revert
By default Undo Changes work on the current location of files and folders that you want to revert the changes for, but now you can specify SongKong looks for songs that were originally in a location that they have since been moved from. This can be very useful if you are looking to revert changes to files that were originally in one folder but have now been moved to multiple folders.Other fixes and Improvements
The full list can of changes can be found at hereMonday, 31 March 2014
How do iTunes options affect SongKong ?
If you are an iTunes user SongKong can automatically update iTunes with any changes it makes -but there are a couple of options in iTunes that can subtlely affect the interaction. I recommend they are disabled, not only do they cause problems for SongKong but they increase disk space usage and they reduce interoperability with many other applications as well.
Both options can be found in the iTunes preferences advanced tab, and they are enabled by default
Keep iTunes Media Folder Organized
Copy files to iTunes Media Folder when adding to library
If you rename these files then they will be seen as new files and added again to iTunes, this could happen if you changed the rename mask or if the first time SongKong only added fingerprints, and second time actually found a match.
If you still want your songs copied to iTunes media folder and you want songs renamed according to their metadata using the iTunes format you can let SongKong rename the files instead of iTunes by selecting a suitable format in the File naming tab
These problems are also relevant to Jaikoz users.
Both options can be found in the iTunes preferences advanced tab, and they are enabled by default
Keep iTunes Media Folder Organized
Copy files to iTunes Media Folder when adding to library
1. Both Options Off
This is the best option, then iTunes makes no modifications to your files.2. Keep iTunes Media Folder Organized enabled only
Just enabling this option has no effect if you have nothing in your iTunes media folder.3. Copy files to iTunes Media Folder when adding to library enabled only
If the files do not exist in iTunes then if they are matched by SongKong then when they are saved copies are added to iTunes. But note if you make more metadata changes to these files then that will have no effect on the ones in iTunes because now iTunes is interested in the versions it has added. So from now on you should be loading the iTunes media library rather than the original file location.If you rename these files then they will be seen as new files and added again to iTunes, this could happen if you changed the rename mask or if the first time SongKong only added fingerprints, and second time actually found a match.
4. Both Options Enabled
Same problems as above, but additionally if you do load the iTunes Media folder into SongKong and match the songs then iTunes will move and rename the files according the latest metadata. This means that the songs may no longer be where SongKong expects them to be so that trying to use Undo Changes and selecting the last location will fail, you will have to select the original location.If you still want your songs copied to iTunes media folder and you want songs renamed according to their metadata using the iTunes format you can let SongKong rename the files instead of iTunes by selecting a suitable format in the File naming tab
Conclusion
Its always best to be in control, but if you want to continues using these iTunes options at least you aware of the potential problems. If you use create multiple iTunes libraries for different scenerios note that the two options will always be enabled by default for every new libraryThese problems are also relevant to Jaikoz users.
Thursday, 27 March 2014
Why do we still call digital music taggers Mp3 Tag Editors ?
Browsing the web this morning I was surprised that most articles about tagging music still refer to software such as Jaikoz or SongKong as a type of MP3 Tag Editor or MP3 Tagger. Whilst MP3 is probably still the most common form of digital music shouldn't we be talking about Digital Music Taggers or simply Music Taggers these days. I hope mp3 doesn't become ingrained as the defacto word for digital like Hoover became for Vacuam Cleaner
Certainly mp3 is not an acceptable format when purchasing digital music these days, for example iTunes uses mp4. And with the availability of cheap disk space ripping CD's to lossless formats like Flac or Apple Lossless is the most sensible option.
Another annoyance about MP3 is capitalization, after all these years I'm very unsure whether to refer to it as MP3 or Mp3
I was also surprised that the wikipedia entry for tag editor has one page for two completely different type of taggers, music taggers and image taggers.
Actually the word Tag was the name given to the metadata format used by Mp3s (ID3) to describe the container for storing metadata, most formats don't actually use this terminology and it would be better to say Music Metadata Editor rather than Music Tag Editor. Many applications have got confused about the word Tag and use it to refer to parts of the metadata within the tag like artist or genre, in fact Ive been guilty of this myself in the past.
Having said that Tagger is a a nice shortening of Tag Editor, so my preferred choice is still Music Tagger, anybody have a better alternative ?
Certainly mp3 is not an acceptable format when purchasing digital music these days, for example iTunes uses mp4. And with the availability of cheap disk space ripping CD's to lossless formats like Flac or Apple Lossless is the most sensible option.
Another annoyance about MP3 is capitalization, after all these years I'm very unsure whether to refer to it as MP3 or Mp3
I was also surprised that the wikipedia entry for tag editor has one page for two completely different type of taggers, music taggers and image taggers.
Actually the word Tag was the name given to the metadata format used by Mp3s (ID3) to describe the container for storing metadata, most formats don't actually use this terminology and it would be better to say Music Metadata Editor rather than Music Tag Editor. Many applications have got confused about the word Tag and use it to refer to parts of the metadata within the tag like artist or genre, in fact Ive been guilty of this myself in the past.
Having said that Tagger is a a nice shortening of Tag Editor, so my preferred choice is still Music Tagger, anybody have a better alternative ?
Wednesday, 26 March 2014
Why does the Artist column in Jaikoz Column Browser not match the values of my artists ?
Today a customer asked why even though they had modified the artist field of some of their songs that Jaikoz still shows the old artists names in the column browser, even after removing the sort artist and sort album artist fields.
The culprit is the artists field, Jaikoz uses the artists field in preference to the artist field for the Column Browser if the artists field is not empty. The artists field is only used by Jaikoz and SongKong currently but other applications are looking at incorporating it. It only differs when a song is attributed to multiple artists. In this case each artist is added separately to the artists field allowing you to search for a song by just the first (main artist) involved in the song.
i.e 'Jackson' by 'Johnny Cash & June Carter'
ARTIST:JohnnyCash & June Carter
ARTISTS[0]:Johnny Cash
ARTISTS[1]:June Carter
So the column browser would have one entry in the artist column containing 'Johnny Cash', but if you delete the artists field you'll then have one entry showing the artists concatenated together as 'Johnny Cash & June Carter' so it is then harder to list all the songs involving Johnny Cash
Now actually when the artists field is intact we want June Carter to be listed as well in the Column Browser but that is not yet done, see http://jthink.net:8081/browse/JAIKOZ-783
The culprit is the artists field, Jaikoz uses the artists field in preference to the artist field for the Column Browser if the artists field is not empty. The artists field is only used by Jaikoz and SongKong currently but other applications are looking at incorporating it. It only differs when a song is attributed to multiple artists. In this case each artist is added separately to the artists field allowing you to search for a song by just the first (main artist) involved in the song.
i.e 'Jackson' by 'Johnny Cash & June Carter'
ARTIST:JohnnyCash & June Carter
ARTISTS[0]:Johnny Cash
ARTISTS[1]:June Carter
So the column browser would have one entry in the artist column containing 'Johnny Cash', but if you delete the artists field you'll then have one entry showing the artists concatenated together as 'Johnny Cash & June Carter' so it is then harder to list all the songs involving Johnny Cash
Now actually when the artists field is intact we want June Carter to be listed as well in the Column Browser but that is not yet done, see http://jthink.net:8081/browse/JAIKOZ-783
Monday, 17 March 2014
Identifying unknown songs in iTunes using SongKong
As you can see:
- The original files contain no metadata, and no useful information in the file path
- The songs can be dragged directly from iTunes to SongKong
- Once the songs have been fixed and saved the changes are reflected in iTunes immediately
Saturday, 15 March 2014
Finding split up tracks from compilation albums revisited
The Problem Revisited
A Mac customer wanted to use Jaikoz to only fix some compilation albums that had been broken up into different folders by another application, so typically each song was the only file within its own folder.A couple of weeks ago I posted a solution to this but since then I have made a couple of improvements:
- The original script would only find files if there was one file in total in the folder, so if there was one music file plus an image file it wouldn't be listed, it has now been fixed to only count music files.
- Folders containing more than one music file may need fixing as well so it has now been extended to allow you to search for x music files in a folder where x is another parameter
Here is the new script ( but no need to copy it there a version available to download)
#!/bin/bash
shopt -s nullglob
find "${1:-.}" -type d | while read dir; do
files=( "${dir}"/*.{mp4,mp3,ogg,flac,wma,m4a} )
IFS=$'\n'
(( ${#files[@]} == ${2:-1} )) && echo "${files[*]}"
done
This
checks all the folders under the provided one looking for folders
containing only the number of music files requested in the folder and list the songs that Jaikoz can process.Then if we redirect the output to a file, and we name it as a unicode playlist we can load the list of files straight into Jaikoz
And this solution should work for Linux as well
The Solution
Finding the list of orphaned files
Save this file to your Documents folderOpen Applications/Terminal,
Within Terminal enter
chmod 777 Documents/findodd2.sh
and press <ENTER> then
./Documents/findodd2.sh MusicFolder NumberOfFiles > playlist.m3u8
and press again.
where MusicFolder is the full path of the root of your music directory and NumberOfFiles is how many music files the folder must contain.
i.e. I might enter:
./Documents/findodd2.sh /Users/paul/Music 2 > playlist.m3u8
to find folders in /Users/paul/Music containing exactly 2 music files
this may take a few minutes depending on how much music you have
Then start Jaikoz and drag playlist.m3u8 onto Jaikoz to load the files
Matching files to releases
Ensure View:Show Column Browser is enabled , and use it or the search so that at any one time you only have the songs from one of the album you want to reconstruct listed.
If you know the release exists in MusicBrainz use Action:Match to Release:Match Songs to Specified MusicBrainz Release and then move onto the next release.
Or if you know the release is in Discogs use Action:Match to Release:Match Songs to Specified Discogs Release and then move onto the next release.
If it doesn't exist in either database you can still use Jaikoz to manually fix the metadata
Then use Action:File and Folder Correct:Correct Filename from Metadata and Correct SubFolder from Metadata to rename the files and folders to bring the files back together.
Now check the results and then Save Changes
Friday, 14 March 2014
Opening Playlists with SongKong
Today we release a new version of SongKong with a number of fixes and improvements, the most important detailed below.
You can create M3U playlists from your iTunes playlists as follows:
1. Select the desired playlist then Control-click / Right-click the playlist and select the Export... option
2. Select Save as type M3U files (.m3u8), selecting this option instead of M3U files (.m3u) ensures that songs in any language can be correctly stored on the playlist, this is particularly important for non European languages such as Arabic, Chinese or Japanese.
3. This newly created playlist file can be dragged and dropped onto SongKong.
But don't forget that although you cannot drag native iTunes playlists directly into SongKong you can drag the songs themselves directly from iTunes.
Playlists
In addition to files and folders SongKong can now open M3U playlists, just drag the playlist onto SongKong and then all the files and folders within the playlist can be fixed or searched for duplicates.You can create M3U playlists from your iTunes playlists as follows:
1. Select the desired playlist then Control-click / Right-click the playlist and select the Export... option
2. Select Save as type M3U files (.m3u8), selecting this option instead of M3U files (.m3u) ensures that songs in any language can be correctly stored on the playlist, this is particularly important for non European languages such as Arabic, Chinese or Japanese.
3. This newly created playlist file can be dragged and dropped onto SongKong.
But don't forget that although you cannot drag native iTunes playlists directly into SongKong you can drag the songs themselves directly from iTunes.
Fix Songs Cancelled
There was an issue in the last couple of versions that could cause SongKong to finish prematurely before it had tried to match all songs, the report would say it had been cancelled even though it was not cancelled. This issue could occur when potentially duplicate files were found in a folder when matching and is now resolved in this version.Report Creation not Finishing
SongKong fixes songs in a pipeline and is very careful not too use very much memory during this process however many songs you are fixing. However at the end of the fixing process a report is created and this does require more memory when more songs are fixed, in extreme cases this could cause SongKong to fail during report creation. Memory usage has now been substantially improved during report creation to prevent this from happening.Other Improvements
The full list of improvements can be found hereFriday, 7 March 2014
Jaikoz versus SongKong
A few times I've been asked me the difference between SongKong and Jaikoz, a good question which I'll answer in this post. I've also been asked which is better,but there is no answer to this question they are just different.
Jaikoz was designed to be very flexible so that all fields can be easily and quickly modified in comparison with other tag editors and players such as iTunes, metadata can be exported to or imported from a spreadsheet
Jaikoz also supports finding and deleting duplicates songs.
Over the years many new features have been added to Jaikoz to deal with almost any situation.
One important principle of Jaikoz was that it would only modify your files when you actively choose to do so. Within Jaikoz you load your files into it, modify your songs then save your songs.
SongKong actually saves songs as they are matched, this has the advantage that there is no delay at the end saving all songs at once, and if anything goes wrong no matches are lost. But to ensure the customer retains control SongKong keeps a full audit of everything modified and allows you to Undo any Changes even after restarting your computer.
SongKong takes automated tagging further as it can continuously monitor your music folder and correct any new files added to the folder.
SongKong also supports finding and deleting duplicates songs.
SongKong also provides a command line interface.
Jaikoz Summary
Jaikoz is an established music tagger encompassing both automated tagging using online databases and manual tagging, development started in 2006. Jaikoz has automated matching to MusicBrainz and Discogs, but also has semi-automatic tagging and matching an album to a specific Musicbrainz or Discogs release. Jaikoz also allows new releases and other information to be added to MusicBrainz.Jaikoz was designed to be very flexible so that all fields can be easily and quickly modified in comparison with other tag editors and players such as iTunes, metadata can be exported to or imported from a spreadsheet
Jaikoz also supports finding and deleting duplicates songs.
Over the years many new features have been added to Jaikoz to deal with almost any situation.
One important principle of Jaikoz was that it would only modify your files when you actively choose to do so. Within Jaikoz you load your files into it, modify your songs then save your songs.
SongKong Summary
SongKong was started in 2012, the main objective was to provide a simpler solution than Jaikoz that would work for the majority of people. With advances in automatic tagging I decided to only provide automated tagging and a simpler more streamlined interface. SongKong starts fixing songs as soon as you start loading them without waiting for all songs to be loaded, it works in a pipeline.SongKong actually saves songs as they are matched, this has the advantage that there is no delay at the end saving all songs at once, and if anything goes wrong no matches are lost. But to ensure the customer retains control SongKong keeps a full audit of everything modified and allows you to Undo any Changes even after restarting your computer.
SongKong takes automated tagging further as it can continuously monitor your music folder and correct any new files added to the folder.
SongKong also supports finding and deleting duplicates songs.
SongKong also provides a command line interface.
Similarities
The automatic matching and duplicate deletion provided by both is the same except for minor differences, both support the same audio file formats and both integrate with iTunes. Development continues for both applications, and typically there are new releases every couple of months.Wednesday, 5 March 2014
Secret genre option in SongKong
Genre support in SongKong is quite simple at the moment, it does not currently have the same genre
options as Jaikoz because I'm looking for something simpler but more effective then we have in Jaikoz.
Currently if you start Fix Songs then select the Update Genres option on the first tab it will update the genre from the Discogs Style field but only if the genre field in your song is empty.
However there is an option you can use that is just missing from the UI to overwrite the genre at all times as follows:
Simply add the line
discogsGenreOverwriteOption=0
to your user copy of songkong.properties
this can be found in the following places depending on OS
OSX
From Finder, hold down Option Button and select Go menu
Select Library from the list
Go to Preferences : SongKong
Windows 7/8
Go to C:\Users\username\AppData\Roaming\SongKong
Linux
Go to $HOME/.songkong
Any suggestions for how you would like genre support implemented, just let me know.
Currently if you start Fix Songs then select the Update Genres option on the first tab it will update the genre from the Discogs Style field but only if the genre field in your song is empty.
However there is an option you can use that is just missing from the UI to overwrite the genre at all times as follows:
Simply add the line
discogsGenreOverwriteOption=0
to your user copy of songkong.properties
this can be found in the following places depending on OS
OSX
From Finder, hold down Option Button and select Go menu
Select Library from the list
Go to Preferences : SongKong
Windows 7/8
Go to C:\Users\username\AppData\Roaming\SongKong
Linux
Go to $HOME/.songkong
Any suggestions for how you would like genre support implemented, just let me know.
Why is SongKong not renaming my music ?
I had three different customers reporting that SongKong is not renaming their files for three completely different reasons so I thought it would be worth sharing this with everyone.
The problem in this case was that although the customer had gone to the File naming tab and checked their rename and compilation masks
They had forgotten to actually check the Rename files based on metadata when matched on the Basic tab, this is required to actually do the rename.
However within iTunes if you have Preferences:Advanced:Keep iTunes Media Folder Organized enabled whenever iTunes is informed of a change to a song it will rename the song based on the new metadata. This is okay if you are not using SongKong to rename your songs but if you are using SongKong to rename (or move) songs you should ensure this iTunes option is not enabled.
Because SongKongs rename mechanism is much more advanced then iTunes and because SongKong logs the changes it makes I recommend not allowing iTunes to organize your media.
It wouldn't make much sense to rename songs that contained poor or non-existent metadata because their current filename may be the only means of identifying them. But you may have songs that SongKong has not matched to MusicBrainz or Discogs but other than that omission contain correct metadata and could be safetly renamed. I'm looking at adding an extra option ion SongKong for this scenario
Rename Files From Metadata Not Enabled
By default SongKong does not rename files, in many ways it would make more sense for renaming to be the default so they reflect the new metadata added to them - however I have had too many occurrences of Jaikoz customers wondering where their files have gone to think this is a good idea.The problem in this case was that although the customer had gone to the File naming tab and checked their rename and compilation masks
They had forgotten to actually check the Rename files based on metadata when matched on the Basic tab, this is required to actually do the rename.
iTunes Organizing Your Music Files
If you have configured SongKong to inform of iTunes of modifications then iTunes will can be kept up to date with metadata modifications, new songs added to iTunes and songs deleted.However within iTunes if you have Preferences:Advanced:Keep iTunes Media Folder Organized enabled whenever iTunes is informed of a change to a song it will rename the song based on the new metadata. This is okay if you are not using SongKong to rename your songs but if you are using SongKong to rename (or move) songs you should ensure this iTunes option is not enabled.
Because SongKongs rename mechanism is much more advanced then iTunes and because SongKong logs the changes it makes I recommend not allowing iTunes to organize your media.
Songs not Renamed because not Matched
The final reason is because of the way SongKong works, when SongKong gets to the save stage it will only rename the file if it has been matched to MusicBrainz or Discogs, This is done by checking the MB_RECORDING_ID and DISCOGS_RELEASE_URL fields so even if the files were not matched during this run previously matched files can be renamed.It wouldn't make much sense to rename songs that contained poor or non-existent metadata because their current filename may be the only means of identifying them. But you may have songs that SongKong has not matched to MusicBrainz or Discogs but other than that omission contain correct metadata and could be safetly renamed. I'm looking at adding an extra option ion SongKong for this scenario
Tuesday, 4 March 2014
New Jaikoz release solves the iTunes update problem
A new release of Jaikoz is now available, the full list of changes can be found here .
Jaikoz uses Applescript to inform iTunes of changes, however because of an oversight later versions of the library Jaikoz uses no longer have the necessary calls to talk to the Applescript library. However if you have earlier versions of the library (as all my test machines do) then Jaikoz can talk to iTunes without problems. Until last week I wasn't aware of the seriousness of this issue, I thought the problem was only on a few isolated installations. If previously you have have been unable to update iTunes from Jaikoz on OSX this release should solve that issue . We have also sped up the time it takes to create the internal model Jaikoz has of iTunes.
Updating iTunes on OSX
This release fixes an important problem with updating iTunes from OSX on some systems.Jaikoz uses Applescript to inform iTunes of changes, however because of an oversight later versions of the library Jaikoz uses no longer have the necessary calls to talk to the Applescript library. However if you have earlier versions of the library (as all my test machines do) then Jaikoz can talk to iTunes without problems. Until last week I wasn't aware of the seriousness of this issue, I thought the problem was only on a few isolated installations. If previously you have have been unable to update iTunes from Jaikoz on OSX this release should solve that issue . We have also sped up the time it takes to create the internal model Jaikoz has of iTunes.
Changes to Release Type
Also the Release Type is now a multiple value field. Previously MusicBrainz only allowed a release to be one type, but now multiple types are supported so a release can be both an album and live or a single and a soundtrack, or even an album, a soundtrack and live.Soundtracks
Usually the first type indicates the length of the release such as album, single or e.p, in MusicBrainz this is known as the primary type. However because it is generally more useful to know if a release is a soundtrack then an album (which can usually be inferred because most soundtracks are albums) if a release has a secondary type of soundtrack we make this the first type added to the song.Opening Playlists
There was a regression on OSX that meant that Open Playlist and Add Playlist no longer work, actually dragging and dropping a playlist continued to work so it was not noticed for a while but this is now fixedApostrophes
Sometimes unusual versions of apostrophes are used when release are added to MusicBrainz, usually by mistake. This causes a problem for Jaikoz when trying to capitalise titles or save files containing these characters so now Jaikoz automatically changes them to the regular apostrophe.Code Signing
For some time the OSX version of Jaikoz has been code signed but not the Windows version, with this release we put that right. This means you can be sure that the installer you download has been created by JThink, and has not been modified by anyone else.Friday, 28 February 2014
A solution for finding split up tracks from compilation albums
The Problem
A Mac customer wanted to use Jaikoz to only fix some compilation albums that had been broken up into different folders by another application, so typically each song was the only file within its own folder.Now Jaikoz can put albums back together again, but the difficulty is finding all those orphaned files in the first place, the following script can help:
find "$1" -type d -exec sh -c '[[ $(find "$0" -mindepth 1 | wc -l) -eq 1 ]] && [[ $(find "$0" -mindepth 1 -type d | wc -l) -eq 0 ]] && find "$0"' {} \; |egrep ".mp4|.mp3|.ogg|.flac|.wma|.m4a"
This checks all the folders under the provided one looking for folders containing only one file, and then filters out any non music files to leave you with a list that Jaikoz can process.
Then if we redirect the output to a file, and we name it as a unicode playlist we can load the list of files straight into Jaikoz
The Solution
Finding the list of orphaned files
Save this file to your Documents folderOpen Applications/Terminal,
Within Terminal enter
chmod 777 Documents/findodd.sh
and press <ENTER> then
./Documents/findodd.sh MusicFolder > playlist.m3u8
and press
where MusicFolder is the full path of the root of your music directory
i.e. I might enter:
./Documents/findodd.sh /Users/paul/Music > playlist.m3u8
this may take a few minutes depending on how much music you have
Start Jaikoz
Drag playlist.m3u8 onto Jaikoz to load the files
Matching files to releases
Ensure View:Show Column Browser is enabled , and use it or the search so that at any one time you only have the songs from one of the album you want to reconstruct listed.
If you know the release exists in MusicBrainz use Action:Match to Release:Match Songs to Specified MusicBrainz Release and then move onto the next release.
Or if you know the release is in Discogs use Action:Match to Release:Match Songs to Specified Discogs Release and then move onto the next release.
If it doesn't exist in either database you can still use Jaikoz to manually fix the metadata
Then use Action:File and Folder Correct:Correct Filename from Metadata and Correct SubFolder from Metadata to rename the files and folders to bring the files back together.
Now check the results and then Save Changes
Thursday, 27 February 2014
Solving the problem of updating iTunes from SongKong on OSX
Hots on the heels of SongKong 1.19 we have
SongKong 1.20 because I have just discovered an important bug in a code
library SongKong uses on OSX for talking to iTunes.
SongKong uses Applescript to inform iTunes of changes, however because of an oversight later versions of the library SongKong uses no longer have the necessary calls to talk to the Applescript library. However if you have earlier versions of the library (as all my test machines do) then SongKong can talk to iTunes without problems..
Until today I wasn't aware of the seriousness of this issue, I thought the problem was only on a few isolated installations. If previously you have have been unable to update iTunes from SongKong on OSX this release should solve that issue .
Linux and Windows users there is no real need to update to this release as it only contains the OSX Applescript/iTunes fix and some minor modifications to debugging output
SongKong uses Applescript to inform iTunes of changes, however because of an oversight later versions of the library SongKong uses no longer have the necessary calls to talk to the Applescript library. However if you have earlier versions of the library (as all my test machines do) then SongKong can talk to iTunes without problems..
Until today I wasn't aware of the seriousness of this issue, I thought the problem was only on a few isolated installations. If previously you have have been unable to update iTunes from SongKong on OSX this release should solve that issue .
Linux and Windows users there is no real need to update to this release as it only contains the OSX Applescript/iTunes fix and some minor modifications to debugging output
Wednesday, 26 February 2014
Organizing music for Plex
I have been experimenting with Plex a media player system consisting of a player application with a 10-foot user interface and an associated media server, its available for all major operating systems.
I have a Mac mini connected to my television and so interestingly this is a very viable alternative to using iTunes on your Mac as your home media centre.
Plex supports all kinds of media but in this blog we are interested in music, and this is where SongKong can help.
Getting your metadata in tip-top condition before uploading into Plex is essential for the best music listening experience so fixing your songs metadata with SongKong is a great start but Plex is rather fussy about how your files are organized so if possible you should ensure you rename your files using the mask now available in SongKong 1.19.
Select Fix Songs, ensure that Rename files based on metadata when matched is checked and that Save Changes to iTunes is unchecked
Then select the File Naming tab and choose the Plex mask for both the Rename mask and the Compilation rename mask, its easy to find it is the last one in the list.
Then select Start and your songs shall have their metadata fixed, and folders and files renamed to best suit Plex.
I have a Mac mini connected to my television and so interestingly this is a very viable alternative to using iTunes on your Mac as your home media centre.
Plex supports all kinds of media but in this blog we are interested in music, and this is where SongKong can help.
Getting your metadata in tip-top condition before uploading into Plex is essential for the best music listening experience so fixing your songs metadata with SongKong is a great start but Plex is rather fussy about how your files are organized so if possible you should ensure you rename your files using the mask now available in SongKong 1.19.
Select Fix Songs, ensure that Rename files based on metadata when matched is checked and that Save Changes to iTunes is unchecked
Then select the File Naming tab and choose the Plex mask for both the Rename mask and the Compilation rename mask, its easy to find it is the last one in the list.
Then select Start and your songs shall have their metadata fixed, and folders and files renamed to best suit Plex.
Sunday, 23 February 2014
Shazam for your computer
Many people use Shazam to identify songs they here in their local cafe or walking down the street, and its a great app for discovering new music, but I have also seen many people asking for Shazam for their desktop computer, now why would you want that you can hardly carry it around with you ?
SongKong can be used to identify as many songs as you like for free, simply look in the Song Changes section of the report to see what the song was matched to.
To keep things simple the screenshot shows the result of just matching a single previously unknown song but SongKong can be used safely on thousands of songs.
SongKong only requires purchase if you want it to add the matching song information to the file itself.
Identifying Music You Already Have
Users understand that Shazam can be used to identify music on their iPhone and want a solution for all those badly labelled tracks on their computer but this is not what Shazam is intended for. Shazam is now available for your Mac and Windows 8, but Shazam requires you to physically play a part of each song in order to identify it, so is not a very good solution if you have more than a handful of songs to identify.A Free alternative
Shazam actually uses acoustic fingerprinting to identify songs but forces you to play them via a microphone, SongKong also uses audio fingerprinting but this is done automatically without you having to play the song, fingerprinting a song only takes a few seconds.SongKong can be used to identify as many songs as you like for free, simply look in the Song Changes section of the report to see what the song was matched to.
To keep things simple the screenshot shows the result of just matching a single previously unknown song but SongKong can be used safely on thousands of songs.
SongKong only requires purchase if you want it to add the matching song information to the file itself.
Rinse My Music Replacement
A few weeks ago I blogged about TuneUp Media going under and my suspicion that it was related to Gracenote being sold. I have not been able to confirm this but guess what, another Mac tagger Rinse My Music powered by Gracenote has gone under as well, coincidence ?
In this case the company itself has not gone under but Rinse My Music is no longer available and no explanation has yet been forthcoming. Existing customers will find the product ceases to work in a couple of months
So I would like to try to help Rinse My Music customers in the same way as TuneUp Media customers by offering SongKong for a 30% discount until the end of March. Simply make the full price purchase in the normal way then email paultaylor@jthink.net with some proof of your Rinse My music product code and I'll refund 30% of the price.
SongKong does everything that Rinse My Music does and more. But it doesn't rely on closed databases like Gracenote instead it uses MusicBrainz and Discogs. I'm an Indie developer so when you deal with Jthink you are dealing with me direct not having to go through layers of anonymous support.
There is a free trial available so you can comprehensively try before you buy.
In this case the company itself has not gone under but Rinse My Music is no longer available and no explanation has yet been forthcoming. Existing customers will find the product ceases to work in a couple of months
So I would like to try to help Rinse My Music customers in the same way as TuneUp Media customers by offering SongKong for a 30% discount until the end of March. Simply make the full price purchase in the normal way then email paultaylor@jthink.net with some proof of your Rinse My music product code and I'll refund 30% of the price.
SongKong does everything that Rinse My Music does and more. But it doesn't rely on closed databases like Gracenote instead it uses MusicBrainz and Discogs. I'm an Indie developer so when you deal with Jthink you are dealing with me direct not having to go through layers of anonymous support.
There is a free trial available so you can comprehensively try before you buy.
Fixing files direct from iTunes
Originally SongKong was very much folder focused, you select a folder and SongKong would fix all the files and sub-folders of that folder. Customers generally went to two extremes, either picking their top level music folder and fixing everything or just picking an album folder and fixing one album at a time.
Now both of these methods are great but we have added a bit more flexibility
Firstly, you can now select multiple folders, if you have five folders under an artist folder now you can just fix two, three or four of the folders in one go instead of matching one at a time or the whole lot. And if the folders are located in different locations SongKong works out the basefolder/sub folder split for each combination as this screenshot shows
Secondly, instead of just fixing complete folders you can now fix files within folders. There are at least three important uses of this
Now both of these methods are great but we have added a bit more flexibility
Firstly, you can now select multiple folders, if you have five folders under an artist folder now you can just fix two, three or four of the folders in one go instead of matching one at a time or the whole lot. And if the folders are located in different locations SongKong works out the basefolder/sub folder split for each combination as this screenshot shows
Secondly, instead of just fixing complete folders you can now fix files within folders. There are at least three important uses of this
Fixing songs direct from iTunes
iTunes does not show folders only files, this was a problem but now files can be dragged directly from iTunes into SongKong.Fixing results of a search
Now you can do a search in Finder or Windows Explorer and drag the list of matching files direct into SongKong.Fixing Songs direct from Windows Explorer
Now all supported music formats can be fixed by just right clicking in Windows and selecting Fix Song in SongKong from the context menuThursday, 6 February 2014
TuneUp Media ceases trading and Gracenote is sold by Sony
Without any notice TuneUp Media has ceased trading leaving their customers high and dry. Knowing that Tuneup had received venture capital investment my first thought was that the investors had pulled the plug on them because they were not making enough profit. However, it was announced a couple of days later that Gracenote had been sold to Tribune Media Services, more details here. Tuneup used the Gracenote service for its song matching so I suspect that under new ownership Gracenote has decided to withdraw that service.
I would like to try to help Tune Up Media customers by offering SongKong for a 30% discount until the end of March (previously February but now extended to March). Simply make the full price purchase in the normal way then email paultaylor@jthink.net with some proof of your Tuneup media license and I'll refund 30% of the price. There is a free trial available so you can comprehensively try before you buy.
Now I would like to reassure you that the problem that have you have encountered with Tuneup Media will not occur with Jthink software because of the following reasons:
1. Jthink is wholly owned by myself Paul Taylor, so there is no pressure exterted on me by external investors. My priority is to provide high quality, useful software at a fair price. JThink software is available with free upgrades and support, so you only pay once. Because I am the designer and developer there are no messy compromises in the design and I am able to be highly responsive to the specific needs of customers, and adapt products to a customers specific requirements.
Jthink has been in business since 2006, my comprehensive tagger Jaikoz is still actively developed and improved but I think the simplicity of SongKong is a better solution for Tuneup customers.
2. SongKong uses the open source MusicBrainz service rather than the closed Gracenote service. The complete database and web service is freely available and I have my own complete copy. Although I currently use the web service hosted by MusicBrainz I could provide my own Musicbrainz server if necessary so even if MusicBrainz was to die a death I could continue the service. MusicBrainz has its own protections in place to make it highly unlikely this could ever happen.
I would like to try to help Tune Up Media customers by offering SongKong for a 30% discount until the end of March (previously February but now extended to March). Simply make the full price purchase in the normal way then email paultaylor@jthink.net with some proof of your Tuneup media license and I'll refund 30% of the price. There is a free trial available so you can comprehensively try before you buy.
Now I would like to reassure you that the problem that have you have encountered with Tuneup Media will not occur with Jthink software because of the following reasons:
1. Jthink is wholly owned by myself Paul Taylor, so there is no pressure exterted on me by external investors. My priority is to provide high quality, useful software at a fair price. JThink software is available with free upgrades and support, so you only pay once. Because I am the designer and developer there are no messy compromises in the design and I am able to be highly responsive to the specific needs of customers, and adapt products to a customers specific requirements.
Jthink has been in business since 2006, my comprehensive tagger Jaikoz is still actively developed and improved but I think the simplicity of SongKong is a better solution for Tuneup customers.
2. SongKong uses the open source MusicBrainz service rather than the closed Gracenote service. The complete database and web service is freely available and I have my own complete copy. Although I currently use the web service hosted by MusicBrainz I could provide my own Musicbrainz server if necessary so even if MusicBrainz was to die a death I could continue the service. MusicBrainz has its own protections in place to make it highly unlikely this could ever happen.
Wednesday, 5 February 2014
Some more important requirements of an automatic music tag editor
Some more important requirements of a rules based music tag editor
Since this post I realised I'd missed some other important requirements for a fully automatic music tagger
1. There needs to be an audit trail
If you are trusting some of the most important data on your computer then you really want to know what it is getting up to.
2. There needs to be a way to roll back changes
However good automated matching is there should always be the possibility to undo the changes
it makes.
3. There needs to be a way to remove duplicates songs
As your song collection gets organised you'll probably find you have duplicates, and finding and removing duplicates is something that automated taggers can potentially be very good at.
4. A way of continuously monitoring and modifying your music collection.
Once you are happy with your automated setup wouldn't it be nice to not even have to think about it anymore.
and this is how SongKong meets these requirements
There needs to be an audit trail
SongKong creates a comprehensive report with details of exactly what has been matched to the Musicbrainz and Discogs databases and also exactly what changes have been made to your files.
There needs to be a way to roll back changes
Every time SongKong makes changes to a file the changes are stored in a database, and because it is in a database those changes are not lost when you close SongKong or restart your computer. If at a later date you decide that you do not like the changes that SongKong has made you can use Undo Changes to change the files back to how they were before changes made by SongKong. This undo facility works even if the files have been moved or renamed.
There needs to be a way to remove duplicates songs
SongKong lets you find duplicates, and when a duplicate is found decide the criteria for which if the duplicates to be deleted. But what is a duplicate, luckily because SongKong stores ids when matching songs it can accurately determine when a song really is a duplicate. SongKong lets you choose any combination of Acoustic Id, Song Id and Album Id to let you decide what is a duplicate. For example if the same song appears on two different albums then you may consider these as different songs or you may consider them the same song.
A way of continuously monitoring and modifying your music collection
Once you have SongKong configured to your liking a good way for working with new music is simply to setup a new folder that you dump new music into and have SongKong detect this and do its thing, this is easy done using the Watch Folder option.
Since this post I realised I'd missed some other important requirements for a fully automatic music tagger
1. There needs to be an audit trail
If you are trusting some of the most important data on your computer then you really want to know what it is getting up to.
2. There needs to be a way to roll back changes
However good automated matching is there should always be the possibility to undo the changes
it makes.
3. There needs to be a way to remove duplicates songs
As your song collection gets organised you'll probably find you have duplicates, and finding and removing duplicates is something that automated taggers can potentially be very good at.
4. A way of continuously monitoring and modifying your music collection.
Once you are happy with your automated setup wouldn't it be nice to not even have to think about it anymore.
and this is how SongKong meets these requirements
There needs to be an audit trail
SongKong creates a comprehensive report with details of exactly what has been matched to the Musicbrainz and Discogs databases and also exactly what changes have been made to your files.
There needs to be a way to roll back changes
Every time SongKong makes changes to a file the changes are stored in a database, and because it is in a database those changes are not lost when you close SongKong or restart your computer. If at a later date you decide that you do not like the changes that SongKong has made you can use Undo Changes to change the files back to how they were before changes made by SongKong. This undo facility works even if the files have been moved or renamed.
There needs to be a way to remove duplicates songs
SongKong lets you find duplicates, and when a duplicate is found decide the criteria for which if the duplicates to be deleted. But what is a duplicate, luckily because SongKong stores ids when matching songs it can accurately determine when a song really is a duplicate. SongKong lets you choose any combination of Acoustic Id, Song Id and Album Id to let you decide what is a duplicate. For example if the same song appears on two different albums then you may consider these as different songs or you may consider them the same song.
A way of continuously monitoring and modifying your music collection
Once you have SongKong configured to your liking a good way for working with new music is simply to setup a new folder that you dump new music into and have SongKong detect this and do its thing, this is easy done using the Watch Folder option.
Sunday, 2 February 2014
Rules Based Music Tagging
With conventional music tagging there is lots of manual editing or semi-automatic tagging but SongKong is a rules based tagger. The basic idea is that rules define how you want your music collection
to be organized, then the rules can be applied to your whole music
collection without any manual editing required giving a totally
consistent music collection with the minimum effort on your part. You can change the rules and then reapply to
the whole collection to maintain a consistent music collection.
There are three main aspects to incorporating rules based tagging, lets summarize the perfect system:
1. It would be possible to correctly identify every file in your music
2. Once a song is identified the database it has been matched to would contain every desired attribute of the song that you are interested.
3. It would be possible to define rules to extract and apply the data in any way required.
And this is SongKongs implementation:
1. SongKong generates Acoustic fingerprints for each song and this can be looked up in the Acoustid database to identify the song, working in a similar way to Shazam. Acoustid currently contains fingerprints for 16 million songs allowing the majority of your songs to be identified, but Acoustid can only identify the song not necessarily the album. However by combining this with comparing existing meta-data in your songs we can match to the album as well in the MusicBrainz database. MusicBrainz provides high quality detailed data onmore than 1 million albums.
You can influence the importance of metadata such as by specifying a preference of matching albums form certain countries of a preferred format such as CD or Vinyl.
So MusicBrainz/Acoustid provides good coverage for most music collections. But we also use the Discogs database as an alternative source.
Even so, we cannot guarantee 100% matching, but testing shows that in the majority of cases approximately 90% of a collection can be matched to MusicBrainz or Discogs.
It has to be said both of these databases do have better coverage for Pop/Rock/Electronic western music than Classical and World Music. But there are many projects ongoing to fill these gaps, for example the Music Technology Group at the Universistat Pompea Fabra is working with MusicBrainz to add Indian Raag Music.
The good news is that MusicBrainz open approach is fast becoming the de-facto standard database of music information.
2. Some Music databases contain only the most basic information such as artist, album, title and year of release. But the MusicBrainz database consists of an incredibly rich semantic model, and allows new relationships to be defined between entities.
When SongKong matches to a song in MusicBrainz it is guaranteed to find at least 19 fields (such as artist, album ecetera) and often an additional 35 fields. This includes high quality artwork, usually at a resolution of at least 600 x 600 pixels.
These fields also includes some MusicBrainz Ids, this means that your songs are compatible with other MusicBrainz enabled applications. It also means you can always look up the original source of the data at any time in the future.
3. SongKong is intended to be easy to use, but there is no one right way to organize your data. Everybody has different requirements based on their personal preferences and how and where they are going to play their music.
Here are a few scenarios handled by SongKong:
You are a DJ only interested in individual songs and has no interest in the album information. SongKong lets you specify exactly how files are named and stored, you can use any piece of metadata for deriving your filename ,and can use powerful Javascript expressions to manipulate the names.
Your music is already stored in iTunes. SongKong can work with iTunes automatically, updating the information in iTunes for songs that are already under iTunes control and adding songs that are not iTunes control to iTunes.
You like to process songs in a pipeline moving them from unmatched to matched location. SongKong allows you to move files as they are matched to a new location, it also allows files that it failed to match to moved as well.
I hope this post helps you understand the basics of rule based tagging. Jaikoz also incorporates rule based tagging , but also provides manual editing and semi-automated tagging.
There are three main aspects to incorporating rules based tagging, lets summarize the perfect system:
1. It would be possible to correctly identify every file in your music
2. Once a song is identified the database it has been matched to would contain every desired attribute of the song that you are interested.
3. It would be possible to define rules to extract and apply the data in any way required.
And this is SongKongs implementation:
1. SongKong generates Acoustic fingerprints for each song and this can be looked up in the Acoustid database to identify the song, working in a similar way to Shazam. Acoustid currently contains fingerprints for 16 million songs allowing the majority of your songs to be identified, but Acoustid can only identify the song not necessarily the album. However by combining this with comparing existing meta-data in your songs we can match to the album as well in the MusicBrainz database. MusicBrainz provides high quality detailed data onmore than 1 million albums.
You can influence the importance of metadata such as by specifying a preference of matching albums form certain countries of a preferred format such as CD or Vinyl.
So MusicBrainz/Acoustid provides good coverage for most music collections. But we also use the Discogs database as an alternative source.
Even so, we cannot guarantee 100% matching, but testing shows that in the majority of cases approximately 90% of a collection can be matched to MusicBrainz or Discogs.
It has to be said both of these databases do have better coverage for Pop/Rock/Electronic western music than Classical and World Music. But there are many projects ongoing to fill these gaps, for example the Music Technology Group at the Universistat Pompea Fabra is working with MusicBrainz to add Indian Raag Music.
The good news is that MusicBrainz open approach is fast becoming the de-facto standard database of music information.
2. Some Music databases contain only the most basic information such as artist, album, title and year of release. But the MusicBrainz database consists of an incredibly rich semantic model, and allows new relationships to be defined between entities.
When SongKong matches to a song in MusicBrainz it is guaranteed to find at least 19 fields (such as artist, album ecetera) and often an additional 35 fields. This includes high quality artwork, usually at a resolution of at least 600 x 600 pixels.
These fields also includes some MusicBrainz Ids, this means that your songs are compatible with other MusicBrainz enabled applications. It also means you can always look up the original source of the data at any time in the future.
3. SongKong is intended to be easy to use, but there is no one right way to organize your data. Everybody has different requirements based on their personal preferences and how and where they are going to play their music.
Here are a few scenarios handled by SongKong:
You are a DJ only interested in individual songs and has no interest in the album information. SongKong lets you specify exactly how files are named and stored, you can use any piece of metadata for deriving your filename ,and can use powerful Javascript expressions to manipulate the names.
Your music is already stored in iTunes. SongKong can work with iTunes automatically, updating the information in iTunes for songs that are already under iTunes control and adding songs that are not iTunes control to iTunes.
You like to process songs in a pipeline moving them from unmatched to matched location. SongKong allows you to move files as they are matched to a new location, it also allows files that it failed to match to moved as well.
I hope this post helps you understand the basics of rule based tagging. Jaikoz also incorporates rule based tagging , but also provides manual editing and semi-automated tagging.
Subscribe to:
Posts (Atom)