MinimServer with SongKong: Part 3: Sorting Conductor and other fields

Go forward to Part 4: Groups and Works
Go back to     Part 2: Sorting Composer by Sort Name

In the last section we saw how to sort Composer by the ComposerSort field, lets now apply the same principle to the Conductor field. But if we look at the tag mapping section of MinimServer User Guide again we find that a ConductorSort field is not listed. 

Some History

At one point the only computer music files commonly found were WAV format but at this time (when harddrives were more expensive and smaller) WAV didn't really support metadata and WAV files were too large to be that useful. But then MP3 was invented, originally with the ID3v1 metadata format and then shortly afterwards the ID3v2 metadata format.

The ID3v2 specification supported alot of fields, and was rather complicated. Music players and tagger software spent years trying to fully support ID3v2. As a result of this when other tag formats came along like Vorbis Comments (used by Flac and Ogg) and Atoms (used by Apple Lossless/Lossy) the software concentrated on ensuring they had support for fields equivalent to those in ID3.

ID3v23 had no sort fields, ID3v24 added ArtistSort , and then Apple added ComposerSort and AlbumArtistSort, and these became a defacto standard. But none of the big players has added ConductorSort and hence it is rare to see it in supported in any format or player.

SongKong Support

But the field does exist in SongKong and Jaikoz, the information is freely available in MusicBrainz so it made sense to add this when tagging. All audio metadata formats do in fact support a standard way to add custom fields, its just that ConductorSort is not listed not as a standard field.

Finding Non standard Field Names

If we look at SongKong Field mapping table  we can see that ConductorSort is listed slightly differently depending on the audio format.

• TXXX:CONDUCTOR_SORT (MP3, WAV, AIF)
• ----:com.apple.iTunes:CONDUCTOR_SORT(MP4)   
• CONDUCTOR_SORT (Flac, Ogg, Wma)

The TXXX and ----:com.apple.iTunes are those formats standard way of creating custom fields and MinimServer understands this and can read such fields, all TXXX fields and all ----:com.apple.iTunes can be read by MinimServer. When you specify such fields in MinimServer you don't specify the TXXX or ----:com.apple.iTunes parts.

Any field in Flac or Ogg format can also be read by MinimServer.  

So we only need to look at the second part. For all the formats this is CONDUCTOR_SORT, this is great because it means that MinimServer can be configured the same whatever format you are using.

But if they were different and you had audio in different formats (e.g FLAC and WAV) then you would need to use tag aliasing.

Adding Conductor Sort to the Configuration

We take the same approach as we did with ComposerSort, but because we are using a non-standard field we use its actual tag name which for all formats is CONDUCTOR_SORT (note the underscore), case does not matter. Because we are adding another rule to the itemTags and tagValue we separate the rule with a comma.

i.e. 

itemTags:ComposerSort, Conductor_sort  
tagValue:Composer.value.sort={ComposerSort},Conductor.value.sort={Conductor_Sort} 

And if we rescan and view in our control point we can see conductors are now sorted by their last name.

Orchestra Sort

We can apply the same mechanism for the Orchestra field. Orchestras are not individuals so the first name / last name rule does not apply. But the problems of sorting non-latin names and avoiding orchestras with 'The' at the start of the name all being grouped under 'T'  do. So simply modify configuration adding the following, as shown in the screenshot.

i.e. 

indexTags:Orchestra
itemTags:Orchestra_sort
tagValue:Orchestra.value.sort={Orchestra_sort}

We can apply sort to the Artist and All Artists index as well, but there is much more to discuss with artists. Next post we change tack and show how to use SongKong with one of MinimServers killer features, Groups and Works.



 

MinimServer with SongKong: Part 2: Sorting Composer by Sort Name

Go forward to Part 3: Sorting Conductor by Sort Name
Go back to     Part 1: Introduction

Continuing with the same dataset  if we double click on the Composer index the composers are displayed alphabetically, but by FirstName rather than by LastName

It is more usual to organize people by the last name, for example consider a telephone directory, and this is the purpose of Sort Fields. A sort field isn't typically used for display but to sort the values for display.  

MinimServer has a set of field names that is has mapped to  logical fields that can be used for any audio format in the Tag mappings section of the User Guide. One of these is the ComposerSort field

MinimServer and Reverse Names

Whilst MinimServer can be configured to guess the sort name by reversing the values using the reverseName option this only works for the usual Western names. Sometimes an artist doesn't have a first name, and composers in non-latin script cannot be handled. It is preferable to use dedicated sort fields rather than guessing the sort value by looking at the non sort field.

 

SongKong and Jaikoz support for Sort Fields

When we look up a person in MusicBrainz they have a dedicated sort version of their name. This means the ComposerSort field always has a value and latin script sort names are provided for all people, even if  their name is in non latin script such as Cyrillic or Chinese. For example the sort name of

Пётр Ильич Чайковский  

is  

Tchaikovsky, Pyotr Ilyich.  

MusicBrainz data is peer reviewed and therefore generally reliable.

MinimServer Configuration

We configure MinimServer as follows:

We add any fields we need MinimServer to read from the files but not index, so we add ComposerSort, case doesn't actually matter we could have added as composersort or COMPOSERSORT.

Then in tagValue a special syntax can be used to tell MinimServer to sort a value by another value, so here we tell it to sort the Composer field by the ComposerSort field using the format

DisplayField.value.sort={SortField} 

i.e
itemTags:ComposerSort
tagValue:Composer.value.sort={ComposerSort}  

Tell MinimServer to rescan and now when we view with our control point we can see the composers are now sorted correctly by their last name.

Other Sort Fields

We can use this approach for any other of people such as conductors and performers in our metadata. Luckily SongKong and Jaikoz provides the required data, for example Conductor Sort is not a standard metadata field but SongKong and Jaikoz provide it so it can be used by MinimServer, we delve into sort fields more in the next post.

MinimServer as your uPNP Media Server: Part 1: Introduction

Go forward to Minim Server:Part 2
Go back to     Introduction to UPnP AV using MinimServer

 

Installation and Start

MinimServer is developed in Java meaning it can be made available for a large number of computer devices, from Macs  and PCs to Network Attached Storage devices (NAS) such as Qnap or Synology and purpose built media servers such as the Melco N1. 

There is a full list of installations available here but for this blog we are going to use a PC based installation. We are also going to do a brief comparison with another popular UPnP server called Asset just to show what difference it makes using a different media server. Since this is a music metadata blog we are currently just concentrating on the metadata features rather than control of the media renderer.

We already have Asset installed and have scanned a folder containing approximately two hundred songs in Flac format. Now with MinimServer installed, we double click it to start in system tray, the first time it starts it asks for a content directory, we then select the same two hundred songs folder to scan for music.

And when it is ready it can be seen in the system tray as a green icon containing of course a Minim

View with Control Point

Now lets view with the Linn Kinsky control point, although branded Linn it seems to work quite reliably with any media server. We have the same set of music files available to be served by both Asset and MinimServer. 

Asset UPnP

Lets select Asset UPnP first

The first thing to notice is that when served by Asset various pseudo lists are available such as Top Artists and Top Albums and we have nice icons for different types. 

We can browse by Artist, Album, Title, Composer and Genre

MinimServer UPnP

Now lets take a look at MinimServer

With MinimServer we don't have these additional list or icons ( I think icons would be a nice to have Simon).

However as well as browsing by Artist, Album, Composer and Genre like Asset we can additionally browse by Conductor and Orchestra. You can probably configure Asset to browse by conductor, I'm not so sure about Orchestra. But the key thing is this demonstrates the different focus of these media servers. MinimServer better supports Classical and is more album orientated. Note that MinimServer doesn't have a Titles index because individual files can be accessed via the n items folder.
 

MinimServer Configuration

Everything can be configured via MinimServers configuration panel, simply right click on MinimServer in the system tray and select Properties.

This will bring up the MinimServer interface

This is not the prettiest interface, nor is it very intuitive since to view the instructions for the various options you need to read the User Guide on the website. 

But it is very effective, and not prescriptive, there is alot of flexibility available to you in this simple interface. The trick is to understand what you want the end result to be, how your songs are tagged and how to properly use the options to achieve what you want.

In the next post we show the first simple customization to make better use of metadata added by Jaikoz and SongKong.

SongKong 4.7 for Melco released 24th of February 2017

We have added some important improvements to the ability of SongKong for Melco to read All Music Guide data from the Naim proprietary WAV metadata format and embed the metadata in the files themselves so that can be read by non Naim applications.

Fix Songs Report - All Music Guide section

 
We now have a separate section in the Fix Song report called All Music Guide, this works in a similar way to Matched to MusicBrainz Release and Matched to Discogs Release  


Selecting All Music Guide opens up an alphabetical index:

Then selecting a letter lists the Album Artist of releases that have been matched to All Music Guide metadata, together with a count of the number of songs that had been matched.

Selecting an Album Artist updates the right hand side of the report with the All Music Guide album(s) that have been matched  

 
Selecting one of these albums lists more information about the album together with a list of all the songs that have been matched on the album and file that has been matched for each track.

Unlike Matched to MusicBrainz Release and Matched to Discogs Release SongKong is not trying to search for a possible match it is simply taking the amginfo.xml file for the files in the folder, parsing this file and embedding the metadata in the files themselves. There should be the same number of music files in the folder as on the release and therefore all the songs on the release should be matched.
  
Please note we talk about the All Music Guide in this article but when the Naim UnitiServe rips a CD and it cannot find a match to the All Music Guide it does a lookup from CDDB instead and creates a cddbinfo.txt file. SongKong also parses and embeds these files as well, but they contain less metadata information than the All Music Guide.

Fix Songs Report - Song Change section 

The song changes report now indicates for each song when they have been matched to the All Music Guide.  In this example all songs have been matched to All Music Guide, MusicBrainz and Discogs !

 

SongKong 4.7 now with View Metadata as Spreadsheet

Today we release SongKong 4.7,this new release has some major improvements to the Fix Songs report, some new options for classical matching and fixes for a number of niggling issues.

View Metadata as Spreadsheet

An often requested feature is now here. The Fix Songs Report now has a new option called View Metadata as Spreadsheet, clicking on this will open a spreadsheet (.xlsx format) in your preferred spreadsheet application listing the complete set of metadata within the set of files processed by SongKong. 

This shows the metadata as it currently is, if in Preview mode it shows the metadata as it would be if not in Preview mode.This spreadsheet view enables you to very quickly view multiple changes on multiple files.
 
Because SongKong can typically add a lot of different metadata we group the metadata into logical groupings over a number of sheets. The sheets are Basic, Release, Classical, People, Sort Fields, MusicBrainz & Discogs, MusicBrainz Work and AcousticBrainz - every sheet lists songs alphabetically by filename, the filename is in the first column of each sheet.
 
When SongKong has added a metadata for a field that did not have any metadata for that field it is shaded yellow, when the field did already have metadata but it has now changed then it is highlighted green, if a field is deleted it is highlighted red.

Classical Improvements

Before this release you could force all releases to be considered not classical but you could not force all releases to be considered. Also you could not allow SongKong to set the IsClassical field without applying Classical options. we have now replaced one option with the two below. 

Identify Classical releases

SongKong uses various way to identify Classical tracks, if a song does appear to be a Classical piece then the IsClassical field is set. Usually it is best to let SongKong to decide if a release is classical or not by keeping the default option of Classical releases identified by SongKong . But some releases are difficult to assess. If you know all the releases you are matching in this run are Classical then you can set this option to All releases treated as Classical to force SongKong to treat all matched songs as classical and set the isClassical field. Alternatively if you consider none of the releases to be Classical then you can set this option to None, no releases treated as Classical to force SongKong to treat all the songs to none classical, clearing any values previously set for the IsClassical field.

Apply these options to releases identified as Classical

If this option is enabled then whenever a song is identified as Classical the options on this tab are applied to it, they are not applied to releases identified as non-Classical.

 

not_classical_release.txt

When SongKong decides if a release is classical or not there are still ambiguities regarding if a release is Classical or not since it is a little subjective.So we have also added an exception list . Adding the MusicBrainz Release Id of a release to the not_classical_release.txt file ensures they are not treated as Classical releases.

Improvements to Song matching

Various improvements and fixes have been made to single song matching, single song matching occurs when an attempted match of a group songs failed to find a match. But then depending on customer options an attempt can  be made to match individual songs. 

 

Other Fixes and Improvements

We have made a number of minor improvements in many areas of SongKong, the full list of fixes can be found on the SongKong News page

Introduction to UPnP AV using MinimServer

Go forward to MinimServer for UPnP:Part1
Go back to     Introduction to AV UPnP

In Introduction to UPnP I explained that there are three components to UPnP AV, the media server, media controller (better known as control point) and media renderer. The metadata displayed in the control point is defined by what is provided to it by the media server rather than the control point itself. A control point cannot display data that the media server doesn't provide it with, although poor control points can limit the data displayed further.

I also said 

We think MinimServer is by far the most flexible media server, offering a significantly more powerful approach to metadata then the others that let you harness all that metadata added by SongKong and Jaikoz, and we will explain how to do just that in a series of blog posts in the near future. 

So in this post I going to explain the advantage of using MinimServer, in a nutshell MinimServer lets you serve up any metadata to your control point in the way you want. In contrast other servers tend to restrict to the standard Pop/Rock conventions of Artist/Album/Track/Genre/Year.

MinimServer was developed with Classical Music in mind, but there is nothing in the design that limits its use to Classical Music it is just that Classical Music simply better demonstrated the problems with other uPNP AV servers.

Jaikoz and SongKong have always provided one of the most comprehensive metadata coverage of any tagging application. But one frustration is that players such as iTunes, Winamp and MediaMonkey do not support many of the fields that are supported by Jaikoz and SongKong, the only way to make this data visible to users within these applications is using the data as part of the filename, and hope they will eventually improve their metadata field support.

But with MinimServer this data can be served to the customers control point, we just have to configure MinimServer correctly. So we can index songs by fields such as performers, groups, movements, orchestras, engineers, conductors, record labels. We can make use of the sort fields we have for not just artists and composers but other roles such as conductors and producers

In fact MinimServer has many configurable aspects, but lets start with the basics, in the next post we show a vanilla MinimServer installation and compare with another popular media server.

A Potted History of Jaikoz and SongKong

Jaikoz was originally released back in 2005, over ten years ago. It got me thinking where has the time gone, and what have we achieved ?

I have always tried to have original ideas rather than take other people ideas and I realized that actually there are quite a few features of Jaikoz and SongKong where I got their first. So at the risk of being self-indulgent here is a chronological list of some of these.

Jan 2005 Jaikoz is released with full spreadsheet interface, this meant instead of having to edit tracks one by one changes could be made rapidly.

June 2005 Jaikoz approach MusicBrainz to become the first commercial tagger to use MusicBrainz for tag matching and help setup the Tagger affiliate program. From the start we always stored the MusicBrainz Ids as well as the song metadata to allow users libraries to be fully MusicBrainz enabled.

June 2007 JThink open source our tagging library for reading and writing metadata to files for different audioformats called jaudiotagger. Jaudiotagger has been very popular and as well as being popular on Pc, Macs and Linux is now the No 1 tagging library for Android devices.

Sep 2008  Jaikoz is the first tagger to add support for Discogs matching when they opened up their metadata without restrictions.

Nov 2008  Jaikoz adds tools for submitting collection to MusicBrainz and seeding MusicBrainz releases.

Feb 2010 Jaikoz takes spreadsheet support further by allowing metadata to be exported to a spreadsheet, edited and then imported back into Jaikoz and hence the files. No other application can do this.

Nov 2011  SongKong released, the only totally automated tag editor with no options for manual editing. SongKong is the first application to use Acoustid for acoustic fingerprinting. 

May 2012 SongKong adds a database based Undo facility that allows metadata changes to be reverted even after a reboot, this feature is unique to SongKong.

July 2014 We create the first combined MusicBrainz/Discogs database server called Albunack. This combines data between the two datasets, includes all metadata that could be used for tagging. It is optimized to provide much better performance than be achieved using MusicBrainz or Discogs directly. Shortly after Jaikoz and SongKong move to use this database.    

Dec 2014 SongKong supports both release based matching and song only matching when the release cannot be accurately determined. Song only matching updates song details such as track title and track artist but not release metadata such as track no or album title. This is in contrast with other applications that either treat songs independently and take no care over their album metadata or applications that do consider songs as parts of albums but then prevent matches when the albums cannot be determined. Im not aware of any other application that can currently do this.

Feb 2015  SongKong is the first application to use AcousticBrainz to support Bpm, Key and Mood analysis. Jaikoz adds support shortly after.

Oct 2016 We release the first version of SongKong to solve many of the metadata issues people have when trying to identify Classical music. Solving issues with identification of Classical, Works, Movements, Composers, Performers and Artists.
   
Dec 2016 SongKong and Jaikoz become the only applications to support automatic updating of the new works and movement fields newly introduced within iTunes 

March 2017 We team up with Melco to solve an issue for Naim UnitiServe customers, namely parsing the proprietary metadata used by Naim and embedding into the files themselves using standard metadata.Unique to SongKong for Melco.

Introduction to uPnP AV for streaming Audio to Hi-Fi

Go forward to Introduction to Av UPnP with MinimServer

 

What Used to Happen

A few years ago digital music was usually confined to a single machine. For example you might download files from iTunes on your Apple mac and then play them on the same mac. You may well have an iPod, in which case you would copy your files onto to your iPod and they would exist as separate files.

At the same time Hi-end audio system typically consisted of a CD player, a DAC (Digital to analogue converter), Amplifier and Speakers.

There was little cross-over between the two

Better networking 

Then we started to get hi-speed internet and wi-fi throughout our homes. Network  attached storage (NAS) allowed us to just have one copy of our music , videos and photographs which could then be accessed by multiple devices. 

Everything then started to become networked, not just computers but TVs, set-top boxes, Cd players and stereo systems. A way of having this wide range of different devices talk to each other was required. Microsoft had already developed Universal Plug and Play or UPnP for computers, gaming consoles and network devices such as routers. So it became the obvious choice as a standard for providing services between devices

What is uPnP AV

AV refers to Audio Visual  - this takes UPnP and develops standard for audio and visual, i.e hi-fi systems, televisions, set-top boxes and the like. When people talk about uPnP they usually mean uPnP AV, on this blog we are interested in the A part.

Hi-end audio manufacturers such as Naim and Linn started to see the potential of using hard drives to store music rather than always playing from the original CD. But customers wanted to play music in multiple rooms this meant a way was required to stream music to players that could reside in different locations to where the music was stored, and they wanted to be able to control this from a single point, uPnP AV addresses these issues.

uPnP

uPnP AV has three main components

1. Media Server
2. Media Renderer
3. Media Controller

The media server streams music to media renderers, control of the media renderers is via the media controller. 

Media Server

The media server is software that could be running on a computer, a NAS or a purpose built hi-fi server such as the Melco N1 or the Naim UnitiServe

The most popular media servers are Asset, MinimServer, Twonky and JRiver . The choice of media server is important because the Media controller can only see the metadata that the media server presents it with, and the media renderer can only play the files that the server provides.

We think MinimServer is by far the most flexible media server, offering a significantly more powerful approach to metadata then the others that let you harness all that metadata added by SongKong and Jaikoz, and we delve into MinimServer in more detailing starting here

Media Renderer

The Media Renderer either plays the music itself or converts it so it can be played by something else further down the line. So the media renderer is usually a piece of hi-fi hardware such as the Linn - DS Series or the Naim NDS

Media Controller

The Media Controller is better known by the curious title of Control Point. Since customers want to be able to control their music where-ever they are in the house the focus of media controller development is definitely for iPads and Android based tablets. But control points are available for
the Mac and PC as well.

Control Points fall into three categories:

1. General purpose software that is designed to work with any compatible server and renderer such as PlugPlayer and SongBook

2. Hi-Fi Branded software that actually seems to work fine with any server, an example of this is Linn Kinsky

3. Hi-Fi software tied to particular hardware such as the Naim App

The choice of control point is not as critical as the other components, at the moment they all work in quite a similar way with not much to distinguish between them.