Home User Forums SongKong Music Tagger Jaikoz Music Tagger Albunack Music Service

Song Tagging and Metadata Blog

Monday, 7 November 2016

Grouping field no longer working in iTunes ?

In October iTunes added some long overdue better support for classical releases in the form of new moment, movement no, total and work fields. This allows movements within a release to be shown together, and really helps for classical releases, especially when you have an album containing multiple works. This is exciting news for SongKong and Jaikoz and we are working on adding support for auto population of these fields in the next release.


However these new fields have caused a problem.




Why is my grouping column blank for newly imported files ?

 

MP3 and AIFF files use the ID3 metadata format for storing information about the music such as the artist or album name. ID3 did not have any fields (known as frames) defined for most of these new fields so Apple has defined some:

MVNM - Movement
MVIN - MovementNo/MovementTotal


But ID3 does already have a field that could possibly be used for storing the work, the TIT1 frame, this is how it is defined in the ID3 specification document.



The 'Content group description' frame is used if the sound belongs to a larger category of sounds/music. For example, classical music is often sorted in different musical sections (e.g. "Piano Concerto","Weather - Hurricane").

This definition is a bit open to interpretation so over the years the frame has been used by different applications in different ways, both as a work and as a way to group multiple tracks in other ways. For example in SongKong it can be used to store genres as a list of values because iTunes Genre can only see the first value, a problem if you want to categorize a song with multiple genres.

In fact until version iTunes 12.5.3 iTunes have always used this frame for the Grouping field. When you update to the version of iTunes your existing files will appear unchanged, but any grouping data that was stored in the TIT1 frame has now silently been moved to iTunes internal database only.


But when you import any new music that had the grouping field set you'll see it no longer shows in the Grouping field of iTunes but the Work field, and this also applies if you update any music outside of iTunes that is already in your iTunes library.
 

You can see this by going to the Songs view and enabling the Work column, and there are all the missing groups.




I created this test from SongKong very simply by 
- Ensuring Update Genres and Save changes to iTunes was enabled on the Basic tab




- Setting Grouping to Always Replace Values on the Genres tab







- and  running against a folder of MP3 files

 

 

So you can't use the Grouping field anymore, right ?


Well no, once you have added your songs to iTunes you can enter data into the Grouping field and it will persist. But it's no longer actually stored in the TIT1 field itself. Instead it is only added to iTunes internal database. But the problem with this is you can only fill in the Grouping field manually from within iTunes after the file has been imported in.  Then once done to get that Grouping data out you would have to use iTunes AppleScript (OS X) or the COM API on Windows, this makes it very difficult to use your organize your songs consistently in both multiple places, such as using with both iTunes and Sonos.





Now for the really crazy part


MP4 files don't use ID3 for metadata they use atoms instead and the ©grp atom does continue to correctly map to the Grouping field if you import MP4 files.

This is because for MP4 Apple has defined the following new fields:

@mvn Movement
@mvi MovemntNo
@mvc MovemntTotal
@wrk Work


So there are separate fields for Grouping and Work !

 

 

Why didn't they define a separate field for Work for ID3 ?


That is the question, by adding new the new frames MVNM and MVIN frames they are already not strictly adhering to the specification anyway so the argument cannot be they did not want to break the standard


What does this mean for me ?


It means that if you use iTunes that when editing metadata for files outside of iTunes you need to consider different audio formats differently.
 

If you have a mix of Classical and non-Classical and you use AIFF or MP3 then it may be wise not to use the Grouping field for anything other than Works.

One question that requires more thought is when importing MP4 does it make sense to add tracks with the same Work as part of the same Grouping or not.

We will be adding options to SongKong and Jaikoz to make this as simple as we possibly can

Within iTunes it is probably best to decide on a format and stick to it, transcoding different audio formats to a single format.


  

Tagging Classical Music:Part 5 - Classical Classification

Go back to Part 4 - Opus and Ranking

In the previous section we saw how that classical composers in centuries gone by often used the opus and ranking system to classify releases, we also saw they didn't do a very good job of it !

But in the case of the major composers after they died often a good soul came along and tried to catalog their works in a more consistent way. This is called classical classification and is an attempt to classify all of a composers works, it is completely separate from the record label catalogue given to a new CD release by the record company.

Usually catalogues are in chronological order but sometimes they are a thematic catalogue.
For example Johann Sebastian Bach, uses the BWV (Bach-Werke-Verzeichnis).  The BWV catalog is thematic so a lower number does not mean it is an earlier composition, for example his cantatas are numbered BWV 1 to 244 and Violin concertos BWV 1046 to 1051. 

There are no other composers whose work is catalogued using BMV numbers so it quite simple to extract them from the song/work  title into the Classical Catalog  field and that is what SongKong does. But things aren't so simple the following composers all use a H (Haydn uses Hob as well) prefix
  • Haydn
  • Hector Berlioz
  • Bohuslav Martinu
  • Gustav Holst
and of course there are song titles that contain H. without it signifying a classical catalogue no.

Luckily because the MusicBrainz database stores the composer separately SongKong only attempts to find a classical catalog no when it has identified the composer of the piece, and then it will search for the correct classical catalog pattern(s) for that composer. This ensures that when SongKong has extracted a classical catalog no it gets it right 

This field offers another way to organize works by the major composers. 

Go back to Part 4 - Opus and Ranking

Wednesday, 2 November 2016

Jaikoz 9 Released 1st November 2016

Pleased to announce a new major version of Jaikoz, this release contains many improvements, fixes and some significant infrastructure work.

Mood, BPM and Key Tagging

Jaikoz Pro users can now make use of AcousticBrainz - analysis has been done on millions of songs that are in the MusicBrainz database, when Jaikoz matches a song to MusicBrainz that has been acoustically analysed it can be update with the AcousticBrainz metadata, and this is all easily visible in the Mood tab of the Detail panel


DSF Support

DSD is a technology first used with Super Audio Cd's (SACD), it uses pulse-density modulation encoding a diffierence approach to other lossles formats like WAV and AIFF. The signal is stored as delta-sigma modulated digital audio; a sequence of single-bit values at a sampling rate of 2.8224 MHz (64 times the CD Audio sampling rate of 44.1 kHz, but only at 132768 of its 16-bit resolution). DSF is a way of storing this format in files with metadata and is gaining in popularity on hi end audio systems.

Jaikoz fully supports DSF, including acoustic fingerprinting and all metadata fields.

 

New Never Modify/Modify If Empty Options

Previously most fields had their own option that allowed you to configure Jaikoz to always modify, modify if empty or never modify the field, and the option was repeated for MusicBrainz match and Discogs match. But it was very cumbersome if you suddenly wanted to change the option for a number of fields. We have now replaced by a list of fields that you can easily modify for each option.

 

Performers

Jaikoz now properly supports editing of the Performer and Involved Person fields, these fields allow you to add additional people involved in the song whatever their role, be it musician or art direction



Classical Tagging

Jaikoz now supports many new fields including many new Classical fields, these Classical fields are all visible on the Detail tab. Auto population of the new Classical fields can currently be done with SongKong  but will be added to a future release of Jaikoz.



Full Change List


The full list of improvements can be found at http://www.jthink.net/jaikoz/jsp/news/start.jsp

Wednesday, 19 October 2016

SongKong 4.2 Released October 18th 2016

Pleased to announce that SongKong 4.2 was released on October 18th 2016. This new release has a number of fixes and improvements


Add Composer to Album Artist


Classical music albums often contains the composer name as a part of the album title but not always. Different composers often named their works simply such as Symphony No 1 so without the composer name it can be difficult to distinguish one album from another. With this option enabled SongKong will add the composer(s) surnames to the start of the album title if the composer is not in the current title (retrieved from MusicBrainz/Discogs) to help with easier album identification.


Other Fixes

  • [SONGKONG-1051] - Ensure AlbumArtist always set even if album has no album artist
  • [SONGKONG-1053] - Add option to include composer in album title for Classical
  • [SONGKONG-1054] - Romanize non-latin artist names should apply to all people
  • [SONGKONG-1058] - Bundle Java with linux headless on website
  • [SONGKONG-1024] - When run songkong from cmdline log output from jaudiotagger goes to console
  • [SONGKONG-1052] - When Part derived from MusicBrainz has a leading space character
  • [SONGKONG-1055] - Discogs match/update ignoring isClassical Never Modify/Modify if Empty option

Thursday, 13 October 2016

Tagging Classical Music:Part 4 - Opus and Ranking

Forward to Part 5 - Classical Classification
Back to Part 3 - More Fields Required

One issue with Classical music is Classical composers were not the most imaginative when naming their works and it can be difficult to differentiate one work from another. Conversely the same single work can be available on many different releases but not consistently named by the record companies making it difficult to identify when you have different versions of the same work.

Some composers used the Opus catalog mechanism to organize their works, if used in conjunction with the composer this can be helpful for organizing your music.

 

Opus

Many Classical composers used the Opus (Op.) number. The concept was to catalogue each work as they were published, so the Opus provides a list of all the composers works in date order.

However some composers only bothered to catalog their best works, some only used it for certain types of works and  some didn't use the system at all. 

An Opus can contain multiple works. most famously Vivaldi’s “Four Seasons”  was part of twelve violin concertos all written between 1723 to 1725 packaged as Opus 8 “Il cimento dell’armonia e dell’inventione”.

Sometimes works were only produced after the composers death. 
 For example Schubert’s last three piano sonatas, which were written in the last months of his life, but were not published until about ten years after his death are written Op.  posth[umous].

 
Ranking

Whereas the Opus should identify all a composers works in date order, the ranking (No.)  is used to catalog all works of a particular type

For example Beethoven wrote 9 symphonies, numbered 1 to 9  in chronological order, and 16 string quartets numbered 1 to 16.

Mozart wrote 40 symphonies, but these are numbered 1-41 because the Symphony No. 37 in G major, K. 444 was found to be composed by Michael Haydn  and was removed from Mozart’s catalogue, so there can be gaps.


Opus with Multiple works

Remember an Opus can contain multiple works. With  L. van Beethoven’s String Quartet No. 8 Opus 59 No. 2 in E minor, there are two numbers No. 2 and No. 8. The No. 8 refers to the  ranking among his sixteen string quartets; but the No. 2 actually represent the sub-opus number (the second piece within the Op 59).


SongKong 

SongKong intelligently extracts Opus and Ranking numbers into their own fields. Now they are in their own fields they can help you to search and organize your collection and they can also be used to construct new filenames for these classical compositions.

But other cataloging methods have been developed for certain composers and we will discuss that next

Forward to Part 5 - Classical Classification
Back to Part 3 - More Fields Required

Monday, 3 October 2016

SongKong 4.1 released 3rd of October 2016

We are pleased to announce that a major new version of SongKong is available today

The main focus of this release is once again support for Classical Music tagging but there are other things as well


  • We now allow you specify almost any fields that you do not want Songkong to modify, and also any fields that you only want SongKong to modify if they are currently empty. Both of these options are on the Format tab
  • We have similar options for just Classical music so that you can configure SongKong to usually modify a field but not if it has been identified as a Classical song.
  • Added option for Classical to remove the composer from the AlbumArtist field, since the composer is not usually a performer.
  • Added option Shorten Work:Movement track titles to just Movement where possible - so your classical tracks just have the movement in the title.
  • Now Classical groups are categorised as Orchestra, Choir or Ensemble, previously the Orchestra field was used for all types.
  • Added isclassical as a field that can be used in rename masks
  • Added Ranking and Classical Catalog No fields, we already had Opus but these fields allow us to add other ways of cataloging classical releases
  • Fixes to allow encoding of filename on filesystems that do not support 16bit characters in filenames.
You can download the latest version from here

Tagging Classical Music:Part 3 - More Fields Required

Go forward to Part 4 - Opus and Ranking  
Go back to  Part 2 - A more Sensible Track Artist
 
How metadata is added to music files is largely determined by what went before. Different audio formats use different metadata mechanisms but the most significant one is ID3. This format was originally created for mp3s but is also used for other formats such as AIFF and WAV. 


ID3 Specification

Now the ID3 specification does consider Classical music a little:


  TIT1
   The 'Content group description' frame is used if the sound belongs to
   a larger category of sounds/music. For example, classical music is
   often sorted in different musical sections (e.g. "Piano Concerto",
   "Weather - Hurricane").

  TIT2
   The 'Title/Songname/Content description' frame is the actual name of
   the piece (e.g. "Adagio", "Hurricane Donna").
 
so in Classical music TIT1 was originally really intended for the Work. However this field is typically called Grouping and used for all kinds of different things not related to a work, typically as as a way of categorising music further with sub-genres or moods.


 TIT3
   The 'Subtitle/Description refinement' frame is used for information
   directly related to the contents title (e.g. "Op. 16" or "Performed
   live at Wembley").
 
TIT3 is usually known as the Subtitle field, we see here that the Opus could be stored in this field but the title can contain all kinds of additional information not just the Opus so it is not very helpful. it is never a good idea to use the same field for different types of information.


User Defined Text Information Frames

So the standard ID3 fields do not properly support Classical music. However ID3 also supports user defined text fields known as TXXX fields and this allows us to store name/value pairs for any additional data. This is also the same as how all metadata is stored in Flac and OggVorbis file formats.


Do not Lose the Semantic Meaning

If we consider the Track Artist again once the performers, orchestra and conductor have been added to this field it is difficult to work out what their role is. So we want to also capture the information in particular fields such as Orchestra or Choir.
 
SongKong can define additional fields that can be used by any other application. Sometimes there are existing defacto standards that we can use, and sometimes we have to define additional new fields. We store this additional textual metadata and then applications and customers can make use of this metadata as required. We also publish details of these fields and make then available in our jaudiotagger software library so they can easily be used by other applications.

Songkong has added many fields that are particularly useful for Classical such as


  • CLASSICAL_CATALOG
  • OPUS
  • RANKING
  • PART 
  • PART_TYPE
  • WORK
  • WORK_TYPE
  • CHOIR
  • ORCHESTRA
  • ENSEMBLE
  • PERFORMER_NAME
  
We will discuss these in a later post 
 
 
Go forward to Part 4 - Opus and Ranking  
Go back to  Part 2 - A more Sensible Track Artist

Friday, 23 September 2016

Tagging Classical Music:Part 2 - A Sensible and Configurable Track Artist

Go back to Tagging Classical Music Part 1
Go Forward to  Part 3:More fields required

We saw in TPart 1 how the Track Artist provided by MusicBrainz is great for Pop/Rock music but not for Classical music and in Part 3: we see how the standard metadata specification is somewhat lacking for Classical music.


What is Classical Music ?

To be able to do something clever for Classical music first of all we need to identify what is Classical music. The definition is not clear cut but SongKong uses various heuristics to try identify if a song is Classical, these include considering the composer (if any), the level of works, acoustic analysis, the record label, identification of orchestras and choirs in the performers and text analysis of the release and song title. We build a probability based on all these factors and if the probability is high enough the track is marked as IsClassical

Note:However it should be said if you have no Classical music or do not want to treat your Classical Music differently you can disable this by disabling the Automatically Identify Songs that are part of Classical releases and apply those changes option on the Classical tab



A Configurable Track Artist

Once a track is defined as being Classical the Classical options within SongKong come into effect. The MusicBrainz Track Artist option allows you to configure from a number of different options, the default is Performers and Choir/Orchestra/Ensemble and Conductor - note the Composer is not included.

SongKong then builds this track artist by looking at the Track Artist Credits, Recording Artist Credits, Release Artist Advanced Relationships and Recording Artist Advanced relationships. 

It also does additional text based analysis, to take a very simplistic example if there is artist credit containing the word Orchestra in the title then that artist would be considered to be an orchestra even if there is no performing orchestra advanced relationship for this track.

Another example is that we have identified over 3000 Classical composers that died before 1900, we can be mostly sure that if one of these artists is credited on a track that their role was composer rather than performer even if there is no composer credit listed.

The data is not always complete but by using the above techniques we can usually create a track artist credit that matches the option chosen.

Go back to Tagging Classical Music Part 1
Go Forward to  Part 3:More fields required

Friday, 16 September 2016

Tagging Classical Music:Part I - The Track Artist

Go to Part 2:A Sensible Track Artist

In Pop/Rock music there is usually a single performer or band who is credited for the track. They are often the composer of the song as well but when they are not the details of the writer are not usually so important, the artist credits on the release itself make it clear who should be credited for each track.

For example 19 the first album by Adele contains mainly songs written by Adele, but track 9 Make You Feel My Love is a cover of a Bob Dylan song, and the main composer of track 3 Chasing Pavements is Ed White. However the Track Artist for all songs is Adele.

But in Classical music, various performers, an orchestra, a conductor and a composer are all usually credited, this means there is potentially a long list of people credited for each track, and we don’t have an accepted order they should be added in.

MusicBrainz approach is to store the Composer in the Track Artist field as can be seen in the release of Piano Concertos whereby all songs are credited to Mozart.


However the the actual performers of each track such as this one are stored in the Recording Artist field, and as relationships between the recording and various people involved.



Until now Taggers usually take the track artist field from the databases such as MusicBrainz  and use it for the Artist field, the result of this was that the Composer was added to both the Composer and Artist fields, and the performers were only stored in specialist fields like Performer and Conductor, if at all. But this rather simplistic approach is not really satisfactory for Classical music

In the Part 2 we explain how SongKong uses a more sensible value for Track Artist in Classical music.

Go to Part 2:A Sensible Track Artist

Friday, 9 September 2016

SongKong 4.0 now released with Classical support

We are pleased to announce that a major new version of SongKong is available today

The main focus of this release is support for Classical Music tagging. Classical Music has never been as well supported in taggers, players or indeed any aspect of recording music as Popular Music but with this version of SongKong we aim to address that.


We will discuss Classical in more detail in other post but there are some key improvements that we list here:
  • SongKong now identifies when a song is classical music 
  • When identified instead of just using the composer as the track artist by default it now adds the performers, orchestra and conductor, but you can configure to keep on using composer if you wish.
  • Works, movements, part, part numbers and part types are now added in their own fields
  • Orchestra and Opus now have their own fields
  • SingleDiscTrackNo now stores the trackno as if it is ona single disc release
  • Performers are added with their instruments, with additional fields to indicate if they are credited soloists or not.
We also make some general matching improvements.

The reports now let you play your songs from within the report to make it easier to check results. This works for all formats that html supports (mp3, wav, ogg, mp4)

We also provide a linux version that can be installed when you only have a command line and no gui (also known as a headless server). SongKong has always had the ability to be run from the command line but previously the installer itself needed a Gui.


Thursday, 18 August 2016

What would you like to see for Classical Track Artists

For Pop/Rock the track artist is clear, generally they are the artist credited for the track on the album cover, they are the performer of song and often the writer as well. If tracks are not credited seperately then they will be the artist credited for the release as a whole, and it is assumed all tracks are performed by the same artist. Usually the artist credited will be just one person or band, and rarely more than three people.

But for Classical it gets more complicated...

 Keeping things simple at release level a Classical release will give usually give credit in this order to
  • Composer
  • Soloist
  • Orchestra
  • Conductor
Of course they may be no soloists and if the release contains multiple works there many be multiple composers or in the case of compilations even multiple orchestras and conductors

At track level clarification can be provided on which tracks a soloist actually performed. When there are multiple works by multiple composers the track artist would only list the composer for that particular track and so on.

So there are more types of people credits on a Classical release, and more people in total.

What should we store as the Release Artist in SongKong and Jaikoz

We use what MusicBrainz provides, which usually corresponds to credits on the cover, and would usually comprise Composer, Soloist (if prominent), Orchestra and Conductor. By having the composer first this has the added advantage that if the AlbumArtist field is used for naming files then all releases by a particular composer would be alphabetically grouped near each other.

But what should we store as the Track Artist in SongKong and Jaikoz

In the past track artist would probably just store the composer field, and this is what MusicBrainz currently provides by default and is what SongKong and Jaikoz currently use.

But in all metadata formats there is a standard Composer field, and this is now well supported by most media players, so do we need to store it again here. There is also a standard Conductor field but this less well supported. What there is not is good support for Soloists and Orchestras

But MusicBrainz does in most cases also store the conductor, orchestra, soloists and also performers

When there is an orchestra performers would probably not be considered of enough signifcance to be part of the track artist, but when it is a classical piece performed without an orchestra then they could be.

I am working to add support to store Soloists, Performers  and Orchestras in their own fields as well to allow users to make use of the data as they wish but we still need a sensible value in the Track Artist field and I would love to hear what you think about this

Some Options

Use Composer as Track Artist
Use Orchestra as Track Artist
Use Soloist/Performer/Orchestra as Track Artist
Use Soloist, Orchestra, Conductor as Track Artist
Use Composer, Soloist, Orchestra, Conductor as Track Artist

But I would rather not add options that are not going to be used so which would be your choice ?

Friday, 1 July 2016

New version (3.25) of SongKong Music Tagger with new features

I am pleased to announce that SongKong 3.25 is now available

This contains some bug fixes and some improvements

1. The rename mask has been extended to allow you to use audio data such as sample rate and bit rate and also to use the original filename as it was when the file was first loaded into SongKong.



2. To help customers with Classical music or home made albums that may use certain fields in a particular way there are now more option to not overwrite these fields on the Artist Format and Album Format tabs




3. We no longer try to guess whether we have permission to edit a file at load time because this does not always work for some filesystem/network combinations. Instead we try to save first and if this fails because of a permissions problem we then display the permissions for that file in the report

4. When using Delete Duplicates you can decide when you find duplicates what criteria to use for deciding the duplicate file to keep and the duplicates to delete. We have now added a new option File Modified Date


Full details here














Friday, 25 March 2016

Tagging DSD Files with Artwork and metadata

What is DSD?


Direct-Stream Digital (DSD) was the name used by Sony and Philips for their system of digitally recreating audible signals for the Super Audio CD (SACD),  But more recently it  has gained traction as a way of storing computer-based audio, especially for playback on hi end audio systems.

Most digital audio use a form of  pulse-code modulation (PCM). But DSD is different in that its underlying audio storage is pulse-density modulation (PDM).

This is how Paul McGowan of PS Audio describes it

1-bit audio is simple to understand in concept.  There are no samples, there are no words, there is no code.  Instead there is a continuous streaming “train” of single identical bits that are either on or off.  The more bits that are on, the higher the eventual output voltage becomes.  The more bits that are off, the lower the eventual output voltage.  We refer to this type of scheme as Pulse Density Modulation because when you have a greater number of on bits it appears as more densely populated.  
....
The speed of the bits is 64 times the sample rate of a CD and some DSD schemes run at 128 times faster than a CD.

Here’s the interesting part of this: if you take a DSD stream and run it through a simple analog lowpass filter to smooth out the on/off transitions, you get music!  This is amazing considering that if you do the same with PCM you get only noise.

DSD is a lot closer to analog than PCM ever thought to be.

this simplicity and direct relationship with analog provides for a potentially better listening experience. 

Converting Super Audio CD to Computer Files ?

When we convert a Super Audio CD to computer files to store on a computer there are two options, DSF and DFF

DSF supports ID3 metadata just like MP3s, but DFF does not and requires use of an separate database. 

So it is always best to convert to DSF not DFF, if you already have DFF files then it is simple to convert to DFF using an Audio converter application.

 

Adding artwork and metadata to DSF files

I said earlier that DSF support the ID3 format, this means that tagging software that support DSF files such as SongKong can add the rich depth of information from MusicBrainz and Discogs that it already does for other formats.

1. Select your DSD folder(s)



2. Start Fix Songs


3. SongKong will identify your songs and add lots of lovely metadata, when it has finished
 a summary dialog will appear



4. Then a Song Changes report is open which details exactly the metadata that has been added, even with two screenshots we cannot fit it all in



5. Now you can properly manage your DSD music with the Music player of your choice.
  

Monday, 14 March 2016

A Discogs Tagger

If you have digitized your Music collection, then there is a real boon in not just identifying your songs but maintaining the links to the Music database they have been matched to. For example you can list your collection on Discogs, you might want to just share your music preferences or you may be interested in trading some of your records.

This is why SongKong and Jaikoz do not only store basic metadata like Artist, Album and Title, but also fields that identify the correct MusicBrainz/Discogs data such as the artist(s) and release.

With previous versions of SongKong songs an attempt was always made to match to MusicBrainz before matching to Discogs. But now in the latest 3.24 release you can disable MusicBrainz matching and only match your songs to Discogs, just disable the Search for a MusicBrainz match.



Now you can even disable the Acoustic fingerprinting stage, but even if you are only matching to Discogs then SongKong can still make use of  acoustic fingerprints, so we do not usually recommend it.

And of course you can just match to MusicBrainz and not update or match to Discogs by disabling Update from Discogs and Search for a Discogs Match.
Jthink blog Jthink Facebook page google_plus Jthink YouTube channel Email Paul at Jthink Subscribe to Weekly Newsletter