Why has running Songkong on NAS left my files in an incomplete state ?

We just received a support request from a  customer who had just moved from using SongKong on a PC to using Synology Disk Station.

Hi Paul I have a technical issue with songkong the server version for Synology. it works but not all the time. Ive attached a screenshot which appears to leave the files in an incomplete state. when I undo and then go to the desktop version of dry it completes properly. it has this error maybe 50/50.

Actually what is happening here is the files are actually fine, but they are not Windows compatible because of one of the characters used.

Problem

Lets look at a folder on my own computer, it exhibits the same problem.

But if we view the same files using File Station on the Synology itself then everything looks fine

The problem is simply that the filenames include the colon character ':' and this is invalid in Windows so Windows just displays an alternative almost random filename. If you try and add such a character on Windows itself you get the following message

The problem is worse for Classical Albums, since convention dictates the album title separates main performers and composer from the title with a colon. So if this is used as part of the filename mask then the colon finds its way into the filename itself.

The same problem also exists for QNAP servers and any linux server.

Short Term Solution

Firstly the files are just fine, you will only see this issue if remotely viewing them from a Windows PC, viewing from a Mac or Linux computer wil be fine since these are both UNIX based.

But if you are a Windows machine  it would be nicer if the files displayed correctly on your PC. So short term the solution are:

• Set Rename files based on metadata  to No on the Basic tab, this is the default setting and will prevent file renaming.
• Edit the filename using Edit Songs Metadata to remove the colon, then they will display again as expected on your PC.

Long term Solution

In the next release we will add a new option to SongKong to enforce Windows compatible filenames when running on non-Windows systems. If SongKong is running on Windows then it only creates windows compatible filenames anyway.

Update:We now have added such an option in SongKong 5.12

SongKong now available For QNAP NAS Servers, Intel and Arm Processors !

Hot on the heels of the SongKong 5.11 release, that was the first to support Synology Disk Station, we now have an installer available for the majority of the QNAP NAS range.

And this versions supports both Intel and Arm based QNAP servers.


Install instructions are available here

 

 

Supported Versions

SongKong is available for all QNAP models that support QTS App Center (v4.2 or above) and the Container Station package, this is currently all recent QNAP models with an Intel CPU and also any QNAP with an Arm CPU as long as they have at least 1GB of memory. 

The following models are supported, full details availble from QNAP Container Station Page

• Intel - TS-x51, TS-x51+, TS-x51A, TS-x53, TS-x53A, TS-x53B, TBS-453A, TS/TVS-x63, TVS-x70, TVS-x71, TS/SS-x79, TS/TVS-x80, TVS-x82 series
• Arm - TS-x28, TS-x31P, TS-x31X, TS-x31+ series and TS-1635 with a minimum of 1GB RAM

Full Version, Full Functionality

This is the complete full version of SongKong, with exact same functionality as the PC, Mac and linux versions. The only difference is that you always control SongKong via the web interface rather than choosing between web and desktop interface.

 

Advantages of using SongKong on a NAS

There are three major advantages of using SongKong on the same NAS as your music resides instead of on your PC or Mac. 

Permanently available

Your NAS is usually on permanently, this means SongKong is instantly available if you want to fix or improve an issue with your metadata. And because it only uses the Remote interface it is available from any device that can connect to your NAS, this could be your iPad or Phone as well as a computer.

In contrast your computer is not probably permanently on, and even if it is it may get rebooted if you have it configured to get auto updates.  Unless you are using SongKong in Remote mode it will not be accessible away from the computer and therefore you'll not be able to quickly dive in with your iPad.

Performance

SongKong performance is based on three factors, cpu, memory and I/O. NAS has a big advantage with I/O since SongKong can access the files locally when reading the file and when writing changes to the file. Whereas a PC has to transfer the contents of your file over the network, and then send it back again when modifications are made. This has most significant effect when using large lossless files such as Wav, Aif and Dsf formats.

Your PC may still have the upper hand with cpu and memory, however the more recent  NAS servers  have much better specifications than earlier machines. And NAS servers are probably not doing that much most of time. So it can be a more efficient use of resources then using a PC and having SongKong competing for resource with other applications that you are using at the same time.

Reliability

 Because the file I/O is local there is no risk of SongKong losing a connection to a file, so it is fundamentally more reliable than modifying files over a wi-fi network.

 

How to add metadata to your music

Part 6: What databases can be used for automatic song identification

Part 8: How do I get Artist Consistency in my Metadata

 

SongKong Tutorial Part 7: How to add metadata to your music

Where does the Album and Artist information, displayed in your music player, come from? In most cases this information is stored within the music files themselves as metadata. This is separate to the filename that may possibly contain some metadata such as Title or Track No but is not a practical way to store more then the bare minimum of information.

Different audio formats have different metadata formats, commonly known as tags. For example Flac and Ogg Vorbis files both use Vorbis Comments to store metadata whereas Mp3, Aif, Wav and Dsf all use ID3. But players and other tools such as SongKong usually try to hide these differences.

Metadata is added  by SongKong once the song (and album) have been identified. The Match tab helps determine how songs are identified whereas the Format tab decides what metadata is added. 

This is an important distinction, for example you may want SongKong to identify all your songs, but add no metadata to your songs apart from MusicBrainz Ids, this would be done by altering the configuration on the Format tab.

Additionally the Artwork tab defines how artwork is added, the Genre tab defines how Genres are added and the Classical tab defines additional special options for Classical music.

But we will begin with the Format tab

 

What databases can be used for automatic song identification ?

Part 5: Fix Songs, Review Match Options
Part 7: How to add Metadata to your Music

 

Part 6: SongKong Tutorial: What databases can be used for automatic song  identification ?

In the previous section we described ways you could modify how songs and albums  were identified. But of course the key things that decides what songs are identified are the databases that SongKong uses, so lets explain these.

SongKong uses MusicBrainz, Discogs and AcoustId for song identification, and by default it uses all of these.

If you have a Melco license it can also use existing Naim Wav metadata files but this is a special case. It only applies to customers with a Melco license who files identified using a Naim server such as a Unitiserve and ripped their music to Wav format, so we will discuss this later on.

MusicBrainz

This is the Gold Standard, metadata is rarely wrong and frequently very detailed. Once songs have been matched to a MusicBrainz album it also makes it very easy to lookup additional databases by the MusicBrainz identifiers, since this has become the defacto standard.

So for example we can look up artwork from the Cover Art Archive and acoustic analysis from AcousticBrainz

We can also frequently lookup Artist and Albums pages from Wikipedia and Discogs.....

Discogs

Discogs is a database that concentrates on physical releases (such as CD's/Vinyl) since it allows people to buy and sales records.
 
When we have matched to a MusicBrainz album then we often have a direct link to an equivalent Discogs release, and this is very useful because Discogs often has additional information that MusicBrainz does not have. And this includes Cover Art so now we have another source for that all important front cover art.

But we can also search Discogs when we do not have a MusicBrainz match. Although there is much overlap,  MusicBrainz has albums that are not in Discogs, and Discogs has albums that are not in MusicBrainz. In fact the Discogs database is larger than MusicBrainz so it is very important to have this second database.

AcoustId

AcoustId creates an audio fingerprint of your songs, this then allows the song to be looked up in the AcoustId database, and the AcoustId database often has a link from an AcoustId to a MusicBrainz Recording Id - so in that way we match songs to MusicBrainz.

So the primary use of AcoustId is to help to match your songs to MusicBrainz, AcoustIds are used in conjunction with existing metadata and folder structure to find the correct match.

But AcoustId database is larger than both MusicBrainz and Discogs. So it can very usefully be used to add basic metadata for songs that could not be found in MusicBrainz or Discogs.

Enabling or Disabling these Databases

You can enable/disable matching from MusicBrainz/Discogs and using the link from MusicBrainz to Discogs releases, options are highlighted below on the Match tab. You need to select Show Advanced to see the Disocgs options.

AcoustIds are always calculated as required but you can force SongKong to calculate for every song even when not neccessary for song identification by enabling Force Acoustic fingerprints even if already matched.

 

 

Fix Songs, Review Match Options

 

Part 4 - Three Things to Understand about Fix Songs Results
Part 6 - What databases can be used for Song Identifcation

SongKong Tutorial: Part 5 - Fix Songs, Review Match Options

So you have run Fix Songs in Preview mode, but maybe it is not quite how you want it. This is the time to take a look at the options and fine tune, lets discuss some of the key options you may want to modify that affect song identification.

All these options are on the Match tab

 

Match: All Existing folders represent a single album

SongKong  generally treats folders as an album, this means it will only allow a match to an album if all songs in the folder match to the same album. But it also tries to recognise the occasional folder full of essentially random songs and treat them differently, allowing songs in such a folder to be matched to different albums or not matched at all;.

But if you know that every folder really does represent an album (or disc of a multi-disc album) then you can enable this option to force SongKong to treat all folders in this way.

Match: Preferred Release Countries

Often Albums often have different versions for different territories and countries, but there may be no physical difference between the track listing to help SongKong decide the release to choose.

For example the Italian and German version of a particular release may have different catalogue numbers or be released by a different record label. But this is information is very unlikely to be in your existing metadata so SongKong cannot use that to decide the release to pick.

But you can select a list of preferred countries that your albums are from and selecting those countries will give releases from those countries a boost making it more likely that they will be selected.

SongKong 5.11 Dry, the Synology Release

We have a new release, named Dry after the seminal first album by P J Harvey released back in 1992, the only artist to ever have won the Mercury Music prize more than once.

Synology Release

This is an exciting release as it is the first to support the Synology Disk Station range of Nas servers ( as long as they have an Intel processor). The better news is we have tested with the most basic model the DS218+ and the results are great, almost the same as our main development PC  !

Full instructions available here 

If you are using Qnap instead, watch this space we should have a Qnap release very soon. 

Naim Metadata Matching 

Customers who have previously used a Naim server to rip their music to Wav have been able to transfer the Naim Wav metadata with our special Melco license.

Sometimes the Naim metadata is for a CD with a different number of tracks then the customer has. We used to just ignore these but we can now extract the album metadata and sometimes the track metadata.

We have also made improvements to parsing metadata edited using the Naim App (this is stored in the UserEdits.xml file).

We also now log to the Errors and Warnings section of the report any files that we had difficulties with.

Report Creation

After completing the task SongKong creates a detailed report, sometimes this could take a while. We have made some massive improvements to the way reports are created so now they are much, much faster.

Linux users, SongKong for Java 8

SongKong is now distributed with Java 10. But as a Linux user you can choose to install SongKong without the distributed version of Java. The songkong.sh is intended to work with Java 9 or Java 10 but you can still run SongKong with Java 8 if you wish, you just need to use songkong8.sh or songkongremote8.sh


Full list of improvements and fixes can be found here.

Three Key Things to understand about Fix Songs Results

Part 3: Fix Songs - Use Preview Mode
Part 5: Fix Songs - Review Match Options

SongKong Tutorial Part 4: Three Key Things to understand about Fix Songs Results

The three most common misunderstandings about the results shown in a Fix Songs Report are as follows:

Metadata is not the Same as Filenames

SongKong fixes the metadata in your music files, by default it does not modify the actual filename so after matching your filenames may not necessarily match the metadata if previously they were badly named.

SongKong can very easily rename filenames based on metadata but it doesn't by default because this can cause problems for other applications and general confusion for the customer.

For example playlists are usually created as a list of filenames so if the name of a file changes this can break a playlist file.

Sometimes an artist changes their name (e.g Prince)  so if SongKong uses a different version of the name than you were using previously this can be confusing

SongKong can easily rename files using the Rename files based on metadata in the Basic tab, but in the early stages we recommend against it.

 

Song Only versus Song matching

SongKong group songs firstly based on the folder structure and if that doesn't give results any existing metadata. If SongKong is unable to find an album match for all songs in a grouping then it does not match any of them to an album. This is important because the last thing you want SongKong to do is break up your albumsjust because it could identify some of the songs.

But if SongKong can correctly identify the song (as it usually can with AcoustIds) but not the album then we can add Song Only fields such as Title but do not modify fields such as Album to ensure we improve the metadata without possibly breaking up albums that SongKong simply could not find (perhaps because missing from MusicBrainz/Discogs databases) .
    

Genres are not changed by Default

Genres are subjective, what one person may simply describe as Rock, another may describe more specifically as Thrash Metal. 

So although we can use genres and styles from Discogs we don't by default. You can easily remedy this by enabling Update Genres on the Basic tab. This will add genres but not replace existing ones.

If you want to replace existing genres then you need to make some adjustments on the Genres tab, we will discuss that later on.

SongKong Tutorial: Part 3 - Fix Songs, Use Preview Mode

Part 2: Status Report Customization
Part 4: Three key Things to Understand about Fix Songs Results


Everyone has different reasons to try SongKong, but it very likely involves Fix Songs, the automated song matching task. 

Preview Mode

Without a license Fix Songs  runs in Preview mode, this means you can see exactly how successful SongKong was in song identification but no files are actually modified.

After installing a license Preview Mode defaults to switched off.

Run Preview Mode on your Music Collection

If you did not run Fix Songs in Preview Mode before buying a license we highly recommend you re-enable preview and then run it now on your whole collection with the default options, or at least a sizeable section of it.

 

Summary

The report generated gives a very useful summary of the percentage of matches to different databases. 

Matched to MusicBrainz

But it also allows you to see exactly what albums your  songs have been matched to:

Song Changes

And what data has been added for songs:

 

Metadata Spreadsheet

Spreadsheet can be easily used to browse changes, new fields are added as Yellow, modified fields are added as Green:

It also gives the chance to find things you don't like about the changes SongKong would make. For example you may use certain metadata fields in a non-standard way and would not wish SongKong to modify these fields.