Custom sort order in music libraries: macOS and Android (en, zh-hans, ja)

在 macOS 和 Android 平台实现音乐库中的自定义排序
macOS と Android での音楽ライブラリーのカスタムソート順

Custom sort order in music libraries is a rather rare need. Most major languages use phonograms in their scripts, where the natural sort order is more or less identical to what is seen in Unicode (probably after some normalizations). On the other hand, languages using logograms (logosyllabic scripts, mainly Chinese characters in our context) does not have their characters sorted in their primary natural (usually phonetic) order in Unicode.

This causes a problem where a list of text sorted in Unicode code point order can be odd and difficult to look up from in these languages. Custom sort order in music libraries is thus useful when you have songs in one of these languages, or even a mix of them.

As this article involves mainly with concepts common among Chinese and Japanese language users, this article is also written in zh-hans and ja. Translated versions are available on my blog.

我的博客里面有本文的中文版。

ブログにはこの記事の日本語バージョンがあります。

Current situations

Currently for speakers of Chinese and Japanese using each platform, there is a different strategy tackling this problem.

Subscription-based platforms. The content provider attaches a sort order key for each name in their music library, and using a (probably proprietary) way to associate them with the music file, and apply them accordingly on web or native platforms. “Sort order keys” are alternative names to title, artist or album that is used in sorting in place of the original name, but never affects how the name is displayed.

Local music in Chinese-made players. Chinese-made music players usually have a built-in Pinyin converter which is used when importing music library. The downside of this is, sometime it would perform badly when dealing with Japanese titles. Some of these players ignores any characters that are neither Simplified Chinese nor Latin alphabet. When you want to look for a Japanese title, you need to first try to think of which character comes first in the title that is also in Simplified Chinese.

Local music in generic players aware of localization problems. (This includes the latest Android when set to the correct language.) These platforms utilizes localized collation to solve this problem. Database systems usually includes localized collation table from the International Components for Unicode (ICU) library. Collation may be a tentative solution for the most, but it doesn’t work if you have a mixed library of both languages.

In Chinese, most character has only one pronunciation. The ICU Pinyin collation is similar to the GB16386 Chinese character encoding, which uses the most common Pinyin of each character to sort alphabetically. This works decently most of the time.

In Japanese, almost all Kanji have multiple pronunciations, and it could change when multiple Kanji is grouped together. Deciding pronunciations for Japanese text is a non-trivial task, and usually involves Natural Language Processing (NLP) techniques. ICU has followed the JIS X 4061 standard (Collation for Japanese Character String), which has defined one common On-yomi for each Kanji. Despite being better than raw Unicode code-point order, this is still considered as against intuition.

Local music in other generic players. Other players that only aware of the English context tend to use the most natural sort order for computers – Unicode code point order – to sort the list. Common Chinese characters are sorted in Unicode by a shape-based language-neutral way called “Kangxi Radical” order. But this order is counter-intuitive for speakers of both languages.

Generating sort keys

As suggested by many previous articles on this matter, tagging manually is the most straightforward way for most of the people, despite introducing a large amount of work when tagging a large library. iTunes has provided a simple interface to add sort keys tags in the “Song Info” menu.

If you have library too large to tags manually, there are still ways to tag it semi-automatically. Both Chinese and Japanese has the case where the same character can be pronounced differently depend on context, and computer generated pronunciation could be incorrect when it can’t detect the right context, or when it simply doesn’t aware of the context at all. Thus, manual review is still necessary for correct tagging.

As a Pythonista, I have written a script to do tagging and reviews in Python. Pinyin is generated using the Python library PyPinyin, while Japanese Yomigana is produced with Yahoo Japanese Morphological Analysis API which requires internet connection at runtime. If you want a local solution, MeCab with mecab-ipadic-NEologd is a really good one, and may perform even better than Yahoo’s API.

Extending a step further, I wrote a simple Flask+Vue+MongoDB web app to enable tagging to be reviewed remotely. Source code of it can be found on GitHub.

Attaching tags to music files

Despite being a minor need, most major music tagging formats has thankfully included property fields for custom sorting order keys. JAudiotagger has provided a comprehensive list of tags used by different formats that represents sort orders.

In my Python script, I have used Mutagen which has a good support to both ID3v2 tags (for MP3 files) and Vorbis Comment (for FLAC files), and even more. For Java/Kotlin users, JAudiotagger is also a good choice. I used that in my Android solution (mentioned later) too.

Use these tags in macOS (and iOS)

Nothing much to highlight here. iTunes in macOS comes with excellent support to those sort key tags, it allow you to edit these tags within itself. One thing to note is that you might want to re-import your music library to refresh if you have done edits outside of iTunes.

iOS, I believe, should also honor these tags as well, since they also use iTunes as the player.

* In later versions of macOS, “iTunes” is renamed to “Music”. I haven’t tried it, but it shouldn’t affect much on the tagging side.

Use these tags in Android

Unlike apps in the Apple ecosystem, Android system has zero awareness of custom tag order. Most music players on Android platform uses the Media Storage API provided by Android. Inside the Media Storage component, it maintains a database that records all media files discovered by the system, including music files. It does have columns like title_key, artist_key and album_key, but what it does is not more than just stripping off stop words (“a”, “an”, “the”, etc.) from the beginning of titles and convert them to collation keys. It would use the localized collation according to the system language, but that is not good enough as explained above.

I have previously tried to find a way to patch the AOSP to add support to these tags, but the media metadata extraction code (in libstagefright) is written in C, and I was kinda lost wrapping my head around it. After that, I have also tried to explore the option of using Xposed Framework to hook on the Java part of updating the Media Storage database. Sadly the section of code calling for collation key generation and writing to database is in the same function. To hook in between them, I have to rewrite the function as a whole which is not a good practice.

In the end, I decided to exploit the fact that Android media storage strongly relies on the *_key columns and to use Root permission to directly update the database with the sort keys we wanted to use.

I then started to write an app to update the database. How the app works is rather simple:

1. Get Root permission and storage permission to read and write to the media storage database, and to read sort tags from music files.
2. Copy out the database file to the data folder of the app for processing.
3. Load the list of music files to check from database.
4. Use JAudiotagger to extract sort key tags from music files.
5. Update the collation keys from database accordingly.
6. Write updated database back to the system.

After updating the database, you can force close your music player and restart it to see the result.

Shortcoming of this method is that the database needs to be updated manually every time when a media file is changed.

Source code and compiled version of this app is also available on GitHub.

The post Custom sort order in music libraries: macOS and Android (en, zh-hans, ja) appeared first on 1A23 Blog.