Building a Personal Aggregated Music Streaming Service with Mopidy, Iris, and Icecast
Like most people over the years, I’ve built up music collections from several different sources, such as MP3s ripped from my CD collection pre-online music distribution days (classic 90s hip-hop & early 00s indie rock, ftw!) and online providers through their applications such as Google Play Music.
Not to mention using a few different streaming services, like Spotify and SoundCloud, and having playlists in each. This is a pretty messy situation that we can fix with a few technical skills by aggregating all the above sources into our personal private music streaming service.
First of all, let’s look at options. While streaming music from files in audio formats such as MP3, FLAC, Vorbis, AAC, and WMA has several popular server applications such as mStream, Subsonic, Ampache, Serviio, even Plex; for also connecting to several existing consumer music streaming services such as Spotify Premium, Tidal, SoundCloud, or Google Play Music, there are few options.
Mopidy, the Python-based music serving software through the use of extensions, allows us to connect to several streaming services, as well as having direct file playback, making it the best choice for a personal service that aggregates several of the above sources with clients for play-back across almost every type of user device.
The next consideration comes with where Mopidy is to run as a server application. You can install it on everything from a high-spec Intel-based server down to a low-powered ARM-based Raspberry Pi 3; for building our music streaming service that compares to consumer music streaming services, a host we can use as a media server already, say for Plex or other video streaming server software is ideal.
This server hardware should have power and connectivity suitable for music streaming. Our choice doesn’t matter too much, though we need a bit of technical knowledge to set up and administer the server.
Next, let’s consider what OS we want to use on our Music Server. We want an operating system that is meant for easily handling media in different formats, designed to be close to the user in an n-tier system structure, has good built-in support of required packages and an easy install mechanism for anything else we may need. For this, a Debian based Linux distribution such as Ubuntu becomes an easy consideration.
Ubuntu is quick and easy to use with great support; adding or upgrading any required packages is trivial. It also offers the full power of a Linux based server OS environment, which means we can use a well-established protocol such as the SSH Filesystem, Unix Network File System, or Windows Server Message Block to access any audio files themselves from another source; alternatively, install any other file sharing & synchronisation software.
Mopidy will aggregate our music library from several different sources, play music, and have clients for every platform you can use to control it. Still, critically, it won’t stream audio output over a network since it’s designed for playing music on the local host it’s being run on. For streaming our audio to our devices, we will need to use Icecast, which Mopidy will use for audio output.
Icecast is a streaming media server that currently supports Ogg, Opus, WebM and MP3 streams. You can use it to create an Internet radio station or a privately running jukebox and many things in between. It is very versatile in that new formats can be added relatively easily and supports open standards for communication and interaction.
Likewise, since Mopidy and Icecast are both just server software, we need a front-end UI to view and play music. A good choice is Iris which provides a nice web client front-end for controlling our Mopidy Server, with support for Spotify, Last.fm, Snapcast, Icecast and many other extensions.
Iris brings all our music into one user-friendly and unified interface that we can use on any device with a web browser. We can even use this setup to control and stream audio to a Raspberry Pi or another low-cost, low-power device that you can use to turn any old-fashioned analogue speaker into an internet-enabled smart speaker.
Before we go ahead and set up the actual music streaming server on our machine, let’s look at how the whole system will hang together. Here we can see a centralised system around our Ubuntu-based server machine running Mopidy, which is aggregating the different music sources, streaming audio using Icecast, and will be controlled by the Iris front-end accessed with web browsers on the user devices.
While I’m generally against such centralised dependencies in distributed systems, in this case, it’s ok and desirable for several reasons. We’re seeking to centralise and aggregate several different existing services for easier usage as well as management.
Right, let’s get to the sexy bit. Installing the Mopidy server application itself using the Linux command-line. First of all, login to your Ubuntu host. Let’s grab the Mopidy archive’s GNU Privacy Guard public key, add the Debian Advanced Package Tool repository for Mopidy to our package sources, then install the Mopidy package along with any dependencies.
Note: these are the packages for Debian stretch (stable), which also work for Raspbian stretch and Ubuntu 16.04 LTS and newer.
wget -q -O - https://apt.mopidy.com/mopidy.gpg | sudo apt-key add -sudo wget -q -O /etc/apt/sources.list.d/mopidy.list https://apt.mopidy.com/stretch.listsudo apt-get updatesudo apt-get install mopidy
Now we have to install Icecast, which, as mentioned, Mopidy will use for streaming audio across networks to our devices for playback:
sudo apt-get install icecast2Hit Yes to configure Icecast, then add your hostname and change the default passwords if you want.
Then let’s grab an extension for an existing audio streaming service we listen to music from. For Spotify, this is easy as we have a nice Debian-based package available for the Mopidy Spotify extension:
sudo apt-get install mopidy-spotifyNext, install the Mopidy Spotify Web extension, which allows us to browse artists and albums in our Spotify Premium account library, from the Python source as a Pip Install’s Package package (gotta love that Unix style recursive naming). You may need to install the pip package first:
sudo apt install python-pipsudo pip install Mopidy-Spotify-Web
Finally, install the Iris pip package to have an actual UI to interact with the server and play music with.
sudo pip install Mopidy-IrisNext, let’s get Mopidy running as a service, so it starts automatically on server reboots and runs happily in the background streaming our music. Run this command:
sudo dpkg-reconfigure mopidyThen on the Package Configuration screen check, you are happy with the description and hit Yes.
Since we don’t need to reboot the whole server right now, let’s just start Mopidy up and check its status:
sudo service mopidy startsudo service mopidy status
We should now be able to see it running happily in the background. By default, it cleverly will not run as root, but it's under its own user. Let’s take a look at the service’s configuration:
sudo mopidyctl config | moreYou should see some output like the following:
[mpd]
enabled = true
hostname = 127.0.0.1
port = 6600[http]
enabled = true
hostname = 127.0.0.1
port = 6680
This shows us that the Mopidy daemon is listening for MPD (i.e. music player) client connections and web connections on the loopback interface for our server. This is no good as we need it to accept connections from the outside world. Let’s edit the Mopidy service config file:
sudo nano /etc/mopidy/mopidy.confThen add or amend the following lines to have Mopidy listen on all IPv4 addresses on this machine, and while we’re here, we might as well set the correct country and language codes for Iris:
[mpd]
enabled = true
hostname = 0.0.0.0
port = 6600[http]
enabled = true
hostname = 0.0.0.0
port = 6680[iris]
country = GB
locale = en_GB
Now restart our Mopidy server application, so it picks up the changes:
sudo service mopidy restartLet’s open up a browser connection from our local machine to our server hostname on port 6680. You should see the following:
Woohoo, our Mopidy music server is up and running. But a server alone isn’t a service. Let’s now configure some music sources.
Go to https://www.mopidy.com/authenticate/#spotify follow the instructions, then add the entries it gives you along with your Spotify Premium username and password to your Mopidy configuration.
sudo nano /etc/mopidy/mopidy.conf[spotify]
username = alice
password = secret
client_id = ... client_id value you got from mopidy.com ...
client_secret = ... client_secret value
bitrate = 320[spotify_web]
client_id = ... client_id value you got from mopidy.com ...
client_secret = ... client_secret value
We can also set the playback bitrate to 320 kbps, higher than the default of 160 kbps. This is quite neat as with regular Spotify, even with Premium, you are limited to 96 kbps playback on mobile devices, with 320 kbps playback only available on desktop; with our personal streaming service, this gives us 320 kbps for Spotify across all devices regardless of being desktop or mobile. Restart our Mopidy server as before, and have a quick check of the log for any errors:
sudo service mopidy restarttail /var/log/mopidy/mopidy.log
Open up your browser and point it at our Mopidy web address; this time, browse to Iris then Playlists. You should see your Spotify playlists.
And be able to browse different Spotify categories, as well as under Spotify Web Browse separate artists and albums in your library:
Now let’s get Icecast up and running to listen to music that we play on our server. Edit the Icecast config and add the following lines to the mount section to provide a fallback stream for Mopidy. You can also optionally disable burst-on-connect . When this is enabled, Icecast will send a connecting client a burst of audio data from the stream. This will have the effect of reducing the startup time for the stream from the listener's perspective. Most media players have local buffers that they must fill before the stream begins to play.
sudo nano /etc/icecast2/icecast.xml<burst-on-connect>0</burst-on-connect><mount>
<mount-name>/mopidy</mount-name>
<fallback-mount>/silence.mp3</fallback-mount>
<fallback-override>1</fallback-override>
</mount>
This may introduce a small latency in the stream (difference in time between when the source plays a clip and the listener hears a clip). If this latency is important to you, then you can disable this feature as above.
Latency is bitrate-dependent, but as an example, for a 128kbps stream, the latency between the source and the player is ~ 1.5 secs without burst on connect, and with burst on connect, the latency is 3 secs. After that, you need to place the silence mp3 file in the default webroot for Icecast then restart it:
cd /usr/share/icecast2/websudo wget https://github.com/anars/blank-audio/blob/master/500-milliseconds-of-silence.mp3 -O silence.mp3sudo systemctl restart icecast2
Browse to port 8000 on your server, and ta-da, you should see Icecast running. If you want, you can login into the Administration panel with the login credentials from the Icecast configuration and see that it has silence.mp3 as a mount point.
You need to tell Mopidy to output audio as a 320 kbps MP3 stream to Icecast by adding the following to the configuration. If you changed the default password or anything, set these accordingly, then restart Mopidy.
sudo nano /etc/mopidy/mopidy.conf[audio]
output = lamemp3enc bitrate=320 ! shout2send async=false mount=mopidy ip=127.0.0.1 port=8000 password=hackmesudo service mopidy restart
Mopidy will now output to Icecast. If we want to listen to the audio, we need to open another separate application like VLC and point it at our Icecast stream. We really should just have one user application to deal with, i.e. our Iris web front-end.
So let’s remedy that. Open your browser to Iris and go to Settings, then make sure Icecast is enabled and the location is pointed at your Icecast stream; it should be the hostname of your machine that the server applications are running on:
While there, check your Spotify settings are Authorised by simply clicking on it under Settings and following the instructions. Now go back to the main view, and your streaming music service should be up and working, with you able to browse to it then be able to play tracks through the web browser with Iris on this or any other device we use:
Be aware that streaming audio at 320 kbps can cause Icecast to drop the listener connection if, for some reason, Icecast cannot maintain the stream bitrate to that listener.
This usually means the client can’t receive fast enough, possibly because their connection is too slow. You can check if this is happening by looking in the Icecast log:
tail /var/log/icecast2/error.log[2019-07-31 21:00:09] INFO source/send_to_listener Client 36 (xxx.xxx.xxx.xxx) has fallen too far behind, removing
The reasons can vary from a bad network connection, latency being too high for the TCP settings, proxies getting in the way or just congestion. It’s normal to get it occasionally with some songs.
If this is happening too much, try outputting audio at a lower bitrate in the Mopidy configuration or increasing the queue-size value in the Icecast configuration.
Now let’s get our streaming music service working with Google Play Music. We have to install the Mopidy Gmusic extension from the source with pip. You may also need to upgrade the pyasn1 Python library, which the mopidy-gmusic extension depends on.
sudo pip install mopidy-gmusicsudo pip install --upgrade pyasn1
Next, you need to create a Google App Password. An app password is a 16-digit passcode that gives a non-Google app or device permission to access your Google account. You can only use app passwords with accounts that have 2-Step Verification turned on. Instructions are here: http://support.google.com/accounts/answer/185833
Once you’ve done that, you need to add the following to your Mopidy config. The username is your regular Google username, and the password is the App Password that you created.
This will allow Mopidy to log in to your Google Play Music account. deviceid should be set as mac as below to identify the device to Google with a unique ID based on its hardware address.
Mopidy will need restarting after this to pick up the Google Play Music login details. As with Spotify by default, the bitrate is set to 160 kbps. You can change this to either 128 or 320 kbps.
sudo nano /etc/mopidy/mopidy.conf[gmusic]
username = google username
password = google app password
deviceid = mac
bitrate = 320sudo service mopidy restart
Browse to Iris, and you’ll see any Google Play Music you have now available to play. If you get any problems, take a look at the Mopidy log to check there were no issues with connecting or logging in.
We now have a personal music service that aggregates our Spotify Premium and Google Play Music accounts which is pretty useful.
Let’s add a final couple of sources, though, so it’s a beastly set-up. First, we’ll hit up SoundCloud by installing the Mopidy SoundCloud extension package for it:
sudo apt-get install mopidy-soundcloudNow making sure you have a SoundCloud account; go to the following page: https://www.mopidy.com/authenticate/#soundcloud and follow the instructions similar to how we did for Spotify. Then add the relevant details to your Mopidy configuration:
sudo nano /etc/mopidy/mopidy.conf[soundcloud]
auth_token = 1-1111-1111111sudo service mopidy restart
Now head back to Iris in your browser, and you can now play your SoundCloud streams under Browse.
The final source configuration is to add our audio file library. For local files, we need to use the Mopidy-Local-SQLite extension, a Mopidy local library extension that uses an SQLite database to keep track of your local media.
This extension lets you browse your music collection by album, artist, composer and performer, and provides full-text search capabilities based on SQLite’s FTS modules.
sudo apt-get install mopidy-local-sqliteNow just open the Mopidy configuration and set the local media folder to the appropriate location. These could even be a remote SSHFS, NFS, or Samba mount, as mentioned above, but that’s beyond the scope of this article.
sudo nano /etc/mopidy/mopidy.conf[local]
media_dir = /var/lib/mopidy/media
scan_flush_threshold = 100sudo mopidyctl local scansudo service mopidy restart
While we’re there, the default scan_flush_threshold can be changed to improve database access during a local scan. Then the Mopidy service needs to scan the updated library, and you should be good to go.
Lastly, let’s get the service integrated with Last.fm and scrobbling music that we listen to. Install the Mopidy Scrobbler extension. And configure it:
sudo apt-get install mopidy-scrobblersudo nano /etc/mopidy/mopidy.conf[scrobbler]
username = alice
password = secretsudo service mopidy restart
Head to Settings in Iris and make sure Last.fm is authorised as well as logged in. This is just for the Iris web front-end and separate from scrobbling, which we enabled above; it’s used for supporting the artist and track information. It also provides album artwork.
So if everything has gone without a hitch, we’ve now combined three different public consumer music streaming services and your audio file library into one private personal music streaming service that will play music on any device with a web browser & sound output at a higher quality on mobile than the regular streaming services. Not bad for a small amount of configuration work and minimal or no capital outlay.
📝 Save this story in Journal.
👩💻 Wake up every Sunday morning to the week’s most noteworthy stories in Tech waiting in your inbox. Read the Noteworthy in Tech newsletter.
