Keeping things going when the user leaves

In the past iOS was always about living in the moment - totally focused on whatever the user was attempting to do. This approach was good news for users as it allowed for a very responsive user experience on performance constrained devices however as app developers it limited what we could do because as soon as an app went into the background iOS suspended it. As devices have become more powerful and energy-efficient, iOS has eased the "living in the moment" restrictions while maintaining responsiveness.

One way that iOS has eased restrictions is allowing apps to start/continue download and upload requests in the background - collectively known as background-transfers. Support for background-transfers was first introduced in iOS 7, so it's not a recent change, but it's such a powerful tool that I wanted to explore it further and especially show a solution for recovering from a terminated state.

Different types of sessions

When Apple introduced the URLSession suite of classes, they addressed a long standing complaint - how to configure the networking stack to handle various types of network requests that an app has to make. Rather than having one networking stack like with NSURLConnection, NSURLSession instead allowed for multiple independent networking stacks (or sessions as they are known). The configurations of these independent sessions are controlled through URLSessionConfiguration. The main session configurations are:

1. Default - allowing for all types of network requests in the foreground.
2. Ephemeral - similar to default but more forgetful (doesn’t write caches, cookies, or credentials to disk).
3. Background - allowing for downloading and uploading content even when the app isn't running.

Of course, it's possible to further differentiate sessions within those configurations by adjusting the properties on each session.

At the time URLSession was introduced I was building a very media-heavy app, and the promise of being able to continue to upload or download content when the app wasn't running by using a background session was very appealing.

Peeking under the hood 🕵️

When scheduling a background-transfer on a background session, that transfer is passed to the nsurlsessiond daemon, which runs as a separate process from the app, to be actioned. As this daemon lives outside of the lifecycle of the app if the app is suspended or killed the transfer will continue unaffected - this is the main difference from scheduling an URLSessionDownloadTask or URLSessionUploadTask request on a default or ephemeral session where the request is tied to the lifecycle of the app. Once a background-transfer is complete, if the app has been suspended/terminated iOS will wake it up (in the background) and allow the app to perform any post-transfer work (within a limited time frame). If the app is in the foreground, control will be passed back to the app as if the transfer has been scheduled on default or ephemeral session (without a limited time frame for post-transfer processing).

Important to note here that you can schedule URLSessionDataTask requests on a background session but they will only be acted on when the app is in the foreground.

You're probably thinking:

"That sounds pretty amazing! Why isn't this the default behaviour for transfers?"

Well, there are a few reasons:

• The more background processing that the device has to support the quicker the battery will drain and the more bandwidth that will be consumed. So while Apple understands that background processing is needed, it wants to ensure that as app developers we use it responsibly and only where it's actually adding value.

• From a developer-POV supporting background-transfers requires a bit more programming than a foreground-only session. Rather than the simple closure based API that foreground-only sessions support, to use background sessions the developer needs to conform to various protocols. This additional programming while not hugely taxing (as we shall see shortly) does require each developer to know more about how URLSession works before they can make a network request. So in true Apple fashion, they have opted to not only keep it simple for the users but also for us developers.

Now that the hood has been peeked under, let's get back to how to use background-transfers to provide a better experience for our users.

Scenarios

Any background-transfer solution should be able to handle 3 different transfer scenarios:

1. Foreground - when the app is open, the transfer should behave the same way as if the session was using the default configuration.
2. Suspended/Backgrounded - when the app is in a suspended state and the asset is downloaded, the app should be able to be woken up to finish the transfer.
3. Terminated - when the app has been terminated by iOS, the app should be able to be relaunched to finish that transfer.

In the rest of this post, we will build an image download system that will look to handle the 3 scenarios described above. This example will populate a collectionview using data retrieved from Imgur with all image retrieval happening on a background session.

This post will focus solely on downloading using a background session as it's much easier to find services that allow us to retrieve content from their system than services that allow us to upload content.

This post will gradually build up to a working example however if you're on a tight deadline and/or there is a murderous look creeping into your manager's eyes 😡, then head on over to the completed example and take a look at BackgroundDownloader, BackgroundDownloaderContext, BackgroundDownloadItem and AppDelegate to see how things end up - in order to run this you will need to follow the instructions below.

Let's get downloading

To support background-transfers we need to grant permission for Background Modes. This is done by opening the Capabilities tab in the target and switching the Background Modes toggle to ON.

As multiple viewcontrollers within the project could be retrieving images I decided to handle these download requests within their own class - BackgroundDownloader. BackgroundDownloader will be responsible for configuring our background session, scheduling download requests against that session and responding to any delegate callbacks.

Lets build the skeleton of the BackgroundDownloader class:

class BackgroundDownloader: NSObject { //1
    private var session: URLSession!

    // MARK: - Singleton

    //2
    static let shared = BackgroundDownloader()

    // MARK: - Init

    private override init() {
        super.init()

        //3
        let configuration = URLSessionConfiguration.background(withIdentifier: "background.download.session")
        session = URLSession(configuration: configuration, delegate: self, delegateQueue: nil)
    }
}

In BackgroundDownloader we create an instance of URLSessionConfiguration and the URLSession instance that will handle any background tasks.:

1. An interesting difference between background sessions and the other session types is that with background sessions we can't use the completionHandler methods when creating a URLSessionDownloadTask task (doing so won't result in the compiler throwing an error, but the app will throw an exception at runtime). Instead of using the closure approach, BackgroundDownloader will need to implement the URLSessionDownloadDelegate protocol (and eventually URLSessionDelegate) - the need to implement these delegates is the reason that BackgroundDownloader is a subclass of NSObject.
2. BackgroundDownloader is singleton to ensure that only one instance of URLSession is configured for background transfer (see next point).
3. configuration is initialised using the background convenience init'er to allow any URLSession instances to support background-transfers. Each URLSessionConfiguration instances needs to have an identifier. This identifier plays a critical role in allowing a background-transfer that started on one instance of URLSession to be completed on another. This happens when the app has been terminated and iOS wakes the terminated app up to finish the transfer - configuration uses a static string as an identifier so it can be recreated (we will see this in action later). It's important to note here that each session needs to have a unique identifier within that app's execution lifecycle.

The more eagle-eyed among you will have noticed that the session property is implicitly unwrapped. I usually try and avoid implicitly unwrapped properties but as session is a private property and is set in the init'er I felt making it implicitly unwrapped resulted in more readable code with very little danger of a crash. The reason that session can't be a let is that when creating the session, the BackgroundDownloader instance sets itself as the session's delegate so the BackgroundDownloader has to exist which means that the initialising of the session has to happen after the super.init() call is made.

So far, we have configured a session for performing background-transfers but don't yet have the ability to download anything so let's add that. A download needs 3 pieces of information:

1. Remote URL - URL of the asset to be downloaded.
2. File path URL - URL of where the downloaded asset should be moved to on the local file system.
3. Completion handler - a closure to be called when the download is complete.

Rather than store these pieces of information about each download in separate arrays, we can create a model class to group these related pieces of information:

typealias ForegroundDownloadCompletionHandler = ((_ result: DataRequestResult) -> Void)

class DownloadItem {
    let remoteURL: URL
    let filePathURL: URL
    var foregroundCompletionHandler: ForegroundDownloadCompletionHandler?

    // MARK: - Init

    init(remoteURL: URL, filePathURL: URL) {
        self.remoteURL = remoteURL
        self.filePathURL = filePathURL
    }
}

An instance of DownloadItem will be created when a new download request is made against BackgroundDownloader. This DownloadItem instance can then be stored in a dictionary to be used once the download is complete:

class BackgroundDownloader: NSObject {

    // Omitted properties

    private var downloadItems: [URL: DownloadItem] = [:]

    // Omitted methods

    func download(remoteURL: URL, filePathURL: URL, completionHandler: @escaping ForegroundDownloadCompletionHandler) {
        print("Scheduling download of: \(remoteURL)")

        let downloadItem = DownloadItem(remoteURL: remoteURL, filePathURL: filePathURL)
        downloadItem.foregroundCompletionHandler = completionHandler
        downloadItems[remoteURL] = downloadItem

        let task = session.downloadTask(with: remoteURL)
        task.resume()
    }
}

It's not unusual to have a scenario where before a download request is finished another request for that same download is made - think of a user scrolling a tableview up and down. There are many articles written on how to prevent these additional (unneeded) download requests so I won't complicate this example about background-transfers by implementing any of them here. However if you are really interested on how to prevent these additional download requests, check out my coalescing closures article on this subject which shows one possible approach.

The URLSessionDownloadDelegate protocol has been mentioned a few times now so lets see how it's implemented:

extension BackgroundDownloader: URLSessionDownloadDelegate {

    func urlSession(_ session: URLSession, downloadTask: URLSessionDownloadTask, didFinishDownloadingTo location: URL) {
        guard let originalRequestURL = downloadTask.originalRequest?.url, let downloadItem = downloadItems[originalRequestURL] else {
            return
        }

        print("Downloaded: \(downloadItem.remoteURL)")

        do {
            //1
            try FileManager.default.moveItem(at: location, to: downloadItem.filePathURL)

            downloadItem.foregroundCompletionHandler?(.success(downloadItem.filePathURL))
        } catch {
            //2
            downloadItem.foregroundCompletionHandler?(.failure(APIError.invalidData))
        }

       downloadItems[originalRequestURL] = nil
    }
}

With support for URLSessionDownloadDelegate, BackgroundDownloader can process the outcome of a download:

1. If the download has been a success, the asset is moved from its temporary location to its permanent location on the file system. Then the completionHandler is triggered with the success case of the DataRequestResult enum containing the permanent location URL.
2. If the download has failed the completionHandler is triggered with the failure case containing an error.

This method is deliberately kept as simple as possible but you can easily imagine how it and DownloadItem could be extended e.g. to update Core Data or perform image manipulation

Finally we have been using DataRequestResult enum as a generic result enum so lets look at it as well:

enum DataRequestResult<T> {
    case success(T)
    case failure(Error)
}

And that's all that's required for performing foreground transfers using a background configuration.

If you want to see this version of BackgroundDownloader running, checkout out the foreground_only_transfers branch on the example project - to run this you will need to follow the instructions below.

Working behind the users back

If you run the app in the foreground you will notice that everything works as expected and the app is populated with downloaded assets however as soon as the app is put into the background, a stream of errors will begin to appear in the Xcode console, similar to:

So far BackgroundDownloader only works when the app is in the foreground, let's extend it to support downloads when the app enters a suspended/backgrounded state.

When an app is in a suspended/terminated state and a download is completed, iOS will wake the app up (in the background) by calling:

class AppDelegate: UIResponder, UIApplicationDelegate {

    // Omitted properties and methods

    func application(_ application: UIApplication, handleEventsForBackgroundURLSession identifier: String, completionHandler: @escaping () -> Void) {
        //TODO: Handle processing downloads
    }
}

Regardless of how many downloads are in progress, application(_:handleEventsForBackgroundURLSession:completionHandler:) is only called once - when all downloads have been completed. If the user foregrounds the app before the above method is called, urlSession(_:downloadTask:didFinishDownloadingTo:) is called for those downloads that have been completed.

In the above method, iOS has provided the app with the identifier of the session that the task was scheduled on and also a completion-closure. As our background session always uses the same identifier, we don't need to do anything with identifier however we do need to pass completionHandler onto BackgroundDownloader so that it can be triggered once all download processing has been completed:

class AppDelegate: UIResponder, UIApplicationDelegate {

    // Omitted properties and methods

    func application(_ application: UIApplication, handleEventsForBackgroundURLSession identifier: String, completionHandler: @escaping () -> Void) {
        BackgroundDownloader.shared.backgroundCompletionHandler = completionHandler
    }
}

As the completionHandler closure is passed to the shared BackgroundDownloader instance we need to add a new property to that class:

class BackgroundDownloader: NSObject {
    var backgroundCompletionHandler: (() -> Void)?

    // Omitted properties and methods
}

When all the downloads are are complete, just like before urlSession(_:downloadTask:didFinishDownloadingTo:) will be called for each download to allow any post-download processing to occur. After all the calls to urlSession(_:downloadTask:didFinishDownloadingTo:) have been made, a special background-transfer only call is made:

extension BackgroundDownloader: URLSessionDelegate {

    func urlSessionDidFinishEvents(forBackgroundURLSession session: URLSession) {
        //1
        DispatchQueue.main.async {
            self.backgroundCompletionHandler?()
            self.backgroundCompletionHandler = nil
        }
    }
}

The above method's main job is to allow backgroundCompletionHandler to be triggered.

1. Triggering backgroundCompletionHandler instructs iOS to take a new snapshot of the app's UI for the app switcher preview - as such it needs to be called from the main queue (urlSessionDidFinishEvents(forBackgroundURLSession:) may be called from a background queue). It's important to ensure that your app does indeed call backgroundCompletionHandler - failure to do so will result in iOS treating the app as a bad citizen app; reducing the opportunity for any background processing in the future and resulting in the error that we seen at the start of this section.

While it's possible to make additional network requests with the app running in the background, they are subject to a rate limiter to prevent abuse of background sessions - this rate limiter delays the start of those network requests. With each separate network request the delay increases - this delay is reset when the user brings the app to the foreground or if your app does not make any additional network requests during the delay.

And that's all the changes required to continue network requests when the app is suspended/backgrounded.

If you want to see this version of BackgroundDownloader running, checkout out the foreground_and_suspended_transfers branch on the example project - to run this you will need to follow the instructions below. In order to better see the downloads happening in the background you may want to add a slight delay before the download actually starts so as to allow you to more leisurely put the app into the background:

task.earliestBeginDate = Date().addingTimeInterval(5)

Coming back from the dead 🧟

After an app has been terminated, all of the memory which was allocated to it is freed. This means that when iOS relaunches the app to complete a background-transfer we don't have the details in memory that we can use to determine how that background-transfer should be completed. Instead, these details need to be persisted in the file system between app executions. In our example, the only information that we really need to persist is filePathURL (as you will see the remoteURL is also persisted but this is primarily for informational purposes). We can take advantage of the Codable protocol to help here. By conforming DownloadItem to Codable we can encode that DownloadItem object into its Data representation which can then be stored in UserDefaults. The only issue with this approach is that the foregroundCompletionHandler property on DownloadItem can't conform to Codable, so the DownloadItem implementation of Codable will be slightly more involved than usual. Thankfully Codable has an easy way - CodingKeys - to declare which properties should and shouldn't be encoded.

class DownloadItem: Codable {

    // Properties omitted

    private enum CodingKeys: String, CodingKey {
        case remoteURL
        case filePathURL
    }

    // Methods omitted
}

With these changes, the DownloadItem class now has a dual nature:

1. Freshly created.
2. Restored from UserDefaults.

This presents a problem with determining which class should be responsible for handling this dual nature - knowing how to save into, load from and clean up UserDefaults. I toyed with placing this responsibility in DownloadItem but wanted to keep this class as simple as possible which ruled that approach out. Then I moved onto BackgroundDownloader but again I ruled that approach out as I felt that it wasn't the correct class for handling this - I didn't want to expose that some DownloadItem instances were freshly created while some had been restored from UserDefaults to BackgroundDownloader whose primary responsibility centred on downloading rather than persistence. In the end, I drew inspiration from Core Data and decided to create a downloading context.

This context class needs to handle 3 different tasks:

1. Loading.
2. Saving.
3. Deleting.

When it comes to loading/retrieving an existing DownloadItem instance, this can be from two possible stores:

1. In-memory.
2. UserDefaults.

class BackgroundDownloaderContext {
    //1
    private var inMemoryDownloadItems: [URL: DownloadItem] = [:]
    //2
    private let userDefaults = UserDefaults.standard

}

1. An in-memory store that is a simple dictionary is being used as an in-memory store which will hold all active DownloadItem instances.
2. A persistent store that will allow DownloadItem instances to live beyond the app lifecycle.

As you can see both of this stores are private as we don't want to expose where a DownloadItem instance comes from so let's add a load method to BackgroundDownloaderContext that can load from either of the two possible stores:

class BackgroundDownloaderContext {

    // Omitted properties

    func loadDownloadItem(withURL url: URL) -> DownloadItem? {
        //1
        if let downloadItem = inMemoryDownloadItems[url] {
            return downloadItem
        //2
        } else if let downloadItem = loadDownloadItemFromStorage(withURL: url) {
             inMemoryDownloadItems[downloadItem.remoteURL] = downloadItem

            return downloadItem
        }

        //3
        return nil
    }

    private func loadDownloadItemFromStorage(withURL url: URL) -> DownloadItem? {
        guard let encodedData = userDefaults.object(forKey: url.path) as? Data else {
            return nil
        }

        let downloadItem = try? JSONDecoder().decode(DownloadItem.self, from: encodedData)
        return downloadItem
    }
}

1. The in-memory store is checked for an existing DownloadItem using the network request's URL as the key.
2. If an existing DownloadItem instance isn't found in the in-memory store, the UserDefaults store is checked. If an object is found in UserDefaults, an attempt is made to decode it from its raw Data representation to a new DownloadItem instance. Decoding can potentially throw an exception, if an exception is thrown it will be silently silenced with the try? statement and nil will be returned.
3. If the url is unknown or an error was thrown then nil is returned.

In order to get DownloadItem instances into UserDefaults we need to save them:

class BackgroundDownloaderContext {

    // Omitted properties and methods

    func saveDownloadItem(_ downloadItem: DownloadItem) {
        //1
        inMemoryDownloadItems[downloadItem.remoteURL] = downloadItem

        //2
        let encodedData = try? JSONEncoder().encode(downloadItem)
        userDefaults.set(encodedData, forKey: downloadItem.remoteURL.path)
        userDefaults.synchronize()
    }
}

In the above code snippet:

1. The DownloadItem instance is stored into the in-memory store
2. The DownloadItem instance is converted to Data and then stored into UserDefaults.

Finally when a download is complete the in-memory and UserDefaults objects need to be cleaned away:

class BackgroundDownloaderContext {

    // Omitted properties and methods

    func deleteDownloadItem(_ downloadItem: DownloadItem) {
      inMemoryDownloadItems[downloadItem.remoteURL] = nil
      userDefaults.removeObject(forKey: downloadItem.remoteURL.path)
      userDefaults.synchronize()
    }
}

By implementing the BackgroundDownloaderContext, the BackgroundDownloader logic can be kept pretty much the same as before despite this significant introduction of new functionality. First we need to add a new property to hold an instance of BackgroundDownloaderContext and remove the existing downloadItems property

class BackgroundDownloader: NSObject {
    private let context = BackgroundDownloaderContext()

    // Omitted properties and methods
}

Now let's use that context when requesting a new download:

class BackgroundDownloader: NSObject {

    // Omitted properties and methods

    func download(remoteURL: URL, filePathURL: URL, completionHandler: @escaping ForegroundDownloadCompletionHandler) {
        print("Scheduling download of: \(remoteURL)")

        let downloadItem = DownloadItem(remoteURL: remoteURL, filePathURL: filePathURL)
        downloadItem.foregroundCompletionHandler = completionHandler
        context.saveDownloadItem(downloadItem)

        let task = session.downloadTask(with: remoteURL)
        task.resume()
    }
}

Then when a download is completed, the DownloadItem instance needs to be deleted from the context:

extension BackgroundDownloader: URLSessionDownloadDelegate {

    func urlSession(_ session: URLSession, downloadTask: URLSessionDownloadTask, didFinishDownloadingTo location: URL) {
        //1
        guard let originalRequestURL = downloadTask.originalRequest?.url, let downloadItem = context.loadDownloadItem(withURL: originalRequestURL) else {
            return
        }

        print("Downloaded: \(downloadItem.remoteURL)")

        do {
            try FileManager.default.moveItem(at: location, to: downloadItem.filePathURL)

            downloadItem.foregroundCompletionHandler?(.success(downloadItem.filePathURL))
        } catch {
            downloadItem.foregroundCompletionHandler?(.failure(APIError.invalidData))
        }

       context.deleteDownloadItem(downloadItem)
    }
}

1. Rather than using the now removed downloadItems property, we instead make use of context.

As with the suspended/backgrounded code changes, when the app is awoken from a terminated state the app-delegate is called:

class AppDelegate: UIResponder, UIApplicationDelegate {

    // Omitted properties and methods

    func application(_ application: UIApplication, handleEventsForBackgroundURLSession identifier: String, completionHandler: @escaping () -> Void) {
      BackgroundDownloader.shared.backgroundCompletionHandler = completionHandler
    }
}

This is actually the exact same code as in the suspended/backgrounded example above but I wanted to highlight that when the app is awoken from a terminated state, this method not only passes the completionHandler along but also sets up the BackgroundDownloader (by calling shared) so that it can respond to the session delegate calls.

An interesting point around terminating the app is that if you manually force quit the app then iOS will take this as definite confirmation that you are finished with the app and so cancel any scheduled background-transfers. This presents a problem when it comes to testing how our solution handles termination. The easiest way I found to overcome this issue was to add a manual exit call to cause the app to terminate without any user involvement:

class AppDelegate: UIResponder, UIApplicationDelegate {

    // Omitted properties and methods

    func applicationDidEnterBackground(_ application: UIApplication) {
        //Exit app to test restoring app from a terminated state. Comment out to test restoring app from a suspended state.
        DispatchQueue.main.asyncAfter(deadline: .now()) {
            print("App is about to quit")

            if let documentsPath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true).first {
                debugPrint("Gallery assets will be saved to: \(documentsPath)")
            }
            exit(0)
        }
    }
}

In the above code snippet, the file path of the directory that the downloaded assets will end up in is printed to the console and then the app terminates. With this file path, you can then navigate to that directory and see the folder fill up with downloaded assets.

Downloads keep going 🤯

With background sessions, we have a powerful weapon in ensuring that we can meet our user's wants without having to trap them in the app. As we seen above adding support for background sessions isn't too tricky and what extra complexity there is, can mostly be hidden from the wider app.

Congratulations on making it to the end 🎉.

---

Running the example project

In "Don't throw anything away with pausable downloads" I used Imgur as an online image database and I've used it again in this post to power the example project. Imgur has a great JSON based API and an extensive library of freely available media. Imgur API, while being free, does require us to register our example project to get a client-id which needs to be sent with each request.

As the example project does not require access to a user's personal data, when registering select the Anonymous usage without user authorization option for Authorization type. At the time of writing, you had to provide a URL in the Authorization callback URL field even for anonymous authentication - I found any URL value would work for this.

After you register, you should be given a unique client-id which will allow you access to Imgur's content. You will need to add your client-id as the value of the clientID property in RequestConfig. After you've added your client-id, you should be able to run the app and see how our background-transfer solution works.

What do you think? Let me know by getting in touch on Twitter - @wibosco