Creating an app centered around video stories

The stories concept popularized by Snapchat and later by Instagram, where you browse recent portrait-oriented photos and videos by scrolling right, can be a compelling way to browse media on a handheld device.

Some prominent apps dedicated to live-streaming also use similar navigation:

In the discovery feeds of Twitter's live-streaming app Periscope, once you've opened up a stream and are watching it in full screen, you can swipe left or right to navigate to the next or previous stream, as an alternative to backing out to the vertically scrollable thumbnail list and selecting the next one from there.

Bigo Live, a popular live-streaming app in Asia, does something similar, with the added twist of using vertical swiping instead.

Let's look at the mechanics of how to add "Story"-inspired video navigation to our own app when using Bambuser's player SDK.

MVP

The simplest possible implementation could forgo animations and use the "tap-near-the-edge" pattern to go back and forward instead of swiping, to avoid having to deal with more than one piece of content at the screen at any one time: It would get us 80% of the way there without having to think as much about when to load preview images or at what point to start or stop a given stream.

Animated transitions

But what if we wanted to add an additional layer of UX pizzazz?

Inspiration: Periscope

Periscope currently uses a flat scroll view and puts the video containers side-by-side, with only the currently selected video visible by default. They also use snapping: the video that occupies the most space when you let go of a swipe gets focus and is automatically centered.

The timing of when you load images and actual streams are important tradeoffs here. Let's analyze what Periscope chose to do:

When your leftwards swipe starts revealing the next video on the righthand side, they start loading the actual stream. Since it typically takes a couple of seconds until the first frame of the stream is playable, the player view needs to show something in the meantime. Periscope's designers opted to use blurring as a stylistic tool: they display a still image similar to the first frame of the stream and blur it heavily. This gives us a nice, contextually relevant cushion to sit on while we wait for the first realtime video frame. They probably preloaded this image: it might even be the thubmnail from the parent video feed.

The previous video is still playing, partially off screen at this point. Once the new video has started, both are playing side-by-side. They clearly have to maintain at least two active video player instances at this point.

Once the user makes up their mind which video to watch, they immediately dismiss the other video, as revealed when you attempt to swipe in the opposite direction directly after letting go of the first swipe and you are met by the same blurring effect and wait time: clearly the previous video now has to start from scratch. Which seems totally reasonable: phones have limited resources after all and your app code gets hard to reason about if you optimize for too many uncommon cases.

To recap Periscope's playback tradeoffs: they preloaded an image as immediately available filler while waiting for the actual stream, but they did not preload the stream itself, to conserve bandwith. They do momentarily keep two streams going simultaneously in the transition phase, but they let go of the irrelevant one at the earliest opportunity, again to conserve bandwith.

Inspiration: Instagram

Let's look at Instagram Stories!

They have a similar scroll view as a base, but they take the animation business one step further and uses a fancy 3d cube animation! Later in this article we'll have a look at how to recreate it in Swift for iOS.

Since Instagram tend to have higher resolution content to work with, they show a preview image straight away on videos. They aren't as realtime-focused and preloading a few images ahead of the current position is a no-brainer. If you swipe really quickly through the stories, you can see that you eventually catch up to the preloader and start seeing spinners. It also seems that they occasionally use lower-res placeholders, not blurred as elegantly as in Periscope. Most of the time you don't see them though.

For story videos that aren't live, Instagram's designers don't feel the urge to start playback until you let go of the swipe, but a subtle spinner illustrates that they are being preloaded as soon as you initiate the swipe, or potentially even before that.

Infinite scroll vs swipe to the left

It is worth noting that both Persicope and Instagram stories require you to let go of the previous swipe before you are able to swipe further. From an implementation standpoint this greatly reduces the complexity, compared to those infinitely scrollable views that have become the default way of displaying content.

When doing infinite scroll, the user is supposed to get the illusion that all items in the feed - or at least the ones above the current scroll position - are present and are occupying their designated space at all times, when in reality you need to drop and reuse them shortly after being dragged off screen, to reduce memory consumption and keep the scrolling smooth. But in a story feed, you get away with only rendering at worst two items at a time:

  • The current one and the next one when you start a leftwards swipe
  • Or the current one and the previous one if you swiped to the right
  • Or just the current one if no swiping is taking place right now

The points at which you need to update your story viewer's state are few and well defined, as opposed to in the endless scrolling scenario where things are more in flux and you need to be careful to not inadvertently introduce performance issues and bugs.

In other words, if you've already tackled the endless scrolling challenge in your app this tutorial will feel easy on some level.

Implementing support for swiping and using a cube animation

Stephen Bodnar has a nice tutorial on how to create a cube animation between UIViews on iOS.

Let's use a similar approach and attempt to render a Bambuser SDK player on the sides of the cube, while keeping the observations above in mind.

Start by following the player SDK's getting started guide for Swift to learn how to install the SDK and get a working player surface.

The BroadcastStory class

Next, let's create a class that represents a single story and is able to manage a corresponding player object. Some ground rules:

  • Whenever play() is called, create a new BambuserPlayer() instance. The SDK does not support reusing a player object between multiple videos anyway.

  • Whenever stop() is called, drop the old instance to free resources.

  • If we accidentally call play or stop twice, do nothing, just keep going in the existing state.

BroadcastStory.swift

import Foundation

class BroadcastStory: NSObject, BambuserPlayerDelegate {
    var view: UIView = UIView()
    var player: BambuserPlayer? = nil
    var resourceUri: String? = nil
    var isPlaying: Bool = false

    func play() {
        if (isPlaying) {
            // ignore consecutive callbacks requesting the same state
            // happens when swiping without releasing
            return
        }

        if (resourceUri == nil) {
            debugPrint("Cannot play without a resourceUri! See https://bambuser.com/docs/key-concepts/resource-uri/")
            return
        }

        debugPrint("Starting playback", resourceUri)
        isPlaying = true

        player = BambuserPlayer()
        player!.applicationId = "REPLACEME"
        self.view.addSubview(player!)
        player!.frame = CGRect(x: 0, y: 0, width: view.bounds.size.width, height: view.bounds.size.height)
        player!.playVideo(resourceUri)
    }

    func videoLoadFail() {
        debugPrint("Failed to load video", player?.resourceUri ?? "");
    }

    func stop() {
        if (!isPlaying) {
            // ignore consecutive callbacks requesting the same state
            // happens when swiping without releasing
            return
        }
        debugPrint("Stopping playback", resourceUri)
        isPlaying = false
        guard let p = player else {
            return
        }
        p.stopVideo()
        p.removeFromSuperview()
        player = nil
    }
}

The player needs to authenticate itself using an applicationId. See the getting starting guide for details on that.

It also needs authorization to play a specific piece of content: this is handled by providing a signed resourceUri to the player instance. For now we will use a set of static demo resourceUri:s. In a production app you would either query your own backend or query bambuser's REST API directly from your app.

Static demo stories

let story1 = BroadcastStory()
story1.resourceUri = "https://cdn.bambuser.net/broadcasts/61b4fbf2-deca-49db-8aaf-6c37119a25e4?da_signature_method=HMAC-SHA256&da_id=9e1b1e83-657d-7c83-b8e7-0b782ac9543a&da_timestamp=1546513833&da_static=1&da_ttl=0&da_signature=f08dbe176d6f6d5378637d8cdd382c8079143e3e967f0278784c9c0d85044319"

let story2 = BroadcastStory()
story2.resourceUri = "https://cdn.bambuser.net/broadcasts/6237a35c-5176-5bd5-113b-69bccb5f237c?da_signature_method=HMAC-SHA256&da_id=9e1b1e83-657d-7c83-b8e7-0b782ac9543a&da_timestamp=1546514104&da_static=1&da_ttl=0&da_signature=3f73ff00ba5865b16e6f8848c46b402bffb1c5d73c012b865efcb8f89f25364c"

let story3 = BroadcastStory()
story3.resourceUri = "https://cdn.bambuser.net/broadcasts/e6c29330-dd00-6e30-3374-a7c8636b3d7d?da_signature_method=HMAC-SHA256&da_id=9e1b1e83-657d-7c83-b8e7-0b782ac9543a&da_timestamp=1546514224&da_static=1&da_ttl=0&da_signature=71060f13bbaf2d8b88b73c7229cf063a0dde50fb0dc5cbc55d41f87dc8969aae"

let story4 = BroadcastStory()
story4.resourceUri = "https://cdn.bambuser.net/broadcasts/90220cf2-1fed-1669-834e-ce80a4fe28b0?da_signature_method=HMAC-SHA256&da_id=9e1b1e83-657d-7c83-b8e7-0b782ac9543a&da_timestamp=1546514239&da_static=1&da_ttl=0&da_signature=9aa21a7bb3d1953515e55db10299332767dc4c0daedd6a5c4f5bbce629a907c0"

The scroll view

Next, let's use modify the scroll view from the cube tutorial so that it can manage our BroadcastStory objects.

StoriesScrollView.swift

import UIKit

class StoriesScrollView: UIScrollView {
    var stories = [BroadcastStory]()

    override init(frame: CGRect) {
        super.init(frame: frame)
        backgroundColor = UIColor.clear
        isPagingEnabled = true
    }

    required init?(coder aDecoder: NSCoder) {
        super.init(coder: aDecoder)
    }

    func setDataSource(with stories: [BroadcastStory]) {
        self.stories = stories
        for i in 0..<self.stories.count {
            let story = self.stories[i]
            let width = frame.width
            let height = frame.height
            let xOffset = width * CGFloat(i)
            story.view.frame = CGRect(x: xOffset, y: 0, width: width, height: height)
            addSubview(story.view)
            contentSize = CGSize(width: xOffset + width, height: height)
        }
        // Start the visible one
        self.stories[0].play()
    }

    func visibleViews() -> [BroadcastStory] {
        let visibleRect = CGRect(x: contentOffset.x, y: 0, width: frame.width, height: frame.height)
        var visibleStories = [BroadcastStory]()
        for story in stories {
            if story.view.frame.intersects(visibleRect) {
                visibleStories.append(story)
                if (!story.isPlaying) {
                    story.play()
                }
            } else {
                if (story.isPlaying) {
                    story.stop()
                }
            }
        }
        return visibleStories
    }
}

The main story view controller

This is either our main view in the app (the one loaded by AppDelegate.swift), or more likely a view opened when we want to enter story mode, if our app contains more than just the story navigation.

This view controller initiates the scrollview which is added as a child view, then proceeds to initiate our BroadcastStory objects and adds them inside the scroll view.

ViewController.swift

import UIKit

class ViewController: UIViewController, BambuserPlayerDelegate {
    var scrollView: StoriesScrollView

    required init?(coder aDecoder: NSCoder) {
        scrollView = StoriesScrollView()
        super.init(coder: aDecoder)
    }

    override func viewDidLoad() {
        super.viewDidLoad()
        view.backgroundColor = .black
        createStories()
    }

    func createStories() {
        scrollView.frame = CGRect(x: 0, y: 0, width: view.frame.width, height: view.frame.height)
        scrollView.delegate = self
        view.addSubview(scrollView)

        let story1 = BroadcastStory()
        story1.resourceUri = "https://cdn.bambuser.net/broadcasts/61b4fbf2-deca-49db-8aaf-6c37119a25e4?da_signature_method=HMAC-SHA256&da_id=9e1b1e83-657d-7c83-b8e7-0b782ac9543a&da_timestamp=1546513833&da_static=1&da_ttl=0&da_signature=f08dbe176d6f6d5378637d8cdd382c8079143e3e967f0278784c9c0d85044319"

        let story2 = BroadcastStory()
        story2.resourceUri = "https://cdn.bambuser.net/broadcasts/6237a35c-5176-5bd5-113b-69bccb5f237c?da_signature_method=HMAC-SHA256&da_id=9e1b1e83-657d-7c83-b8e7-0b782ac9543a&da_timestamp=1546514104&da_static=1&da_ttl=0&da_signature=3f73ff00ba5865b16e6f8848c46b402bffb1c5d73c012b865efcb8f89f25364c"

        let story3 = BroadcastStory()
        story3.resourceUri = "https://cdn.bambuser.net/broadcasts/e6c29330-dd00-6e30-3374-a7c8636b3d7d?da_signature_method=HMAC-SHA256&da_id=9e1b1e83-657d-7c83-b8e7-0b782ac9543a&da_timestamp=1546514224&da_static=1&da_ttl=0&da_signature=71060f13bbaf2d8b88b73c7229cf063a0dde50fb0dc5cbc55d41f87dc8969aae"

        let story4 = BroadcastStory()
        story4.resourceUri = "https://cdn.bambuser.net/broadcasts/90220cf2-1fed-1669-834e-ce80a4fe28b0?da_signature_method=HMAC-SHA256&da_id=9e1b1e83-657d-7c83-b8e7-0b782ac9543a&da_timestamp=1546514239&da_static=1&da_ttl=0&da_signature=9aa21a7bb3d1953515e55db10299332767dc4c0daedd6a5c4f5bbce629a907c0"

        scrollView.setDataSource(with: [story1, story2, story3, story4])
    }

    override func viewWillLayoutSubviews() {
        scrollView.frame = CGRect(x: 0, y: 0, width: view.frame.width, height: view.frame.height)
    }
}

extension ViewController: UIScrollViewDelegate {
    func scrollViewDidScroll(_ scrollView: UIScrollView) {
        let visibleViews = self.scrollView.visibleViews().sorted(by: {$0.view.frame.origin.x > $1.view.frame.origin.x} )
        let xOffset = scrollView.contentOffset.x // 2
        let rightViewAnchorPoint = CGPoint(x: 0, y: 0.5)
        let leftViewAnchorPoint = CGPoint(x: 1, y: 0.5)
        var transform = CATransform3DIdentity
        transform.m34 = 1.0 / 1000
        let leftSideOriginalTransform = CGFloat(90 * Double.pi / 180.0)
        let rightSideCellOriginalTransform = -CGFloat(90 * Double.pi / 180.0)
        if let viewFurthestRight = visibleViews.first, let viewFurthestLeft =  visibleViews.last {
            let hasCompletedPaging = (xOffset / scrollView.frame.width).truncatingRemainder(dividingBy: 1) == 0
            var rightAnimationPercentComplete = hasCompletedPaging ? 0 :1 - (xOffset / scrollView.frame.width).truncatingRemainder(dividingBy: 1)
            if xOffset < 0 { rightAnimationPercentComplete -= 1 }
            viewFurthestRight.view.transform(to: rightSideCellOriginalTransform * rightAnimationPercentComplete, with: transform)
            viewFurthestRight.view.setAnchorPoint(rightViewAnchorPoint)
            if  xOffset > 0 {
                let leftAnimationPercentComplete = (xOffset / scrollView.frame.width).truncatingRemainder(dividingBy: 1)
                viewFurthestLeft.view.transform(to: leftSideOriginalTransform * leftAnimationPercentComplete, with: transform)
                viewFurthestLeft.view.setAnchorPoint(leftViewAnchorPoint)
            }
        }
    }
}

The UIScrollViewDelegate extension from the cube tutorial is what turns our flat scroll view into a cube. We could leave that part out and get a naviagtion more similar to Periscope's.

We also need to add some helper extensions from the cube tutorial:

Extensions.swift

import UIKit

extension UIView {

    func transform(to radians: CGFloat, with transform: CATransform3D) {
        layer.transform = CATransform3DRotate(transform, radians, 0, 1, 0.0)
    }

    func setAnchorPoint(_ point: CGPoint) {
        var newPoint = CGPoint(x: bounds.size.width * point.x, y: bounds.size.height * point.y)
        var oldPoint = CGPoint(x: bounds.size.width * layer.anchorPoint.x, y: bounds.size.height * layer.anchorPoint.y)

        newPoint = newPoint.applying(transform)
        oldPoint = oldPoint.applying(transform)

        var position = layer.position
        position.x -= oldPoint.x
        position.x += newPoint.x

        position.y -= oldPoint.y
        position.y += newPoint.y

        layer.position = position
        layer.anchorPoint = point
    }
}

Demo

This is what our story implementation looks like at this stage.

Very promising, but still a bit crude. For starters, it would benefit a lot from showing that static preloaded image until the video is ready to play. To accomplish that we could put a UIImageView behind the BambuserPlayer and load a jpeg provided by the REST API.

What's next?

In future additions to this article, we will look at how to extend this live stories concept by:

  • Showing a blurred preview image before the video loads, like Periscope, to make the experience feel more polished.

  • Loading content dynamically from the REST API, so that we get a useful and result and not just a static demo.

  • Adding positional indicators and automatically transitioning to the next video when the current one ends

Make sure to also have a look at how you can add live broadcasting to your app.