In the last few months, I’ve been building Hoot, a private video messaging app for Android, with my business partner Christina. Part of a video messaging app is hosting and serving – you guessed it – videos. Hoot’s videos are served off the filesystem by nginx on a shared media server, separate from our application servers that process Hoot’s API requests.

Serving media files with nginx is dead-simple. Just throw the files on your filesystem, set up a location block to point at it using root or alias, and you’re good to go. For example:

location /videos/ {
    root /srv;
}

Accessing yourwebser.ver/videos/bill/to_jane/cute_puppies.mp4 will load whatever’s in /srv/videos/bill/to_jane/cute_puppies.mp4. Great! Unfortunately, this setup doesn’t work for media that’s meant to be private, as any video is accessible to anyone.

One solution is just to make the media request within your application server (Rails, Django, Flask, etc) and proxy the data back to nginx, but don’t do that. It hogs a bunch of application resources, and, in my experience, is a 3-4x slowdown when compared to serving files from nginx. What we really want is for the application to decide whether a request should be allowed to access a file and leave it up to nginx to handle serving it.

Fortunately, nginx allows this through the X-Accel-Redirect header, documented in the X-Sendfile feature. If a proxy server sets this header, nginx rewrites the request and returns the result of the location listed in that header. The media location could be either on the same machine on which nginx is hosted (e.g. /srv/videos/...) or on some machine that nginx can access (192.168.1.100/media/videos/...). That’s a mouthful, so let’s look at some code. This example uses Flask, but the concepts are easily applied to other web frameworks.

Let’s assume your nginx configuration has video files accessible either on the local file system or on some other server in your network. As I mentioned above, in Hoot’s case, our video files live on an external media server.

# nginx.conf
server {
    # videos locally hosted on this machine
    location /raw_videos/local {
        internal;      # only allowed for internal redirects
        alias /srv/videos;
    }
    # videos hosted elsewhere by HTTP
    location ~* /raw_videos/url/(.*) {
        internal;      # only allowed for internal redirects
        proxy_pass $1; # redirect to the specified URL at the end of this location
    }
    # your app (assumes that your Flask server is running locally on port 5000)
    location / {
        proxy_pass http://localhost:5000;
    }
}

And your upstream Flask file is:

from flask import Flask, abort, make_response
app = Flask(__name__)
@app.route("/videos/<video_id>")
def view_video(video_id):
    # check logged-in, eligible to see video, etc
    if not user_is_allowed_to_view_video(video_id):
        abort(404)
    # figure out the real path for this video_id
    # (e.g. /abc/def/awesome_video.mp4)
    real_path = get_path_for_video_id(video_id)
     # hosted locally... (/srv/videos/abc/def/awesome_video.mp4)
    redirect_path = "/raw_videos/local" + real_path
    # ...or on an external server (http://videoserver/videos/abc/def/awesome_video.mp4)
    media_url = "http://videoserver/videos" + real_path
    redirect_path = "/raw_videos/url/" + media_url
    # empty response with X-Accel-Redirect header
    response = make_response("")
    response.headers["X-Accel-Redirect"] = redirect_path
    return response

In the case where host videos using the same nginx instance as where our app server is running, nginx parses X-Accel-Redirect from the app, sees that it’s of the form /raw_videos/local/some/great/path.mp4, and so matches it with this location block:

location /raw_videos/local {
    internal; # only allowed for internal redirects
    alias /srv/videos;
}

Nginx will then look for a local file: /srv/videos/some/great/path.mp4.

In the case where we want to return some media that’s hosted elsewhere, the value of X-Accel-Redirect is something like /raw_videos/url/http://somegreaturl.com/whee/media.mp4, which nginx matches with this location block:

location ~ /raw_videos/url/(.*) {
    internal; # only allowed for internal redirects
    proxy_pass $1; # redirect to the specified URL at the end of this location
}

Since we used a regular expression to match the location and indicated a group using parentheses, the URL indicated by our app is available to our location block as the variable $1. We want nginx to serve the media at this URL, so we simply use this variable as the value for the proxy_pass directive.

Neat trick, huh? There are a couple of gotchas that are worth mentioning.

The first gotcha is that in processing an X-Accel-Redirect, nginx treats the redirected URL similar to how it processes the rewrite directive. If you do anything fancy with your appserver access logging or add particular headers to your app’s response, please note that nginx processes the redirect this as a separate request, so you’ll need to specify those directives again in your media location blocks.

Second, since the media redirect is processed as a separate request, in the case where you proxy to an external media server, the upstream headers ($upstream_http_HEADER) come from the external media server, not from the initial request to the app! That is, if you set a special header like $upstream_http_x_myapp_username) and use that value in your access logs, you’ll need to save the headers from the initial request to your app before you proxy_pass to your media server.

And finally, please note that nginx has its own DNS implementation that doesn’t look at /etc/hosts when it makes a proxied request to your external media server (or any proxied request, for that matter). That is, if you’ve set up /etc/hosts within your production network with short hostnames that you reference elsewhere in the app (e.g. web-1, web-2, media-1, db-1), you have to use something like dnsmasq as your DNS server on each machine and use the resolver directive to point at your local DNS server. For example, in your nginx.conf:

http {
    resolver 127.0.0.1;
}

Have fun keeping your media safe! If you’d like to see this in action, make sure to check out Hoot.