Talking About Django App Deployment and Performance

Introduction

As I've gone deeper into my work, I've increasingly found that there are many pitfalls in Python web development, and I've always envied the executable-file deployment and high performance of AspNetCore and Go, as well as the richness of the Spring ecosystem. But since work uses Django, life still has to go on—Django is at least a web framework with a large market share, and it's not that bad; at least in development speed it utterly beats a bunch of other frameworks~

In a previous article I introduced how to deploy a Django app with Docker, but that deployment approach is only suitable for going live for debugging: it serves with Django's built-in web server, single-threaded, with very low performance. For a real production service, you still need to deploy with uwsgi plus nginx.

Clarifying the Approach

I previously wrote an article introducing how to deploy a Django app directly on a Linux server with uwsgi+nginx. Now, for easier management, we'll use Docker to deploy the Django app.

I've seen many solutions online that also put nginx in Docker, which feels a bit off. We may have many projects on a server; one nginx per project not only uses extra resources, but is also quite a hassle in terms of configuration.

A single nginx installed on the system is completely sufficient and convenient to configure. For our team, the company's network security is quite strict, with restrictions on open ports on the server and domain names we can apply for. We can't open a port per application; we can only have nginx listen on one port and configure reverse proxies to serve multiple projects, so one nginx is very convenient.

Our approach is to put Django and uwsgi in Docker—the usual way—use docker-compose to orchestrate related dependent database, cache, and monitoring services, then use nginx for reverse proxy and static file serving. For static files and hot-reloading of code, we also need volume mapping.

Next I'll walk through, step by step, how to build a convenient uwsgi+django image.

Writing the Dockerfile

First is our image file. Compared to the previous development image, it adds a uwsgi module; everything else stays the same.

FROM python:3.7

# 设置 python 环境变量
ENV PYTHONUNBUFFERED 1

# 创建 code 文件夹并将其设置为工作目录
RUN mkdir /code
WORKDIR /code

# 更新 pip
RUN pip install -i https://mirrors.aliyun.com/pypi/simple pip -U
# 设置国内源
RUN pip config set global.index-url https://mirrors.aliyun.com/pypi/simple

# 将 requirements.txt 复制到容器的 code 目录
ADD requirements.txt /code/

# 安装库
RUN pip install -r requirements.txt

# 将当前目录复制到容器的 code 目录
ADD . /code

# 安装uwsgi
RUN pip install uwsgi

uwsgi ini Configuration

For uwsgi configuration I chose the ini format; the file contents are as follows:

[uwsgi]
socket = :8000

# the base directory (full path)
chdir = /code

# Django s wsgi file
module = config.wsgi

# 启用主进程，当进程挂掉会再spawn一个（以下三个指标，实现“并发”）
master = true
# 进程个数
processes = 4
# 每个进程中的线程个数/workers
threads = 2
# 一个工作进程最大请求数
max-requests = 5000

vacuum = true

# pid文件，用于脚本启动、停止该进程
pidfile = /code/uwsgi.pid

# 当文件改变时，优雅的重启uWSGI
touch-reload = /code/readme.md

One thing to note: in the previous configuration we used daemonize = log/uwsgi.log to set the log file path so uwsgi would write logs, but if you do that with Docker, docker-compose will treat the container as finished and exit automatically when it starts. So as a workaround we can only skip writing a log file and output to the console, which keeps the container from shutting down (if I find a better approach later I'll update the article to record it)

I also used touch-reload to watch for file changes and automatically restart uwsgi, which gives us hot-reloading of code (strictly speaking it's not real hot-reloading haha. But it's good enough) Every time you update the code, also tweak the readme file—you can record what changed, and the service restarts automatically. Super convenient~

However, this kind of hot-reload will cause brief bad requests, so try to update the service during low-traffic periods~

The current system is still fairly simple, so my approach is: once testing on the test server looks fine, sync the code to the production server~ Since the frontend is a mini program and the user base isn't large yet, users basically don't notice~

Docker-Compose

Not much to say here—just changed the command~

version: "3"
services:
  redis:
    image: redis
    expose:
      - 6379
  web:
    restart: always # 除正常工作外，容器会在任何时候重启，比如遭遇 bug、进程崩溃、docker 重启等情况。
    build: .
    environment:
      - ENVIRONMENT=docker
    command: uwsgi uwsgi.ini
    volumes:
      - .:/code
    ports:
      - "19001:8000"
    depends_on:
      - redis

After the steps above, a simple docker-compose up command gets our service running. Then configure reverse proxy and static files in nginx and you're all set. As for nginx configuration, I plan to cover that in the next article.

Finally, about performance: after the above deployment, tests show this system can currently handle up to 1800 QPS with no optimizations at all. Pros might think that's too low—yeah, Python's performance is indeed… but for our current product it's more than enough (overkill). If not, have the company throw more hardware at it; it's cloud anyway, easy to scale (the company has plenty of servers and money haha) Optimize as needed later… I'm out haha

Feel Free to Discuss

I've put together a series of technical articles and materials. Reply with linux, flutter, c#, netcore, android, kotlin, java, python, etc. in the backend of the WeChat official account 「程序设计实验室」 to get related technical articles and materials. If you have any questions you can also leave a message in the official account backend~