Project Summary: Using Docker to Migrate Services to an Offline Server

Foreword

The scenario I came across recently required migrating a system I previously developed to an offline server. This server was somewhat challenging—access required connecting through VPN and then a jump server (堡垒机), and the speed was quite slow. I encountered several pitfalls along the way. This article documents the migration process.

Basic Information

Originally, this system was quite simple – a Django backend API paired with a Vue+Echarts large-screen dashboard, using MySQL as the database and Redis for caching. All of these were deployed using Docker on the original server.

This offline server runs CentOS 7.9. Although it cannot access the internet, it fortunately has a local yum repository. However, the official yum sources are very limited, allowing only a few select software installations.

Now, besides the frontend which does not need to be migrated, all other services need to be transferred to this offline server.

Migration Steps:

• Install Docker on the offline server
• Export container images from the original server
• Upload and import the images to the offline server
• Upload various Docker configurations and start the containers on the offline server
• Configure surrounding components, such as Nginx

Offline Docker Installation

As mentioned earlier, this server has a local yum mirror, but it doesn't include Docker, so you'll need to download Docker's RPM installation packages manually.

At this address: https://download.docker.com/linux/centos/7/x86_64/stable/Packages/

You need to download the following installation packages (the versions are the latest at the time of writing; please download the current latest versions yourself)

• containerd.io-1.6.21-3.1.el7.x86_64
• docker-buildx-plugin-0.10.5-1.el7.x86_64
• docker-ce-24.0.2-1.el7.x86_64
• docker-ce-cli-24.0.2-1.el7.x86_64
• docker-ce-rootless-extras-24.0.2-1.el7.x86_64
• docker-compose-plugin-2.18.1-1.el7.x86_64

Then upload all these installation packages to the server.

Execute the command to install

sudo yum install containerd.io-1.6.21-3.1.el7.x86_64.rpm docker-ce-24.0.2-1.el7.x86_64.rpm # and others...

This is to place all files in the same install command, because these packages have interdependencies. If installed one by one, errors may occur due to ordering issues.

After installation, start Docker

sudo systemctl enable docker
sudo systemctl start docker

Adding user groups is also a standard operation after installing Docker

sudo groupadd docker
sudo usermod -aG docker $USER
newgrp docker

After that, you can run Docker commands without using the root user.

Exporting Images

There are two ways to export images

• docker export - corresponds to docker import for importing images
• docker save - corresponds to docker load for importing images

PS: The export and import commands must be matched and cannot be interchanged

The former is mainly used to create base images. After export, there are no history records or metadata, and there is no entrypoint, so they cannot be run directly. For this service migration, the save method is used.

Command to operate

docker save <container_id> -o image.tar

You can also use redirection

docker save <container_id> > image.tar

Specific differences between these two methods

• File size: The image file exported with export is smaller than the one saved with save, because save exports all layers.
• Multiple images: save can package multiple images into a single file, while export cannot.
• export (import): Retrieves the image from a container. Upon re-importing, all image history and metadata are lost (i.e., only the container's snapshot state is preserved), making rollback operations impossible.
• save (load): Preserves the image history, allowing rollback to a previous layer (层).

Application Scenario Differences

• docker export Application Scenario: Primarily used to create base images. For example, we start a container from an ubuntu image, install some software and make some configurations, then use docker export to save it as a base image. Subsequently, this image can be distributed to others for use, such as a basic development environment.
• docker save Application Scenario: If our application is composed of multiple images orchestrated using docker-compose.yml, but the customer server we need to deploy to cannot access the external network, we can use docker save to package the required images, then copy them to the customer server and load them using docker load.

Pack Data

On the old server, Docker was used to deploy the MySQL service, with the data directory mapped out. You need to migrate this data along with everything else to the new server.

MySQL Docker volumes configuration

volumes:
	- ./data:/var/lib/mysql

Because the VPN bandwidth is very low, this time tar + bzip2 is used for packing and compression.

tar -cjvf ./data data.tar.bz2

Command parameter explanation:

• -c: Parameter to create an archive (create)
• -j: Use bzip2 compression
• -v: Display details during compression (verbose)
• -f: Specify the filename, no other parameters can follow this one

Then, upload this data.tar.bz2 to the new server.

I'll also include the decompression command.

tar -xjvf data.tar.bz2

This command will generate the data directory in the current directory.

Command parameter explanation:

• -x: Indicates extraction (extract)
• The other parameters are the same as above.

Importing Images

After uploading the previously exported Docker image, execute the import command

docker load < image.tar

Then run docker images to verify if the import was successful.

Starting the Container

Upload the original docker-compose configurations.

Previously, for easier updates, the Django project was built on the server directly after uploading the code, like this:

version: "3.6"
services:
  web:
    restart: always
    build: .
    environment:
      - ENVIRONMENT=docker
      - URL_PREFIX=demo/api
      - DEBUG=false
      - DEFAULT_DATABASE=mysql
    command: uwsgi uwsgi.ini
    volumes:
      - .:/code

However, on an offline server, it's impossible to build without a network.

Therefore, we need to change build . to image: image_name.

After making this change, it started successfully~

Configure Peripheral Components

Just providing the interface isn't a big deal; all you need to do is install nginx.

However, as everyone knows, the default yum repository doesn't even have nginx. To install nginx, you need to add the epel-release repository.

sudo yum install epel-release

Fortunately, the local repository of the offline server has this package.

sudo yum update
sudo yum install nginx

Done~

Then it's the same old routine, use systemctl to start nginx, which I won't repeat.

This server has been quite a hassle. Since it's all set up, you need to test it, but the interfaces aren't open, and the SSH connection is through multiple jumps. It seems impossible to enable SSH-tunnel forwarding to test locally.

So, you can only use tools like curl to test.

However, this time I tried a different approach and used a command-line browser to see if I could open the project's webpage.

I tried these two:

• w3m
• lynx

Successfully opened the webpage, and the effect of both browsers was similar.

References

• https://www.hangge.com/blog/cache/detail_2411.html
• https://www.cnblogs.com/52linux/archive/2012/03/04/2379738.html
• https://docs.docker.com/engine/install/centos/#install-from-a-package