基于OSS搭建私有(跨区域)Docker镜像仓库

基于OSS搭建私有(跨区域)Docker镜像仓库

公司目前在使用阿里云的容器服务,但是镜像仓库在香港没有部署,我们在香港的集群拉取镜像的速度比较慢,加速二次拉取速度,决定做一个镜像站:
repo(hangzhou) -- repo mirror(hongkong) -- user
                          |
                         oss 
示例如上,构建好的镜像存储在杭州的镜像仓库
  • 首次拉取,mirror做中转功能,并存储一份到oss
  • 二次拉取,直接从oss中读取

一,配置

我们使用nginx做反向代理,并配置认证模块,镜像使用docker提供的registry2.x版本

编排模板:

registry:
  restart: always
  image: "registry:2.6"
  # ports:
  #  - 5000:5000 
  # 关闭端口吧,只能使用认证账号登录
  environment:
    - REGISTRY_STORAGE=oss
    - REGISTRY_STORAGE_OSS_ACCESSKEYID={AK}
    - REGISTRY_STORAGE_OSS_ACCESSKEYSECRET={SK}
    - REGISTRY_STORAGE_OSS_REGION=oss-cn-hongkong
    - REGISTRY_STORAGE_OSS_BUCKET={BUCKER}
    - REGISTRY_STORAGE_OSS_INTERNAL=false
    - REGISTRY_STORAGE_OSS_SECURE=false
    - REGISTRY_PROXY_REMOTEURL=https://registry.cn-hangzhou.aliyuncs.com
    - REGISTRY_PROXY_USERNAME={USERNAME}
    - REGISTRY_PROXY_PASSWORD={PASSWORD}
    
nginx:
  image: "nginx:1.9"
  ports:
    - 443:443
  links:
    - registry:registry
  volumes:
    - ./ext:/etc/nginx/conf.d
    - ./ext/nginx.conf:/etc/nginx/nginx.conf:ro

生成密码:

➔ htpasswd -bn testuser testpassword > ext/nginx.htpasswd
testuser:$apr1$SO5gw8Pp$Q1ILVkpcYzURmvt.G3/xy0

复制证书:

cp ~/ssl/insta360.com/insta360.com.chained.crt ext/insta360.com.crt
cp ~/ssl/insta360.com/insta360.com.key ext/insta360.com.key
nginx配置:
events {
    worker_connections  1024;
}

http {

  upstream docker-registry {
    server registry:5000;
  }

  ## Set a variable to help us decide if we need to add the
  ## 'Docker-Distribution-Api-Version' header.
  ## The registry always sets this header.
  ## In the case of nginx performing auth, the header will be unset
  ## since nginx is auth-ing before proxying.
  map $upstream_http_docker_distribution_api_version $docker_distribution_api_version {
    '' 'registry/2.0';
  }

  server {
    listen 443 ssl;
    server_name registry-cn-hk.insta360.com;

    # SSL
    ssl_certificate /etc/nginx/conf.d/insta360.com.crt;
    ssl_certificate_key /etc/nginx/conf.d/insta360.com.key;

    # Recommendations from https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html
    ssl_protocols TLSv1.1 TLSv1.2;
    ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';
    ssl_prefer_server_ciphers on;
    ssl_session_cache shared:SSL:10m;

    # disable any limits to avoid HTTP 413 for large image uploads
    client_max_body_size 0;

    # required to avoid HTTP 411: see Issue #1486 (https://github.com/moby/moby/issues/1486)
    chunked_transfer_encoding on;

    location /v2/ {
      # Do not allow connections from docker 1.5 and earlier
      # docker pre-1.6.0 did not properly set the user agent on ping, catch "Go *" user agents
      if ($http_user_agent ~ "^(docker\/1\.(3|4|5(?!\.[0-9]-dev))|Go ).*$" ) {
        return 404;
      }

      # To add basic authentication to v2 use auth_basic setting.
      auth_basic "Registry realm";
      auth_basic_user_file /etc/nginx/conf.d/nginx.htpasswd;

      ## If $docker_distribution_api_version is empty, the header will not be added.
      ## See the map directive above where this variable is defined.
      add_header 'Docker-Distribution-Api-Version' $docker_distribution_api_version always;

      proxy_pass                          http://docker-registry;
      proxy_set_header  Host              $http_host;   # required for docker client's sake
      proxy_set_header  X-Real-IP         $remote_addr; # pass on real client's IP
      proxy_set_header  X-Forwarded-For   $proxy_add_x_forwarded_for;
      proxy_set_header  X-Forwarded-Proto $scheme;
      proxy_read_timeout                  900;
    }
  }
}
太懒了直接复制了docker官网的文档,改了一下域名,有需要的自行配置吧。

目录结构:

➔ tree
.
├── ext
│   ├── insta360.com.crt
│   ├── insta360.com.key
│   ├── nginx.conf
│   └── nginx.htpasswd
└── docker-compose.yml

1 directory, 5 files

二,运行

➔ docker-compose up -d 

这样一个镜像站就配好了,但是使用之前记得登录,比如像下面这样:

➔ docker login registry-cn-hk.insta360.com
Username: 
Password:

接下来直接拉取你在杭州镜像仓库的镜像吧,除了域名其他都一样了~

三,其他

Docker Registry : Authenticate proxy with nginxDocker Registry: Configuration基于OSS搭建跨区域部署的分布式Docker镜像仓库-博客-云栖社区-阿里云

参考过阿里的方案,不过我司的Global服务,由于众所周知的原因以及服务上线区域的差异,并不能很好的使用OSS的跨区域同步功能,目前的方案可以适当提高集群内应用二次拉取的速度,极大缩短镜像的更新速度。

编辑于 2017-05-17

文章被以下专栏收录