Deploying Docker Registry on Kubernetes using S3 Storage

Rahul Mahale

By Rahul Mahale

on May 3, 2018

In today's era of containerization, no matter what container we are using we need an image to run the container. Docker images are stored on container registries like Docker hub(cloud), Google Container Registry(GCR), AWS ECR, etc.

We can also self-host docker registry on any docker platform. In this blog post, we will see how to deploy docker registry on kubernetes using storage driver S3.


As per docker registry documentation, We can simply start the registry using docker image registry.

Basic parameters when deploying production registry are:

  • Authentication
  • SSL
  • Storage

We will use htpasswd authentication for this post though registry image supports silly and token based authentication as well.

Docker registry requires applications to use SSL certificate and key in the registry. We will use kubernetes service, which terminates SSL on ELB level using annotations.

For registry storage, we can use filesystem, s3, azure, swift etc. For the complete list of options please visit docker site site.

We need to store the docker images pushed to the registry. We will use S3 to store these docker images.

Steps for deploying registry on kubernetes.

Get the ARN of the SSL certificate to be used for SSL.

If you don't have SSL on AWS IAM, upload it using the following command.

1$aws iam upload-server-certificate --server-certificate-name registry --certificate-body file://registry.crt --private-key file://key.pem

Get the arn for the certificate using the command.

1$aws iam get-server-certificate --server-certificate-name registry  | grep Arn

Create S3 bucket which will be used to store docker images using s3cmd or aws s3.

2$s3cmd mb s3://myregistry
3Bucket 's3://myregistry/' created

Create a separate namespace, configmap, deployment and service for registry using following templates.

2apiVersion: v1
3kind: Namespace
5name: container-registry
8apiVersion: v1
9kind: ConfigMap
11  name: auth
12  namespace: container-registry
14  htpasswd: |
15    admin:$2y$05$TpZPzI7U7cr3cipe6jrOPe0bqohiwgEerEB6E4bFLsUf7Bk.SEBRi
18apiVersion: extensions/v1beta1
19kind: Deployment
21  labels:
22    app: registry
23  name: registry
24  namespace: container-registry
26  replicas: 1
27  strategy:
28    type: RollingUpdate
29  template:
30    metadata:
31      labels:
32        app: registry
33    spec:
34      containers:
35        - env:
36            - name: REGISTRY_AUTH
37              value: htpasswd
38            - name: REGISTRY_AUTH_HTPASSWD_PATH
39              value: /auth/htpasswd
40            - name: REGISTRY_AUTH_HTPASSWD_REALM
41              value: Registry Realm
42            - name: REGISTRY_STORAGE
43              value: s3
44            - name: REGISTRY_STORAGE_S3_ACCESSKEY
45              value: <your-s3-access-key>
46            - name: REGISTRY_STORAGE_S3_BUCKET
47              value: <your-registry-bucket>
48            - name: REGISTRY_STORAGE_S3_REGION
49              value: us-east-1
50            - name: REGISTRY_STORAGE_S3_SECRETKEY
51              value: <your-secret-s3-key>
52          image: registry:2
53          name: registry
54          ports:
55            - containerPort: 5000
56          volumeMounts:
57            - name: auth
58              mountPath: /auth
59      volumes:
60        - name: auth
61          configMap:
62            name: auth
64apiVersion: v1
65kind: Service
67  annotations:
68 <your-iam-certificate-arn>
69 http
70 "443"
71  labels:
72    app: registry
73  name: registry
74  namespace: container-registry
76  ports:
77    - name: "443"
78      port: 443
79      targetPort: 5000
80  selector:
81    app: registry
82type: LoadBalancer

Let's launch this manifest using kubectl apply.

2kubectl apply -f registry-namespace.yml registry-configmap.yml registry-deployment.yaml registry-namespace.yml
3namespace "registry" created
4configmap "auth" created
5deployment "registry" created
6service "registry" created

Now that we have created registry, we should map DNS to web service ELB endpoint. We can get the webservice ELB endpoint using the following command.

2$kubectl -n registry get svc registry -o wide
4NAME       CLUSTER-IP      EXTERNAL-IP                                                               PORT(S)         AGE       SELECTOR
5registry   443:30494/TCP   1h       app=registry

We will point DNS to this ELB endpoint with domain

Once we have registry running, now it's time to push the image to a registry.

First, pull the image or build the image locally to push.

On local machine run following commands:

2$docker pull busybox
3latest: Pulling from busybox
4f9ea5e501ad7: Pull complete
5ac3f08b78d4e: Pull complete
6Digest: sha256:da268b65d710e5ca91271f161d0ff078dc63930bbd6baac88d21b20d23b427ec
7Status: Downloaded newer image for busybox:latest

Now login to our registry using the following commands.

2$ sudo docker login
4Username: admin
8Login Succeeded

Now tag the image to point it to our registry using docker tag command

1$ sudo docker tag busybox

Once the image is tagged we are good to push.

Using the docker push command let's push the image.

2$ sudo docker push
4The push refers to a repository []
605732a3f47b5: Pushed
730de36c4bd15: Pushed
85237590c0d08: Pushed
9latest: digest: sha256:f112e608b2639b21498bd4dbca9076d378cc216a80d52287f7f0f6ea6ad739ab size: 205

We are successfully able to push image to registry running on kunbernetes and stored on S3. Let's verify if it exists on S3.

Navigate to our s3 bucket and we can see the docker registry repository busybox has been created.

1$ s3cmd ls s3://myregistry/docker/registry/repositories/
2DIR   s3://myregistry/docker/registry/repositories/busybox/

All our image related files are stored on S3.

In this way, we self-host container registry on kubernetes backed by s3 storage.

Stay up to date with our blogs. Sign up for our newsletter.

We write about Ruby on Rails, ReactJS, React Native, remote work,open source, engineering & design.