Ever since I started working with Gitlab, I immediately took a liking to their build system. The CI process is defined programmatically via YAML files (ok, apart from that bit), and it runs on their build farm which is pay per use. I’m not sure if this is their original idea, or whether it was first conceived by someone else (CircleCI comes to mind), but having a build process and maintaining that being someone else’s problem sounds appealing to me.
Our build process builds an artefact in the form of a docker image - that image is then used to execute tests, and, if those pass - gets published to an AWS ECR repository. Getting this to work nicely and seamlessly has proven a bit challenging.
Step 1 - build a custom CI image
I’ve started by building a custom image, which in addition to having docker installed, also has awscli installed, and a bunch of other common dependencies. What to include is largely up to you, but you’ll need to install the amazon-ecr-credential-helper binary. This component will talk to the AWS API and generate credentials which other containers can use to talk to ECR.
FROM docker:19
RUN curl -o /usr/bin/docker-credential-ecr-login https://amazon-ecr-credential-helper-releases.s3.us-east-2.amazonaws.com/0.4.0/linux-amd64/docker-credential-ecr-login
RUN chmod a+x /usr/bin/docker-credential-ecr-login
RUN mkdir /root/.docker
RUN echo '{"credsStore":"ecr-login"}' > /root/.docker/config.json
ENV AWS_DEFAULT_REGION eu-west-2
Additionally, you need to tell docker that there is a credential helper to use - this is accomplished by modifying ~/.docker/config.json. You may need to set the default AWS region, unless you intend to set it dynamically through the variables section of the gitlab CI config file. The region setting is used by the credential helper to generate credentials - and has nothing to do with anything else.
The image you build and push to a repository will be used by other build processes, so it’s best if you note down its reference.
Step 2 - use your custom build image
Simply replace any image declaration you might have, or add one to your .gitlab-ci.yml file, like this:
image: 340000007665.dkr.ecr.eu-west-2.amazonaws.com/ci:latest
This will tell gitlab-runner to use that image instead of the stock GitLab CI image.
Step 3 - provision some boxes and set up IAM roles
Use your preferred method to bring up a number of EC2 boxes, and run gitlab-runner on them. There are many tutorials on this. I’ve used npalm’s module for terraform which works very well.
We’re going to use an EC2 instance role which effectively allows an EC2 instance to behave like a user with permissions. This role can be assigned to any number of EC2 instances, giving them permission to access only a subset of resources which they must access.
Create a new instance role, and give it permissions to use your AWS ECR repository as follows:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"ecr:PutImageTagMutability",
"ecr:StartImageScan",
"ecr:ListTagsForResource",
"ecr:BatchDeleteImage",
"ecr:UploadLayerPart",
"ecr:ListImages",
"ecr:CompleteLayerUpload",
"ecr:TagResource",
"ecr:DescribeRepositories",
"ecr:BatchCheckLayerAvailability",
"ecr:GetLifecyclePolicy",
"ecr:PutLifecyclePolicy",
"ecr:DescribeImageScanFindings",
"ecr:GetLifecyclePolicyPreview",
"ecr:GetDownloadUrlForLayer",
"ecr:PutImageScanningConfiguration",
"ecr:DeleteLifecyclePolicy",
"ecr:PutImage",
"ecr:UntagResource",
"ecr:BatchGetImage",
"ecr:DescribeImages",
"ecr:StartLifecyclePolicyPreview",
"ecr:InitiateLayerUpload",
"ecr:GetRepositoryPolicy"
],
"Resource": [
"arn:aws:ecr:${region}:${registry_id}:repository/*"
]
},
{
"Sid": "VisualEditor1",
"Effect": "Allow",
"Action": "ecr:GetAuthorizationToken",
"Resource": "*"
}
]
}
You may also want to add a policy for it to access an S3 bucket, if you use that for sharing caches between runners.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:AbortMultipartUpload",
"s3:DeleteObject",
"s3:ListBucket"
],
"Resource": [
"${bucket_arn}/*",
"${bucket_arn}"
]
}
]
}
Step 4 - Try it out!
Running with gitlab-runner 13.1.1 (6fbc7474)
on docker-default 0Fc827a
Preparing the "docker" executor
00:35
Using Docker executor with image 340000007665.dkr.ecr.eu-west-2.amazonaws.com/ci:latest ...
Starting service docker:18.09-dind ...
Using locally found image version due to if-not-present pull policy
Using docker image sha256:27105793dc2bbf270ec87d6a9ba041bba52cfef66384251980a4d55d03ed736c for docker:18.09-dind ...
Starting service postgres:11-alpine ...
Using locally found image version due to if-not-present pull policy
Using docker image sha256:1b3d58586baa5a8e28d89666d94b7170c8f7a2811eac047e8fbac7e1790ac3ef for postgres:11-alpine ...
Waiting for services to be up and running...
Using locally found image version due to if-not-present pull policy
Using docker image sha256:99bec339004251e8639fe81487959d98deb8fda932191fbd9e253a92d6e9c57e for 340000007665.dkr.ecr.eu-west-2.amazonaws.com/ci:latest ...
Preparing environment
00:01
Running on runner-0Fc827a-project-1283723-concurrent-0 via ip-10-200-1-7.eu-west-2.compute.internal...
Getting source from Git repository
00:02
Fetching changes with git depth set to 50...
...
