How Can We Upload the Files and Jar to S3 Bucket via Gitlab CI/CD

3 min readDec 3, 2022

First, I want to explain why I needed to investigate and develop this project. I face the following issue;
- If you have worked on AWS Glue and developing in Scala. When you want to add some file to read under the resources file, you have faced the issue that glue can not read the file :(
But this blog post can help other developers who are working on GitLab ci/cd tools and AWS.

If I sort the topics;
- Create S3 Bucket
- Create a new user to use AWS S3 Bucket access from GitLab
- Create variable on GitLab (AWS credentials)
- GitLab CI/CD script

Let’s start cracking on it!

Create S3 Bucket

I created my package with the name api-deployment-files

Create a new user to use AWS S3 Bucket access from GitLab

Go to IAM -> Users -> Add users and Click Add users button

Add a name and select the Access key

Choose Attach existing policies directly and choose AmazonS3FullAccess

After creating the user, copy Acess key ID and Secret access key to use on Gitlab

Create variable on GitLab (AWS credentials)

Go to your Gitlab repository -> Settings -> CI/CD -> Variables
Add your creating S3_BUCKET -> api-deployment-files in this article
AWS_ACCESS_KEY_ID -> created user’s key id on AWS
AWS_SECRET_ACCESS_KEY -> created user’s secret key on AWS

NOTE: If you have different environments, you can define the variables for each environment such as dev/qa/prod or you can choose environment = All(default).

GitLab CI/CD script

It’s the last part of my article

Let’s look at the gitlab-ci.yml file

01_build_project_dev
- build with ./gradlew clean shadowJar command
- add the jar with the artifact keyword to keep the jar file.

02_deploy_files_dev
- add the aws-base image to use AWS commands
- the first script is to copy the jar file to the S3 bucket
- the second script is to copy any file to theS3 bucket

Let’s run the pipeline