AWS DevOps Engineer Professional Certification Guide: Hands-on guide to understand, analyze, and solve 150 scenario-based questions (English Edition)

C

HAPTER

1

Continuous Integration with CodeCommit and CodeBuild

Introduction

Continuous Integration and Continuous Delivery/Deployment (CI/CD) are two important parts of DevOps. In this process, when commit or changes happen in source code, they go through automated stage gates, all the way from building, testing, deploying the applications from development to production environments, across cloud accounts.

In this chapter, we learn how AWS CodeCommit makes it easy for developers to collaborate on code using secure code repositories and AWS CodeBuild compiles source code, runs tests and produces software packages that are ready to deploy, on a dynamically created build server.

Structure

In this chapter, we will discuss about the following topics in AWS CodeCommit:

Setup AWS CodeCommit repository

Security requirements for AWS CodeCommit repository

Tagging repositories in CodeCommit

Setting up CodeCommit trigger for SNS Topic

Setting up CodeCommit trigger for Lambda Function

Creating a pull request in AWS CodeCommit

Creating an approver rule for pull request

AWS CloudShell for DevOps tasks

Working with private NPM packages in CodeArtifact

Automated code review with AWS CodeGuru

Here are the topics that we will cover in AWS CodeBuild:

Building and deploying source code with AWS CodeBuild and S3, building and pushing docker images to AWS Elastic Container Registry (ECR), and configuring an App Runner service

Objectives

After reading this chapter, we will be able to efficiently set up and manage an AWS CodeCommit repository, implement security measures, and utilize various features like tagging and triggers. We will gain a deep understanding of how to create and manage pull requests and approver rules, as well as how to leverage AWS CloudShell and CodeArtifact for DevOps tasks. Additionally, we will master the art of building and deploying source code with AWS CodeBuild and S3, working with Docker images and AWS ECR, and configuring an App Runner Service. By the end of this chapter, we will have acquired the essential skills and knowledge to implement a seamless CI process using AWS CodeCommit and CodeBuild, ultimately enhancing your proficiency in DevOps and cloud-based development practices.

Setup AWS CodeCommit repository

This section discusses the key features of AWS CodeCommit such as creating a repository, commit files to the repository, cloning a repository, pushing files to the repository, creating pull requests, tagging repositories and adding triggers into the repository.

Creating a Repository using console

We can either use AWS Console or AWS Command Line Interface (AWS CLI) to create an empty repository. Go to the Repositories page from CodeCommit console and click on Create Repository. Enter the name of repository, description (optional), add tags (optional), select Enable Amazon CodeGuru Reviewer for Java and Python (optional), and click Create as shown in Figure 1.1:

Figure 1.1: Create a repository

Creating a repository using AWS CLI

Use create-repository command to create a repository from AWS CLI by providing a unique name for the repository (with –repository-name option), and some optional comments about the repository (with –repository-description option):

1.  aws codecommit create-repository --repository-name MyFirstRepo \

2.   --repository-description My First CodeCommit Repository

If the command is successful, it displays the following output:

3.  {

4.     repositoryMetadata: {

5.         accountId: 958651443844,

6.         repositoryId: 8d9914d9-bcb1-4f3a-bbd5-62320273a956,

7.         repositoryName: MyFirstRepo,

8.         repositoryDescription: My First CodeCommit Repository,

9.         lastModifiedDate: 2022-10-13T21:32:10.432000+00:00,

10.         creationDate: 2022-10-13T21:32:10.432000+00:00,

11.         cloneUrlHttp: https://git-codecommit.us-east-1.amazonaws.com/v1/repos/MyFirstRepo,

12.         cloneUrlSsh: ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/MyFirstRepo,

13.         Arn: arn:aws:codecommit:us-east-1:958651443844:MyFirstRepo

14.     }

15.  }

Creating a commit

We can use Git client, AWS CLI or CodeCommit Console to create a commit in CodeCommit repository. In this example, we will show how to commit using Git client which is the most preferred method used by the developers to push the changes in AWS CodeCommit repository.

Use the following steps to commit a file in MyFirstRepo CodeCommit repository:

Ensure you are on the right branch otherwise, run git branch which will display the list of branches. If you are not in the right branch run git checkout branch-name to switch to the intended branch.

Make a change to the branch like adding, updating, or deleting a file. For example, create a test file (test.txt) with some text: Hello to DevOps World!.

Run git status command and it will alert you that there are some untracked files.

Run git add test.txt. This command will include a change in the working directory to the staging area.

If we run git status, it will alert that there are some changes to be committed.

Finally run git commit -m Some comments to the commit. If we run git status now git will alert that commit is ready to be pushed from local repo to CodeCommit repository.

If the changes in the file look good, then run git push origin remote-branch which will push the file to CodeCommit repository

Cloning a repository

Select the repository which you want to connect and choose one of the following methods shown in the following drop-down populated:

Figure 1.2: Clone a repository

Clone HTTPS: Use this option if you want to use your git credentials obtained along with your IAM user details from AWS Administrator:

https://git-codecommit.us-east-1.amazonaws.com/v1/repos/MyFirstRepo

Clone SSH: Use this option if you want to use SSH public/private key pair with your IAM User:

ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/MyFirstRepo

Clone HTTPS (GRC): Use this option if you want to use git-remote-codecommiton your local machine. This method is recommended if you want to support connections made with federated access, identity providers, and temporary credentials:

codecommit::us-east-1://MyFirstRepo

Security requirements for AWS CodeCommit repository

CodeCommit repositories are automatically encrypted at rest. No action is needed from the customer in this regard. CodeCommit also encrypts data in transit automatically. The first time we create a repository in CodeCommit, CodeCommit creates an AWS-managed key (aws/codecommit) in the same region and stores it in your AWS account. CodeCommit uses this key to encrypt and decrypt the data in this and other repositories in the same AWS account. Please note that we cannot use a customer-managed key created in AWS KMS Service to encrypt or decrypt data in CodeCommit repositories.

We are allowed to create our own custom IAM policies by allowing permissions to CodeCommit actions and resources.

In this section, we will show how to create a Deny policy that prevents users from making changes (Push/DeleteBranch/PutFileMerge) to a particular branch named master in a repository MyFirstRepo. By attaching the following IAM policy, we can prevent the users from pushing any commit(s) to the master branch.

1.  {

2.   Version: 2012-10-17,

3.   Statement: [{

4.   Effect: Deny,

5.   Action: [

6.     codecommit:GitPush,

7.     codecommit:DeleteBranch,

8.     codecommit:PutFile,

9.     codecommit:Merge*

10.   ],

11.   Resource: arn:aws:codecommit:us-east-1:111111111111:MyDemoRepo,

12.   Condition: {

13.     StringEqualsIfExists: {

14.     codecommit:References: [

15.       refs/heads/master

16.     ]

17.     },

18.     Null: {

19.     codecommit:References: false

20.     }

21.   }

22.   }]

23.  }

Tagging repositories in CodeCommit

Tags are custom key-value pairs that are assigned to any AWS resource. These are different from Git tags that are applied to any commit. For example, we can use tag-resource AWS CLI to add a tag to the CodeCommit repository.

1.  aws codecommit tag-resource --resource-arn arn:aws:codecommit:us-east-1:111111111111:MyFirstRepo \

2.    --tags Status=Alert,Team=Snowflake

If the command is successful, it will not return nothing as on output. You can use list-tags-for-resource AWS CLI to return the tag which was added earlier.

3.  aws codecommit list-tags-for-resource \

4.   –resource-arn arn:aws:codecommit:us-east-1:111111111111:MyFirstRepo

The output will display the tags that have been added:

1.  {

2.   tags: {

3.   Status: Alert,

4.   Team: Snowflake

5.   }

6.  }

Setting up CodeCommit trigger for SNS topic

Using CodeCommit Triggers, we can capture certain events that occur in the repository and launch an action in other AWS services.

In general, triggers are configured in CodeCommit to:

• Send emails to subscribed users when the developer pushes the code to the repository

• Notify to external build system to start a build when a developer merges code from one branch to another branch in CodeCommit repository

• Invoke an AWS lambda function

In this section, we will provide you with a step-by-step procedure using AWS CLI commands to create a trigger for a CodeCommit repository so that events in repository trigger notification from a SNS topic. The users who have subscribed to that SNS topic are notified via an email about the creation of new branches in the repository.

We have created two shell scripts to automate this entire process. The architecture diagram of this example is illustrated in Figure 1.3.

The first shell script create-repo-and-snstopic.sh creates a CodeCommit repository and an SNS topic with email subscription.

Figure 1.3:AWS CodeCommit Integration with AWS Simple Notification Service (SNS)

create-repo-and-snstopic.sh

1.  #!/bin/sh

2.  #

3.  # create-repo-and-snstopic.sh

4.  #

5.  # Create a new codecommit repository

6.  awscodecommit create-repository --repository-name MySecondRepo \

7.    --repository-description My Second CodeCommit Repository

8. 

9.  # Create a new SNS topic with an email subscription

10.  my_sns_toppic_arn=$(awssns create-topic \

11.    --name my_sns_topic \

12.    --tags Key=name,Value=testapp \

13.    --query 'TopicArn' \

14.    --output text)

15. 

16.  # create a sns topic email subscription

17.  awssns subscribe \

18.    --topic-arn $my_sns_toppic_arn \

19.    --protocol email \

20.    --notification-endpoint [email protected]

Output:

21.  {

22.     repositoryMetadata: {

23.         accountId: 958651443844,

24.         repositoryId: a01a65ae-7175-42cb-aa87-92e49e5ee04d,

25.         repositoryName: MySecondRepo,

26.         repositoryDescription: My Second CodeCommit Repository,

27.         lastModifiedDate: 2022-10-15T16:11:10.516000+00:00,

28.         creationDate: 2022-10-15T16:11:10.516000+00:00,

29.         cloneUrlHttp: https://git-codecommit.us-east-1.amazonaws.com/v1/repos/MySecondRepo,

30.         cloneUrlSsh: ssh://git-codecommit.us-east-1.amazonaws.com/v1/repos/MySecondRepo,

31.         Arn: arn:aws:codecommit:us-east-1:111111111111:MySecondRepo

32.     }

33.  }

34.  {

35.     SubscriptionArn: pending confirmation

36.  }

After we run the script, we receive an email notification to confirm the email subscription. Click on Confirm Subscription as shown in Figure 1.4:

Figure 1.4:Confirm subscription email notification

After we confirm the subscription, we receive a pop-up confirmation that subscription has been confirmed as shown in Figure 1.5:

Figure 1.5:SNS subscription confirmation email

The other shell script sns_trigger_for_codecommit_repo.sh creates a SNS trigger for your CodeCommit repository MySecondRepo:

sns_trigger_for_codecommit_repo.sh:

1.  #!/bin/sh

2.  #

3.  # sns_trigger_for_codecommit_repo.sh

4.  #

5.  # Get previous created SNS topic ARN

6.  AWS_SNS_TOPIC_ARN=$(aws sns list-topics | jq -r '.Topics[].TopicArn')

7. 

8.  # create the trigger definition

9.  cat < my_trigger_def.json

10.  {

11.    repositoryName: MySecondRepo,

12.    triggers: [

13.        {

14.            destinationArn: $AWS_SNS_TOPIC_ARN,

15.            branches: [],

16.            name: test_sns_trigger,

17.            customData: SNS Trigger ForCodeCommit,

18.            events: [

19.                createReference

20.            ]

21.        }

22.    ]

23.  }

24.  EOF

25. 

26.  ## create the trigger

27.  awscodecommit put-repository-triggers \

28.  --repository-name MySecondRepo" \

29.  --cli-input-son file://my_trigger_def.json

30. 

31.  ## get trigger details

32.  awscodecommit get-repository-triggers \

33.  --repository-name MySecondRepo"

34. 

35.  ## perform a test trigger

36.  awscodecommit test-repository-triggers \

37.  --repository-name MySecondRepo" \

38.  --cli-input-son file://my_trigger_def.json

Output:

1.  {

2.     configurationId: d8e4a5b2-38d8-4ea3-9b39-2b6ed9716d6d

3.  }

4.  {

5.     configurationId: d8e4a5b2-38d8-4ea3-9b39-2b6ed9716d6d,

6.     triggers: [

7.         {

8.             name: test_sns_trigger,

9.             destinationArn: arn:aws:sns:us-east-1:958651443844:my_sns_topic,

10.             customData: SNS Trigger For CodeCommit,

11.             branches: [],

12.             events: [

13.                 createReference

14.             ]

15.         }

16.     ]

17.  }

18.  {

19.     successfulExecutions: [

20.         test_sns_trigger

21.     ],

22.     failedExecutions: []

23.  }

So far, we have completed all the setup to complete this automation now is the time to test our trigger. Go to AWS CLI and create a new git branch by executing the following commands:

1.  git checkout -b new_branch

2.  touch test_file

3.  touch test_file.txt

4.  echo hello > test_file.txt

5.  git add test_file.txt

6.  git commit -m Added test file

7.  git push origin new_branch

Upon successful completion of preceding command, we will receive the following email notification that new branch has been created successfully as illustrated in Figure 1.6:

Figure 1.6: SNS trigger for CodeCommit repository events

Setting up CodeCommit trigger for Lambda function

We can also invoke lambda function for certain events which occur on AWS CodeCommit repository. In this section, we will create a lambda function that returns the commit ID of the push event to the AWS CloudWatch logs.

Follow the given steps to create AWS Lambda function from AWS Console:

Figure 1.7: AWS CodeCommit Integration with AWS Lambda

Sign-in to AWS Management Console and go to AWS Lambda Console.

On Lambda function page, choose Create function. On the Create function, choose Author from Scratch. In function name, provide a name for the function LambdaFuncToDisplayCommitId.

On Configuration tab, choose Add trigger.

In Trigger configuration, choose CodeCommit from the services drop-down menu as shown in the following screenshot:

Figure 1.8:Create an AWS CodeCommit trigger for an AWS Lambda function

In Repository name, provide the name of the repository: MyFirstRepo

In Trigger name, provide the name for the trigger: TriggerLambdaFunc

In events, choose Push to existing branch event and click on Add.

Choose Python 3.9 as the Runtime for Lambda Function.

Go to Code tab and replace hello world code with the following code and deploy the lambda function:

1.  import json

2.  import os

3.  import boto3

4.  codecommit = boto3.client('codecommit')

5.  def lambda_handler(event, context):

6.     try:

7.          repo_name = event['Records'][0]['eventSourceARN'].split(':')[-1]

8.         commit_hash = event['Records'][0]['codecommit']['references'][0]['commit']

9.         branchName = os.path.basename(

10.             str(event['Records'][0]['codecommit']['references'][0]['ref']))

11.         print(Commit hash is: + commit_hash)

12.     except Exception as e:

13.         print(fError processing event: {e})

Go to AWS CLI and clone the MyFirstRepo Repository and push an empty file to the repository as shown below:

1.  git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/MyFirstRepo

2.  cd MyFirstRepo/

3.  touch newfile

4.  git add newfile

5.  git commit -m added new file

6.  git push origin test_branch

Go to AWS CloudWatch logs and check the timestamp of the log stream and you will notice commit id of git push event in the logs. Refer to Figure 1.9:

Figure 1.9:CloudWatch logs showing the commit id

Grab the commit ID from Step 11 and pass it to the next CLI command get-commit:

1.  aws codecommit get-commit –repository-name MyFirstRepo \

2.   –commit-id fd46e59f4a250f6d1e0c9cb9082793ce46c6090c

If you look at the output, you will notice that the value for message field, that is, added new file is the same as it was provided in step 10: for git commit.

Output:

1.  {

2.     commit: {

3.         commitId: fd46e59f4a250f6d1e0c9cb9082793ce46c6090c,

4.         treeId: 3c22ead769a428098f713faee5fcd47ec981d8ed,

5.         parents: [

6.             5ccd628c11c947d43f559fe09462268985f20f98

7.         ],

8.         message: added new file\n,

9.         author: {

10.             name: abc def,

11.             email: [email protected],

12.             date: 1666059317 +0000

13.         },

14.         committer: {

15.             name: abc def,

16.             email: [email protected],

17.             date: 1666059317 +0000

18.         },

19.         additionalData:

20.     }

21.  }

Creating a pull request in AWS CodeCommit

By creating a pull request, users can collaborate. They can review code, provide comments, and merge changes from one branch to another. We can also create approver rules for the pull request. For example, we can create an approver rule where at least two users need to approve the pull request before the code is merged from one branch to another. Now, we will show you an example where we will create a pull request in an AWS CodeCommit repository using AWS CLI.

In this section, we will provide you with two examples. In the first example, Example-1 we create a pull request named My Test Pull Request with a valid description that targets new_branch source branch and is to be merged in the default branch in the AWS Code Repository named MySecondRepo.

Example 1:

1.  aws codecommit create-pull-request \

2.      --title My Test Pull Request \

3.      --description Please review these changes by COB today \

4.      --client-request-token testtoken١2٣ \

5.      --targets repositoryName=MySecondRepo,sourceReference=new_branch

Output:

6.  {

7.     pullRequest: {

8.         pullRequestId: 4,

9.         title: My Test Pull Request,

10.         description: Please review this changes by COB today,

11.         lastActivityDate: 2022-10-18T21:21:59.912000+00:00,

12.         creationDate: 2022-10-18T21:21:59.912000+00:00,

13.         pullRequestStatus: OPEN,

14.         authorArn: arn:aws:iam::111111111111:user/testuser,

15.         pullRequestTargets: [

16.             {

17.                 repositoryName: MySecondRepo,

18.                 sourceReference: refs/heads/new_branch,

19.                 destinationReference: refs/heads/test_branch,

20.                 destinationCommit: 7a88cec01c6a4e3394cdf7634a9adf64cb275b94,

21.                 sourceCommit: 355dda141c36c73ee1daa65a3bb17afc64817f2a,

22.                 mergeBase: 78f5bbf8f48d367adddbc3011c41f38269edbdbb,

23.                 mergeMetadata: {

24.                     isMerged: false

25.                 }

26.             }

27.         ],

28.         clientRequestToken: testtoken123,

29.         revisionId: 7fd367432e80521fec28f7ff8955e6cc611df4a47da2595c3a0df6796db8c2f0,

30.         approvalRules: []

31.     }

32.  }

Creating an approver rule for pull request

In the next example, we will create an approver rule for pull request using AWS CLI.

To create the approver rule, first we need to find out how many pull requests are currently OPEN by running the following AWS CLI:

1.  aws codecommit list-pull-requests --author-arn arn:aws:iam::111111111111:user/testuser \

2.   --pull-request-status OPEN --repository-name MySecondRepo

Output:

1.  { pullRequestIds: [ 4, 3 ] }

Now, we pick one pull request ID and pass it to the next AWS CLI to create an approver rule:

1.  aws codecommit create-pull-request-approval-rule  \

2.     --pull-request-id  4 \

3.     --approval-rule-name We need two approver to approve the pull request  \

4.     --approval-rule-content {\"Version\": \"2018-11-08\",\"Statements\": [{\"Type\": \"Approvers\",\"NumberOfApprovalsNeeded\": 2,\"ApprovalPoolMembers\": [\"CodeCommitApprovers:111111111111:xyz\", \"arn:aws:sts::111111111111:assumed-role/CodeCommitReview/*\"]}]}

Output:

1.   {

2.     approvalRule: {

3.         approvalRuleId: 3f95dfd5-add7-4fbf-8b39-4602bafa6b9f,

4.         approvalRuleName: We need two approver to approve the pull request,

5.          approvalRuleContent: {\"Version\":\"2018-11-08\",\"Statements\":[{\"Type\":\"Approvers\",\"NumberOfApprovalsNeeded\":2,\"ApprovalPoolMembers\":[\"CodeCommitApprovers:111111111111:xyz\",\"arn:aws:sts::111111111111:assumed-role/CodeCommitReview/*\"]}]},

6.         ruleContentSha256: ae4ab7001045fc5415a366e88021c8dbe56173434a9ce226ea06b6b71d20d134,

7.         lastModifiedDate: 2022-10-18T22:07:36.976000+00:00,

8.         creationDate: 2022-10-18T22:07:36.976000+00:00,

9.         lastModifiedUser: arn:aws:iam::111111111111:user/testuser

10.     }

11.  }

AWS CloudShell for DevOps tasks

AWS CloudShell is a browser-based, pre-authenticated shell environment that provides users with direct access to AWS services, resources, and infrastructure. It comes pre-configured with AWS CLI, SDKs, and other useful tools to help manage and automate tasks related to DevOps and continuous integration.

To access AWS CloudShell, navigate to the AWS Management Console and search for CloudShell, or click on the CloudShell icon in the menu bar, as shown in Figure 1.10.

Figure 1.10: Accessing AWS CloudShell in the AWS Management Console

Once the CloudShell terminal opens, you can run the following commands in the CloudShell terminal:

Check the AWS CLI version:

  aws --version

Retrieve information about your AWS account:

  aws sts get-caller-identity

Check the AWS Serverless Application Model (SAM) CLI version:

  sam --version

Figure 1.11 illustrates the output of the preceding commands when run in CloudShell:

Figure 1.11: CloudShell Command outputs

Working with private NPM packages in CodeArtifact

AWS CodeArtifact fetches software packages on-demand from public repositories such as npm registry, Maven central, Python Package Index (PyPI), and NuGet. CodeArtifact securely publishes these private packages to a central organizational repository and shares them across organizations. Automated builds, such as CodeBuild pipelines, pull dependencies from CodeArtifact for use and publish newer versions of the packages. We can also build automated approval workflows for software releases using CodeArtifact APIs and gain full visibility into the packages by integrating with AWS CloudTrail.

In this example, we will show how to create, publish, and download a private npm package using AWS CodeArtifact. The architecture diagram of this solution is shown in Figure 1.12:

Figure 1.12: Publishing private npm packages with AWS CodeArtifact

To complete this use case in your AWS account, open the CloudShell terminal and follow the step-by-step instructions below:

Create a new domain in CodeArtifact:

  aws codeartifact create-domain --domain my-test-domain

Create a new CodeArtifact repository in the domain we just created:

  aws codeartifact create-repository --domain my-test-domain --repository my-test-repo

Log in to the CodeArtifact repository using the AWS CLI:

  aws codeartifact login --tool npm --domain my-test-domain --repository my-test-repo

This command will return a temporary authorization token and configure our local npm to use the CodeArtifact repository. The token is valid for 12 hours.

Create a new directory for your package and navigate to it:

  mkdir hello-world-package

  cd hello-world-package

Initialize the package with npm init and the specified scope:

  npm init --scope=@mycompany -y

This will create a package.json file in your package directory.

Create an index.js file in the package directory with the following content:

  1.  function showGreeting() {

  2.   console.log('Hello World, DevOps Gurus!');

  3.  }

  4. 

  5.  module.exports = showGreeting;

This file exports a simple function that displays Hello World, DevOps Gurus! message.

Create a .npmrc file in the root directory of your npm package, if you do not already have one, and add the following lines:

  registry=https://my-test-domain-[your-aws-accountid].d.codeartifact.us-east-1.amazonaws.com/npm/my-test-repo/

  always-auth=true

Publish your npm package to the CodeArtifact repository:

  npm publish

Figure 1.13shows the output after we published the package:

Figure 1.13:Successful Package Publication to CodeArtifact Repository

Now, if we navigate to the CodeArtifact console and check the repository, we can see that our new package is ready to be downloaded, as shown in Figure 1.14:

Figure 1.14:Newly published package visible in CodeArtifact repository

Install private npm package:

a.  Create a new directory for your project and navigate to it:

  mkdir hello-world-app

  cd hello-world-app

b.  Initialize the project with npm init -y:

  npm init -y

c.  Install the published package as a dependency in this project:

  npm install --save @mycompany/hello-world-package

d.  Create an index.js file in the project directory and import the showGreeting function from the installed package:

  const showGreeting = require('@mycompany/hello-world-package');

  showGreeting();

e.  Run the index.js file to see the Hello World, DevOps Gurus! message output by the showGreeting function:

  node index.js

Automated code review with AWS CodeGuru

AWS CodeGuru is a developer tool that uses machine learning to improve code quality and optimize application performance. It consists of two main components: CodeGuru Reviewer and CodeGuru Profiler. CodeGuru Reviewer scans code repositories, analyzing pull requests to find potential issues, security vulnerabilities, and deviation from best practices. CodeGuru Profiler, on the other hand, analyzes the runtime behavior of applications, finding performance bottlenecks and suggesting optimizations to improve efficiency and reduce infrastructure costs.

In this real-world example, a manager who acts as a code approver needs help in setting up a tool for reviewing the codebase to check for any code issues that may contribute to poor application performance. AWS CodeGuru will be used to review the code for any potential bugs. As a DevOps engineer, we will associate a CodeCommit repository with AWS CodeGuru Reviewer. Whenever a pull request is created to merge the code from the development branch to the main branch, CodeGuru Reviewer will automatically start analyzing the code and provide recommendations. This process ensures that the manager receives valuable insights into the code quality, enabling informed decision-making before approving any changes.

The architecture diagram of this use-case is shown in Figure 1.15:

Figure 1.15: Configure Automated Code Review with AWS CodeGurur

Open CloudShell terminal and follow step-by-step instructions below to set up a repository, add CodeGuru reviewer, and creating a pull request with a Java example:

Create a new repository in AWS CodeCommit:

a.  Sign into the AWS Management Console and open the AWS CodeCommit console.

b.  Click Create repository and name the repository codeguru-reviewer-java-example.

c.  Click Create.

Clone the repository:

1.  git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/codeguru-reviewer-java-example

Add a Java file:

Create a simple Java file named Example.java in the cloned repository with the following code:

1.  public class Example {

2. 

3.     public static int addNumbers(int a, int b) {

4.         int result = a + b;

5.         return result;

6.     }

7. 

8.     public static void main(String[] args) {

9.         int x = 5;

10.         int y = 10;

11.         int result = addNumbers(x, y);

12.         System.out.println(The sum of + x + and + y + is + result);

13.     }

14.  }

Commit and push the changes:

Commit and push the changes to the remote repository:

1.  git add Example.java

2.  git commit -m Add example Java file

3.  git push

Connect AWS CodeGuru reviewer:

a.  Sign into the AWS Management console and open the CodeGuru console.

b.  In the navigation pane, choose CodeGuru Reviewer.

c.  Click on Repositories and then Associate repository.

d.  Select AWS CodeCommit as the repository provider.

e.  Choose the codeguru-reviewer-java-example repository from the list

f.  Enter the name of the Source branch as mainto scan and click on Associate repository and run analysis.

Please refer to Figure 1.16to understand the instructions for a visual representation of the steps described above:

Figure 1.16:Associate CodeCommit Repository with AWS CodeGuru Reviewer

Create a new branch:

Create a new branch for the feature or bugfix you want to work on:

  git checkout -b dev

Make changes:

Modify the Example.java file by introducing a resource leak issue. Update the file with the following code:

1.  import java.io.*;

2. 

3.  public class Example {

4. 

5.     public static int addNumbers(int a, int b) {

6.         int result = a + b;

7.         return result;

8.     }

9. 

10.     public static void readFile(String fileName) {

11.         // Intentionally introduce an issue: resource leak

12.         try {

13.             FileReader fileReader = new FileReader(fileName);

14.             BufferedReader bufferedReader = new BufferedReader(fileReader);

15.             String line = bufferedReader.readLine();

16.             System.out.println(First line of the file: + line);

17.         } catch (IOException e) {

18.             System.out.println(Error reading file: + e.getMessage());

19.         }

20.         // The file reader and buffered reader are not properly closed

21.     }

22. 

23.     public static void main(String[] args) {

24.         int x = 5;

25.         int y = 10;

26.         int result = addNumbers(x, y);

27.         System.out.println(The sum of + x + and + y + is + result);

28. 

29.         String fileName = example.txt;

30.         readFile(fileName);

31.     }

32.  }

Commit and push the changes:

Commit and push the changes to the remote feature branch:

1.  git add Example.java

2.  git commit -m Introduce resource leak issue in readFile method

3.  git push origin dev

Create a Pull Request:

a.  Go to the AWS CodeCommit console and navigate to the codeguru-reviewer-java-example repository.

a.  Click on Create pull request.

b.  Choose dev as the source branch and main as the destination branch.

c.  Add a title and description for the pull request.

d.  Click Create to submit the PR.

Once the PR is created, CodeGuru Reviewer will analyze the code and provide recommendations on the Pull Request. It will take a few minutes for CodeGuru Reviewer to complete the analysis. Once the analysis is finished, navigate back to the CodeGuru Console, and select Code reviews on the left navigation pane. Click on the Incremental code review tab, and then click on the code review which we’re interested in. This will take us to a new page where can see the recommendations provided by CodeGuru Reviewer. Figure 1.17 displays those findings:

Figure 1.17: Viewing Recommendations from AWS CodeGuru Reviewer in Incremental Code Reviewer

Building and deploying source code with AWS CodeBuild and S3

In this example, we use AWS CodeBuild to build a collection of sample source code build input files into a deployable version of build code artifact. We are instructing CodeBuild to use Apache Maven which is a build tool for Java applications. For this tutorial we do not need to be familiar with Maven build tool. Figure 1.10 shows how AWS CodeBuild gets the source code from S3, downloads source code into the build environment, uses build specification to build the project and finally uploads the build output artifact to S3. Build information is logged into AWS CloudWatch logs.

Figure 1.18: CodeBuild use S3 as input/output to get source code/produce artifacts

Following is the step-by-step procedure that should be followed while using AWS CodeBuild Service in this build process:

Creating source code files: Create a root project directory named HelloWorldMavenTest and cd HelloWorldMavenTest. Under the directory create following files:

  HelloWorld.java

1.  package helloworld;

2.  public class HelloWorld {

3.    public static void main(String[] args) {

4.    Greeteing greeting = new Greeteing();

5.          System.out.println(greeting.showMessage());

6.      }

7.  }

Greeting.java

1.  package helloworld;

2.  public class Greeting {

3.      public String showMessage() {

4.          return Hello world!;

5.      }

6.  }

pom.xml:

1.  http://maven.apache.org/POM/4.0.0

2.      xmlns:xsi=http://www.w3.org/2001/XMLSchema-instance

3.      xsi:schemaLocation=http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd>

4.    4.0.0

5.    org.example

6.    hello-world-from-codebuild

7.    1.0

8.    jar

9.    Display Hello World Utility Java Sample App

10.   

11.     

12.        junit

13.        junit

14.        4.11

15.        test

16.   

17.     

18.       

19.          org.apache.maven.plugins

20.          maven-compiler-plugin

21.          3.8.0

22. 

Creating buildspec file: A buildspec file is a collection of build commands in yaml format that is used by AWS CodeBuild to run a build.

Create and place buildspec.yml, inside HelloWorldMavenTest subdirectory.

1.  version: 0.2

2. 

3.  phases:

4.    install:

5.      runtime-versions:

6.        java: corretto11

7.    pre_build:

8.      commands:

9.        - echo Nothing to do in this pre_build phase...

10.    build:

11.      commands:

12.        - echo Build started on `date`

13.        - mvn install

14.    post_build:

15.      commands:

16.        - echo Build completed on `date`

17.  artifacts:

18.    files:

19.      - target/helloWorldFromCodeBuild-1.0.jar

Creating two S3 buckets: Create two S3 buckets. One bucket will store the build input and the other bucket stores the build output. These two buckets should be in the same region where build exists.

Use following naming convention for creating input and output buckets.

Input bucket name: codebuild-[region]-[your-aws-accountid]-input-bucket

Output bucket name: codebuild-[region]-[your-aws-accountid]-output-bucket

Figure 1.19:S3 Input and Output Buckets

Uploading source code and buildspec file into input S3 bucket: Run the following AWS CLI to package the files in a .zip format and upload the ZIP file into input S3 bucket:

1.  zip -r HelloWorld.zip .

2.  aws s3 cp HelloWorld.zip s3://codebuild-us-east-1-111111111111-input-bucket

Creating CodeBuild Project: Create a build project that AWS CodeBuild uses to run the build. A build project includes information on how to run a build, the path where to get the source code, which commands to run and where to upload the build output.

On Create build project page, enter the following information before you hit on Create build project.

•  Project Name: codebuilddemo-project

•  Source Provider: Amazon S3

•  Bucket: codebuild-us-east-1-111111111111-input-bucket

•  S3 object key: HelloWorld.zip

•  Environment | Environment Image: Managed Image

•  Operating System: Amazon Linux 2

•  Runtime: Standard

•  Image: aws/codebuild/amazonlinux2-x86_64-standard:3.0

•  Service Role: New Service Role

•  BuildSpec: buildspect

•  Artifacts Type: Amazon S3

•  For bucket name: codebuild-us-east-1-111111111111-output-bucket

Leave Name and Path blank. Finally, click on Create Build Project.

An illustration of Step 5 can be seen in the following figure:

Figure 1.20:Create build project in AWS CodeBuild

Creating notification rule: Select codebuild project and choose Create notification rule.

a.  Enter notification name: build-notification-sns

b.  Detail type: Basic

c.  Events that trigger notifications: Failed, Stopped

d.  Choose targe type: SNS

e.  Choose Target: Specify SNS topic name i.e., my-sns-topic

An illustration of Step 6 can be seen in the following figure:

Figure 1.21: Configure notification rule

Choose target type as SNS topic and target name as my-sns-topic as illustrated in Figure 1.22:

Figure 1.22:Configure SNS Target for notification rule

1.   {

2.       Sid: AWSCodeStarNotifications_publish,

3.       Effect: Allow,

4.       Principal: {

5.         Service: codestar-notifications.amazonaws.com

6.       },

7.       Action: SNS:Publish,

8.       Resource: arn:aws:sns:us-east-1:111111111111:my-sns-topic,

9.       Condition: {

10.         StringEquals: {

11.           aws:SourceAccount: 111111111111

12.         }

13.        }

14.     }

Selectcodebuilddemo-projectfrom navigation paneand click on Start Build and watch the logs by clicking ontail logs.

Once the build is successful, you will see that UPLOAD_ARTIFACT state is SUCCEEDED in AWS CloudWatch logs.

Figure 1.23:CloudWatch logs show created artifact name by CodeBuild

To see the build output artifact, go to the output S3 bucket: codebuild-us-east-1-111111111111-output-bucket. The following figure illustrates the name of the build output artifact that is uploaded to S3 bucket.

Figure 1.24:Build output uploaded to S3 bucket

Building and pushing Docker image to AWS ECR

In this example, we use AWS CodeBuild to produce a build output as Docker image and push the image to Amazon Elastic Container Registry (ECR) image repository.

Here, we use Node.js application into a Docker container. See below for a step-by-step procedure to run this example on AWS account. The flowchart representation of the process can be seen in the following figure:

Figure 1.25: Upload docker image to AWS ECR by CodeBuild

Create a private image repository in ECR. Ensure that ECR repository and build environment are in the same AWS region. The name of the repository should be the same as the value for environment variable IMAGE_REPO_NAME which will be used in environment settings in CodeBuild in subsequent steps.

Figure 1.26:Create a private ECR repository

Create a repository in CodeCommit with the name nodejs-codecommit-repo as shown in Figure 1.19:

Figure 1.27:Create CodeCommit repository

Add the following files into nodejs-codecommit-repo CodeCommit repository as illustrated in Figure 1.20:

Figure 1.28:The source code pertaining to nodejs application

package.json: This file consists of nodejs app and its dependencies.

1.  {

2.   name: my-nodejs-webapp,

3.   version: 1.0.0,

4.   description: My Node.js Application on Docker,

5.   author: [email protected]>,

6.   main: server.js,

7.   scripts: {

8.     start: node server.js

9.   },

10.   dependencies: {

11.     express: ^4.16.1

12.   }

13.  }

server.js: This file defines a web application using Express.js framework.

1.  'use strict';

2. 

3.  const express = require('express');

4. 

5.  // Constants

6.  const PORT = 8080;

7.  const HOST = '0.0.0.0';

8. 

9.  // App

10.  const app = express();

11.  app.get('/', (req, res) => {

12.   res.send('Hello World From CodeBuild');

13.  });

14. 

15.  app.listen(PORT, HOST, () => {

16.   console.log(`Running on http://${HOST}:${PORT}`);

17.  });

Dockerfile: Dockerfile is a text document that contains all the commands to assemble an image for NodeJs application. This file consists of following commands:

1.  FROM node:16

2. 

3.  WORKDIR /usr/src/app

4. 

5.  COPY package*.json ./

6. 

7.