Tag AWS

JMeter Setup | AWS Linux AMI (AWS EC2)

prashanthsams Leave a comment
  • SSH into the Amazon EC2 machine

ssh -i yourkey.pem ec2-user@11.111.11.11

  • Download Apache JMeter

wget https://downloads.apache.org/jmeter/binaries/apache-jmeter-5.6.3.tgz

  • Extract the downloaded compressed file

tar -xzf apache-jmeter-5.6.3.tgz

  • Move folder into the /opt/ folder

sudo mv apache-jmeter-5.6.3 /opt/

  • Set JMeter environment variable and path

export JMETER_HOME=/opt/apache-jmeter-5.6.3
export PATH=$PATH:$JMETER_HOME/bin
source ~/.bashrc

  • Download and install custom JDK

yum search java | grep "17"
sudo yum install java-17-amazon-corretto.x86_64

  • Copy certificate and test plan into the AWS EC2 machine

sudo scp -i yourkey.pem certificate.pfx ec2-user@11.111.11.11:/home/ec2-user
sudo scp -i yourkey.pem yourtestplan.jmx ec2-user@11.111.11.11:/home/ec2-user

  • Download JMeter plugins manager directly into the /ext folder; else, download and copy into the EC2 machine from your local machine

cd /opt/apache-jmeter-5.6.3/lib/ext
curl -O https://repo1.maven.org/maven2/kg/apc/jmeter-plugins-manager/1.9/jmeter-plugins-manager-1.9.jar

or

sudo scp -i yourkey.pem jmeter-plugins-manager-1.9.jar ec2-user@11.111.11.11:/opt/apache-jmeter-5.6.3/lib/ext

  • Download Throughput Shaping Timer plugin from the link and copy the below mentioned jar files into the respective folders

sudo scp -i yourkey.pem jmeter-plugins-tst-2.6.jar ec2-user@11.111.11.11:/opt/apache-jmeter-5.6.3/lib/ext
sudo scp -i yourkey.pem jmeter-plugins-cmn-jmeter-0.7.jar ec2-user@11.111.11.11:/opt/apache-jmeter-5.6.3/lib/

  • JMeter Test runner with certificate and it’s password exporting results into the .jtl file

jmeter -Djavax.net.ssl.keyStore=/home/ec2-user/certificate.pfx -Djavax.net.ssl.keyStorePassword=111111 -n -t /home/ec2-user/yourtestplan.jmx -l /home/ec2-user/results.jtl

  • Copy results.jtl from AWS EC2 machine into the local machine

sudo scp -i yourkey.pem ec2-user@11.111.11.11:/home/ec2-user/results.jtl /local-machine-path/

  • Generate HTML report from the generated results.jtl file

jmeter -g /local-machine-path/results.jtl -o ./destination-folder/

AWS S3 Bucket

prashanthsams Leave a comment

AWS CLI

LIST
# list all the available s3 buckets
aws s3 ls
[list with bucket name]
aws s3 ls s3://bucket-name/

# list all the sub-folders and files
aws s3 ls s3://bucket-name/ --recursive
(i.e., aws s3 ls s3://prashanth-sams --recursive)

# list all the bucket names with it's size
aws s3 ls s3://bucket-name/ --summarize

CREATE
# create new bucket; here mb is 'make bucket'
aws s3 mb s3://bucket-name/
(i.e., aws s3 mb s3://prashanth-sams)

# create new bucket with specific region
aws s3 mb s3://bucket-name/ --region us-east-1

COPY | MOVE
# copy a file inside bucket
aws s3 cp source-file s3://bucket-name/
(i.e., aws s3 cp /file.html s3://prashanth-sams)

# move a file inside bucket
aws s3 mv source-file s3://bucket-name/
(i.e., aws s3 mv /file.html s3://prashanth-sams)

DELETE
# delete all the data inside a bucket
aws s3 rm s3://bucket-name/ --recursive

# delete all files and folders excluding a specific file pattern
aws s3 rm s3://bucket-name/ --recursive --exclude "*.html"

# delete all files and folders excluding a specific folder
aws s3 rm s3://bucket-name/ --recursive --exclude "folder/*"

# delete a bucket which is empty; here, rb is 'remove bucket'
aws s3 rb s3://bucket-name

# delete a bucket which is not empty
aws s3 rb s3://bucket-name --force 

SYNC
# upload or sync your local data to remote s3 bucket
aws s3 sync . s3://bucket-name

# upload data by excluding files/folders with specific pattern to remote s3 bucket
aws s3 sync . s3://bucket-name --exclude "*.tgz"
aws s3 sync . s3://bucket-name --exclude "folder/*"

# download or sync your remote s3 bucket data to local
aws s3 sync s3://bucket-name .

# download data by excluding files/folders with specific pattern to local
aws s3 sync s3://bucket-name . --exclude "*.tgz"
aws s3 sync s3://bucket-name . --exclude "folder/*"

# copy or sync your remote s3 bucket data to another s3 bucket
aws s3 sync s3://bucket-name1 s3://bucket-name2

# update and deleted file/folder in remote s3
aws s3 sync . s3://bucket-name --delete

DYNAMIC URL
# make data public and open to users with dynamic access key
[default expire time of the link will be 3600 secs]
aws s3 presign s3://bucket-name/file.html

# make data public for a specific time period
aws s3 presign s3://bucket-name/file.html --expires-in 30

STATIC WEBSITE
# static url for a html file
aws s3 website s3://bucket-name --index-document index.html

# static url for a html file with working and not working document 
aws s3 website s3://bucket-name --index-document index.html --error-document error.html

OUTPUT
http://bucket-name.s3-website-us-east-1.amazonaws.com/
(i.e., http://prashanth-sams.s3-website-us-east-1.amazonaws.com/)

AWS Console

CREATE BUCKET

  • Go to S3 in aws console
  • Click on Create bucket

  • Enter bucket name, Region, and click on the create button

  • Select Bucket name and click on Edit public access settings

  • Untick Block all public access and click on the save button

  • Now, click on the bucket-name and upload files
  • Select the file and make it public

  • Now, click on the file and open the link

AWS Key-Pair

prashanthsams Leave a comment

Automatic (AWS web interface)

Configure AWS key-pair from AWS web interface

  • Go to AWS console and click on Key Pairs link
  • Click on the Create key pair button

  • Enter Key pair name and choose pem file format and click on the Create key pair button
  • Now, a pem (private key) file will be downloaded in your local machine, which was actually generated in the AWS console

  • Generate the public key out of the generated private key
sudo ssh-keygen -y -f ~/Downloads/prashanth.pem > prashanth.pub

 

Manual (AWS CLI)

Configure AWS key-pair from local machine (terminal)

  • Generate AWS key-pair
# generate key-pair
aws ec2 create-key-pair --key-name prashanth

# generate key-pair by exporting the values in a .pem file
aws ec2 create-key-pair --key-name prashanth --output text > prashanth.pem

# generate key-pair with a query cli for filter
aws ec2 create-key-pair --key-name prashanth --query 'filter_name'


  • Restrict permission access to read only
chmod 400 prashanth.pem
  • Verify the recently generated key-pair
aws ec2 describe-key-pairs --key-name prashanth
  • Delete a key-pair
aws ec2 delete-key-pair --key-name prashanth

Create an EC2 instance | Terraform

prashanthsams Leave a comment
This post helps you to create a basic EC2 instance (free tier) through Terraform AWS provisioning DSL scripting language
  • Download Terraform CLI
https://www.terraform.io/downloads.html
  • Extract and move terraform inside bin folder
mv ~/Downloads/terraform /usr/local/bin/

terraform version
  • Create a new project and a file with extension .tf
  • Copy and paste the below script
provider "aws" {
version = "~> 2.0"
region = "us-west-2"
shared_credentials_file = "~/.aws/credentials"
profile = "prashanth"
}
data "aws_ami" "amazon-linux-2" {
most_recent = true
owners = ["amazon"]
filter {
name = "name"
values = ["amzn2-ami-hvm*"]
}
filter {
name = "architecture"
values = ["x86_64"]
}
}
resource "aws_instance" "test" {
ami = "${data.aws_ami.amazon-linux-2.id}"
associate_public_ip_address = true
instance_type = "t2.micro"
}
view raw create_ec2_machine.tf hosted with ❤ by GitHub
  • Now, initialize terraform
terraform init
  • Compile the terraform file for any issues
terraform plan
  • Finally apply terraform
terraform apply -auto-approve

AWS CLI configuration

prashanthsams Leave a comment
  • Login to your AWS account
  • Download access key ID and secret access key as a file or obtain the keys from the below link:

https://console.aws.amazon.com/iam/home?#/security_credentials

  • Click on Create New Access Key button to generate the security key

  • Open terminal and configure aws cli
aws configure

# if profile applied
aws configure --profile profile_name

  • Now, you can verify the list of aws config profiles
cat ~/.aws/config


  • And you can also verify the list of aws credentials that you’ve set earlier
cat ~/.aws/credentials

  • Verify aws configuration
# view the active profile config
aws configure list

# view specific profile's config
aws configure list --profile default
aws configure list --profile profile_name
  • Switch between the aws profiles
export AWS_PROFILE=default
export AWS_PROFILE=prashanth
export AWS_DEFAULT_PROFILE=prashanth

[verify if needed]
aws configure list
aws s3 ls

Create own Docker images in AWS ECR

prashanthsams Leave a comment

Now, you can register you own custom docker image in AWS ECR instead of hub.docker.com. Secure your docker image through AWS ECR.

  • Install aws cli library 
pip3 install --upgrade awscli
  • Configure aws in your local machine
aws configure

  • After configuration, you can validate these details as seen below
aws configure list

  • Build a Dockerfile to create an image locally
  • Login to remote AWS and create a repository as you do in the GitHub
  • Now, open the terminal and login to AWS ECR from cli
aws ecr get-login --no-include-email --region ap-southeast-1
  • Copy and paste the auto-generated login details
  • Build docker image as normal
docker build -t your-image-name .
  • Create tag for the image you create (here, xxxxxxxxxxxxxx is to be copied from the remote aws ecr repo)
docker tag your-image-name:latest xxxxxxxxxxxxxx.dkr.ecr.ap-southeast-1.amazonaws.com/your-image-name:latest
  • Push it to the remote AWS ECR
docker push xxxxxxxxxxxxxx.dkr.ecr.ap-southeast-1.amazonaws.com/your-image-name:latest

 

Archive/Move Jenkins test artifacts to Amazon S3 bucket

prashanthsams Leave a comment

Wanted to stop losing the test artifacts at the end of every Jenkins regression test? here you go to archive it

  • Open a Jenkins job and add shell from Configure > Add build step > Execute shell 
  • Make sure the below snippet is copied and modified wherever the field is mentioned in RED. Insert the AWS access key and secret key
export AWS_ACCESS_KEY_ID=$your_aws_access_key
export AWS_SECRET_ACCESS_KEY=$your_aws_secret_key
export AWS_DEFAULT_REGION="ap-southeast-1"

DATE=`date '+%Y-%m-%d'`
DATE_W_TIME=`date '+%Y-%m-%d-%H-%M'`
JOB_NAME="you_jenkins_job_name"

# Store the reports in S3
aws s3 cp --recursive reports s3://your_constant_folder_in_s3_bucket/$DATE/$JOB_NAME/$DATE_W_TIME/reports
aws s3 cp --recursive rerun_reports s3://your_constant_folder_in_s3_bucket/$DATE/$TEST_NAME/$DATE_W_TIME/rerun_reports
  • Run the Jenkins job and wait for the reports to be populated
  • Now, you can access the s3 reports in two ways; it can be either through Cyberduck (an application to view the stored files from AWS machine) or through machine terminal
  • Download and install Cyberduck
  • Launch Cyberduck, and select Open Connection > Amazon S3

  • Enter the access key id, secret access key and click connect button

  • Finally, access the reports as shown in the below image

  • To access the reports through the terminal, follow this: