Oct 5, 2017 3:00 AM

Awless tutorial: Try a smarter CLI for AWS

A guided tour of the Awless CLI for Amazon Web Services, an alternative to the AWS CLI with simpler syntax, smart defaults, and powerful templates


When the cloud was just about virtual machines, tools like Chef or Puppet helped us easily prepare our VMs. The only thing that mattered was to provision instances that contained all of the code and data required. But now that Amazon Web Services has ballooned to more than 90 services, interacting with the AWS API becomes the major part of the work.

How should we manage AWS infrastructure, and what interfaces should we use? Most beginners start with AWS Console, the default GUI, while seasoned sysadmins generally prefer a command-line interface (CLI). Problem is, the AWS CLI is not user friendly. Because it integrates the entire AWS API, it exposes an enormous surface area in terms of commands, flags, and options.

Awless is born from our need for a fast, powerful, and easy-to-use CLI to manage AWS. With Awless, you can create and run an AWS infrastructure, starting from scratch, and always get readable output (for both humans and programs), explore and query all cloud resources (even offline), connect to instances, and create, update, and delete cloud resources. Beyond single command lines, Awless supports templates that enable higher levels of automation. Last, but not least, Awless aims to ensure smart defaults and security best practices.

Because there are so many AWS services, it is often important to find and display a hierarchy of services from the command line. We can group services by functionality—such as compute and database. But going through each of them exhaustively is tedious as there are, as of this writing, no fewer than 15 services around storage and database, not counting four data migration services and nine analytics services that are directly related to data usage.

We find it easier to group services by cloud readiness. In this article, we will detail how to use Awless to create and manage cloud resources for a real use case, the deployment of production-ready WordPress instances. We will use the following AWS resources:

  1. VM services EC2 (Elastic Compute Cloud) and ELB (Elastic Load Balancing);
  2. High-level services that run in VMs but are managed by AWS, such as RDS (Relational Database Service) or ElastiCache (for queues);
  3. “Serverless” services that run in multi-tenant VMs, such as S3 (object storage) or Lambda (single function execution).

Cloud readiness levels start with VMs and extend to managed services and serverless services. We will use all layers in this article.

Get started with Awless

Sign up for AWS and create a first account with AdministratorAccess rights. Carefully note your access key and secret key.

Install Awless

Awless is available at GitHub. We provide pre-built binaries and Homebrew packages for MacOS:

>brew tap wallix/awless 
>brew install awless

You can check that Awless is properly installed by running:

>awless version

Awless is modeled after popular command-line tools such as Git. Most commands are in the form of:

>awless verb [entity] [parameter=value ...]

This article will give a 360-degree overview of real production workloads on AWS, starting from scratch. For clarity, we omit all confirmation and some output steps, as Awless always asks to confirm commands that create, update, or delete resources.

First steps with Awless

We can issue our first Awless command by listing our Virtual Private Clouds (VPCs). Because this is our first run, we will need to input some necessary data to configure Awless:

>awless list vpcs 
Welcome to awless! Resolving environment data...

Please choose an AWS region:
ap-northeast-1, ap-northeast-2, ap-south-1, ap-southeast-1, ap-southeast-2, ca-central-1, cn-north-1, eu-central-1, eu-west-1, eu-west-2, sa-east-1, us-east-1, us-east-2, us-gov-west-1, us-west-1, us-west-2
Value ? > us-west-2

Syncing region ‘us-west-2’...

Cannot resolve AWS credentials (AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) Please enter access keys and choose a profile name (stored at /Users/john/.aws/credentials):
AWS Secret Access Key? hYWZBVOusePEPSr5PkscplskB84fjbgUEXAMPLE
Choose a profile name? admin
✓ /Users/john/.aws/credentials created
✓ Credentials for profile ‘admin’ stored successfully

All done. Enjoy!

You can review and configure awless with `awless config`.

Now running: awless list vpcs
|     ID ▲     | NAME | DEFAULT |   STATE   |     CIDR      |
| vpc-1d1df679 |      | true    | available | |

Create an AWS user

We will now use Awless to create a new AWS user and give him sufficient rights using the admin profile. We create the user John and his access key:

>awless create user name=john 
>awless create accesskey user=john aws_access_key_id = AKIAIOSFODNN7EXAMPLE
Do you want to save in your .aws/credentials? (y/n) y
Entry name in .aws/credentials? [default] john

Now that John exists, he needs a set of permissions. We will give John full access to the EC2, RDS, Auto Scaling, CloudFront, and S3 services that we will use in this article:

>awless attach policy service=ec2 access=full user=john 
>awless attach policy service=rds access=full user=john
>awless attach policy service=s3 access=full user=john
>awless attach policy service=autoscaling access=full user=john
>awless attach policy service=cloudfront access=full user=john

Now that John is a fully functional user, we’ll switch to his profile for the next steps:

>awless config set aws.profile john

We will use AWS to set up a highly available, managed WordPress deployment, combining VMs, managed, and serverless services. Our main goal is pictured below. We will have to address three “devops challenges” to reach it, making use of AWS infrastructure services, managed services, and serverless services, respectively.


We will deploy a production WordPress blogging application that relies on managed services and serverless improvements.

Challenge 1: Lift and shift an application to EC2

Lift and shift is the quickest to migrate legacy applications to the cloud and benefit from the flexibility and cost advantages of cloud platforms. In this case, we will start by deploying a WordPress engine and its database in a single VM. Clients will connect directly to the VM.


The first step is deploying our blog application in a single EC2 instance.

Create a VPC

Before we proceed with the VM creation, we first need to create network resources:

  • A private network (or VPC)
  • An Internet gateway for this VPC
  • A subnet using the Internet gateway

Awless will prompt for any missing parameters with autocompletion. Here we use a mix of both provided (param=value) and prompted parameters:

>awless create vpc cidr= name=wordpress-vpc 
>awless create internetgateway
[OK] id=igw-1234567
>awless attach internetgateway
Please specify (Ctrl+C to quit, Tab for completion):
internetgateway.id? [Tab]
internetgateway.id? igw-1234567
internetgateway.vpc? @wo[Tab]
internetgateway.vpc? @wordpress-vpc

Awless puts forward the best practice to use names rather than resource IDs. As such, @resource-name is the identifier of the resource named “resource-name.”

Let’s create a public subnet to host our WordPress instance, and attach a route table that routes the Internet traffic to the VPC’s Internet gateway:

>awless create subnet cidr= vpc=@wordpress-vpc name=wordpress-public-subnet 
>awless update subnet id=@wordpress-public-subnet public=true

>awless create routetable vpc=@wordpress-vpc
>awless attach routetable subnet=@wordpress-public-subnet
        Please specify (Ctrl+C to quit, Tab for completion):
        *select the ID of the routetable you created above*

>awless create route cidr=
        Please specify (Ctrl+C to quit, Tab for completion):
route.gateway? *the ID of the internet gateway you attached to the VPC above*
route.table? *the ID of the routetable you created above*

Note that each action in Awless is about as simple as it can get. Although we follow a comprehensive step-by-step approach, Awless allows us to get through the tedious process of setting up an infrastructure much faster than with the graphical console or the AWS CLI.

Create an SSH keypair and a security group

The cloud network is now ready. Before creating the instance, we need an SSH key pair, to connect to the instance later. In a single command, Awless generates an SSH key pair locally and registers it on AWS:

>awless create keypair name=johnkey

A best practice is to give minimal access to any resource, so we will only accept HTTP connections from all the Internet and SSH from our outgoing IP address. To do that, we create and configure a security group:

>awless create securitygroup vpc=@wordpress-vpc description=\”HTTP public + SSH access\” name=wordpress-secgroup 
>MY_IP=$(awless whoami —ip-only)
>awless update securitygroup id=@wordpress-secgroup inbound=authorize cidr=$MY_IP/32 portrange=22
>awless update securitygroup id=@wordpress-secgroup inbound=authorize cidr= portrange=80

Provision the application with AWS user data

We will now provision our WordPress instance through AWS user data. Here we will use the script available on GitHub:

>awless create instance subnet=@wordpress-public-subnet keypair=johnkey name=wordpress-instance userdata=https://raw.githubusercontent.com/zn3zman/AWS-WordPress-Creation/16a52aef4f618d558d61197df4e4eca4c015277f/WP-Setup.sh securitygroup=@wordpress-secgroup

You can use awless show to get information about any resource, such as the public IP address of our WordPress instance:

>awless show wordpress-instance

You can connect to the IP address from the command output to access your WordPress service (though you might have to wait a few minutes for the instance to be provisioned properly).

WordPress Foundation

Our initial launch of WordPress.

By default, Awless will create a type t2.micro (1 vCPU, 1GB RAM) using Amazon Linux. You can update default values by using awless config set:

>awless config set instance.type m4.large 
>UBUNTU_AMI=$(awless search images canonical:ubuntu —id-only —silent)
>awless config set instance.image $UBUNTU_AMI

To this point, we have built several resources. Using awless list, we can list users, instances, subnets, and all other types of resources (provided your AWS profile has sufficient rights of course). For example, we can list instances:

>awless list instances 
|       ID ▲        |   ZONE   |        NAME        | UPTIME  |
|i-00268db26b0d0393c|us-west-1c| wordpress-instance | 57 mins |

Awless provides a powerful feature that enables easy connections to instances with SSH. Behind the scenes, Awless will automatically get the instance IP address, guess the username, and connect with the keypair we created earlier:

>awless ssh wordpress-instance

If you want to delete the WordPress instance, you can run awless delete instance id=@wordpress-instance. You can do it now, as we will create a more advanced deployment in the next challenge.

How to use Awless templates

All the steps in this challenge can be described as a sequence of Awless commands, where the results of previous commands (for instance, the ID of the Internet gateway) are used as inputs to subsequent commands. Because Awless provides a built-in templating system, you could encapsulate all of Challenge 1 in a template and run it with:

>awless run https://raw.githubusercontent.com/wallix/awless-templates/bcd0dd41b1524eeac1e53d12b2998bc56689c517/simple_wordpress_infra.aws

Awless offers a powerful feature that enables you to revert most changes applied to an AWS infrastructure. For instance, you can delete the whole infrastructure created by a template in a single command: awless revert revert-id. To find a given revert-id, awless log lists all of the commands previously applied to the cloud infrastructure, with both their output and their ID:

>awless log # find the ID to revert >awless revert 01BM6D1YRZ5SSN5Z09VEEGN0HV

Challenge 2: Use AWS managed services

Our previous deployment is functional, but pretty artisanal. Our blog is powered by a single instance in a single Availability Zone (AZ). We now want to build a highly available blog, with a load balancer, two instances in different AZs, and a replicated database that is shared by our instances. Instead of running our own database in an instance, we will use AWS RDS, Amazon’s managed service for SQL databases. Using a managed service provides many advantages including clustering, managed security, and backups.


Our goal for Challenge 2 is to set up a highly available infrastructure with a managed and replicated database.

In order to have highly available resources, we need to distribute them in subnets in different availability zones (AZs) and balance the load through Elastic Load Balancing.


We want our WordPress instances to be able to fail gracefully (one at a time).

For this challenge, we will create the following:

  • One load balancer to distribute the load between the instances
  • Two public subnets to associate with the Internet-facing load balancer
  • Two private subnets in different AZs (e.g. us-east-1a, us-east-1e) to host the instances
  • One auto scaling group to manage the scaling of WordPress instances
  • One NAT gateway in one public subnet to enable outbound calls for instances provisioning
  • One public fixed IP (Elastic IP) for the NAT gateway
  • One RDS for MariaDB instance automatically replicated in the private subnets

We will build this infrastructure by running Awless templates. The first template creates subnets and routing. The {hole} notation allows parameters to be filled dynamically during the running of the template. The $reference notation enables back references of created resources.

Use an Awless template to create subnets and routing

Create a text file called subnets_public_and_private_with_NAT.aws with the following content:

# Filename: “subnets_public_and_private_with_NAT.aws” 
# 2 public subnets in different AZs attached to internet gateway
pubSub1 = create subnet cidr= vpc=@wordpress-vpc name=wordpress-public-subnet-1 availabilityzone={az1}
update subnet id=$pubSub1 public=true
pubSub2 = create subnet cidr= vpc=@wordpress-vpc name=wordpress-public-subnet-2 availabilityzone={az2}
update subnet id=$pubSub2 public=true rt = create routetable vpc=@wordpress-vpc
create route table=$rt cidr= gateway={vpc.internetgateway}
attach routetable id=$rt subnet=$pubSub1
attach routetable id=$rt subnet=$pubSub2

# 2 private subnets in different AZs
privSub1 = create subnet cidr= vpc=@wordpress-vpc name=wordpress-private-subnet-1 availabilityzone={az1}
privSub2 = create subnet cidr= vpc=@wordpress-vpc name=wordpress-private-subnet-2 availabilityzone={az2}  

# NAT Gateway in public subnet with a fixed IP
ip = create elasticip
natgw = create natgateway elasticip-id=$ip subnet=@wordpress-public-subnet
check natgateway id=$natgw state=available timeout=180  

# Routing between private subnets and NAT gateway
natgw_rtable = create routetable vpc=@wordpress-vpc
attach routetable id=$natgw_rtable subnet=$privSub1
attach routetable id=$natgw_rtable subnet=$privSub2
create route cidr= gateway=$natgw table=$natgw_rtable

Run this template with the awless run command, providing the file path and the required parameters on the command line. If you don’t know the list of available AZs, Awless can list them for the current region:

>awless list availabilityzones 
|   NAME ▲   |   STATE   |  REGION   | MESSAGES |
| us-east-1a | available | us-east-1 |          |
| us-east-1b | available | us-east-1 |          |
>awless run subnets_public_and_private_with_NAT.aws az1=us-east-1a az2=us-east-1e

Use an Awless template to configure load balancing

Now that the network is set up, we will use a second template to configure the load balancer:

# Filename: “provision-lb.aws” 
# Create the load balancer security group
lbsecgroup = create securitygroup vpc=@wordpress-vpc description="authorize HTTP from the internet" name=wordpress-lb-securitygroup
update securitygroup id=$lbsecgroup inbound=authorize protocol=tcp cidr= portrange=80
# Provision the load balancer listening in the public subnets, with its target group and HTTP listener
tg = create targetgroup name=wordpress-workers port=80 protocol=HTTP vpc=@wordpress-vpc
update targetgroup id=$tg stickiness=true
lb = create loadbalancer name=wordpress-loadbalancer subnets={public.subnets} securitygroups=$lbsecgroup
create listener actiontype=forward loadbalancer=$lb port=80 protocol=HTTP targetgroup=$tg

We just need to grab the IDs of the public subnet and run the template:

>PUBLIC_SUBNETS=$(awless show wordpress-public-subnet-1 —values-for ID),$(awless show wordpress-public-subnet-2 —values-for ID) >awless run provision-lb.aws public.subnets=$PUBLIC_SUBNETS

Use an Awless template to set up the database

Now that our load balancer is set up, we create the RDS instance in a third template:

# Filename: “rds_database.aws” 
# Database security group (authorize access from private subnets to port 3306)
dbsec = create securitygroup name=wordpress-database-securitygroup vpc=@wordpress-vpc description="authorize mysql port 3306 from private subnets"
update securitygroup id=$dbsec inbound=authorize cidr= portrange=3306
update securitygroup id=$dbsec inbound=authorize cidr= portrange=3306
# Distribute the MariaDB database in the 2 subnets
dbsub = create dbsubnetgroup name=wordpress-dbsubnets description="subnets for wordpress databases" subnets={private.subnets}
create database engine=mariadb id=wordpress-database password={dbpassword} size=8 type=db.t2.micro username=wordpress dbname=wordpress multiaz=true subnetgroup=$dbsub vpcsecuritygroups=$dbsec

Run this template with:

>PRIVATE_SUBNETS=$(awless show wordpress-private-subnet-1 —values-for ID),$(awless show wordpress-private-subnet-2 —values-for ID) 
>awless run rds_database.aws private.subnets=$PRIVATE_SUBNETS dbpassword=mysqlpassword

The provisioning of the database may take a few minutes before it is fully usable. We will need to wait until the database is available:

>awless check —force database id=wordpress-database state=available timeout=600

The —force flag disables the confirmation step before executing the command. You can now grab a cup of coffee as the database provisioning may take more than 10 minutes in some regions.

Use an Awless template to enable Auto Scaling

We will use an AWS Auto Scaling group to ensure that there are at least two WordPress instances running. The following template creates the Auto Scaling group and provisions the instances, reusing best practices we already saw in Challenge 1 (security groups, user data provisioning, etc.).

# wordpress.aws : 
# Create the security group and launch configuration for the instances
instSecGroup = create securitygroup vpc=@wordpress-vpc description="HTTP + SSH within VPC" name=wordpress-private-secgroup
update securitygroup id=$instSecGroup inbound=authorize cidr= portrange=22
update securitygroup id=$instSecGroup inbound=authorize cidr= portrange=80
launchconf = create launchconfiguration image={instance.image} keypair={keypair} name=wordpress-launch-configuration type=t2.micro userdata=https://raw.githubusercontent.com/wallix/awless-templates/bcd0dd41b1524eeac1e53d12b2998bc56689c517/userdata/wordpress.sh securitygroups=$instSecGroup
create scalinggroup desired-capacity=2 launchconfiguration=$launchconf max-size=2 min-size=2 name=wordpress-scalinggroup subnets={private.subnets} targetgroups=@wordpress-workers

We get the database host name and run the template:

>DBHOST=$(awless show wordpress-database —values-for PublicDNS) 
>awless run wordpress.aws private.subnets=$PRIVATE_SUBNETS keypair=johnkey dbname=wordpress dbhost=$DBHOST dbuser=wordpress dbpassword=mysqlpassword

To connect to our domain, we wait until the load balancer is active to get its domain name:

>awless check loadbalancer id=@wordpress-loadbalancer state=active timeout=180 
>awless show wordpress-loadbalancer —values-for PublicDNS

When accessing the URL, you might receive a 50X error at first (“The instances are provisioning”). Refresh periodically until success.

Well done! You are running a secure, highly available WordPress deployment with a replicated and managed database, ready for production use (provided you also set up HTTPS).

Challenge 3: Use AWS serverless services

Serverless services are multi-tenant instances operated by Amazon. There is no allocated instance for the AWS customer and no low-level access. As a result, the cost of these services is much lower than dedicated alternatives and proportional to real usage. These services—S3, CloudFront, Lambda, DynamoDB, etc.—represent a new way of architecting software.

Let’s optimize our WordPress deployment with serverless services. Our database is managed and can grow easily, but serving static resources through an instance and a database is an inefficient use of resources. Furthermore, it would require users from all around the world to query a single region that might be slow to reach.

It is vastly more efficient to use a dedicated service, together with a content delivery network (CDN), to host static resources. Amazon’s object storage service, S3, allows you to store files in buckets and make them available through direct HTTP access. An additional CDN service, CloudFront, allows you to distribute and cache the content in 70 locations worldwide. Naturally, CloudFront is pluggable in front of S3.

Before we begin, let’s remove the resources we created for Challenge 2 that we no longer need, using awless revert. We can use awless log to locate the IDs of the last two template executions, which created the RDS database and provisioned the EC2 instances:

>awless revert 01BM9DBTH1Q1579HXHVF377GF7 # Revert of wordpress.aws
>awless revert 01BM9CYN3TAYGK5FYQMHWCH83Y # Revert of rds_database.aws

Note that you can do the two reverts in parallel in two terminals—a handy option as the database deletion may take a while.

The infrastructure for Challenge 3 is slightly different than for Challenge 2. Note just a few changes:

  • A bucket on S3 to store the WordPress files
  • A CloudFront CDN distribution to deliver the objects stored in the S3 bucket
  • A role enabling the WordPress instances to write to the bucket
  • A plug-in for WordPress that uploads static resources to S3

Create an S3 bucket and CloudFront distribution

Using Awless, applying all of these changes is straightforward. Generally, one line per resource is all that is needed:

>awless create bucket name=awless-wordpress-bucket 
>awless create distribution origin-domain=awless-wordpress-bucket.s3.amazonaws.com

While waiting for the CloudFront distribution to propagate (awless check distribution state=Deployed timeout=600), let’s deploy a new database using the template from Challenge 2:

>PRIVATE_SUBNETS=$(awless show wordpress-private-subnet-1 —values-for ID),$(awless show wordpress-private-subnet-2 —values-for ID) 
>awless run rds_database.aws private.subnets=$PRIVATE_SUBNETS dbpassword=mysqlpassword

Assign an IAM role to EC2 instances

The WordPress instances need to write to the S3 bucket. An AWS best practice recommends assigning an IAM role to EC2 resources to get privileges. However, our user John has insufficient permissions to assign a role to the instances. We use -p admin to temporarily use the admin profile to grant John sufficient permissions:

>awless -p admin create policy name=AllowCreateAttachPolicy effect=Allow resource=\* description=\”Authorize the creation of policies\” action = iam:CreatePolicy, iam:DeletePolicy, iam:AttachRolePolicy, iam:DetachRolePolicy, iam:CreateRole, iam:DeleteRole, iam:CreateInstanceProfile, iam:DeleteInstanceProfile, iam:AddRoleToInstanceProfile, iam:RemoveRoleFromInstanceProfile, iam:PassRole, iam:ListInstanceProfiles 
>POLICY_ARN=$(awless -p admin show AllowCreateAttachPolicy —values-for arn)
>awless -p admin attach policy arn=$POLICY_ARN user=john

Use an Awless template to deploy our application

The template for the production-ready deployment of WordPress with S3 is available from a public gist. We will grab the required parameters and run the template:

>DBHOST=$(awless show wordpress-database —values-for PublicDNS) 
>PRIVATE_SUBNETS=$(awless show wordpress-private-subnet-1 —values-for ID),$(awless show wordpress-private-subnet-2 —values-for ID)
>URL=$(awless show wordpress-loadbalancer —values-for PublicDNS)
>CLOUDFRONT_URL=$(awless list distributions —filter origins=wordpress —sort modified —format tsv |tail -n1 |cut -f2)
>awless run https://gist.githubusercontent.com/fxaguessy/c09ad9d9452afb5e1885e7963fca7987/raw/5d55aac230d88c11ff9b44cfcf1f2feb13ce6c33/wordpress-with-S3-plugin.aws dbhost=$DBHOST private.subnets=$PRIVATE_SUBNETS wordpress.url=$URL cdn.URL=$CLOUDFRONT_URL

Please specify (Ctrl+C to quit, Tab for completion):
dbpassword? mysqlpassword
keypair? johnkey
wordpress.email? test@test.com
wordpress.password? wordpresspassword

Congratulations, your highly available blog application is now ready to publish worldwide and draw visitors around the globe, while efficiently keeping up with the load. If you’re willing to go beyond the already advanced concepts that we applied in this tutorial, I’ll leave you with two more tasks to try on your own.

  1. If you own a domain name, you can easily use Awless to set up a DNS entry with AWS Route53. Try awless create record —help to learn about the creation of DNS records.
  2. Create CloudWatch alarms to automatically increase the number of instances in Auto Scaling groups. Hint: You just have to follow the template detailed here.

Our hands-on tour shows how Awless simplifies IT operations in the AWS cloud. The project has gained traction quickly, collecting more than 3,000 stars on GitHub in only a few months of existence. You can learn more about the project on GitHub and in my introductory blog post on Medium. Awless already supports more AWS services beyond those featured in this quick overview, as well as more features for each service.

The project exists thanks to sponsorship by Wallix, a publicly traded cybersecurity software vendor, and is fully open source under the liberal Apache license. As such, Awless is open to contributions from the community, be it in the form of code contributions, documentation improvements, or just spreading the word.

New Tech Forum provides a venue to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to newtechforum@infoworld.com.