If you are an avid Amazon Web Services (AWS) consumer and have been thinking about ways to integrate these services into third party-tools (such as CyBot - https://github.com/cylance/CyBot), you have come to the right place. In this article we will cover how to setup and use the aws2 client (AWS CLI) to easily interact with AWS' APIs. Per Amazon: "AWS Command Line Interface (AWS CLI) is an open source tool that enables you to interact with AWS services using commands in your command-line shell." Our example below will illustrate just one possibility by using the Cost Explorer (ce) API determine your current and past AWS bills via the command line or third-party tool.
In other words, we are ditching the Web interface shown below:
 |
| Figure 1: AWS Billing Web Interface |
In favor of querying the information via the command line:
 |
| Figure 2: AWS Billing query via command line |
Or better yet, a ubiquitous tool such as your favorite chat application using CyBot:
 |
| Figure 3: Enabling CyBot to query the AWS API |
- Configuring AWS Permissions
- Policy creation
- Group creation
- User creation
- Installing the aws2 client
- Configuring the aws2 client
- Using the aws2 client
Cost Estimation Requirements
Before we dive into the configuration steps, there are two main requirements:- Access to the following endpoint: https://ce.us-east-1.amazonaws.com
- aws2 client
- A while back, we tried version 1 of the aws client, but it lacked the cost estimator (ce) command, thus we had to upgrade to version 2 (both referenced via the aws command)
Configuring AWS Permissions
Note: This step generates your AWS Access Key ID and AWS Secret Access Key. Copy these down in a safe place because you will need this info in the AWS client setup sectionFirst create and retrieve your AWS credentials via IAM:
https://console.aws.amazon.com/iam/home
Policy Creation
Once, authenticated, create a policy to allow only cost estimation queries (following the principle of least privilege it may be possible to restrict this even more, but that is an exercise left up to the reader):
- Click "Customer Managed Policies"
- Create policy button
- JSON tab, and copy and paste the following:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ce:*"
],
"Resource": [
"*"
]
}
]
}
 |
| Figure 4: Policy Creation via JSON |
Group Creation
Now that the policy exists, create a Cost Estimator Group and assign it the newly created policy by performing the following steps:
- Click "Groups"
- Click Create New Group Button
- Name the group: CostEstimatorAPI
- Attach a policy: CostExplorerPolicy
- Next
- Apply
User Creation
Now that the group exists, create a user and assign it to the newly created group by performing the following steps:
- Click "Users"
- Add user button
- Name the user
- Access type: Programmatic access
- Select the CostEstimatorAPI Group we created
- Click next button
- Click Create user button
 |
| Figure 5: User is created and tied to the group that has cost estimator permissions |
Installing the aws2 Client
We installed the aws client on an Ubuntu Linux VM. If you are using a different operating system, feel free to follow the instructions contained in the following documentation: https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-linux.htmlcurl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
Now check the version, you might see something like the
/usr/local/bin/aws --version
You might see something like the following:
aws-cli/2.0.33 Python/3.7.3 Linux/4.4.0-31-generic botocore/2.0.0dev37
Note: This needs to be version 2.x or higher for the cost explorer functionality to exist.
Configuring the aws2 Client
Note: Have your AWS Access Key ID and AWS Secret Access Key ready from the Setup AWS Permissions sectionaws configure
AWS Access Key ID [None]: AK[redacted]YY
AWS Secret Access Key [None]: vn[redacted][redacted][redacted]Rm
Default region name [None]:
Default output format [None]:
Using the aws2 Client
When using the aws client, you can use the following syntax to get help in trying to figure out how to achieve your goals.
aws help
aws <command> help
aws <command> <subcommand> help
Be sure to also consult the docs for syntax help and example usage. In our case, we are looking at cost explorer functionality:
Specifically for our example, we will look to retrieve the cost and usage information:
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ce/get-cost-and-usage.html
After a bit of trial an error to get the exact format of required parameters, we have the following:
aws ce get-cost-and-usage --time-period Start=2020-07-01,End=2020-08-01 --granularity MONTHLY --metrics "BlendedCost" "UnblendedCost" "UsageQuantity"
Which yields the following output:
{
"ResultsByTime": [
{
"TimePeriod": {
"Start": "2020-07-01",
"End": "2020-08-01"
},
"Total": {
"BlendedCost": {
"Amount": "xx.6272352241",
"Unit": "USD"
},
"UnblendedCost": {
"Amount": "xx.6272352241",
"Unit": "USD"
},
"UsageQuantity": {
"Amount": "xxx.1172177568",
"Unit": "N/A"
}
},
"Groups": [],
"Estimated": true
}
]
}
Two things to note about the time period parameter (--time-period Start=2020-07-01,End=2020-08-01)
- It would be ideal if the aws client accepted terms such as: "This year", "this month", "this week", or "today", but now that this is integrated into our third party tool, we can create that logic.
- But at least the client allows you to select a date in the future (ex: The first of next month) for calculating the present time period.
Conclusion
We hope this article serves as a quick introduction and jumpstart to your interactions with the AWS API using the freely available client. While not perfect, this client is quite powerful and can be used to quickly interact with the Amazon Web Services API to build integrations into third-party tools. Feel free to leave any comments in the section below. Happy Hacking.
References
These are references which we found to be useful:
- https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-linux.html
- https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ce/index.html
- https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ce/index.html#cli-aws-ce
- https://awscli.amazonaws.com/v2/documentation/api/latest/reference/ce/get-cost-and-usage.html
