So you are creating EC2 instances from the AWS CLI and\or Python using BOTO 3 and you want to get the InstanceId afterwards.  Below are the method(s) I use in each scenario.

Create 1 EC2 Instance with AWS CLI:

Example Command:

aws ec2 run-instances --profile <value> --image-id <value> --security-group-ids <value> --count 1 --instance-type <value> --subnet-id <value> --query 'Instances[0].InstanceID'

Create multiple EC2 Instances with Python\Boto3:

Example Command:

ec2_session_client = session.client('ec2')
response = ec2_session_client.run_instances(ImageID="value",SecurityGroupIds=["value"],MaxCount=value,MinCount=value,InstanceType="value",SubnetId="value")
for instance in response["Instances"]:
   if 'InstanceId' in instance"
      print (instance['InstanceId'])

So these are just a couple ways to grab the InstanceId for use later in your script.

This free Open Source tool, Cloud Custodian, is an interesting program that can be used help manage your AWS environment(s), ensuring compliance via policies written using YAML.

Thinking about this more, it seems like you could execute the polices via Lambda, or even a local Jenkins instance via a reoccurring schedule.

I would suggest storing your YAML policy files in Git and pulling them from there as needed.

Useful Links:

• Python Home Instance – https://pypi.python.org/pypi/c7n
• Capital One Custodian Home – https://developer.capitalone.com/opensource-projects/cloud-custodian/
• Capital One Custodian Docs – http://www.capitalone.io/cloud-custodian/docs/overview/index.html
• Git Location – https://github.com/capitalone/cloud-custodian/blob/master/README.rst

I listened to a Tech Talk on AWS S3 recently.  They covered some high-level stuff, and then some low-level stuff.

High-Level (S3 Storage Classes)

• Amazon S3 Standard – Active Data (S)
• Amazon S3 Standard-Infrequent Access Data (SIA)
• Amazon Glacier – Archive Data

Low-Level (Storage Class Analysis)

• Use storage class analysis to determine when to move from (S) to (SIA)
• Use to create a life cycle transition policy based on the data
• Allows you to deep dive into buckets, tags, prefix, etc…
• Allows you to measure object age

I love the AWS CLI –query option.  It allows you to pull out the relevant data you are looking for and display it in a nice table format.

There are a couple different ways to determine what the correct query parameters are.

#1 – Run your command and parse through the JSON that is returned.  The JSON will show you exactly what element(s) you can query on.

Example: aws ec2 describe-instances –query ‘Reservations[*].Instances[*].[InstanceId, Monitoring.State]’ –output table

#2 – Review the AWS CLI documentation. (e.g. describe-instance-status)  On this page you will want to examine the “Output” section, which gives you the same information as #1, but with a different view.

Example: aws describe-instance-status –query ‘InstanceStatuses[*].AvailabilityZone’ –output table

Overall using the query option is pretty easy and at the same time pretty powerful!

Traditional REST API Setup in AWS

• Multiple EC2 instances
• Part of an Auto Scaling group
• Setup with an Elastic Load Balancer
• Code is manually deployed with your tool of choice (e.g. Ansible)
• CloudWatch for monitoring

Serverless REST API Setup in AWS

• Amazon API Gateway (front door)
• API Gateway handles monitoring, access, and authorization
• AWS Lamda is configured as the back-end (Pay compute time only)
• No EC2 servers to manage, AWS handles it
• Manually setup or extensive SDK scripting needed

Chalice Serverless REST API Setup in AWS

• Chalice is a Python package with syntax similar to Flask.
• Main Chalice components
  • App object, routes, and app.py file
• Chalice allows you to “quicky” deploy your Python API via the Chalice CLI to AWS
• Auto generated IAM policy
• API Gateway and Lamda are used “behind” the scenes and automatically configured