AWS: Solutions Architect Professional (SAP) study notes

Completed this one on a wintry Sydney morning. Despite a lack of sleep, I felt much better prepared vs. DevOps Pro, and it showed in the grade, too:

Overall Score: 75%

Topic Level Scoring:
1.0  High Availability and Business Continuity: 72%
2.0  Costing: 75%
3.0  Deployment Management: 57%
4.0  Network Design: 42%
5.0  Data Storage: 72%
6.0  Security: 85%
7.0  Scalability & Elasticity: 90%
8.0  Cloud Migration & Hybrid Architecture: 85%

  • ASG;
  • CloudFormation, using YAML/JSON for solution architecture:
  • Which Services are supported?;
  • Templates vs Stacks;
  • “Fn::getAtt”: [ “WebServerHost”, “PublicIp”];
  • Chef/Puppet integration OK, or use bootstrapping for application-level dependencies/config;
  • By default, automatic rollback on error (resource provisioning charges still apply);
  • WaitCondition;
  • Resource deletion policy;
  • Resource update policy;
  • IAM Role definition and assignment;
  • VPC creation and customisation (and pretty much everything within a VPC);
  • EIPs and private IPs;
  • Multiple VPCs (for peering) within the same Account only;
  • Create/update Route 53 hosted zones;
  • CloudFront, web/RTMP, geo black-/white-list, SSL (SNI/dedicated), PPPD HTTP methods never cached (proxied to origin), CNAME?, Invalidation API, naked domain alias, dynamic content support (cookie), Origin Access Identity (OAI);
  • CloudHSM, key GSM within a VPC, non fault-tolerant (needs to be clustered), accessible via peering;
  • CloudSearch?;
  • CloudTrail, from multiple Accounts to an S3 bucket;
  • CloudWatch, indefinite by default, 14 days for alarm history;
  • Data Pipeline, use on AWS (EC2/EMR)or on-premise, pipeline container, data node (end dest), activity, precondition, schedule;
  • Direct Connect, 802.1q, can address both public/private environments by relying on an underlying VIV, sub-1/1/10 Gbps dark fibre to AWS vs establishing a VPN, CGW vs VGW, also configure BGP failover, US-only: 1 Direct Connect will work for all Regions;
  • Directory Services, AD Connector (existing), Simple AD (new);
  • DR as a part of BC (eg hardware/software failure, network/power outage, physical damage). Spectrum of options:
  • Backup & Restore: (i) backup (to AWS), (ii) retention policies; and (iii) security measures; eg access policies, encryption.
  • Pilot Light: (i) Pre-configure functional (eg app/web, database) servers as AMIs for various functions; (ii) fire drill; (iii) consider automation (via CloudFormation).
  • Warm Standby: (i) Run apps in an ASG (and/or other infrastructure); and (ii) keep ’em up-to-date (eg patches, config files).
  • Multisite/active-active: (i) Duplicate non-AWS environment; and (ii) Configure weighted routing (Route 53) to route traffic on-premise/AWS environments;
  • DynamoDB (cross-Region replication);
  • EBS point-in-time (eg using the CLI), attached to a single EC2 only;
  • EC2;
  • D (dense storage);
  • I (IOPS);
  • R (RAM);
  • T (t2.micro);
  • M (main choice);
  • C (compute);
  • G (graphics);
  • F (FPGA);
  • P (mining);
  • X (HANA/Spark);
  • ECS?;
  • EFS?;
  • ElasticCache, Memcached (scale out, multi-threaded) vs Redis (scale up, persistency, multi-AZ);
  • Elastic Beanstalk (EB), relatively simple (vs CloudFormation);
  • Applications vs environments;
  • Supported languages:
  • Docker (single-/multi-container);
  • Go 1.6;
  • Java w/ Tomcat;
  • Java SE (7, 8);
  • .NET (IIS 7.5+);
  • Node.js;
  • PHP;
  • Python 2.6+;
  • Ruby 1.9+;
  • Supported AWS services include:
  • CloudWatch;
  • IAM;
  • RDS;
  • S3;
  • VPC (within a Region only);
  • Elastic Transcoder?;
  • ELB, CLB (single/multiple AZs, health checks, associate SGs, SSL offload, sticky sessions, IPv{4,6}, CloudWatch metrics, optional logging to S3, CloudTrail support, layer 4) vs ALB (single AZ, content-/host-/path-based routing, ECS dynamic port integration, HTTP/2, Web Sockets, HA {2 or more AZs}, WAF support, delete protection, X-Amzn-Trace-Id, layer 7), Proxy Protocol;
  • EMR?;
  • ENI?;
  • FSMO role?;
  • Glacier, cheap/slow data archival (3+ hours);
  • HA:
  • MySQL (async. replication);
  • Oracle Database (DataGuard, RAC);
  • SQL Server (AlwaysOn Availability Groups, clustering, mirroring);
  • HTTP Live Streaming (HLS);
  • HPC, Jumbo (Ethernet) Frames via Enhanced Networking (selected HVM instance types), PGs within an AZ;
  • IAM:
  • Cross-Account access, segregation of access for Dev., vs Test via pre-configured inline policy.; ie no need to remember a separate Account ID/username/password, can also be used to store/deploy SSL (in lieu of ACM);
  • IDS/IPS, watch the AlertLogic video;
  • Kinesis Data Streams, real-time data streaming (1-7 days);
  • KMS, to generate signed certificates on demand for a requesting instance;
  • Multicast?;
  • NAT scaling
  • OpsWorks, Chef 11+ deployments on AWS;
  • Stacks, Layers (eg apps, caching, databases, load balancers), and Recipes;
  • ELBs must be separately started up and attached initially, but subsequently are managed via OpsWorks;
  • ELBs and SGs must be separately torn down after layer/stack deletion;
  • Instances may be: 24/7(default), Time-based, and Load-based;
  • Organizations:
  • All Features vs Consolidated Billing: the former merely enables policy-based service controls for Accounts (eg deny EC2 in a bid to encourage Serverless computing);
  • Consolidated Billing; ie a single bill for multiple Accounts, and with volume discounting too (eg EC2 RIs, S3):
  • Alerts can still be individually configured at either level of the hierarchy;
  • CloudTrail must be configured individually, logging to a Cross-Account S3 bucket, though;
  • Promiscuous?;
  • RDS:
  • Multi-AZ (sync., durable) vs RRs (async., scalable);
  • Multi-AZ tech.: AWS (Aurora/MariaDB/MySQL, Oracle Database, PostgresSQL), vs Microsoft (SQL Server mirroring);
  • RRs in another Region OK, except Oracle Database/SQL Server;
  • RRs can also be configured as Multi-AZ;
  • RRs of RRs for MySQL only, and this will increase replica lag;
  • Supports snapshotting to a different Region;
  • Redshift (snapshot to S3, or copy to another Region), WLM;
  • RIs:
  • EC2, reserve within an AZ:
  • On Demand: Unpredictable;
  • Dedicated;
  • Spot: Flexible provisioning, only if the bid price is met only;
  • Reserved (Standard, Convertible, Scheduled): up to 75% discounts for 1-/3-year terms, for steady-state use (eg Production):
  • May be split into multiple instances if the footprint remains the same;
  • Restricted within the same family (eg T2) unless Convertible;
  • Restricted for Linux only, excl. RHEL and SUSE;
  • RDS, reserve within a Region, supports Multi-AZ and RRs (same Region only);
  • Route 53;
  • Routing symmetrically vs asymmetrically (ie round-trip data path);
  • RTO vs RPO;
  • S3 (11 9s durability), can be a VPC endpoint;
  • Scale up (ie vertical), vs out (ie horizontal). Latter is preferred to minimise downtime;
  • SES;
  • SG, cannot setup explicit deny rules (NACLs can);
  • Snowball/Snowmobile, or previously (data) Import/Export;
  • SNS;
  • SQS;
  • Storage Gateway (on-prem {ESXi/Hyper-V} bandwidth-throttled, or as an EC2; also works with Direct Connect):
  • File (NFS), up to 5 TB per file;
  • Tape (iSCSI):
  • Library (S3: instant);
  • Shelf (Glacier: 1d);
  • Volume (iSCSI):
  • Cached (subset only, most frequently used, up to 32 volumes {32 TB ea}; ie 1 PB);
  • Stored (full set, up to 32 volumes {16 TB ea}; ie 512 TB);
  • SR-IOV?;
  • STS, AD-based identity federation for 1-36 hour access (to some resource, eg S3) without having to create new IAM creds, LDAP authentication first (then STS), 4 fields (access key, secret access key, token, duration);
  • SWF?;
  • Tags are key/value pairs attached to resources, usable in Resource Groups;
  • VM Import/Export;
  • VPC tenancy (default vs dedicated) and its impact on EC2 instances;
  • Route table (created by default), subnet to AZ (1:1), private vs public subnets, assign a public IP within a public subnet to make an instance internet-facing (behind an ELB also works), 5 reserved IPs per subnet (.0-.3, .255), CIDR block; “local” route within a VPC, IGW to VPC (1:1), route table (for n subnets), IGW/NAT target (for destination 0.0.0.0/0) route, SGs can span multiple subnets but not the other way around, NAT instance disable source/destination check, VPC peering: use private IPs to address instances within the same Region (50-125 VPCs), 1:1 relationship, private DNS names won’t resolve, routes/SGs/NACLs config required on both ends, multicast vs unicast?;
  • WAF, managed layer 7 sandwich;
Advertisements

AWS: DevOps Pro study notes

So I covered my eyes for a bit when I clicked ‘Finish’ (the test attempt), as this was the toughest exam I’d faced thus far, and I was maybe 70% satisfied with my body of work. Fortuitously, I passed, albeit with an overall score of 65%:-

1.0  Continuous Delivery and Process Automation: 47%
2.0  Monitoring, Metrics, and Logging: 93%
3.0  Security, Governance, and Validation: 75%
4.0  High Availability and Elasticity: 83%

I really need to backtrack, figure out this CI/CD thing, then.

  • ASG (lifecycle hooks {Terminating > Terminating:Wait}, span AZs evenly by default, Launch Configs cannot be edited, suspense AddToLoadBalancer and subsequent manual reg., Termination Policy {Default|OldestInstance});
  • CI tooling (e.g., Jenkins) can perform syntax/build tests;
  • CloudFormation (CreationPolicy {post-config}, ::CustomResource, {RDS} DeletionPolicy=Retain, nested stacks, UpdatePolicy=AutoScalingRollingUpdate);
  • CloudTrail;
  • CloudWatch (dimensions {per-ASG}, retention period, aggregation, Logs {agent}, Log Filters, subscriptions);
  • DynamoDB (cache S3 object metadata);
  • EB (Applications > Versions > Environments, Container Commands {leader-only}, Docker containers, Saved Configs., Swap URLs, .ebextensions);
  • EBS (unencrypted to encrypted, pre-warming);
  • ECS (Dockerrun.aws.json);
  • ElastiCache;
  • Elasticsearch?;
  • IAM (Database Authentication {Aurora|MySQL}, Instance Profile > Role);
  • Kinesis Streams (real time);
  • OpsWorks; i.e., Chef+ (Configure {custom cookbook});
  • RDS (Multi-AZ, Read Replicas, sharding);
  • S3 (key-based naming scheme, store developer’s public keys, MFA Delete);
  • SNS;
  • SQS;
  • WiF (via some IdP {e.g., Google});

#aws

AWS: SysOps Administrator (SOA) study notes

Hello, world. Penned down some keywords after passing my recent AWS SOA exam, and then expanded on ’em below. Perhaps you’ll find ’em useful then.


EBS
RAID 0 (striped) vs. 1 (mirrored); i.e., the lower the number, the higher the risk, see https://www.diffen.com/difference/RAID_0_vs_RAID_1.

Just like EC2 instances, EBS volumes reside in a specific AZ of a Region; i.e., they can only be attached to a running instances within the same AZ. To switch AZs, use snapshots, see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html.

EC2
AMIs can be referred to as being backed by EBS, or ephemeral/instance store. Ephemeral/instance AMIs are stored in S3; i.e., terminating an EC2 instance running the S3-based AMI means that data in the root volume is gone forever, see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ComponentsAMIs.html.

EBS optimized; i.e., minimizing contention between EBS I/O and other traffic from your EC2 instance, see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSOptimized.html.

Cluster-type placement groups: low-latency grouping (of EC2 instances) within a single AZ, see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/placement-groups.html.

RDS
Automated Backups allow users to restore to data within about 5 minutes of the current time, see https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_PIT.html.

S3
TA, as its name suggests, allows users to accelerate file transfers to S3, for when users are underutilizing available Internet bandwidth at upload time, see https://docs.aws.amazon.com/AmazonS3/latest/dev/transfer-acceleration.html.

VPC
Tenancy is typically default (i.e., shared) tenancy. Users cannot change from default to dedicated/host, and vice-versa, see https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/dedicated-instance.html#change-tenancy-vpc.

IPv4 CIDR blocks can range from large (/16 netmask, 65k addresses) to small (/28 netmask, 16 addresses), see https://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Subnets.html.

Direct Connect, use private (virtual interface) to connect to your VPC, public for services that aren’t in a VPC (e.g., Glacier), see https://docs.aws.amazon.com/directconnect/latest/UserGuide/WorkingWithVirtualInterfaces.html.

Within VPCs, there is a “local” route allowing communication between subnets using private IP addresses only, see https://medium.com/@mda590/aws-routing-101-67879d23014d, https://acloud.guru/forums/aws-certified-solutions-architect-professional/discussion/-KGl5vgVKjHuXcpWM0S6/communication_between_subnets.

Windows
Active Directory and AWS, see https://aws.amazon.com/blogs/security/how-to-connect-your-on-premises-active-directory-to-aws-using-ad-connector/, https://docs.aws.amazon.com/directoryservice/latest/admin-guide/ms_ad_tutorial_setup_trust.html.

Windows EC2 instances can be configured using EC2Config (2.2.10+) to export data to CloudWatch, see https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/ec2config-service.html.

#aws

AWS: Analysis of the Certified SysOps Administrator – Associate exam

I recently became fully AWS certified (at the Associate Level), most recently passing the the SysOps Administrator – Associate exam at my third attempt.

  1. In late 2016, I failed with a score of 61% (or  67%). It was my first failure — I deleted the “unsuccessful” notification email in a fit of rage;
  2. In early 2018, I failed with a score of 71%;
  3. In April 2018, I passed with a score of 80%.

My experience was that the Certified SysOps Administrator – Associate exam was the toughest of the lot. In the table below, I compare the weighted scores between attempts #2 and #3 for individual domains:

SysOps Administrator

While I improved in 3 domains, I obtained the same score in the other 4 domains (Monitoring and Metrics, Deployment and Provisioning, Security, & Networking). This gelled with my observation that many of the questions for attempt #3 I’d actually attempted 2 weeks earlier!

Of course, your own mileage may vary, plus you still need to spend time/effort reading through and understanding the AWS FAQs — th9/: are really in-depth, and helped me answer 5 questions correctly; i.e., 9% of 55 ~= 4.95.

In a subsequent post I’ll discuss particular areas that showed up at exam time.

#aws

AWS Certified Associate

After an extended hiatus away from AWS certification, finally I’m certified at the Associate level, 3 times over!

Photo 18-4-18, 1 16 13 PM.png

To date I’ve taken and passed:

  1. Certified Developer – Associate
  2. Certified Solutions Architect – Associate
  3. Certified SysOps Administrator – Associate

AWS has a nice road map, so I just re-purposed the following image off their site:

AWS_Certification_Roadmap_April_2018.d51f56ef22f8d98ad54423c132a976eab2b94abf

I’m told that the Professional level exams are much harder, but that’s a post for another day…

#aws

Docker containers can’t resolve DNS

I’ve recently switched over to using Docker for dev. work on a Windows 10 host, and it’s worked pretty well. Today, apt-get somehow stopped working; e.g.,

...RUN apt-get update && apt-get install...
---> Running in ...
Err:1 http://security.ubuntu.com/ubuntu xenial-security InRelease
Temporary failure resolving 'security.ubuntu.com'
Err:2 http://archive.ubuntu.com/ubuntu xenial InRelease
Temporary failure resolving 'archive.ubuntu.com'
Err:3 http://archive.ubuntu.com/ubuntu xenial-updates InRelease
Temporary failure resolving 'archive.ubuntu.com'
Err:4 http://archive.ubuntu.com/ubuntu xenial-backports InRelease
Temporary failure resolving 'archive.ubuntu.com'
Reading package lists...
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial/InRelease Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial-updates/InRelease Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial-backports/InRelease Temporary failure resolving 'archive.ubuntu.com'
W: Failed to fetch http://security.ubuntu.com/ubuntu/dists/xenial-security/InRelease Temporary failure resolving 'security.ubuntu.com'
W: Some index files failed to download. They have been ignored, or old ones used instead.

This StackOverflow.com post suggests it might be DNS-related, so I changed it from the (default) Google DNS config to Cloudflare’s. Google’s seem to have been (very recently) blocked for whatever reason. Here’s how my Settings -> Network looks like now:

docker-settings-network

Forced choice (VirtualBox, or Docker)

In order to run Docker, it seems I must discontinue my use of VirtualBox. Great (not great); e.g.,

#windows