Moved to GitHub Pages

Going forward, I’ll no longer be using WordPress. Instead, I’ve moved on to GitHub Pages, served up using my own custom domain name.

I go into a bit more detail in this post.

AWS: Solutions Architect Professional (SAP) study notes

Completed this one on a wintry Sydney morning. Despite a lack of sleep, I felt much better prepared vs. DevOps Pro, and it showed in the grade, too:

Overall Score: 75%

Topic Level Scoring:
1.0  High Availability and Business Continuity: 72%
2.0  Costing: 75%
3.0  Deployment Management: 57%
4.0  Network Design: 42%
5.0  Data Storage: 72%
6.0  Security: 85%
7.0  Scalability & Elasticity: 90%
8.0  Cloud Migration & Hybrid Architecture: 85%

  • ASG;
  • CloudFormation, using YAML/JSON for solution architecture:
  • Which Services are supported?;
  • Templates vs Stacks;
  • “Fn::getAtt”: [ “WebServerHost”, “PublicIp”];
  • Chef/Puppet integration OK, or use bootstrapping for application-level dependencies/config;
  • By default, automatic rollback on error (resource provisioning charges still apply);
  • WaitCondition;
  • Resource deletion policy;
  • Resource update policy;
  • IAM Role definition and assignment;
  • VPC creation and customisation (and pretty much everything within a VPC);
  • EIPs and private IPs;
  • Multiple VPCs (for peering) within the same Account only;
  • Create/update Route 53 hosted zones;
  • CloudFront, web/RTMP, geo black-/white-list, SSL (SNI/dedicated), PPPD HTTP methods never cached (proxied to origin), CNAME?, Invalidation API, naked domain alias, dynamic content support (cookie), Origin Access Identity (OAI);
  • CloudHSM, key GSM within a VPC, non fault-tolerant (needs to be clustered), accessible via peering;
  • CloudSearch?;
  • CloudTrail, from multiple Accounts to an S3 bucket;
  • CloudWatch, indefinite by default, 14 days for alarm history;
  • Data Pipeline, use on AWS (EC2/EMR)or on-premise, pipeline container, data node (end dest), activity, precondition, schedule;
  • Direct Connect, 802.1q, can address both public/private environments by relying on an underlying VIV, sub-1/1/10 Gbps dark fibre to AWS vs establishing a VPN, CGW vs VGW, also configure BGP failover, US-only: 1 Direct Connect will work for all Regions;
  • Directory Services, AD Connector (existing), Simple AD (new);
  • DR as a part of BC (eg hardware/software failure, network/power outage, physical damage). Spectrum of options:
  • Backup & Restore: (i) backup (to AWS), (ii) retention policies; and (iii) security measures; eg access policies, encryption.
  • Pilot Light: (i) Pre-configure functional (eg app/web, database) servers as AMIs for various functions; (ii) fire drill; (iii) consider automation (via CloudFormation).
  • Warm Standby: (i) Run apps in an ASG (and/or other infrastructure); and (ii) keep ’em up-to-date (eg patches, config files).
  • Multisite/active-active: (i) Duplicate non-AWS environment; and (ii) Configure weighted routing (Route 53) to route traffic on-premise/AWS environments;
  • DynamoDB (cross-Region replication);
  • EBS point-in-time (eg using the CLI), attached to a single EC2 only;
  • EC2;
  • D (dense storage);
  • I (IOPS);
  • R (RAM);
  • T (t2.micro);
  • M (main choice);
  • C (compute);
  • G (graphics);
  • F (FPGA);
  • P (mining);
  • X (HANA/Spark);
  • ECS?;
  • EFS?;
  • ElasticCache, Memcached (scale out, multi-threaded) vs Redis (scale up, persistency, multi-AZ);
  • Elastic Beanstalk (EB), relatively simple (vs CloudFormation);
  • Applications vs environments;
  • Supported languages:
  • Docker (single-/multi-container);
  • Go 1.6;
  • Java w/ Tomcat;
  • Java SE (7, 8);
  • .NET (IIS 7.5+);
  • Node.js;
  • PHP;
  • Python 2.6+;
  • Ruby 1.9+;
  • Supported AWS services include:
  • CloudWatch;
  • IAM;
  • RDS;
  • S3;
  • VPC (within a Region only);
  • Elastic Transcoder?;
  • ELB, CLB (single/multiple AZs, health checks, associate SGs, SSL offload, sticky sessions, IPv{4,6}, CloudWatch metrics, optional logging to S3, CloudTrail support, layer 4) vs ALB (single AZ, content-/host-/path-based routing, ECS dynamic port integration, HTTP/2, Web Sockets, HA {2 or more AZs}, WAF support, delete protection, X-Amzn-Trace-Id, layer 7), Proxy Protocol;
  • EMR?;
  • ENI?;
  • FSMO role?;
  • Glacier, cheap/slow data archival (3+ hours);
  • HA:
  • MySQL (async. replication);
  • Oracle Database (DataGuard, RAC);
  • SQL Server (AlwaysOn Availability Groups, clustering, mirroring);
  • HTTP Live Streaming (HLS);
  • HPC, Jumbo (Ethernet) Frames via Enhanced Networking (selected HVM instance types), PGs within an AZ;
  • IAM:
  • Cross-Account access, segregation of access for Dev., vs Test via pre-configured inline policy.; ie no need to remember a separate Account ID/username/password, can also be used to store/deploy SSL (in lieu of ACM);
  • IDS/IPS, watch the AlertLogic video;
  • Kinesis Data Streams, real-time data streaming (1-7 days);
  • KMS, to generate signed certificates on demand for a requesting instance;
  • Multicast?;
  • NAT scaling
  • OpsWorks, Chef 11+ deployments on AWS;
  • Stacks, Layers (eg apps, caching, databases, load balancers), and Recipes;
  • ELBs must be separately started up and attached initially, but subsequently are managed via OpsWorks;
  • ELBs and SGs must be separately torn down after layer/stack deletion;
  • Instances may be: 24/7(default), Time-based, and Load-based;
  • Organizations:
  • All Features vs Consolidated Billing: the former merely enables policy-based service controls for Accounts (eg deny EC2 in a bid to encourage Serverless computing);
  • Consolidated Billing; ie a single bill for multiple Accounts, and with volume discounting too (eg EC2 RIs, S3):
  • Alerts can still be individually configured at either level of the hierarchy;
  • CloudTrail must be configured individually, logging to a Cross-Account S3 bucket, though;
  • Promiscuous?;
  • RDS:
  • Multi-AZ (sync., durable) vs RRs (async., scalable);
  • Multi-AZ tech.: AWS (Aurora/MariaDB/MySQL, Oracle Database, PostgresSQL), vs Microsoft (SQL Server mirroring);
  • RRs in another Region OK, except Oracle Database/SQL Server;
  • RRs can also be configured as Multi-AZ;
  • RRs of RRs for MySQL only, and this will increase replica lag;
  • Supports snapshotting to a different Region;
  • Redshift (snapshot to S3, or copy to another Region), WLM;
  • RIs:
  • EC2, reserve within an AZ:
  • On Demand: Unpredictable;
  • Dedicated;
  • Spot: Flexible provisioning, only if the bid price is met only;
  • Reserved (Standard, Convertible, Scheduled): up to 75% discounts for 1-/3-year terms, for steady-state use (eg Production):
  • May be split into multiple instances if the footprint remains the same;
  • Restricted within the same family (eg T2) unless Convertible;
  • Restricted for Linux only, excl. RHEL and SUSE;
  • RDS, reserve within a Region, supports Multi-AZ and RRs (same Region only);
  • Route 53;
  • Routing symmetrically vs asymmetrically (ie round-trip data path);
  • RTO vs RPO;
  • S3 (11 9s durability), can be a VPC endpoint;
  • Scale up (ie vertical), vs out (ie horizontal). Latter is preferred to minimise downtime;
  • SES;
  • SG, cannot setup explicit deny rules (NACLs can);
  • Snowball/Snowmobile, or previously (data) Import/Export;
  • SNS;
  • SQS;
  • Storage Gateway (on-prem {ESXi/Hyper-V} bandwidth-throttled, or as an EC2; also works with Direct Connect):
  • File (NFS), up to 5 TB per file;
  • Tape (iSCSI):
  • Library (S3: instant);
  • Shelf (Glacier: 1d);
  • Volume (iSCSI):
  • Cached (subset only, most frequently used, up to 32 volumes {32 TB ea}; ie 1 PB);
  • Stored (full set, up to 32 volumes {16 TB ea}; ie 512 TB);
  • SR-IOV?;
  • STS, AD-based identity federation for 1-36 hour access (to some resource, eg S3) without having to create new IAM creds, LDAP authentication first (then STS), 4 fields (access key, secret access key, token, duration);
  • SWF?;
  • Tags are key/value pairs attached to resources, usable in Resource Groups;
  • VM Import/Export;
  • VPC tenancy (default vs dedicated) and its impact on EC2 instances;
  • Route table (created by default), subnet to AZ (1:1), private vs public subnets, assign a public IP within a public subnet to make an instance internet-facing (behind an ELB also works), 5 reserved IPs per subnet (.0-.3, .255), CIDR block; “local” route within a VPC, IGW to VPC (1:1), route table (for n subnets), IGW/NAT target (for destination route, SGs can span multiple subnets but not the other way around, NAT instance disable source/destination check, VPC peering: use private IPs to address instances within the same Region (50-125 VPCs), 1:1 relationship, private DNS names won’t resolve, routes/SGs/NACLs config required on both ends, multicast vs unicast?;
  • WAF, managed layer 7 sandwich;

AWS: DevOps Pro study notes

So I covered my eyes for a bit when I clicked ‘Finish’ (the test attempt), as this was the toughest exam I’d faced thus far, and I was maybe 70% satisfied with my body of work. Fortuitously, I passed, albeit with an overall score of 65%:-

1.0  Continuous Delivery and Process Automation: 47%
2.0  Monitoring, Metrics, and Logging: 93%
3.0  Security, Governance, and Validation: 75%
4.0  High Availability and Elasticity: 83%

I really need to backtrack, figure out this CI/CD thing, then.

  • ASG (lifecycle hooks {Terminating > Terminating:Wait}, span AZs evenly by default, Launch Configs cannot be edited, suspense AddToLoadBalancer and subsequent manual reg., Termination Policy {Default|OldestInstance});
  • CI tooling (e.g., Jenkins) can perform syntax/build tests;
  • CloudFormation (CreationPolicy {post-config}, ::CustomResource, {RDS} DeletionPolicy=Retain, nested stacks, UpdatePolicy=AutoScalingRollingUpdate);
  • CloudTrail;
  • CloudWatch (dimensions {per-ASG}, retention period, aggregation, Logs {agent}, Log Filters, subscriptions);
  • DynamoDB (cache S3 object metadata);
  • EB (Applications > Versions > Environments, Container Commands {leader-only}, Docker containers, Saved Configs., Swap URLs, .ebextensions);
  • EBS (unencrypted to encrypted, pre-warming);
  • ECS (;
  • ElastiCache;
  • Elasticsearch?;
  • IAM (Database Authentication {Aurora|MySQL}, Instance Profile > Role);
  • Kinesis Streams (real time);
  • OpsWorks; i.e., Chef+ (Configure {custom cookbook});
  • RDS (Multi-AZ, Read Replicas, sharding);
  • S3 (key-based naming scheme, store developer’s public keys, MFA Delete);
  • SNS;
  • SQS;
  • WiF (via some IdP {e.g., Google});


AWS: SysOps Administrator (SOA) study notes

Hello, world. Penned down some keywords after passing my recent AWS SOA exam, and then expanded on ’em below. Perhaps you’ll find ’em useful then.

RAID 0 (striped) vs. 1 (mirrored); i.e., the lower the number, the higher the risk, see

Just like EC2 instances, EBS volumes reside in a specific AZ of a Region; i.e., they can only be attached to a running instances within the same AZ. To switch AZs, use snapshots, see

AMIs can be referred to as being backed by EBS, or ephemeral/instance store. Ephemeral/instance AMIs are stored in S3; i.e., terminating an EC2 instance running the S3-based AMI means that data in the root volume is gone forever, see

EBS optimized; i.e., minimizing contention between EBS I/O and other traffic from your EC2 instance, see

Cluster-type placement groups: low-latency grouping (of EC2 instances) within a single AZ, see

Automated Backups allow users to restore to data within about 5 minutes of the current time, see

TA, as its name suggests, allows users to accelerate file transfers to S3, for when users are underutilizing available Internet bandwidth at upload time, see

Tenancy is typically default (i.e., shared) tenancy. Users cannot change from default to dedicated/host, and vice-versa, see

IPv4 CIDR blocks can range from large (/16 netmask, 65k addresses) to small (/28 netmask, 16 addresses), see

Direct Connect, use private (virtual interface) to connect to your VPC, public for services that aren’t in a VPC (e.g., Glacier), see

Within VPCs, there is a “local” route allowing communication between subnets using private IP addresses only, see,

Active Directory and AWS, see,

Windows EC2 instances can be configured using EC2Config (2.2.10+) to export data to CloudWatch, see


AWS: Analysis of the Certified SysOps Administrator – Associate exam

I recently became fully AWS certified (at the Associate Level), most recently passing the the SysOps Administrator – Associate exam at my third attempt.

  1. In late 2016, I failed with a score of 61% (or  67%). It was my first failure — I deleted the “unsuccessful” notification email in a fit of rage;
  2. In early 2018, I failed with a score of 71%;
  3. In April 2018, I passed with a score of 80%.

My experience was that the Certified SysOps Administrator – Associate exam was the toughest of the lot. In the table below, I compare the weighted scores between attempts #2 and #3 for individual domains:

SysOps Administrator

While I improved in 3 domains, I obtained the same score in the other 4 domains (Monitoring and Metrics, Deployment and Provisioning, Security, & Networking). This gelled with my observation that many of the questions for attempt #3 I’d actually attempted 2 weeks earlier!

Of course, your own mileage may vary, plus you still need to spend time/effort reading through and understanding the AWS FAQs — th9/: are really in-depth, and helped me answer 5 questions correctly; i.e., 9% of 55 ~= 4.95.

In a subsequent post I’ll discuss particular areas that showed up at exam time.


AWS Certified Associate

After an extended hiatus away from AWS certification, finally I’m certified at the Associate level, 3 times over!

Photo 18-4-18, 1 16 13 PM.png

To date I’ve taken and passed:

  1. Certified Developer – Associate
  2. Certified Solutions Architect – Associate
  3. Certified SysOps Administrator – Associate

AWS has a nice road map, so I just re-purposed the following image off their site:


I’m told that the Professional level exams are much harder, but that’s a post for another day…


Docker containers can’t resolve DNS

I’ve recently switched over to using Docker for dev. work on a Windows 10 host, and it’s worked pretty well. Today, apt-get somehow stopped working; e.g.,

...RUN apt-get update && apt-get install...
---> Running in ...
Err:1 xenial-security InRelease
Temporary failure resolving ''
Err:2 xenial InRelease
Temporary failure resolving ''
Err:3 xenial-updates InRelease
Temporary failure resolving ''
Err:4 xenial-backports InRelease
Temporary failure resolving ''
Reading package lists...
W: Failed to fetch Temporary failure resolving ''
W: Failed to fetch Temporary failure resolving ''
W: Failed to fetch Temporary failure resolving ''
W: Failed to fetch Temporary failure resolving ''
W: Some index files failed to download. They have been ignored, or old ones used instead.

This post suggests it might be DNS-related, so I changed it from the (default) Google DNS config to Cloudflare’s. Google’s seem to have been (very recently) blocked for whatever reason. Here’s how my Settings -> Network looks like now:


Forced choice (VirtualBox, or Docker)

In order to run Docker, it seems I must discontinue my use of VirtualBox. Great (not great); e.g.,


CentOS: Forced shutdown

Yesterday, we encountered a disk issue on one of our CentOS servers. Some of the disks had either failed, or were predicting failure, so our vendor swooped in, changed some of the disks, as well as the RAID controller. Unfortunately, this worked for only a short time, before we started seeing “input/output error” verbiage in the console. The concern was data loss, so we tried to shutdown, reboot: same “input/output error”.

And then I learnt that it’s possible to force a shutdown via the Magic SysRq key. I mean, magic!!

echo 1 > /proc/sys/kernel/sysrq
echo o > /proc/sysrq-trigger

New disks are incoming as I type this, we’ll have to keep a close look on this developing situation.


Oracle: Cleaning up SYS_EXPORT_SCHEMA_xx jobs

Sometimes expdp jobs fail for any number of reasons. Re-running expdp, I noticed that the number had increased since the prior run; e.g., SYS_EXPORT_SCHEMA_02 instead of SYS_EXPORT_SCHEMA_01. As it turns out, these are potentially orphaned jobs, so clean ’em up!

Generally, copied from Anar Godjaev’s excellent blog post:

SQL> select owner_name, job_name, operation, job_mode, state, attached_sessions from dba_datapump_jobs;

SQL> drop table {owner_name}.SYS_EXPORT_SCHEMA_01;
SQL> purge table {owner_name}.SYS_EXPORT_SCHEMA_01;