Operations Automation and Developer Support

I started building and selling PC’s for a side hobby back in the early 2000’s. That led led me to web design since I needed a place for customers to configure their orders. From there, I moved on to creating apps such as KASL WebHost and KASL Portal which were fully dynamic, database driven sites (they don’t really exist anymore except in random corners of the web, and are severely outdated as those projects have died).

Later on, after I was really good with web design and basic networking, I went to school, learned a lot more, and turned my bad habits into good ones.

From there, my career turned into automation and developer support. Here is what I do for a living now:

Develop and Build the stack

If you aren’t familiar with the term “stack”, here is my definition: A bunch of computers (servers) on the same local network that work together to provide a working application. For example, a web server, a database server, a job server, and a caching server can be considered a stack.

When I build a stack, I do it in roughly this order:

  • Receive information about the desired outcome of a project, including what platforms and frameworks to use, and how much traffic is ultimately expected
  • Create a plan that includes level of effort (LOE) for completing the project
  • Launch the cloud-based servers in multiple data centers to achieve redundancy, including development and staging environments if budget permits
  • Develop Chef cookbooks that will install required server packages, configure users, manage roles and permissions, configure server applications, automate offsite backups, automate disaster recovery (to a point – we don’t want automated DR to start on its own without human intervention), set up monitoring, and configure deployment endpoint info as needed.
  • Run the Chef cookbooks on the servers as relevant, configuring each server (aka chef-client) for its specific role, such as database or application server
  • Test the stack to ensure everything is communicating properly
  • Configure load balancing via either HAProxy or any DNS traffic management system available (DynECT, Route53, freedns.afraid.org, etc)
  • Ensure automatic failover and recovery
  • Set up a “fail whale” page which is served as a last resort if all of the geographically redundant stacks go down. The fail whale page will simply say something about the site being broken, and that we’re working on it.

Developer Support

Developers will always need somebody around who can assist them with day to day server access, instruction, usage and troubleshooting. As a result, the job is never done as long as the developers are using the servers built in the above section. If developers are building their applications out on the server stacks I build, they are sure to run into circumstances where they need a package installed and configured, or their access isn’t working, or they broke the server and need me to fix it. When packages need to be installed or server configurations need to change, I update my Chef cookbooks by adding in code that is relevant towards the desired outcome.

There are days where developers don’t say a word, and there are days where 10 developers are all asking me the same questions. The nice thing about this is that on quiet days, you know there are no problems (and hopefully the monitoring systems reflect this as well). On the days where I get pinged left and right, I know it might get rough, but luckily any troubleshooting issues are resolved by my fully tested Chef cookbooks going through and fixing the broken configurations. Otherwise, back to the logs…

DevOps for hire

I do have a full-time job where I work for a huge company that everybody knows about. With that said, I do have enough free time where I’m willing to work on side projects. If you are interested in chatting with me about a project, please use the contact form on this site and I’ll get back to you.


%d bloggers like this: