Category: Automation

The case against GUIs (again)…

Recent events have made me think about this post again…

Software Vendors

I can’t explain how much I despise being forced to use a GUI to do something that could be scripted.

If you are a software vendor, please make sure you offer some form of scriptable API to interact with your product, and make sure it’s documented properly. I don’t care how much time and effort you put into your GUI, I don’t want to use it. I want everything in a script that can be checked into Git and automated.

If you are a software vendor that doesn’t provide a scriptable way to interact with your system, you are going to the bottom of my list. Even if I am forced to use your product now, I will switch at the first possible opportunity.

Staff

I’m sure this will ruffle a few feathers, but as I said in the linked article, when I see people using a GUI to perform certain maintenance operations my immediate reaction is they are wasting time. It is very rare a manual operation will be as fast and accurate as a scripted operation.

In the past we have hired “experts” to do work for us, and they’ve taken days working with GUIs to accomplish something that could have been scripted and run in much less time. If they are truly experts I would have expected them to have scripts for everything they do anyway.

I realise some consultants are running up chargeable hours by taking the long route, and some are not the experts they claim to be. It is noticed!

Why the rant?

The further down the rabbit hole I go with automation, the less I can stand doing manual operational work. I’m reaching the point where the mere sight of an unnecessary GUI gives me toxic shock…

GUIs have their place, but not for operational tasks IMHO!

Cheers

Tim…

Why Automation Matters : The cloud may not be right for you, but you still have to automate!

A few days ago I tweeted this link to an article about some workloads being better suited to on-prem infrastructure.

Jared Still sent me this link.

The executive summary in both cases is, if you have defined workloads that don’t require elastic resource allocation, and you are not making use of cloud-only platforms, you might find it significantly cheaper to run your systems on-prem compared to running it in the cloud.

With reference to the first article Freek D’Hooge responded with this.

“I agree that cloud is not always the best or most cost effective choice, but I find the article lacking in what it really takes to run on-prem equipment.”

I responded to Freek D’Hooge with this.

“Yes. On-prem works well if you have Infrastructure as Code and have automated all the crap, making it feel more like self-service.

For many people, that concept of automation only starts after they move to the cloud though, so they never realise how well on-prem can work…”

I’m assuming these folks who are moving back to on-prem are doing the whole high availability (HA) and disaster recovery (DR) thing properly.

There are many counter arguments, and I don’t want to start a religious war about cloud vs on-prem, but there is one aspect of this discussion that doesn’t seem to be covered here, and that is automation.

But you still have to automate!

Deciding not to go to the cloud, or moving back from the cloud to on-prem, is not an excuse to go back to the bad old days. We have to make sure we are using infrastructure as code, and automating the hell out of everything. I’ve mentioned this before.

Of course, servers in racks are a physical task, but for most things after that we are probably using virtual machines and/or containers, so once we have the physical kit in place we should be able to automate everything else.

Take a look at your stack and you will probably find there are Terraform providers and Ansible modules that work for your on-prem infrastructure, the same as you would expect for your cloud infrastructure. There is no reason not to use infrastructure as code on-prem.

For many people the “step change” nature of moving to the cloud is the thing that allows them to take a step back and learn automation. That’s a pity because they have never seen how well on-prem can work with automation.

Even as I write this I am still in the same situation. I’m currently building Azure Integration Services (AIS) kit in the cloud using Terraform. I have a landing zone where I, as part of the development team, can just build the stuff we need using infrastructure as code. That’s great, but if I want an on-prem VM, I have to raise a request and wait. I’ve automated many aspects of my DBA job, but basic provisioning of kit on-prem is still part of the old world, with all the associated lost time in hand-offs. For those seeking to remain on-prem, this type of thing can’t be allowed to continue.

In summary

It doesn’t matter if you go to the cloud or not, you have to use infrastructure as code and automate things to make everything feel like self-service. I’m not suggesting you need the perfect private cloud solution, but you need to provide developers with self-service solutions and let them get on with doing their job, rather waiting for you.

Check out the rest of the series here.

Cheers

Tim…

Why Automation Matters : Your automation is your documentation

How many times have you been following a process defined in a knowledge base note, only to find something has been omitted, or is unclear? This may be because of empire building, laziness or more often oversight, but the result is the same. Unless your processes are well documented, you always run the risk of progress drawing to a halt when “the right person” is not present.

One of the great things about automation is, by definition, every step of the process must be defined. If person X is on holiday, you can be 100% sure all the steps to complete the automation are present.

Of course, this doesn’t stop people writing stupid, ugly and hard to understand code, but your development process should have some control over that. Even if it doesn’t, you know the answer is there. It must be there because the process works.

Does that mean you don’t need to document automations?

No. The automations should be self documenting. I don’t mean that in the sense that “my code is so good it’s self-documenting”, which is the calling card of the lazy developer. I mean that automation code in your source control system should be documented. Markdown is a quick and easy tool that allows us to easily document our code, and the good thing about it is it remains close to the code. It’s right next to it in the repository. When we change our code, we should revise our documentation where necessary. The documentation becomes a living document, rather than some 1000 page word document that nobody ever reads, and nobody updates.

But documentation sucks!

Documentation gets a really bad rap because most people are doing it wrong. They fall into one of these traps.

  • They produce too little, which means people are unlikely to find what they are looking for.
  • They produce too much, which makes it daunting to look at, so nobody bothers.
  • It’s overly formal, which is dry and boring.
  • It’s hidden, or at least separate to the code, so people might not even know it exists.

Basic pointers and how-to examples are good enough for 90% of the cases, so make these the focus of your documentation. You can always give links to more detailed documentation for those people that need a little more. The context is slightly different, but this post on Structuring Content should give you some clues about how to structure your documentation. After all, documentation is content. 🙂

Conclusion

For some companies an automation or infrastructure as code project may well be the first time in their company history that they have got everything about a process documented. That has to be a positive result for the company!

Check out the rest of the series here.

Cheers

Tim…

Why Automation Matters : It doesn’t get much simpler than this!

Just a little story about something that happened recently…

The July Oracle quarterly security patches dropped, along with the downstream OpenJDK releases a little later. As soon as the OpenJDK release dropped I updated all our non-production Oracle REST Data Services (ORDS) middle tier to the latest and greatest. Production will follow shortly. I figured that was the end of it…

On Friday evening I noticed a post by Jeff Smith saying version 22.2.1 of SQLcl and ORDS had dropped. Now it was Friday evening, and I typically don’t work on Fridays, but I really didn’t want to have this hanging over me all weekend, so this is what I did…

  • Download the software.
  • Upload the software to Artifactory.
  • Change a couple of files in Git, specifying the new software versions.
  • Press a button in TeamCity to rebuild the ORDS Docker image and push the new image to JFrog Platform.
  • Press a button in TeamCity to redeploy all the non-production ORDS containers.

The whole process took a few minutes, and I was confident it would work because it’s all automated and works every time. I sent this message to Jeff.

I {expletive} love automation. I just noticed your post about 22.2.1 of SQLcl and ORDS. I logged on to work. Updated a couple of files in git and pressed a button in TeamCity. New ORDS Docker image built and pushed to all non-prod systems. Boom…

This is why automation matters. The combination of picking the right platform, in this case containers, and investing a little time in the automation means we can react really quickly to new releases. It doesn’t get much simpler than this!

Check out the rest of the series here.

Cheers

Tim…

PS. This post explains the application topology we use for ORDS and APEX.

Video : Ansible Playbooks : Roles

In today’s video we demonstrate how to use role with Ansible Playbooks.

The video is based on the following article.

You might find some useful stuff here.

The star of today’s video is Anton Els, who I first met when he worked for DBvisit.

Cheers

Tim…

Video : Ansible Playbooks : Users and Groups

In today’s video we demonstrate how to manage users and groups with Ansible Playbooks.

The video is based on the following article.

You might find some useful stuff here.

The star of today’s video is Connor McDonald of AskTom fame. You can check out his YouTube channel here.

Cheers

Tim…

Video : Ansible Playbooks : Tags


In today’s video we demonstrate how to use handlers in Ansible Playbooks.

The video is based on the following article.

You might find some useful stuff here.

The star of today’s video is Tanel Poder, who thinks he’s at the bow of a Viking ship by the look of it. 🙂

Cheers

Tim…

ORDS and APEX 22.1 : Vagrant and Docker Builds

I was on top of the recent ORDS 22.1 release, but somehow I managed to miss the APEX 22.1 release. Here’s an update on what’s been going on with my builds…

Vagrant Builds

A number of my Vagrant database builds include APEX. Those that do have been updated to use APEX 22.1. You can find them here.

They had already been updated to use the latest versions of Java, Tomcat, ORDS and SQLcl where appropriate, so the APEX upgrade was a small change. The ORDS update was done 11 days ago. It was a bigger update, as the installation process for ORDS 22.1 has changed quite a bit. I wrote about that here.

You can read my beginner’s guide to Vagrant here.

Docker/Container Builds

I have some ORDS containers that include the APEX images, so those image builds have been updated to use ORDS 22.1 and APEX 22.1 images now.

APEX is also included in the database builds. You can find them here.

I was a little slow to add the updates to the Docker builds. The Tomcat and SQLcl changes were done a couple of weeks ago, but I’ve only added the Java, ORDS and APEX changes today. I held back a little on the ORDS 22.1 changes, as I was rewriting a bunch of articles that were affected by the installation and configuration changes.

You can read my Docker/Container articles here.

Real Life

We probably won’t be putting APEX 22.1 live until the next patching cycle, which will be July. In the past I could be quite aggressive about the upgrades, but as APEX is becoming more important in the organisation, the rollout of updates has to be a bit more considered. 🙂

The same will probably be true for the ORDS 22.1 rollout. We run ORDS in containers, so it’s really quick and easy to replace all the infrastructure, but I wanted to be a bit cautious because of all the changes in the new version. Having played with it a bit now, I’m feeling a lot more confident, but at the time we were starting our patching cycle I was too nervous to include it.

As far as home systems are concerned, I only use the latest and greatest, unless there is a compelling reason not to. 😉

Cheers

Tim…

Video : Ansible Playbooks : Handlers

In today’s video we demonstrate how to use handlers in Ansible Playbooks.

The video is based on the following article.

You might find some useful stuff here.

The star of today’s video is my dad, Graham Wood, who spend most of his time galivanting and spending my inheritance, which I’ve been waiting patiently for!

Cheers

Tim…

Video : Ansible Playbooks : Host and Group Variables

In today’s video we demonstrate how to use lists and loops in Ansible Playbooks.

The video is based on the following article.

You might find some useful stuff here.

The star of today’s video is Toon Koppelaars, famous for The Helsinki Declaration (IT Version), which you may know better these days as the SmartDB approach to building applications.

Cheers

Tim…