Category: Technology

Oracle VirtualBox 7.0.18, Vagrant 2.4.1 and Packer 1.10.3

Oracle VirtualBox 7.0.18

VirtualBox 7.0.18 has been released.

The downloads and changelog are in the usual places.

I’ve installed it on my Windows 10 and 11 machines. Both gave me issues, which I put down to the new version of VirtualBox, but on further investigation it was actually because of my new Vagrant box builds.

If I used the new version of VirtualBox with the old version of my Vagrant boxes the builds worked fine. If I tried to use one of the newly built Vagrant box versions it failed with this error.

The SSH connection was unexpectedly closed by the remote end. This
usually indicates that SSH within the guest machine was unable to
properly start up. Please boot the VM in GUI mode to check whether
it is booting properly.

I tried using “config.ssh.keep_alive = true” in the Vagrantfile, but that didn’t help. I’m continuing to investigate the issue, but it seems like VirtualBox 7.0.18 is working fine. It’s something with my box builds which is the problem.

Vagrant 2.4.1

Releases of VirtualBox prompt me to check for new versions of Vagrant. The current version is Vagrant 2.4.1. All my test systems are built with Vagrant, so I installed it as well.

If you are new to Vagrant and want to learn, you might find this useful.

Once you understand that, I found the best way of learning more was to look at builds done by other people. You can see all my Vagrant builds here.

I’ll be doing some updates to my Oracle builds over the coming days, so this will get a lot of testing.

Packer 1.10.3

I use Packer to rebuild my Vagrant boxes (Oracle Linux 7, 8 and 9) so they have the latest guest additions. For the reasons mentioned above I’ve not released the new version of the Vagrant boxes yet. You can see the old ones here.

If you are interested in creating your own Packer builds, you might take inspiration from mine, available here.

Once I get to the bottom of the SSH issues on the new builds I’ll update the boxes.

How did it all go?

Most things worked fine. As mentioned there is an issue with my Vagrant box builds, but I’m assuming that is something on my side of things. 🙂

What about the VirtualBox GUI?

Just a quick warning. I do everything using Vagrant, so I rarely look at the VirtualBox GUI. Remember, when I say everything worked fine, I mean for what I use.

Cheers

Tim…

What is the end goal for automation and AI?

I’ve been a big proponent of automation over the years. I recognise that automation will cause the loss of some jobs, but I’ve always assumed people would move on to other things. A consistent theme of mine was, why waste humans on mundane work when they could do something of more value?

I recently read The Naked Sun by Isaac Asimov and now I feel rather uneasy about automation and AI…

I don’t want to give too much of the story away, but we are told fairly early that the planet Solaria only has 20,000 humans, and millions of robots. The robots do everything as far as the economy is concerned, and the humans are essentially idle custodians. Rather than having millions of people doing a variety of occupations, there are only the elite sitting at the top of the pile…

As I was reading this I was thinking about the economic elite on our planet and I started to feel some concern. My dream of a post-scarcity society is one where we are all equal, but I can’t imagine there are many people at the top that want to be equal. They would always want to be more than equal. Call me a pessimist, but when I think of Elon Musk calling himself the emperor of Mars, I wonder if a future post-scarcity society will be less like Star Trek, and more like Solaria…

Of course, the AI could just turn into Skynet and kill us all, but I have this feeling that greedy humans using AI and automation against the majority of the population poses the greater threat. Why have “them and us”, if there could just be “us”? 🙁

All of a sudden automation has lost some of its lustre…

Cheers

Tim…

AI Prompt Engineer (AI-fu). The new Google-fu?

The other day I came across the term AI Prompt Engineer. It was in the context of being the next big thing in the job market. I did a bit of Googling and sure enough there are courses about it, and a number of jobs being offered. These seem to break down into two main types of role.

  1. A technical AI development role, involving understanding of AI and coding against various AI backends via APIs.
  2. A person who types in stuff to get the right outcome from specific AI tools.

As mentioned, the first is a technical development role, but the second seems like AI-fu to me…

Google-fu

Here is the Wikipedia definition of Google-fu.

“Skill in using search engines (especially Google) to quickly find useful information on the Internet.”

https://en.wiktionary.org/wiki/Google-fu

After all these years I’m still surprised how weak people’s Google-fu is. Searching for information requires knowing what to include in your search criteria, as well as evaluating the results to make sure they are actually what you need.

If you don’t understand how to filter the results using targeted search terms, you may get incorrect or misleading results. If you don’t understand the subject matter, you have no way of validating correctness of the responses.

AI-fu

I’ve made up the term AI-fu. I can’t see any references to it on the internet. The closest I can find is ChatGPT-fu, which I also made up here. 🙂 I said Google-fu is about providing the appropriate inputs, and validating the outputs. I would suggest that AI-fu is very similar.

If you’ve played around with ChatGPT you will know there is a degree of “garbage in, garbage out” involved. You have to refine your inputs, either by altering the original question, or following a chain of thought (CoT) process.

How do you know you have a suitable result? Well, you have to understand the subject matter, and do some fact checking to make sure you are not generating garbage. If you don’t understand that a human hand typically has 5 digits, you can’t tell if that 7 fingered monstrosity you’ve just generated is wrong. 🙂

Thoughts

I suspect those people that have acquired good Google-fu, will also acquire good AI-fu. Those people who have proved to have consistently weak Google-fu, are unlikely to ever developer strong AI-fu.

Cheers

Tim…

PS. If you want a certificate in AI-fu, please send £500 to my PayPal… 😉

Automation : Increasing pressure on an existing constraint

Yesterday I tweeted that I was reminded of this post.

I was reminded of it because of something that is happening to me at work, so I thought I would talk about it here.

Production lines

If you’ve read anything about DevOps you will know it came from manufacturing. If you didn’t know that, check out The Goal, which was the basis for The Phoenix Project.

Manufacturing typically uses production lines made up of multiple stations, where each station performs a specific task, and the product moves forward through the stations until it is complete. If one station is slower than the others, it will become a blocker. Product will start to queue up behind it, and downstream stations will become starved. So production lines only work well if they are planned to enable a consistent flow.

What’s more, you can only sell the product when it is completed, so we could describe the product as having no value until it is finished and with the customer.

It’s not just about manufacturing

The processes born out of manufacturing also work really well for other industries. I would suggest most things can be described like a production line, and as soon as you do that, a similar approach can be adopted to identify and fix the constraints.

Tech is an obvious one, and variations on DevOps have grown in popularity because of that. My sister in law works in a medical practice, and we’ve discussed the processes used in the administration side of it. As a result of our discussions she’s started to use Kanban boards to visualise the flow of work.

Back to my problem

So with the concept of production lines and constraints in mind, we jump back to my problem…

We are in the process of replacing loads of Oracle Linux 7 servers with either Oracle Linux 8 or Oracle Linux 9, depending on vendor support. The first three links in that production line are as follows.

  • VMs are provisioned.
  • Operating system customisations are run.
  • Database or app server is installed and configured.

We are not perfect, but we’ve got pretty good at this part of the production line. When we are finished, the systems have to be tested, and go through various processes to get them live and used by the business. Those parts of the production line that follow us are slow due to a number of factors. So our improvements in the production line have just made things harder for those steps that follow us. A simplified view of the Kanban board looks something like this.

The obvious thing to do here is focus on the constraints and start working on downstream links in the chain, to improve the overall flow. That’s where we hit organisation and culture barriers, so we are pretty much stumped…

Thoughts

I’m pretty happy with what we’ve done over the last few years. We’ve definitely improved several aspects of our systems because of automation, but at the same time I can’t help thinking we’ve achieved nothing because ultimately the work is not getting completed as fast as it’s started.

I wrote here about reframing the goal, and I have to do that as copium. Unfortunately copium only goes so far… 🙂

Cheers

Tim…

Continuous Improvement : It’s easy to get lazy…

I like to think that continuous improvement is always part of my process, but occasionally things crop up and remind me that I am just as lazy and set in my ways as everyone else. Here’s a little story to illustrate that…

The Issue

The main thing I’m supposed to be focussed on at work is server rebuilds. We are moving a bunch of stuff from Oracle Linux 7 (OL7) to Oracle Linux 9 (OL9), or OL8 if the vendor doesn’t support OL9 yet.

Anyone who follows my Vagrant builds knows I’ve got loads of scripts that build things, but because of our company organisation structure, there is only so much I can do without pinging requests to other teams, so full automation on-prem is not currently possible for me at work.

As a result of this limitation, there is a bunch of things I can do in Vagrant that I can’t do at work. The fact that work is semi-automated for builds has indirectly made me lazy in respect to continuously improving my build processes. In my head I guess I’ve been saying, “What’s the point if I can’t finish it?”

Making Progress

Due to the large number of rebuilds we are working through now, I’ve taken a step back and started again. I created a new build repo at work, and populated it with all the relevant stuff from our existing build processes, and some bits from my Vagrant builds. It’s still not fully automated, due to the company organisation issues mentioned before, but every step in the process is up to date with the best we can do, and the process is documented in the README.txt of the repo.

Along the way I noticed a bunch of things that were pretty crap in my original approach. It worked, but it wasn’t something to be proud of. 🙂

So What?

This just goes to illustrate the point I made in the title of this post. It’s easy to get lazy and let things start to slip, even when you know better. This is especially true if you have a “completionist” mentality like me. The thought of being able to finish something is quite compelling to me. Unfortunately, when I know I will not be able to complete something, I find it really hard to make the effort, because it all feels kind-of pointless. 🙂

Reframe The Goal

I guess this is really about reframing the goal. Rather than thinking that completion is when I have a fully automated process, it should be that I’ve got everything up to date and as far down the automation path as I can get at the moment.

The funny thing is I wrote about this in a previous post, but it seems I’m not always capable of taking my own advice. 🙂

Cheers

Tim…

UNIX/Linux Time Command : Record elapsed time

In a recent post I mentioned using a scratchpad to record everything I do. As part of that process I try to make regular use of the UNIX/Linux “time” command to record elapsed times of long running commands.

It’s really simple to use. All you do is put “time” in front of the command and it will display how long it takes to complete the command. In this example I do a sleep for 10 seconds and use the time command to report the elapsed time. 🙂

$ time sleep 10

real    0m10.002s
user    0m0.001s
sys     0m0.001s
$

Clearly that’s a silly example, but it gives you an idea of how this works.

If you get into the habit of using this with all long running processes, you can get accurate timings for steps. That way, when someone asks you how long something takes you can give them a real answer, rather than making something up and hoping for the best.

Just remember, timings can vary if the load on the system varies between runs. Even so it’s always nicer to have some real data to inform your decisions going forward. Especially when planning for something that will cause disruption in production. 🙂

Cheers

Tim…



Using a scratchpad…

Followers of the blog know I’m a big advocate for writing things down. The main reason I do this is because I want a record of everything I do.

I rarely type a command directly into the command line. I nearly always type it in a scratchpad first. Currently I have 67,250 lines in my work scratchpad and 12,309 lines in my personal scratchpad.

When I say scratchpad, I just mean a text file, which I edit using a text editor. Nothing fancy.

Why do I do this?

Inspiration

Most of my articles and blog posts start life as notes in my personal scratchpad. At work some of my scratchpad notes become more formal documentation, like knowledge base notes and how-to files in Git etc.

I know if I don’t make the notes as I go along, I will forget what I did, and struggle to write the documentation later.

If something makes it as far as being written up, it gets removed from my scratchpads, so what’s in there at the moment are notes that have not made the cut, so to speak. 🙂

One of the reasons I’ve been able to produce content for so many years is there is a constant stream of stuff added to my scratchpads. Of course, some of it is junk, but some of it is not.

If you are struggling with documentation or inspiration, I think taking this approach will really help.

Reflection

One of the things that I find really useful about taking notes is it allows me to look back and reflect on what I did to complete something. For example I might search through my scratchpad to see what happened over the lifetime of a server. I can see all tickets that were raised and what firewall rules and configuration changes were required. When I get a similar request this allows me to estimate the amount of work that needs to be done, and I can see what teams will be involved in the process.

I could search though our ticketing system for much of this information, but I find it a lot easier to keep a record of my actions in a scratchpad, then drill into the tickets if I need more info, which I rarely do.

Rewrites

Much like my articles, if I read back through some notes and they aren’t 100% clear, I often rewrite them. Maybe adding some more text, or a clearer example. This process may result in something graduating into being a separate document, but sometimes it just stays in the scratchpad forever.

Give it a go

If you don’t already do this, give it a go and see how you feel about it. Especially you content creators.

Cheers

Tim…

URGENT : Why you should {almost} never put URGENT in your message

Just a little note about something that rubs me up the wrong way.

I quite often get messages with the word URGENT in the subject or text. I scan through the content, and if it doesn’t seem truly urgent to me, I put it at the bottom of my list of things to do. Why?

You are not the central character in my life

When someone is communicating with me, they are thinking it’s a 1-to-1 interaction. What they forget is that I am working on many different things. As a result, for me it is a 1-to-many relationship.

Just because something is urgent to you, it doesn’t mean it takes priority over the other work I am doing. You don’t know what I’m doing, so you can’t possibly know how your issue sits in my list of priorities. Assuming your needs are more important than the needs of others is really rude.

This is even more annoying when it comes from someone outside of work. If you are not paying me, you have no business sending me an “urgent” request.

Your bad planning is not my emergency

In many cases these “urgent” issues could have been solved well in advance. It’s bad planning that has caused this issue, so I don’t see why it should have a negative impact on my life.

Sometimes there are genuine reasons for something to be classed as an emergency, like P1 incidents, but that’s not what I’m talking about.

There are some people that bounce from one emergency to the next. It soon becomes obvious that these people are just really bad at planning, and as a result are constantly in the weeds, and asking you to help drag them out.

Personal heroics don’t help the company long term

Occasionally you have to dig deep to get through a real emergency, but for the constant stream of self-inflicted emergencies, the only solution is to let things fail so people can see the root cause.

Personal heroics may feel good to you in the short term, but in the long term it is bad for your company and for you. The company needs to know what is failing and do something about it. Relying on a small number of people to pull them out of the weeds is not a long term strategy. Sooner or later this will stop working because the “heroes” will get annoyed and leave, or quiet quit.

What does urgent even mean to you?

I once got a message late on a Friday about an “urgent” issue. I felt sorry for the person in question, so I cancelled my plans, worked on the issue and sent them back the solution. I then got a reply saying, “Great, I’ll have a look at it on Monday”. Needless to say I lost my shit. That clearly was not an urgent issue.

I’m not alone

Over the years I’ve had this conversation many times, and I know I’m not the only person that gets annoyed by messages marked as urgent. I also know I’m not the only person that puts them to the bottom on my to-do list if they are not truly urgent.

Conclusion

As you can see, unnecessarily marking things as urgent is a bad idea, and likely to result in a longer resolution time, so next time you consider adding that little word into your message, just don’t!

Cheers

Tim…

PS. Rant over…

Answering some questions about Vagrant

Someone on YouTube asked me some general questions about my experience of Vagrant, so I thought I would write them down as a blog post.

Could you share the story of when and how you first encountered Vagrant, and how did you feel about it at the time?

I was quite late to the party. In 2017 I was at a VMware workshop in Cork, Ireland. I was sitting in the hotel and Frits Hoogland was showing me his Vagrant build for a test Oracle database. Like most things when they are unfamiliar, it seemed a little complex. He gave me access to his Vagrant repository, but I hardly looked at it. It was on my list of things to do, but there is always so much on my to-do list. When Frits talks you should listen, but unfortunately I failed that mission. 🙂

About a year later a colleague at work asked me what Vagrant was, and I struggled to give a reasonable answer. That evening I Googled it, and tried a couple of really simple builds. As someone with lots of VMs at home it totally blew my mind. From that point on I was hooked. I wrote Vagrant builds for all my test databases, so I could rebuild them whenever I wanted to. I went from never using it, to never shutting up about it overnight.

Now my PC is a lot less bloated. I don’t have to keep loads of VMs for different database versions, RAC and Data Guard etc. If I need something to do a test I just build it from scratch.

Another benefit is it makes live demos feel a lot less stressful. I remember being in my hotel in India, and a few minutes before I was due to start presenting I was having some issues with my demo VM. I just typed “vagrant destroy -f” followed by “vagrant up” and my demo system was rebuilt and I was good to go. Nightmare averted, and no need for loads of backups and snapshots.

How long does it usually takes for you to create the builds that are used to create the database?

The first couple of real builds took some time as there was a learning curve with Vagrant. Fortunately I have a lot of Oracle skills and some basic system administration skills, so it wasn’t too bad. It would have been a lot harder if I was trying to pick up several new skills at the same time.

All my Vagrant builds are based on articles, so I already know what to do. I’m basically tweaking the instructions from my articles to form the Vagrant builds.

The more vagrant builds you do, the quicker you get at doing them, because you have a repository of previous builds to pull ideas from. I’m now at the point where most new builds are slight variations of a previous build, so they are really quick to write.

How are you using Vagrant at work? I assume that some companies will require us to install the database with its options in a bare metal server, and not with VMs.

I use Vagrant at home. All my writing is based on VMs I’m running at home. Those VMs are built using Vagrant.

Automation at work is a little tricky because there is a separation of duties between virtualization, system administration and DBA teams. Configuring a complete automation is quite time consuming and political. Terraform and Ansible are more commonly used at work, but we are still on that journey. We are less DevOps and more DevHopeful. 🙂

Our cloud automations all use Terraform. Terraform is similar to Vagrant, as both are produced by Hashicorp.

The tools you use for automation are not as important as the attitude. Once you get into automation you can switch between tools a lot more easily, because you understand the approach you need to take.

How does using Vagrant help you in serving your customers?

Using Vagrant at home makes it quick to set up new environments, which allows me to learn new stuff faster. I don’t like doing anything at work unless I’ve already tried it at home. I guess me knowing more helps me do my job better, which ultimately benefits the people depending on me at work, so I guess there is an indirect relationship. 🙂

How do you learn Vagrant?

If you want to know more about Vagrant you can start here.

I find the easiest way to learn about Vagrant is to build things. There are loads of builds on the internet to use for inspiration. You can find mine here.

Cheers

Tim…

The case against GUIs (again)…

Recent events have made me think about this post again…

Software Vendors

I can’t explain how much I despise being forced to use a GUI to do something that could be scripted.

If you are a software vendor, please make sure you offer some form of scriptable API to interact with your product, and make sure it’s documented properly. I don’t care how much time and effort you put into your GUI, I don’t want to use it. I want everything in a script that can be checked into Git and automated.

If you are a software vendor that doesn’t provide a scriptable way to interact with your system, you are going to the bottom of my list. Even if I am forced to use your product now, I will switch at the first possible opportunity.

Staff

I’m sure this will ruffle a few feathers, but as I said in the linked article, when I see people using a GUI to perform certain maintenance operations my immediate reaction is they are wasting time. It is very rare a manual operation will be as fast and accurate as a scripted operation.

In the past we have hired “experts” to do work for us, and they’ve taken days working with GUIs to accomplish something that could have been scripted and run in much less time. If they are truly experts I would have expected them to have scripts for everything they do anyway.

I realise some consultants are running up chargeable hours by taking the long route, and some are not the experts they claim to be. It is noticed!

Why the rant?

The further down the rabbit hole I go with automation, the less I can stand doing manual operational work. I’m reaching the point where the mere sight of an unnecessary GUI gives me toxic shock…

GUIs have their place, but not for operational tasks IMHO!

Cheers

Tim…

Exit mobile version