UNIX/Linux Time Command : Record elapsed time

In a recent post I mentioned using a scratchpad to record everything I do. As part of that process I try to make regular use of the UNIX/Linux “time” command to record elapsed times of long running commands.

It’s really simple to use. All you do is put “time” in front of the command and it will display how long it takes to complete the command. In this example I do a sleep for 10 seconds and use the time command to report the elapsed time. 🙂

$ time sleep 10

real    0m10.002s
user    0m0.001s
sys     0m0.001s
$

Clearly that’s a silly example, but it gives you an idea of how this works.

If you get into the habit of using this with all long running processes, you can get accurate timings for steps. That way, when someone asks you how long something takes you can give them a real answer, rather than making something up and hoping for the best.

Just remember, timings can vary if the load on the system varies between runs. Even so it’s always nicer to have some real data to inform your decisions going forward. Especially when planning for something that will cause disruption in production. 🙂

Cheers

Tim…



Using a scratchpad…

Followers of the blog know I’m a big advocate for writing things down. The main reason I do this is because I want a record of everything I do.

I rarely type a command directly into the command line. I nearly always type it in a scratchpad first. Currently I have 67,250 lines in my work scratchpad and 12,309 lines in my personal scratchpad.

When I say scratchpad, I just mean a text file, which I edit using a text editor. Nothing fancy.

Why do I do this?

Inspiration

Most of my articles and blog posts start life as notes in my personal scratchpad. At work some of my scratchpad notes become more formal documentation, like knowledge base notes and how-to files in Git etc.

I know if I don’t make the notes as I go along, I will forget what I did, and struggle to write the documentation later.

If something makes it as far as being written up, it gets removed from my scratchpads, so what’s in there at the moment are notes that have not made the cut, so to speak. 🙂

One of the reasons I’ve been able to produce content for so many years is there is a constant stream of stuff added to my scratchpads. Of course, some of it is junk, but some of it is not.

If you are struggling with documentation or inspiration, I think taking this approach will really help.

Reflection

One of the things that I find really useful about taking notes is it allows me to look back and reflect on what I did to complete something. For example I might search through my scratchpad to see what happened over the lifetime of a server. I can see all tickets that were raised and what firewall rules and configuration changes were required. When I get a similar request this allows me to estimate the amount of work that needs to be done, and I can see what teams will be involved in the process.

I could search though our ticketing system for much of this information, but I find it a lot easier to keep a record of my actions in a scratchpad, then drill into the tickets if I need more info, which I rarely do.

Rewrites

Much like my articles, if I read back through some notes and they aren’t 100% clear, I often rewrite them. Maybe adding some more text, or a clearer example. This process may result in something graduating into being a separate document, but sometimes it just stays in the scratchpad forever.

Give it a go

If you don’t already do this, give it a go and see how you feel about it. Especially you content creators.

Cheers

Tim…

Planning our next Oracle database upgrades. A customer perspective…

Upgrading a database is not about the technical side of things. It’s about the planning that is required. I can upgrade a database in a few minutes, but the project to upgrade all the environments for a specific application can take months to complete. In this post I want to discuss some of the issues we are discussing at the moment regarding our future Oracle 23c upgrades.

What support dates are relevant?

Here are the support dates for the Oracle database.

ReleasePremier Support (PS) EndsFree Extended Support (ES) Ends
19cApril 30 2024April 30 2025
21cApril 30 2024N/A

Release Schedule of Current Database Releases (Doc ID 742060.1)

Here are the current support dates for Oracle Linux.

ReleaseGA DatePremier Support EndsExtended Support Ends
OL7Jul 2014Jul 2024Jun 2026
OL8Jul 2019Jul 2029Jul 2031
OL9Jun 2022Jun 2032Jun 2034

Lifetime Support Policy: Coverage for Oracle Open Source Service Offerings

What database versions are we currently running?

To upgrade directly to 23c we must be running Oracle 19c or 21c.

All our databases are on 19c, so this puts us in a good position. It took a lot of pain and effort to get us to 19c, but it was worth it!

PDB or non-CDB architecture?

The non-CDB architecture was deprecated in 12.1.0.2, but it has remained supported up to, and including, 19c. So Oracle 23c will be the first long term release where the non-CDB architecture is not an option. If you’ve not got up to speed on pluggable databases, you better get started soon! (Multitenant Articles)

With one exception, we have PDBs across the board, so there is nothing new for us here. It sometimes felt like I was swimming against the tide by pushing PDBs so hard over the years, but it all seems worth it now.

What OS are you running on?

I’m going to conveniently forget that anything other than RHEL/OL exist, because other operating systems don’t exist for me in the context of running Oracle databases.

It took us a long time to migrate from OL6 to OL7. The majority of our Oracle databases are currently still running on OL7, which is fast approaching end of life. Since Oracle 23c will not be supported on OL7, we are going to need to migrate to a newer operating system. I wrote about my scepticism around in-place RHEL/OL upgrades (here), so that leaves us two choices.

  • Move our existing databases to a new OS now, then upgrade to 23c later.
  • Wait for the 23c upgrade, and do a big-bang OS migration and database upgrade.

What’s stopping us from doing the first option now? Nothing really. We could migrate our 19c databases to OL8 servers. It would be nicer to migrate them to OL9, but it is not supported for 19c yet. I recently wrote a rant about product certifications on Oracle Linux 9, which resulted in this response from Oracle.

“Oracle Database product management has confirmed that when Oracle Database 23c ships, it will be certified for both OL8 and OL9. Also, Oracle Database 19c will be certified on OL9 before end of 2023.”

That’s really good news, as it gives us more options when we move forward.

Will Oracle Linux exist in the future? Yes!

Just as I thought I had got my head around the sequence of events, RHEL dropped the bombshell about how they would distribute their source in future (see here). This raised concerns about if RHEL clones such as Oracle Linux, Rocky Linux and AlmaLinux could even exist in future.

Without knowing the future of our main operating system, we were questioning what to deploy on new servers. Do we continue with OL or switch to RHEL? Rocky and Alma released some “don’t panic” messages, but Oracle were very quiet. I wasn’t surprised by that, because Oracle don’t say anything until it has passed through legal, but as a customer it was a very nervy time.

A couple of days ago we got a statement from Oracle (here), with a firm commitment to the future of Oracle Linux. I immediately spoke to our system admins and said OL8/OL9 is back on the table. Phew!

I have my own opinions on the RHEL vs clones situation, but as a customer it’s not about politics. I just need to know the OS we are using is still going to exist, and will be supported for our databases. In that respect the statement from Oracle was very welcome!

Do you need a hardware refresh?

If you are running on physical kit, you need to check your maintenance agreements. You may need a hardware fresh to keep everything up to date.

We run everything on virtual machines (VMs), so the hardware changes to the clusters have no impact on our VMs. We’ve had at least one hardware refresh during the lifespan of some of our database VMs.

Thanks to Ludovico Caldara for mentioning this point.

What versions do vendors support?

We use a lot of third party applications, and some of the vendors are really slow to certify their applications on new versions of the database and operating systems.

Ultimately we will make a choice on destination versions and timings based on application vendor support.

Manual or AutoUpgrade?

In Oracle 23c manual upgrades are deprecated (but still supported). I was late to the party with AutoUpgrade, but now I’ve used it I will never do manual upgrades again. We will definitely be using AutoUpgrade for our 23c upgrades!

If you are new to AutoUpgrade I have some examples of using it when I was doing 21c upgrades (see here). That should help you get started.

What are you going to test?

Testing is always a big stumbling block for us. We are not very far down the path of automated testing, which means we need bodies to complete testing. The availability of testing resource is always an issue. There are times of the year when it is extremely unlikely people will be made available, so planning this resource is really important.

So what’s the plan?

It’s always a balancing act around support for the OS, database and application vendors. Ultimately each project will have to be dealt with on a case by case basis, as the allocation of testing resources and potential disruption to the business have to be factored in. Everything is open to change, but…

  • Our default stance is we will upgrade to Oracle 23c on OL9. We will build new OL9 servers and install 23c on them, then use AutoUpgrade to migrate and upgrade the databases. For some of our internal developments I feel this could happen relatively quickly (kiss of death).
  • Application vendor support is often a sticking point for us, and timing will have to factor in the OL7 end of life. If support for 19c on OL9 comes in time, we may migrate our 19c databases to OL9, while we wait for a vendor to support Oracle 23c. Alternatively we could pay for extended support for OL7, and do the OS and database in one go once the application vendor is happy.

I realise this has been a bit of a ramble, but I just wanted to write it down to get things straight in my own head. 🙂

Cheers

Tim…

PS. I have some technical posts on upgrading to 23c that will be released once the on-prem version of 23c goes GA.

Why are RHEL/Oracle Linux upgrades still so unreliable?

The RHEL distribution is still really popular in the enterprise Linux market. The stats sometimes look worse because the users are split across both RHEL and all of its many clones, but it still represents a massive chunk of the enterprise market. With that in mind, why are RHEL upgrades still so unreliable?

My history of OS upgrades

Over the years I’ve done loads of operating system upgrades. In “recent” times I’ve done Windows upgrades (8 -> 8.1 -> 10 -> 11) and loads of Intel macOS upgrades with minimal drama. That’s not the case with Linux.

About a decade ago I was using Fedup to upgrade between versions of Fedora. That was later replace by the DNF upgrade process. More recently I’ve tried my hand using Leapp to upgrade Oracle Linux. I’m no expert, but these upgrades always feel really janky. You never really know what you are going to get until the upgrade is complete. The milage varies depending on configuration of the server, and what software has been installed on it. In some cases you have a system that is running fine. In some cases you have a running server, but a bunch of the configuration has to be redone to make your application work. In some cases you have bricked your server. Not exactly confidence building.

So what is the alternative?

My overall feel has always been don’t upgrade! Build new kit, migrate across to it, then ditch the old kit.

In the past we would typically make that process fit with the lifespan of the physical kit, but now when we use virtual machines we could potentially have a VM that outlives the physical kit many times over, so the desire to migrate is not so pressing, and a reliable in-place upgrade process is more desirable.

Migrating to a new server comes with a hole bunch of overheads. It’s a lot more work than an in-place OS upgrade.

Infrastructure as Code

If you have true infrastructure as code, you can build new systems really quickly, including all the VMs, networking and firewalls, which just leaves you with the data migration to worry about. If you are not in that position, it’s a nightmare of service tickets and endless waiting.

Questions

Is it too much to ask for such a dominant operating system to have reliable upgrades?

I would love to know your experiences of OS upgrades for RHEL and its clones. Have I just been really unlucky with Leapp, or are upgrade really as bad as I think?

Cheers

Tim…

Update 1: I ran a poll on Twitter. There weren’t many responses, but at least I can see I’m not completely alone. 🙂

Update 2: There are some interesting responses in the comments. Well worth you giving them a look.

Oracle product certifications on Oracle Linux 9. What is going on?

See update at the end.

Warning. Rant mode enabled…

It’s over a year since Oracle Linux 9 was released and we still haven’t seen any certifications of Oracle products on it. What is going on?

Check out the really important update at the end of the post.

Setting the scene

Here are the current support dates for Oracle Linux.

ReleaseGA DatePremier Support EndsExtended Support Ends
OL6Feb 2011Mar 2021Jul 2024
OL7Jul 2014Jul 2024Jun 2026
OL8Jul 2019Jul 2029Jul 2031
OL9Jun 2022Jun 2032Jun 2034

So unless you want to start paying for extended support, you need to be getting rid of OL7 soon. The problem is, you can only move to OL8, because OL9 is not supported for Oracle 19c yet.

So if you are trying to do the right thing by migrating off OL7, you are being forced on to an old version of the OS, which in turn reduces the amount of time you can run before the next OS move.

The next long term release of the Oracle database is Oracle 23c. It’s still in beta, so we don’t know what it will be supported on yet, but 23c Free is only available for RHEL8/OL8 at this time.

Your choices?

So your choices are:

  • Migrate on to OL8, which solves your OS support issue, and will allow you to upgrade to 23c when it is released.
  • Wait for 23c to be released and hope it is supported on OL9. If it is, do a big bang migration to OL9 and 23c, hoping you can get it all done before the OL7 support deadline.

I don’t like either of these choices.

What do I want?

Apart from unlimited wishes, I want:

  • A definitive statement about if Oracle 19c will ever be supported on OL9. If it will be, when is that likely to happen? My guess is this will never happen, but I want an official statement ASAP.
  • A definitive statement about support for Oracle 23c on OL9. Will it be supported when 23c goes GA?
  • I guess it would also be nice if RHEL/OL upgrades actually worked reliably, but that is the subject for another rant!

Conclusion

I think like most people, I just want some clarity. It’s hard to plan when you don’t have any of the vital information. The thought of having to move to OL8 rather than OL9 is kind-of depressing… 😞

Cheers

Tim…

Update from Oracle: “Oracle Database product management has confirmed that when Oracle Database 23c ships, it will be certified for both OL8 and OL9. Also, Oracle Database 19c will be certified on OL9 before end of 2023.”

Update: Oracle have now certified Oracle 19c on Oracle Linux 9, as discussed here.

Fedora 38 and Oracle

Fedora 38 was released recently. Here comes the standard warning.

Here are the usual things I do when a new version of Fedora comes out.

This is not a recommendation. I just like messing about with this stuff, as explained in the first link.

I pushed Vagrant builds to my GitHub.

If you want to try these you will need to build a Fedora 38 box. You can do that using Packer. There is an example of that here.

What’s New?

So what’s new with Fedora 38? You can read about it here.

Cheers

Tim…

Bringing new life to an old MacBook Pro

I’ve got an old MacBook Pro (13-inch, Mid 2009) that sits next to my bed. I use it for watching films and streaming rain sounds while I sleep. It stopped getting OS updates a while ago, and more recently it doesn’t want to play Prime videos, so I figured it was time to take action.

It’s got a Core 2 Duo processor, 8G RAM and a 500G SSD, so it’s not great, but it’s more than capable of doing what I need it to do…

I would like to say I did a lot of research to decide the best OS for it, but that would be a lie. I figured I wanted Ubuntu, and I wanted to run a simple window manager, so I picked Ubuntu Mate.

  • I downloaded the ISO to my Windows 11 desktop.
  • Used the Portable Apps version of Rufus to burn it onto a USB stick.
  • Put the USB stick into my old MBP.
  • Restarted the MBP and held down the “alt” key to give me the boot menu, and picked the USB stick.
  • The live version of Ubuntu Mate started and I clicked the install icon.
  • A few questions later and I had a running Ubuntu Mate installation.

The first problem was I didn’t have wifi access, so I had to plug directly into my router to get internet access. I installed a suitable driver for my wifi card, did an update and I was ready to go. If I had read the installer pages properly, I could have loaded the proprietary drivers as part of the install, but randomly clicking the “Continue” button without reading is the way I roll. 🙂

I installed KeePassXC and DropBox to get access to my passwords, and that’s about it. It’s working really well, and saved me having to think about what to replace the old laptop with.

So my current operating system landscape looks like this:

  • My main workstation is a Dell XPS laptop running Windows 11.
  • I also have a MacBook Pro 15-inch on my desk running macOS Big Sur, which I use to stream films while I work, and test my Vagrant builds on macOS.
  • I have a server running Oracle Linux 7, which I tend not to use much these days as my XPS laptop has more than enough power to do most things I want to do in VirtualBox VMs.
  • I have the old MacBook Pro 13-inch next to my bed running Ubuntu Mate.

I believe in operating system diversity… 🙂

I know people become fanboys/fangirls/fanpeople of their operating system of choice, but I really can’t be bothered with all that anymore. I used Red Hat Linux, then Fedora as my main desktop for 10+ years. I spent a few years with macOS as my daily driver. Now I’m happily using Windows. I’m not even that partisan about my Linux distros these days. I use Oracle Linux for all my Oracle software, but I’ll use any Linux distro for non-Oracle stuff. It’s all the same but different… 😉

Cheers

Tim…

Video : Convert CentOS to Oracle Linux

In today’s video we’ll demonstrate how to convert a CentOS installation to an Oracle Linux installation.

The video is based on this post.

The star of today’s video is Dan Norris, in a clip from a previous OpenWorld, which seems like a long distant memory now.

Cheers

Tim…

PS. I know I said there would be no more videos this year, but this was very much “of the moment”, so I thought it was worth doing.

Video : Using Podman With Existing Dockerfiles (Oracle Database and ORDS)

Today’s video shows me using some of my existing Docker builds with Podman. Specifically a 19c database container and an Oracle REST Data Services (ORDS) container.

For those with an understanding of Docker, it should look really familiar, but it does introduce a twist in the form of a pod.

The video is based on this article.

You can see more information about containers here.

The star of today’s video is Bart Sjerps. It was really hard to find a piece of this recording that didn’t have James Morle wittering over everyone on it. 🙂

Cheers

Tim…

Video : Install Podman on Oracle Linux 8 (OL8)

In today’s video we’ll take a look at installing Podman on Oracle Linux 8 (OL8).

This is based on the article here.

You can see more information about containers here.

The star of today’s video is John King. John’s been on the channel a couple of times before. Once to do a message to one of his super-fans, a work colleague of mine who was impressed that I know John, and once for a regular “.com” appearance. I blame the wife for the terrible audio. 🙂

Cheers

Tim…