APEX : Keeping up to date is so easy…

Over the years I’ve extolled the virtues of Oracle Application Express (APEX) because of the ease of development. I think low code tools are a massive boon to productivity. Of course there are some tasks that need alternative tools, but for many scenarios low code tools are awesome.

Something else I find really appealing about APEX is the ease of upgrades. I’m not talking about how easy it is to apply the upgrade itself, because updating Java and Tomcat versions on a server is really easy too. I mean how simple it is from a wider perspective.

I was the first person in my company to use APEX. I used it to write some utility type applications, when it was still “forbidden”. Some of these applications were written over a decade ago, and they are still working fine. In that time we’ve had regular APEX upgrades, and they’ve just kept going. No refactoring. No drama.

Of course, they aren’t using all of the new features that were added in subsequent releases, but the important thing is all that development investment was not impacted by staying on the latest APEX release and patch set. In comparison, updating some of our other platforms and frameworks is a nightmare, requiring substantial development effort and testing.

So it’s not just about improving productivity during the development phase. It’s also about the reduction in the total cost of ownership (from a development perspective) over the lifespan of the application.

Just thought I would share that thought, as I upgrade & patch some production systems… 🙂

Cheers

Tim…

Oracle VirtualBox 7.0.14

VirtualBox 7.0.14 has been released.

The downloads and changelog are in the usual places.

I’ve installed it on my Windows 10 and 11 machines with no drama.

Vagrant 2.4.0

Vagrant 2.4.0 is still the current release, so no change there.

If you are new to Vagrant and want to learn, you might find this useful.

Once you understand that, I found the best way of learning more was to look at builds done by other people. You can see all my Vagrant builds here.

I’ve updated my Vagrant builds to include the latest Oracle patches, along with GraalVM, ORDS, SQLcl and Tomcat.

Packer 1.10.0

I use Packer to rebuild my Vagrant boxes (Oracle Linux 7, 8 and 9) so they have the latest guest additions. The current set of boxes were built using Packer 1.10.0. The new version of the boxes can be seen here.

If you are interested in creating your own Packer builds, you might take inspiration from mine, available here.

Packer has been warning about using built in plugins for some time. This version doesn’t use built in plugins at all, so I had to do the following (in bold) to load the plugins, so I could perform a build.

set PATH=%USERPROFILE%\u01\software\hashicorp\packer_1.10.0_windows_amd64;%PATH%
packer plugins install github.com/hashicorp/vagrant
packer plugins install github.com/hashicorp/virtualbox

cd \git\oraclebase\vagrant\packer\ol9

packer build -only virtualbox-iso ol9.json

How did it all go?

The new versions of VirtualBox and Packer worked fine. 🙂

What about the VirtualBox GUI?

Just a quick warning. I do everything using Vagrant, so I rarely look at the VirtualBox GUI. Remember, when I say everything worked fine, I mean for what I use.

Cheers

Tim…

Oracle Database 19c on Oracle Linux 9 (OL9): Installation Articles and Vagrant Builds

Earlier this year I wrote a rant about the lack of product certifications on Oracle Linux 9 (OL9).

One of the points I made was we are having to replace OL7 servers, but were forced to go to OL8 because Oracle 19c was not certified on OL9, and Oracle 23c on-prem is not available.

This blog post by Mike DieTrich changed all that because now 19c is certified on OL9, provided you are on patch 19.19 or above, and are on the correct version of UEK or the RHEL kernel. See Mike’s post for details.

Installation Articles

Of course, this triggered some installation articles.

Vagrant Builds

There are database, RAC and Data Guard vagrant builds here.

Odd Occurrence

I noticed something a little odd when doing these builds using the 19.21 RU patches.

For the database and Data Guard builds I used the DB RU and OJVM combo patch, and I was still forced to fake the distribution using the CV_ASSUME_DISTID environment variable. For the RAC build I used the GI RU and OJVM combo patch, and I didn’t need to fake the distribution.

I went back to the DB build, and instead used the GI RU and OJVM combo patch, and I no longer needed to fake the distribution. So it looks like there is something different about the database patches between these two types of RUs that slightly affect the installation process. It’s no big deal, but it might catch you out.

Oracle 19c is old. Why do you care?

We are in the process of replacing a load of VMs that are currently running OL7, and we want to go to OL9. Prior to this announcement were were going to have to do one of two things.

  • Migrate to 19c on OL8, which would be OK for 23c when it drops, but not ideal as building an OL8 box now seems like a fail.
  • Wait for 23c on-prem to drop and move to 23c on OL9. The problem here is we could run out of time waiting for 23c to come.

This announcement gives us a new option.

  • Migrate to 19c on OL9, then upgrade to 23c when the on-prem version drops.

This third option is way better for us!

Remember

There are a couple of things to remember.

  • You need to be on 19c to upgrade to 23c, so getting your 19c database on an OS that is supported for 23c is important. We’ve had confirmation that 23c will be available for OL8 and OL9 on release.
  • The extended support waiver for 19c was increased from 1 year to 2 years. Mike also wrote about this here. That means you get free extended support for 19c until April 30, 2026.

Conclusion

This is massive for us. I’m very happy!

Cheers

Tim…

Oracle VirtualBox 7.0.12, Vagrant 2.4.0 and Packer 1.9.4

Oracle VirtualBox 7.0.12

VirtualBox 7.0.12 has been released.

The downloads and changelog are in the usual places.

I’ve installed it on my Windows 10 and 11 machines with no drama.

Vagrant 2.4.0

Releases of VirtualBox prompt me to check for new versions of Vagrant. The current version is Vagrant 2.4.0. All my test systems are built with Vagrant, so I installed it as well.

If you are new to Vagrant and want to learn, you might find this useful.

Once you understand that, I found the best way of learning more was to look at builds done by other people. You can see all my Vagrant builds here.

I’ll be doing some updates to my Oracle builds over the coming days, so this will get a lot of testing.

Packer 1.9.4

I use Packer to rebuild my Vagrant boxes (Oracle Linux 7, 8 and 9) so they have the latest guest additions. The current set of boxes were built using Packer 1.9.4. The new version of the boxes can be seen here.

If you are interested in creating your own Packer builds, you might take inspiration from mine, available here.

How did it all go?

The new versions of VirtualBox, Packer and Vagrant all did their jobs fine. 🙂

What about the VirtualBox GUI?

Just a quick warning. I do everything using Vagrant, so I rarely look at the VirtualBox GUI. Remember, when I say everything worked fine, I mean for what I use.

Cheers

Tim…

Joel Kallman Day 2023 : Announcement

Since 2016 we’ve had an Oracle community day where we push out content on the same day to try and get a bit of a community buzz. The name has changed over the years, but in 2021 it was renamed to the “Joel Kallman Day”. Joel was big on community, and it seems like a fitting tribute to him.

When is it?

The date is Wednesday October 11th. That’s just over a week away from today!

How do I get involved?

Here is the way it works.

  • Write a blog post. The title should be in the format “<insert-the-title-here> #JoelKallmanDay“.
  • The content can be pretty much anything. See the section below.
  • Tweet/X out the blog post using the hashtag #JoelKallmanDay.
  • Publishing the posts on the same day allows us to generate a buzz. In previous years loads of people were on twitter retweeting, making it even bigger. The community is spread around the world, so the posts will be released over a 24 hour period.
  • Oracle employees are welcome to join in. This is a community day about anything to do with the Oracle community.

Like previous years, it would be really nice if we could get a bunch of first-timers involved, but it’s also an opportunity to see existing folks blog for the first time in ages! 

The following day I write a summary post that includes links to all the posts that were pushed out through the day. You can see examples here.

What Should I Write About?

Whatever you want to write about. Here are some suggestions that might help you.

  • My favourite feature of {the Oracle-related tech you work on}.
  • What is the next thing on your list to learn.
  • Horror stories. My biggest screw up, and how I fixed it.
  • How the cloud has affected my job.
  • What I get out of the Oracle Community.
  • What feature I would love to see added to {the Oracle-related tech you work on}.
  • The project I worked on that I’m the most proud of. (Related to Oracle tech of course)

It’s not limited to these. You can literally write about anything Oracle-related. The posts can be short, which makes it easy for new people to get involved. If you do want to write about something technical, that’s fine. You can also write a simple overview post and link to more detailed posts on a subject if you like. In the previous years the posts I enjoyed the most were those that showed the human side of things, but that’s just me. Do whatever you like. 

Do I have to write in English?

No! It’s great to see people contributing to their own community. Google Translate does a pretty good job of translating them, so we can still read them.

Do I need to write about Joel or APEX?

I’m sure people would be happy to read stories about Joel, or content about APEX, but you don’t have to write about that. You can write about whatever you want, so long as it has an Oracle and/or community spin…

So you have a little over a week to get something ready!

Cheers

Tim…

Oracle Database 23c Generally Available – Sort Of

Oracle Cloud World is happening, which typically means lots of announcements. One of the welcome announcement was the release of Oracle 23c on OCI Oracle Base Database Service, so there is a production version generally available… Sort of… Why do I say “sort of”?

OCI Oracle Base Database Service

This is a single cloud service, and it’s not available on the free tier, so it’s only available for paying customers that want to use this particular service.

At the time of writing there is no Autonomous Database service for this version, and there is still no full on-prem release.

I thought it was unavailable in my data centre, but Jeff Smith told me 23c GA is only available for Intel shapes at the moment. Once I switched from the default AMD shape to an Intel shape and 23c release was in the database version list. Happy days.

Oracle Database 23c Free

In addition to the OCI Oracle Base Database Service, the announcement post mentions a new version of Oracle Database 23c Free. It is now a “Developer Release”, not a “Developer Preview”. You can get hold of it here.

The slightly confusing thing is there is no difference in the file name, so my immediate impression was it had not actually been released yet. I downloaded the file and did an installation, and sure enough it was version 23.3. It would have been nice if there was an indication of the update on the page, or a version number in the file name…

For those that previously used Oracle XE, Oracle Database 23c Free is now the natural replacement, so go crazy with it. 🙂

Here’s how you can get started.

  • Install documentation here.
  • My installation article here.
  • My Vagrant build here.
  • VirtualBox appliance from Oracle here.
  • Docker image from Oracle here.

Documentation

The documentation links we’ve been using for 23c Free are no longer marked as “Free”. It is the normal GA documentation now.

My 23c Articles

You can see all my 23c articles here. There are still some I can’t publish until the on-prem GA release happens…

On-Prem Release

As mentioned previously, there is no full on-prem release for Oracle 23c, so you can’t start planning your upgrades from 19c yet. It’s really good to have an updated version of Oracle Database 23c Free so quickly for home use, but from a work perspective 23c won’t really exist for me until there is an on-prem release.

I’m hoping that won’t be long, but time will tell.

Cheers

Tim…

Update: Someone pointed me to Release Schedule of Current Database Releases (Doc ID 742060.1), which has now been updated to include 23c. It says the on-prem releases for 23c will start appearing during 1H CY2024, which is the first half of 2024.

Oracle databases on other clouds?

Oracle just announced the expansion of their partnership with Microsoft to deliver Oracle database services in Azure. You can read the blog post here.

Oracle and Microsoft expand partnership to deliver Oracle database services in Azure

This is a very interesting development for a number of reasons. Here are some of my thoughts…

The database is not a driving factor in cloud provider selection

Over the years Oracle have been playing the game of making Oracle Cloud look like the most attractive place to run Oracle databases. What I think they had lost sight of is the database is not the driving factor in the choice of which cloud provider to pick. It might not even be part of the decision process. Quite often there are other factors that have much more sway.

This move is a welcome step, but I feel like it should just be the beginning!

Software should run on every cloud

I’m sure some people in Oracle now consider themselves a “cloud company”, but I think most of us still consider Oracle as a software company. Oracle rely on sales/licensing to make their money. As a result, anything that blocks the sale of a product is a problem.

Whatever cloud provider I pick, Oracle should be hoping I choose their software to run on my systems.

Not only are companies multi-cloud, but they already use multiple database engines. If there is any friction to using your product, they can go elsewhere.

A welcome start, but…

I’m really glad Oracle have taken this step. Microsoft are the second largest cloud provider, and anything that simplifies using Oracle databases on Azure is a good thing. IMHO this should be the start of the journey. Oracle should be trying to get similar partnerships with other cloud providers too.

Unless I’m missing something, or it’s not been amended yet, this document looks unchanged to me.

The pricing of Oracle on Azure seems to be unchanged, and we are still limited to AWS and Azure as “Authorized Cloud Environments”.

What would I do?

There are two main things:

  • I would make the pricing consistent across all cloud providers and on-prem.
  • I would increase the number of “Authorized Cloud Providers”.

The stats vary, but a quick Google shows me the following market share information.

  • AWS : 32%
  • Azure : 22%
  • Google : 11%
  • Alibaba : 4%
  • Oracle : 2%

Just adding Google and Alibaba would add another 15% of the cloud market as potential customers.

What do I know?

I’m sure someone will tell me I don’t know what I’m talking about, and maybe they are right. I just think Oracle should be making sure most/all of their software is available to run anywhere people want to run it.

As I said before, I’m really happy about this announcement, but I think it needs to be the first step on a longer journey.

Cheers

Tim…

When Overlapping CRON Jobs Attack…

We recently had an issue, which I suspect was caused by overlapping CRON jobs. By that I mean a CRON job had not completed its run by the time it was scheduled to run again.

CRON

If you’ve used UNIX/Linux you’ve probably scheduled a task using CRON. We’ve got loads of CRON jobs on some of our systems. The problem with CRON is it doesn’t care about overlapping jobs. If you schedule something to run every 10 minutes, but the task takes 30 minutes to complete, you will get overlapping runs. In some situations this can degrade performance to the point where each run gets progressively longer, meaning there are more and more overlaps. Eventually things can go bang!

Fortunately there is a really easy solution to this. Just use “flock”.

Let’s say we have a job that runs every 10 minutes.

*/10 * * * * /u01/scripts/my_job.sh > /dev/null 2>&1

We can use flock protect it by providing a lock file. The job can only run if it can lock the file.

*/10 * * * * /usr/bin/flock -n /tmp/my_job.lockfile /u01/scripts/my_job.sh > /dev/null 2>&1

In one simple move we have prevented overlapping jobs.

Remember, each job will need a separate lock file. In the following example we have three separate scripts, so we need three separate lock files.

*/10 * * * * /usr/bin/flock -n /tmp/my_job1.lockfile /u01/scripts/my_job1.sh > /dev/null 2>&1
*/10 * * * * /usr/bin/flock -n /tmp/my_job2.lockfile /u01/scripts/my_job2.sh > /dev/null 2>&1
*/10 * * * * /usr/bin/flock -n /tmp/my_job3.lockfile /u01/scripts/my_job3.sh > /dev/null 2>&1

Oracle Scheduler (DBMS_SCHEDULER)

The Oracle Scheduler (DBMS_SCHEDULER) doesn’t suffer from overlapping jobs. The previous run must be complete before the next run can happen. If we have a really slow bit of code that takes 30 minutes to run, it is safe to schedule it to run every 10 minutes, even though it may seem a little stupid.

begin
  dbms_scheduler.create_job (
    job_name        => 'slow_job',
    job_type        => 'plsql_block',
    job_action      => 'begin my_30_min_procedure; end;',
    start_date      => systimestamp,
    repeat_interval => 'freq=minutely; interval=10; bysecond=0;',
    enabled         => true);
end;
/

The Oracle Scheduler also has a bunch of other features that CRON doesn’t have. See here.

Conclusion

I’m not a massive fan of CRON. For many database tasks I think the Oracle Scheduler is far superior. If you are going to use CRON, please use it safely. 🙂

Cheers

Tim…

Planning our next Oracle database upgrades. A customer perspective…

Upgrading a database is not about the technical side of things. It’s about the planning that is required. I can upgrade a database in a few minutes, but the project to upgrade all the environments for a specific application can take months to complete. In this post I want to discuss some of the issues we are discussing at the moment regarding our future Oracle 23c upgrades.

What support dates are relevant?

Here are the support dates for the Oracle database.

ReleasePremier Support (PS) EndsFree Extended Support (ES) Ends
19cApril 30 2024April 30 2025
21cApril 30 2024N/A

Release Schedule of Current Database Releases (Doc ID 742060.1)

Here are the current support dates for Oracle Linux.

ReleaseGA DatePremier Support EndsExtended Support Ends
OL7Jul 2014Jul 2024Jun 2026
OL8Jul 2019Jul 2029Jul 2031
OL9Jun 2022Jun 2032Jun 2034

Lifetime Support Policy: Coverage for Oracle Open Source Service Offerings

What database versions are we currently running?

To upgrade directly to 23c we must be running Oracle 19c or 21c.

All our databases are on 19c, so this puts us in a good position. It took a lot of pain and effort to get us to 19c, but it was worth it!

PDB or non-CDB architecture?

The non-CDB architecture was deprecated in 12.1.0.2, but it has remained supported up to, and including, 19c. So Oracle 23c will be the first long term release where the non-CDB architecture is not an option. If you’ve not got up to speed on pluggable databases, you better get started soon! (Multitenant Articles)

With one exception, we have PDBs across the board, so there is nothing new for us here. It sometimes felt like I was swimming against the tide by pushing PDBs so hard over the years, but it all seems worth it now.

What OS are you running on?

I’m going to conveniently forget that anything other than RHEL/OL exist, because other operating systems don’t exist for me in the context of running Oracle databases.

It took us a long time to migrate from OL6 to OL7. The majority of our Oracle databases are currently still running on OL7, which is fast approaching end of life. Since Oracle 23c will not be supported on OL7, we are going to need to migrate to a newer operating system. I wrote about my scepticism around in-place RHEL/OL upgrades (here), so that leaves us two choices.

  • Move our existing databases to a new OS now, then upgrade to 23c later.
  • Wait for the 23c upgrade, and do a big-bang OS migration and database upgrade.

What’s stopping us from doing the first option now? Nothing really. We could migrate our 19c databases to OL8 servers. It would be nicer to migrate them to OL9, but it is not supported for 19c yet. I recently wrote a rant about product certifications on Oracle Linux 9, which resulted in this response from Oracle.

“Oracle Database product management has confirmed that when Oracle Database 23c ships, it will be certified for both OL8 and OL9. Also, Oracle Database 19c will be certified on OL9 before end of 2023.”

That’s really good news, as it gives us more options when we move forward.

Will Oracle Linux exist in the future? Yes!

Just as I thought I had got my head around the sequence of events, RHEL dropped the bombshell about how they would distribute their source in future (see here). This raised concerns about if RHEL clones such as Oracle Linux, Rocky Linux and AlmaLinux could even exist in future.

Without knowing the future of our main operating system, we were questioning what to deploy on new servers. Do we continue with OL or switch to RHEL? Rocky and Alma released some “don’t panic” messages, but Oracle were very quiet. I wasn’t surprised by that, because Oracle don’t say anything until it has passed through legal, but as a customer it was a very nervy time.

A couple of days ago we got a statement from Oracle (here), with a firm commitment to the future of Oracle Linux. I immediately spoke to our system admins and said OL8/OL9 is back on the table. Phew!

I have my own opinions on the RHEL vs clones situation, but as a customer it’s not about politics. I just need to know the OS we are using is still going to exist, and will be supported for our databases. In that respect the statement from Oracle was very welcome!

Do you need a hardware refresh?

If you are running on physical kit, you need to check your maintenance agreements. You may need a hardware fresh to keep everything up to date.

We run everything on virtual machines (VMs), so the hardware changes to the clusters have no impact on our VMs. We’ve had at least one hardware refresh during the lifespan of some of our database VMs.

Thanks to Ludovico Caldara for mentioning this point.

What versions do vendors support?

We use a lot of third party applications, and some of the vendors are really slow to certify their applications on new versions of the database and operating systems.

Ultimately we will make a choice on destination versions and timings based on application vendor support.

Manual or AutoUpgrade?

In Oracle 23c manual upgrades are deprecated (but still supported). I was late to the party with AutoUpgrade, but now I’ve used it I will never do manual upgrades again. We will definitely be using AutoUpgrade for our 23c upgrades!

If you are new to AutoUpgrade I have some examples of using it when I was doing 21c upgrades (see here). That should help you get started.

What are you going to test?

Testing is always a big stumbling block for us. We are not very far down the path of automated testing, which means we need bodies to complete testing. The availability of testing resource is always an issue. There are times of the year when it is extremely unlikely people will be made available, so planning this resource is really important.

So what’s the plan?

It’s always a balancing act around support for the OS, database and application vendors. Ultimately each project will have to be dealt with on a case by case basis, as the allocation of testing resources and potential disruption to the business have to be factored in. Everything is open to change, but…

  • Our default stance is we will upgrade to Oracle 23c on OL9. We will build new OL9 servers and install 23c on them, then use AutoUpgrade to migrate and upgrade the databases. For some of our internal developments I feel this could happen relatively quickly (kiss of death).
  • Application vendor support is often a sticking point for us, and timing will have to factor in the OL7 end of life. If support for 19c on OL9 comes in time, we may migrate our 19c databases to OL9, while we wait for a vendor to support Oracle 23c. Alternatively we could pay for extended support for OL7, and do the OS and database in one go once the application vendor is happy.

I realise this has been a bit of a ramble, but I just wanted to write it down to get things straight in my own head. 🙂

Cheers

Tim…

PS. I have some technical posts on upgrading to 23c that will be released once the on-prem version of 23c goes GA.

The rise and fall of read-only Oracle homes

Cast your mind back to the olden days of Oracle 18c, where one of the new features introduced was read-only Oracle homes. I wrote about it here.

Read-Only Oracle Homes in Oracle Database 18c

What was the problem?

Mixing executables, log files and configuration files is a really bad idea. Configuration files tend to have a long lifespan, while executables change all the time due to patches and upgrades. It can be difficult to find log files when they are spread across multiple subdirectories in the Oracle home, although the Automatic Diagnostics Repository (ADR) solved a lot of those problems for us.

Historically Oracle have had this mixed approach, with directories such as “dbs”, “network”, and some of the subdirectories of “rdbms” amongst others under the Oracle home.

The solution?

Read-only Oracle homes solve this problem by splitting out most of the common “problem files” into a separate location, leaving the contents of the Oracle home in a mostly read-only state.

The solution was nice enough, and didn’t require mental gymnastics to understand if you paid a little attention. If you have never tried it, check out the article linked about for the basics.

The Rise

When Oracle 21c was released one of the behaviour changes was that read-only Oracle homes were the default. You could still choose to go read/write, but there was a clear statement of direction. Read-only Oracle homes were the future!

The Fall

I noticed during the 23c beta that the read/write Oracle homes were the default again. I raised a question about it a couple of times. I noticed that Oracle 23c Free used a read/write Oracle home too, but figured that wasn’t a “proper installation”, so whatever.

More recently I was going through the 23c installation guide and I saw this.

“With Oracle Database 23c, an Oracle home is available in read/write mode by default. However, you can choose to configure an Oracle home in read-only mode after you have performed a software-only Oracle Database installation.”

https://docs.oracle.com/en/database/oracle/oracle-database/23/ladbi/about-read-only-oracle-home.html#GUID-D848002A-DBAD-48FA-8467-E849630B8E42

So it looks like we’ve flipped back to the read-write Oracle homes by default in 23c. Read-only Oracle homes are still available. Just not by default.

So what?

I prefer the read-only Oracle homes, and of course I can still choose to use them. The difference now is I expect the vast majority of people will use the read/write homes, as people tend to stick with the path of least resistance. So the question is, do I want to turn myself into a minority?

I understand why this is better for backwards compatibility, but I’m a little disappointed. Forcing the change of the default behaviour would have been nice, and better for the product in the long run.

I’ve got plenty of time to consider my options

Ah well. Better to have loved and lost, than to have your eyes gouged out with rusty spoons… 🙂

Cheers

Tim…

PS. I think I may have given some people the impression that read-only Oracle homes are going away. That wasn’t my intention. I’m just talking about the change in default behaviour since Oracle 21c.