Packer by HashiCorp : Second Steps?

In a previous post I mentioned my first steps with Packer by HashiCorp. This is a brief update to that post.

I’ve created a new box called “oracle-7” for Oracle Linux 7 + UEK. This will track the latest OL7 spin. You can find it on Vagrant Cloud here.

I’ve altered all my OL7 Vagrant builds to use this box now.

You will see a new sub-directory called “ol7” under the “packer” directory. This contains the Packer build for this new image.

Cheers

Tim…

Packer by HashiCorp : First Steps

A few days ago I wrote about some Vagrant Box Drama I was having. Martin Bach replied saying I should build my own Vagrant boxes. I’ve built Vagrant boxes manually before, as shown here.

The manual process is just boring, so I’ve tended to use other people’s Vagrant boxes, like “bento/oracle-8”, but then you are at the mercy of what they decide to include/exclude in their box. Martin replied again saying,

“Actually I thought the same until I finally managed to get around automating the whole lots with Packer and Ansible. Works like a dream now and with minimum effort”

Martin Bach

So that kind-of shamed me into taking a look at Packer. πŸ™‚

I’d seen Packer before, but had not really spent any time playing with it, because I didn’t plan on being in the business of maintaining Vagrant box images. Recent events made me revisit that decision a little.

So over the weekend I spent some time playing with Packer. Packer can build all sorts of images, including Vagrant boxes (VirtualBox, VMware, Hyper-V etc.) and images for Cloud providers such as AWS, Azure and Oracle Cloud. I focused on trying to build a Vagrant box for Oracle Linux 8.2 + UEK, and only for a VirtualBox provider, as that’s what I needed.

The Packer docs are “functional”, but not that useful in my opinion. I got a lot more value from Google and digging around other people’s GitHub builds. As usual, you never find quite what you’re looking for, but there are pieces of interest, and ideas you can play with. I was kind-of hoping I could fork someone else’s repository and go from there, but it didn’t work out that way…

It was surprisingly easy to get something up and running. The biggest issue is time. You are doing a Kickstart installation for each test. Even for minimal installations that takes a while to complete, before you get to the point where you are testing your new “tweak”. If you can muscle your way through the boredom, you quickly get to something kind-of useful.

Eventually I got to something I was happy with and tested a bunch of my Vagrant builds against it, and it all seemed fine, so I then uploaded it to Vagrant Cloud.

I’ve already made some changes and uploaded a new version. πŸ™‚

You will see a couple of older manually built boxes of mine under oraclebase. I’ll probably end up deleting those as they are possibly confusing, and definitely not maintained.

I’ve also altered all my OL8 Vagrant builds to use this box now.

You will also see a new sub-directory called “packer”. I think you can guess what’s in there. If I start to do more with this I may move it to its own repository, but for now this is fine.

I’m not really sure what else I will do with Packer from here. I will probably do an Oracle Linux 7 build, which will be very similar to what I already have. This first image is pretty large, as I’ve not paid much attention to reducing it’s size. I’ve looked at what some other builds do, and I’m not sure I agree with some of the stuff they remove. I’m sure I will alter my opinion on this over time.

I’m making no promises about these boxes. That same way I make no promised about any of my GitHub stuff. It’s stuff I’m playing around with, and I will mostly try to keep it up to date, but I’m not an expert and it’s not my job to maintain this. It’s just something that is useful for me, and if you like it, great. If not, there are lots of other places to look for inspiration. πŸ™‚

Cheers

Tim…

Vagrant Box Drama

I had a little bit of VirtualBox and Vagrant drama today.

I was doing my normal thing of recreating some test VMs and I started to get errors like this during the first part of the VM build, before the config scripts ran.

==> default: Machine booted and ready!
Got different reports about installed GuestAdditions version:
Virtualbox on your host claims:
VBoxService inside the vm claims: 6.1.12
Going on, assuming VBoxService is correct…
[default] GuestAdditions seems to be installed (6.1.12) correctly, but not running.
Got different reports about installed GuestAdditions version:
Virtualbox on your host claims:
VBoxService inside the vm claims: 6.1.12
Going on, assuming VBoxService is correct…
Redirecting to /bin/systemctl start vboxadd.service
Job for vboxadd.service failed because the control process exited with error code.
See "systemctl status vboxadd.service" and "journalctl -xe" for details.
Redirecting to /bin/systemctl start vboxadd-service.service
Job for vboxadd-service.service failed because the control process exited with error code.
See "systemctl status vboxadd-service.service" and "journalctl -xe" for details.
Got different reports about installed GuestAdditions version:
Virtualbox on your host claims:
VBoxService inside the vm claims: 6.1.12
Going on, assuming VBoxService is correct…
VirtualBox Guest Additions: Starting.
VirtualBox Guest Additions: Building the VirtualBox Guest Additions kernel
modules. This may take a while.
VirtualBox Guest Additions: To build modules for other installed kernels, run
VirtualBox Guest Additions: /sbin/rcvboxadd quicksetup
VirtualBox Guest Additions: or
VirtualBox Guest Additions: /sbin/rcvboxadd quicksetup all
VirtualBox Guest Additions: Kernel headers not found for target kernel
5.4.17-2011.5.3.el8uek.x86_64. Please install them and execute
/sbin/rcvboxadd setup
ValueError: File context for /opt/VBoxGuestAdditions-6.1.12/other/mount.vboxsf already defined
modprobe vboxguest failed
The log file /var/log/vboxadd-setup.log may contain further information.
==> default: Checking for guest additions in VM…
default: No guest additions were detected on the base box for this VM! Guest
default: additions are required for forwarded ports, shared folders, host only
default: networking, and more. If SSH fails on this machine, please install
default: the guest additions and repackage the box to continue.
default:
default: This is not an error message; everything may continue to work properly,
default: in which case you may ignore this message.
The following SSH command responded with a non-zero exit status.
Vagrant assumes that this means the command failed!
/usr/sbin/rcvboxadd setup
Stdout from the command:
VirtualBox Guest Additions: Starting.
VirtualBox Guest Additions: Building the VirtualBox Guest Additions kernel
modules. This may take a while.
VirtualBox Guest Additions: To build modules for other installed kernels, run
VirtualBox Guest Additions: /sbin/rcvboxadd quicksetup
VirtualBox Guest Additions: or
VirtualBox Guest Additions: /sbin/rcvboxadd quicksetup all
VirtualBox Guest Additions: Kernel headers not found for target kernel
5.4.17-2011.5.3.el8uek.x86_64. Please install them and execute
/sbin/rcvboxadd setup
Stderr from the command:
ValueError: File context for /opt/VBoxGuestAdditions-6.1.12/other/mount.vboxsf already defined
modprobe vboxguest failed
The log file /var/log/vboxadd-setup.log may contain further information.

The VM had booted, but because the guest additions weren’t working it couldn’t mount the shared folders, so none of the setup scripts had run.

Reading the output I figured the kernel-uek-devel package was missing from the Vagrant box, so I did the following…

I connected to the VM, installed the “kernel-uek-devel” package and exited from the VM.

vagrant ssh

sudo dnf install -y kernel-uek-devel
exit

Then I restarted the VM.

vagrant halt
vagrant up

During this second startup of the VM the problem kernel module was rebuilt, and the rest of the configuration steps ran the way you would expect from a normal first-time startup.

One of the issues about using someone else’s Vagrant box is you are at the mercy of what they decide to do with it. In this case I was using the ‘bento/oracle-8’ Vagrant box, which was built using Oracle Linux 8.2 with UEK 6, but the installed guest additions were not happy, and the packages were not present to allow the kernel module to be rebuilt on the fly.

If you are trying to use my Vagrant builds, which mostly use the ‘bento/oracle-8’ Vagrant box, and you are getting this type of issue, now you know what to do about it. Hopefully the next release of this Vagrant box will be less problematic.

Cheers

Tim…

Update: I spent some time figuring out Packer, and now I’ve switched all my OL8 builds to use my own image called ‘oraclebase/oracle-8’.

Vagrant and Docker Builds : ORDS 20.2 and SQLcl 20.2 Updates

The recent Oracle REST Data Services (ORDS) 20.2 release prompted my usual reaction. I’ve gone through my Vagrant and Docker builds, and updated them to use ORDS 20.2 and SQLcl 20.2.

The Vagrant database builds, which include ORDS, can be found here.

The Docker ORDS builds can be found here.

There were also some small Tomcat mods.

  • Tomcat upgraded to 9.0.37.
  • HTTP/2 enabled.
  • Compression enabled.
  • Cache-Control enabled for images, CSS and Javascript.

All that went pretty well so as soon as I got to work yesterday I rolled ORDS 20.2 to all non-production environments, and a few “not yet production” environments. If you follow the blog you will know we use Docker for ORDS (similar to my Github builds). It makes rolling out a new version really simple. Just throw away all the containers and replace them with the spangly new ones.

If it’s looking OK after a few days we’ll push it out to the remaining production installations.

Cheers

Tim…

Oracle Linux 8 (OL8) : Vagrant and Docker builds for 19c Database

A couple of days ago I mentioned the certification of Oracle database 19c on Oracle Linux 8 (OL8) with UEK6.

I’ve had a bunch of OL8 articles and builds for a while, but up until now they’ve included warnings to say they weren’t certified. Over the last couple of evenings I’ve made some changes, so I thought I would summarise then here.

Database 19c on OL8 Article

My article on installing 19c on OL8 (from July last year) now includes the new preinstall package. It also mentions that the installation will work without patches, but it will not be supported unless you include the 19.7 patches, and are using UEK6.

Oracle Database 19c Installation On Oracle Linux 8 (OL8)

Vagrant Build

I’ve had a vagrant build of 19c on OL8 since last year. This has been amended to use the new preinstall package, and to optionally include the 19.7 patches if you’ve downloaded them. By default the patch script is commented out, so folks without a support contract can still use the build. This isn’t meant to be a “supported build”, so I’m not personally bothered about the patches for it, but it seemed a little wrong to not include them, even if it is lip-service.

https://github.com/oraclebase/vagrant/tree/master/database/ol8_19

The base box is ‘bento/oracle-8’, which hasn’t been updated to 8.2 and UEK6 yet. Once again, this doesn’t phase me. The ‘bento/oracle-8’ image tracks the latest release (8.0, 8.1, 8.2 etc.), so at some point it will updated to the latest spin and UEK6.

My go-to Vagrant build has typically been the “ol7_19” build. I’m now going to switch to the “ol7_19” build.

I’ve also added 19c Data Guard build on OL8.

https://github.com/oraclebase/vagrant/tree/master/dataguard/ol8_19

Container (Docker/Podman) Build

Similar to the Vagrant build, I’ve updated by Docker build. It also uses the new preinstall package and includes and optional patch script. I’ve also switched back from the “oraclelinux:8” image to the “oraclelinux:8-slim” base image, which means I had to make some changes, like using “microdnf” instead of “dnf”. Similar to the vagrant build, I’ve left the patch script commented out by default, because I only use this build for playing and demos.

https://github.com/oraclebase/dockerfiles/tree/master/database/ol8_19

My go-to container combination was “ol7_19” plus “ol7_ords”. I’m now going to switch to “ol8_19” plus “ol8_ords” for running APEX 20.1 using containers.

What’s Next?

I’ve got a few things in the pipeline.

  • RAC on OL8 Vagrant Build. I’m unsure at the moment if I will include the patching for this, as it makes it more complicated. I might just stick with the base release. It’s definitely not production, so I’m not sure how much I care about making the build slower and more complicated.
  • I’ve got some Podman stuff I want to talk about that relates to both Vagrant and Docker, but that is better served in a separate post.

Cheers

Tim…

PS. I don’t feel I should need to say this, but experience tells me I should. This stuff on GitHub is just a playground for me. There is no error checking. It’s purposely kept simple so people can read it easily. It’s not production ready. I don’t support it. It’s not scripture. If you find value in it, that’s great. If you don’t, don’t use it. πŸ™‚

PPS. I used the pirate costume as I thought it looked funny. There was no deeper meaning behind this. It was nearly a Pharaoh.

APEX 20.1 : Vagrant and Docker Builds (and Some Comments)

About 2 days ago we saw the announcement of the release of APEX 20.1.

I normally set myself quite an aggressive timetable to get new APEX releases to production. So much of what we do lags behind the curve, to the point where I just want to see it burn, so when I get the opportunity to force the issue, like I do with APEX, I do.

In my typical fashion I move all my builds to the latest release and kind-of forget any prior release ever existed. As result, you will see all my Vagrant and Docker builds have been updated to use APEX 20.1, along with updates of OpenJDK and Tomcat.

The basic installation is the same as always (here), so there is no drama there.

Next week I’ll start upgrading a few of our development environments, and check everything looks OK. Assuming I hit no problems, it will be full steam ahead. Fingers crossed.

The new features are always a big draw for most people, but I tend to start by looking at the deprecated features and desupported features.

  • When I saw APEX_UTIL.STRING_TO_TABLE and APEX_UTIL.TABLE_TO_STRING had been deprecated I was about to lose my shit, then I saw they have just been relocated to the APEX_STRING package. Composure regained. πŸ™‚
  • The desupport of some of the productivity apps is a good thing. Some of them were not so good and having a long list of Meh, is not as good as a smaller list that offer something better. Just my opinion.

As far as the new features in the announcement, here are my initial (and uneducated) thoughts.

  • Redwood. I feel a little “Meh” about this. I don’t love it. I don’t hate it. I guess it makes sense to bring it in line with the current Oracle thang! πŸ™‚ I would have had a little more contrast on the icons. They look quite washed out, but I am not renowned for my design aesthetic. πŸ™‚
  • Faceted Search Enhancements. Love this! When I saw Mike Hichwa demonstrate the first iteration of this I had a When Harry Met Sally – Restaurant Scene moment in my head. All additional functionality is welcome. I think Faceted Search is a great and some might say killer feature.
  • Friendly URLs. My first reaction was, “Oh thank God!”. This was quickly followed by, “Oh my God!”. The thought of what some people will name their page aliases fills me with dread, but it is a welcome addition.
  • Native PDF Printing. Nice.
  • Mega Menus. Love it. The traditional side-bar menu is fine for little apps, but it’s a bit dated, takes up a lot of room and is nowhere near as flexible as this looks. I think I would have made it the default choice, but I can see why sticking with the old style is probably the safer option. πŸ™‚

There are a bunch of other things in the release notes that sound interesting, including remote application deployments using REST Enabled SQL, but I’ll leave you to discover those for yourself.

It’s early days, but this looks like a really nice release… Again…

Cheers

Tim…

APEX 19.2 : Vagrant and Docker Builds

I’m sure anyone who cares knows that APEX 19.2 was officially released on Friday. I did an upgrade of one of our development instances straight away and it worked fine. it’s subsequently gone to a bunch of other development instances. I’ll be pushing to get this out to production as quickly as possible.

Over the weekend I worked through a bunch of my GitHub stuff.

Vagrant : I’ve updated all my Vagrant builds to use APEX 19.2 and the latest versions of Tomcat 9 and OpenJDK 11. I was using newer versions of OpenJDK, but it seemed a bit silly, so I reverted back to the long term support release. I tried updating the base box to ‘bento/oracle-7.7’, but it kept giving me timeouts, so I’ve reverted back to ‘bento/oracle-7.6’ for the moment.

Docker : Same as above, I’ve updated all my Docker builds to use APEX 19.2 and the latest versions of Tomcat 9 and OpenJDK 11. I noticed oraclelinux:8-slim was behaving a little strangely. I thought it was a PATH issue, but I need to spend some time to understand what is happening. It seems you can’t run basic commands like dnf during the build phase. It’s probably something stupid I’m doing, but for now I’ve switched from oraclelinux:8-slim to oraclelinux:8. Just making that switch made everything work as expected.

My Docker builds within the company have gone through a similar process, so as I’m rolling out APEX 19.2 to the databases, I’m also switching the ORDS containers over to the new images. You gotta love containers!

I guess I’ll be working through all this again when the next version of ORDS and SQLcl drop. πŸ™‚

Cheers

Tim…

Vagrant Build of AWX on Oracle Linux 7 Using Docker-Compose Method

I may need to do a bunch of scripting related to our load balancers, and I have the choice of using the API from the servers directly, Ansible Core or the web services exposed by AWX. I wanted to play around with AWX anyway, so that seemed like a good excuse…

First step was to install AWX. It’s pretty easy, but I must admit to spending a few minutes in a state of confusion until I rebooted my brain and started again. Turning things off and on always works. I’m an Oracle Linux person and “I do Docker”, so the obvious choice was to install it using the Docker-Compose method on Oracle Linux 7 (OL7).

The post includes the basic Docker setup, but if you need something a little more, check out the installation article and video.

If you don’t care about the build and just need AWX up quickly, you can use this Vagrant build that does everything for you, including Docker and AWX on Oracle Linux. πŸ™‚

Cheers

Tim…

Video : Vagrant : Oracle Database Build (19c on OL8)

Today’s video is an example of using Vagrant to perform an Oracle database build.

In this example I was using Oracle 19c on Oracle Linux 8. It also installs APEX 19.1, ORDS 19.2, SQLcl 19.2, with ORDS running on Tomcat 9 and OpenJDK 12.

If you’re new to Vagrant, there is an introduction video here. There’s also an article if you prefer to read that.

If you want to play around with some of my other Vagrant builds, you can find them here.

If you want to read about some of the individual pieces that make up this build, you can find them here.

The star of today’s video is Noel Portugal. It’s been far too long since I’ve seen you dude!

Cheers

Tim…

Enterprise Manager Cloud Control 13.3 Vagrant Build

A little short of a year ago I knocked up a Vagrant build to prepare an environment for practising Cloud Control 13.3 installations and upgrades. This just automated the creation of the environment and installation of the database, ready for me to start playing around with the Cloud Control bit. At the time I released these articles.

I didn’t mention anything about the Vagrant build as it didn’t do much more than build the database, so it didn’t seem worth mentioning. It was just a convenience for me.

More recently someone pointed it out on Twitter and I made a note to myself to “finish it off” and make it do a full silent build, then kind-of forgot again.

A few days ago I had a self-induced problem with our Cloud Control server, and I realised I didn’t have the best plan of action for a complete rebuild scenario. I had backups, so I didn’t need to do a rebuild, but that doesn’t stop me wanting to be able to do it, so I did the following things…

I scripted a silent build of the work environment we use. I put together a general article to show how to do a silent build of a simple installation too. If you’re interested you can see it here.

I wrote some EMCLI scripts to do most of the tasks I needed for a complete rebuild. We already use EMCLI for some of the stuff, like jobs, but I filled in the gaps where I had been a bit lazy. Those are all checked into a company Git repo, and they are quite specific to what we need, but there are some basic EMCLI examples available here, if you are interested in getting into EMCLI.

Finally, I made my Vagrant build a fully automated Cloud Control 13.3 build on Oracle database 18c. According to the certification matrix, Oracle 19c is not yet certified for the repository database (but someone on Oracle-L said this certification is imminent (See Update)). If you are interested in playing around with Vagrant, you can find it here. I’ve managed to get away with 6G of memory, but that makes it chronically slow. The more memory you can throw at it the better. πŸ™‚

I didn’t really expect to be revisiting this stuff a year down the line, but it was born out of necessity, or at least necessity for my peace of mind. πŸ™‚

Cheers

Tim…

Update: The OMS is now supported on Oracle database 19c.