VirtualBox 6.1.2

About a month after the release of VirtualBox 6.1 we get the release of VirtualBox 6.1.2, a maintenance release.

The downloads and changelog are in the usual places.

So far I’ve only tried it on a Windows 10 host at work, but it looks fine.

Remember, if you use Vagrant 2.2.6 and this is your first time using VirtualBox 6.1.x you will need to do a couple of config changes to Vagrant, as discussed in this post by Simon Coter. I’m sure once Vagrant 2.2.7 is released this will no longer be necessary.

Happy upgrading! πŸ™‚

Cheers

Tim…

Update: Once I got home I installed VirtualBox 6.1.2 on Windows 10, Oracle Linux 7 and macOS Catalina hosts. It worked fine. πŸ™‚

Video : Oracle REST Data Services (ORDS) : SQL Developer Web

Today’s video is a quick run through SQL Developer Web, introduced in Oracle REST Data Services (ORDS) 19.4.

For those that prefer the written word, this is based on the following article.

You can find all my other ORDS content here.

The reluctant star of today’s video is Tuomas Pystynen, who was held at gunpoint whilst filming this. πŸ™‚

Cheers

Tim…

Automating SQL and PL/SQL Deployments using Liquibase

You’ll have heard me barking on about automation, but one subject that’s been conspicuous by its absence is the automation of SQL and PL/SQL deployments…

I had heard of some products that might work for me, like Flyway and Liquibase, but couldn’t really make up my mind or find the time to start learning them. Next thing I knew, SQLcl got Liquibase built in, so I figured that was probably the decision made for me in terms of product. This also coincided with discussions about making a deployment pipeline for APEX applications, which kind-of focused me. It’s sometimes hard to find the time to learn something when there is not a pressing demand for it…

Despite thinking I would probably be using the SQLcl implentation, I started playing with the regular Liquibase client first. Kind of like starting at grass roots. If you are working in a mixed environment, you might prefer to use the regular client, as it will work with multiple engines.

Once I had found my feet with that, I essentially rewrote the article to use the SQLcl implementation of Liquibase. If you are focused on Oracle, I think this is better than using the standard client.

Both these articles were written more than 3 months ago, but I was holding them back on publishing them for a couple of reasons.

  1. I’m pretty new to this, and I realise some of the ways I’m suggesting to use them do not fall in line with the way I guess many Liquibase users would want to use them. I’m not trying to make out I know better, but I do know what will suit me. I don’t like defining all objects as XML and the Formatted SQL Changelogs don’t look like a natural way to work. I want the developer to do their job in their normal way as much as possible. That means using DDL, DML and PL/SQL scripts.
  2. I thought there was a bug in one aspect of the SQLcl implementation, but thanks to Jeff Smith, I found out it was a problem between my keyboard and seat. πŸ™‚

With a little cajoling from Jeff, I finally released them last night, then found a bunch of typos that quickly got corrected. Why are those never visible until you hit the publish button? πŸ™‚

The biggest shock for most people will probably be that it’s not magic! I’m semi-joking there, but I figure a lot of people assume these products solve your problems, but they don’t. Both Flyway and Liquibase provide a tool set to help you, but ultimately you are going to need to modify the way you work. If you are doing random-ass stuff with no discipline, automation is never going to work for you, regardless of products. If you are prepared to work with some discipline, then tools like Liquibase can help you build the type of automated deployment pipelines you see all the time with other languages and tech stacks.

The ultimate goal is to be able to progress code through environments in a sensible way, making sure all environments are left in the same state, and allow someone to do that promotion of code without having to give them loads of passwords etc. You would probably want a commit in a branch of your source control to trigger this.

So looking back to the APEX deployments, we might think of something like this.

  • A developer finishes their work and exports the current application using APEXExport. It doesn’t have to be that tool, but humans have a way of screwing things up, so having a guaranteed export mechanism makes sense.
  • Code gets checked into your source control. This includes any DDL, DML, packages, and of course the APEX application script.
  • A new changelog is created for the work which includes any necessary scripts, including DDL and DML, as well as the APEX script, all included in the correct order. That new changelog for this piece of work is included in the master changelog, and these are committed to source control.
  • That commit of the changelog, or more likely a merge into a branch triggers the deployment automation.
  • A build agent pulls down the latest source, which will probably include stuff from multiple repositories, then applies it with Liquibase, using the changelog to tell it what to do.

That sounds pretty simple, but depending on your company and how you work, that might be kind-of hard.

  • The master changelog effectively serialises the application of changes to the database. That has to be managed carefully. If stuff is done out of order, or clashes with another developer, that has to be managed. It’s not always a simple process.
  • You will need something to react to commits and merges in source control. In my company we use TeamCity, and I’ve also used GitLab Pipelines to do this type of thing, but if you don’t have any background in these automation tools, then that part of the automation is going to be a learning curve.
  • We also have to consider how we handle actions from privileged accounts. Not all changes in the database are done using the same user.

Probably the biggest factor is the level of commitment you need as a team. It’s a culture change and everyone has to be on board with this. One person manually pushing their stuff into an environment can break all your hard work.

I’m toying with the idea of doing a series of posts to demonstrate such a pipeline, but it’s kind-of difficult to know how to pitch it without making it too specific, or too long and boring. πŸ™‚

Cheers

Tim…

Midlands Microsoft 365 and Azure User Group – January 2020

Last night I went to the Midlands Microsoft 365 and Azure User Group. It was co-organised by Urfaan Azhar and Lee Thatcher from Pure Technology Group, and Adrian Newton from my company.

First up was Matt Fooks speaking about “Microsoft Cloud App Security”. Matt covered a number of use cases, including shadow IT detection, log collection, checking compliance of applications and using policies to protect systems. He demoed a few of these. The flexibility of the cloud is great, but it also allows you to create a security nightmare as your cloud estate grows. MCAS gives you visibility and control over that. I guess the value of this will depend how far down the cloud journey you are. If you’ve got a bit of IaaS that’s being managed centrally, this isn’t going to sounds too interesting. Once you open the gates and let other people/teams get involved in provisioning services, you are going to need something like this to keep some level of control over the sprawl.

I heard one of the attendees mention Snowflake, so I collared him during the break to discuss it. I’m not so interested in the headline stuff. I care more about the boring day-to-day stuff, as people tend not to talk about it. It was a really interesting. Networking is great.

Next up was Richard Harrison with “The Journey from being a DBA Guy for 20 years to becoming an Azure Guy”. Richard was a fellow Oracle ACE before he moved in a new direction, so it was good to see him again. We spent some time chatting before his session, and I kept him for ages after the session chatting about a bunch of cloud related stuff. As the name of the session suggests, this was about his transition. What made him decide to make the move. His first opening into the world of the cloud. Some of the steps along the way and the amount of work involved. Richard is a smart guy, so when he says it’s hard work to keep on top of things due to the rate of change in the cloud, that should be a warning sign for people who are focused on the status quo.

There were some pieces that related nicely to the first session. For example, he discussed the control/governance aspect. To paraphrase, some services like security, budget management and databases are kept under central control, because of the critical nature of them, but people are given pretty much a free reign with platforms within their resource group. Why? Because there are loads of services with loads of features and trying to manage them centrally is practically impossible. I think of this as a move to a high-trust culture. Of course, you have tools to monitor what’s going on to stop people doing crazy stuff, but ultimately you have to start releasing control (where it’s appropriate) or people will look elsewhere. πŸ™‚

I’m hoping Richard will come back again and tell us some more about what he is doing, and I know he’s got some really interesting Oracle-related content that will work really well at Oracle events, so hopefully he’ll throw his hat into that arena too. I don’t want to say anything more because I don’t want to steal his thunder.

Thanks to everyone who turned up to support the event, the speakers, and the sponsor Pure Technology Group. See you at the next event.

Cheers

Tim…

Video : Oracle REST Data Services (ORDS) : Including Hyperlinks in JSON Output

In today’s video we’ll demonstrate how to include hyperlinks in JSON output delivered by Oracle REST Data Services (ORDS).

This is based on this article, which includes some more complete examples.

Good API design is not as simple as you might think, and making sure you pass back relevant information, like URLs for navigating through the services and maybe even service documentation links can make things a lot clearer.

The star of today’s video is James Morle. One of the OGs of performance!

You can check out my ORDS articles and YouTube ORDS playlist here.

Cheers

Tim…

Oracle REST Data Services (ORDS) 19.4 : A quick life update…

Almost 2 weeks ago I wrote about the release of Oracle REST Data Services (ORDS), SQLcl, SQL Developer and SQL Developer Data Modeler 19.4.

I spent the holidays playing around with ORDS quite a bit, so I came back to work today and pushed it out across all Dev and Test installations.

As I’ve mentioned before, at work we run ORDS on Tomcat inside Docker containers. The build we use is very similar to this one I put on GitHub, but with some extra work-related bits added.

What did I have to do for this update?

Two things:

  • Build a new version of our ORDS Docker image with version 19.4 of the ORDS and SQLcl software.
  • Remove all the containers based on this image and fire up new containers.

How long did it take to deploy this to all Dev and Test instances?

The build of the new Docker image took about 5 minutes. It’s mostly just unzipping the software. This can be done before we touch any running containers, so there is no downtime associated with this.

The removal and creation of all the containers took about 5 minutes as well. Each container is created in a second, but the first run with a new version of ORDS has to do the ORDS upgrade in the database, which takes a few minutes sometimes. If there were no ORDS upgrade, the containers start really quickly.

So effectively, in 5 minutes we replaced all the “kit” and ran the ORDS upgrade across everything. I could have done production in that same 5 minute span too, but I’m not allowed to yet. πŸ™‚

Why am I talking about this?

It’s just another example of why containers make more sense than conventional app servers for this type of stuff.

To throw away kit and rebuild it from scratch takes an eternity here. I can do the equivalent with containers in seconds.

Once I’ve tested a new image and proved it works, I can roll that same image out across everything with no worries. If it works against one database, it will work against all the others. That’s the great thing about standardising the approach you take!

And another thing!

I’ve enabled SQL Developer Web on every Dev/Test installation too. Now all I’ve got to do is wait for the right opportunity to use it to save the day when someone is waiting for a firewall change, and act all casual like it’s no big thing! πŸ™‚

So in summary

Containers good! ORDS good!

If you are interested in playing with Docker, you can find more information here.

If you want to learn about ORDS, you can find more information here.

Cheers

Tim…

VirtualBox 6.1 : No compatible version of Vagrant yet! (or is there?)

VirtualBox 6.1 was released on the 11th of December and I totally missed it.

The downloads and changelog are in the usual places.

I spotted it this morning, downloaded it and installed in straight away. I had no installation dramas on Windows 10, macoS Catalina and Oracle Linux 7 hosts.

The problem *for me* was the current version of Vagrant (2.2.6) doesn’t support VirtualBox 6.1 yet. I can’t live without Vagrant these days, so I installed VirtualBOx 6.0.14 again and normal life resumed. See Update.

I’m sure there will be a new release of Vagrant soon that supports VirtualBox 6.1, but for now if you use Vagrant, don’t upgrade to VirtualBox 6.1 yet. I’m sure you won’t have to wait long… See Update.

Cheers

Tim…

Update 1 : A couple of people Peter Wahl and Andrea Cremonesi pointed me at this post by Simon Coter, which contains config changes to allow Vagrant 2.2.6 to run with VirtualBox 6.1.

Update 2 : I’ve followed Simon’s post and it worked fine. If you are using Windows 10 as the host and have done a default installation of Vagrant, the files he’s discussing are in these directories.

C:\HashiCorp\Vagrant\embedded\gems\2.2.6\gems\vagrant-2.2.6\plugins\providers\virtualbox\driver\

C:\HashiCorp\Vagrant\embedded\gems\2.2.6\gems\vagrant-2.2.6\plugins\providers\virtualbox\

Update 3 : I updated by work PC also. It required a couple of reboots to get things working. I think it may be something to do with the way we do security here. It’s working fine now.

2019 : A Year in Review

Well, it seems 2019 was another slightly bizarre year for me.

I just looked back on last year’s review (here) and I’m guessing I had “resting bitch face” while I was reading it…

Conferences

One of the things I mentioned last year was I was taking a year off presenting. I had two events I had already committed to at the start of the year, then dropped off the face of the earth for the rest of the year. I came out of retirement for OpenWorld. I was originally going to pull out of that also, but the wife persuaded me I should go. I’m glad I did.

The reason for not presenting was really to give myself a break. I had been having a lot of trouble travelling, which I suspect is mostly down to being so fat. It was good not to have the hassle of travelling, but I did miss seeing folks. I’ve done some presentations at work during the year, so it’s not like I’ve done nothing…

The problem with not doing conferences was that work was very busy and I didn’t take many holidays, so I was basically swapping one set of stress for another.

I’m thinking I might do a few things this year. I’m not going to go mad and try and speak at loads of events, but I’ve got to get back on the horse. Of course, the first thing is to think of something to present…

If you’ve followed the blog, you’ll know I’ve been going to a few local meetups for Docker, DevOps and Azure. I’m a tourist, rather than a speaker. It’s good to do something different!

Videos

Last year I said I wanted to start doing some videos on my YouTube Channel again. I was “on a break” for a while before that. Well I started at the end of January and I think I’ve done about 44 videos this year. There are a few new playlists, and some additions to existing ones listed here.

I like doing the videos, and it alleviates my guilt at not presenting, but they do take up a lot of time. I know some people have asked for longer, more in-depth videos, but I would estimate that for every minute of the final video, it takes me about 60 minutes of work, so a 5 minute video takes me about 5 hours. With that in mind, the chances of me doing a long-form video are remote.

I know some people can just record themselves talking, but presenting takes a lot of prep for me. For a live presentation I work for days/weeks so I can look casual. For videos it’s a little easier as you can edit stuff and trim it down, but it still takes time…

If you’ve watched my videos, you’ll know I put in little cameos of people saying “.com” to finish my introduction line. I went to put out the latest collection of them (Volume 3) and noticed I had never uploaded Volume 2, so today I posted two short videos. Thanks to everyone who has helped.

I know it’s stupid, but I like it. πŸ™‚

Work

Work continues to be problematic. I could blame the company, but ultimately it comes down to me. I’ve got to learn to walk away. I keep doing stuff in an attempt to make a difference, but things progress at a glacial pace and I get frustrated and think if I do more I can get things moving faster. I can’t. You can lead a horse to water, but that doesn’t make it a duck.

I’m going to try and stop moaning about work, and focus on fixing me. It’s too easy to get a victim mentality about this. I don’t need the job. I know the problems there are not my fault. Fuck ’em!

Website

The website it ticking along as usual. The numbers are similar to last year. I did over 130 blog posts and a hundred articles. I’ve got a bunch of other stuff waiting in the wings, but I can’t publish them yet. In a couple of cases I’ve got multiple related articles, but they are kind-of blocked until some bugs get fixed. It’s a little frustrating, but…

Maintenance of the website takes more time each year. The de-support of the non-CDB architecture in Oracle 20c has got people into a PDB frame of mind, and a lot more people have started to read the multitenant articles on my website. I’ve had to go through them a few times making changes to make things a little clearer, or put in some updates where a feature has changed slightly. Today Patrick Jolliffe sent me a message about a change, so I had to work through it, check it and amend the article. It’s a dirty job, but someone’s got to do it!

Similar to last year, quite a lot of effort has been going into my Docker and Vagrant stuff on GitHub. It’s something I find odd, as it takes up a lot of time and doesn’t really result in anything for my website. Sure, I use them to test stuff and I get a lot of good feedback about them, but I’ve spent nearly 20 years putting stuff on the website, and it feels odd to “produce content” for something that’s not my website.

Other Stuff

Nephew #1 has just turned 18 and nephew #2 has turned 15. December is a busy month. It’s hard to believe how time flies. I bought my house when the couple I shared a house with got pregnant with their first child. That child is now at Uni doing dentistry, and I’ve not got round to buying furniture or curtains yet. I’m not even going to pretend I’m going to do something about it by next year.

Physically I’m a bit of a wreck. I’ve really got to get my shit together, especially if I’m going to start travelling again!

Everything changes, and yet nothing changes… Let’s see what 2020 brings. I hope it’s good for everyone!

Cheers

Tim…