Updates to Vagrant and Docker Builds : ORDS and SQLcl 21.2

The 21.2 version of ORDS and SQLcl dropped at the start of the month. I guess I missed that, as the first I noticed was Alex Nuijten talking about SQLcl 21.2 nearly two weeks later. As soon as I realised they had arrived I downloaded them and went to work.

All of the relevant Vagrant and Docker builds were updated to use ORDS 21.2, SQLcl 21.2 and Tomcat 9.0.50.

You can check out what’s new in this post by Jeff Smith.

The Oracle security patches come out next week, so these builds will be updated again to include the latest versions of OpenJDK (AdoptOpenJDK) and the Oracle database patches where necessary.

SQL Developer and Database Modeler

These aren’t anything to do with my builds, but thought is was worth mentioning. The 21.2 version of SQL Developer and Data Modeler have been available for the last few days. You can read Jeff’s announcement here.

Cheers

Tim…

Vagrant and Docker Builds: ORDS 20.4, SQLcl 20.4 and Database Patches

The January Oracle quarterly patches were released yesterday, which prompted me to do some new builds.

We got Oracle REST Data Services (ORDS) 20.4 and SQLcl 20.4, which I use in a number of my Vagrant and Docker builds, so I updated them and ran some builds.

The Vagrant database builds, which include ORDS, can be found here.

The Docker ORDS builds can be found here.

I also updated Tomcat to 9.0.41 and OpenJDK to 11.0.10_9 from AdoptOpenJDK.

Once I finished those I decided to try out the Oracle database 19c (19.10) OJVM+DB combo patch on a single instance build. That went fine. You can see that build here.

Since that went OK, I figured it was worth trying to update my OL8 19c RAC build with the 19.10 OJVM+GI combo patch. That also went fine. You can see that build there.

I wasn’t really expecting to cover so much ground so quickly, but that’s the great thing about automation. πŸ™‚

Tomorrow I’ve got to start putting together all the patch scripts for work. It’s always a bit tedious because I have to deal with a lot more products and variations, and I have to make sure I don’t screw up. Happy patching… πŸ™‚

Cheers

Tim…

PS. If you are interesting in ORDS, SQLcl, Vagrant or Docker, these might help.

Vagrant & Docker Builds : ORDS and SQLcl 20.3

In a previous post I discussed the recent release of APEX 20.2 and the subsequent builds it triggered. Last night I pulled down ORDS 20.3 and SQLcl 20.3, so I updated my Vagrant and Docker builds again.

The ORDS download page is here. At the time of writing, the SQLcl download page is still showing 20.2, but the SQL Developer download page has a link for the 20.3 SQLcl download at the bottom. Both these versions have been available for about a week. Update: It’s showing 20.3 on both SQLcl pages now. πŸ™‚

Vagrant

All my GitHub Vagrant builds that include ORDS and SQLcl have been updated to use version 20.3.

I had previously updated Tomcat, and a few days ago I updated Java as soon as AdoptOpenJDK was available.

Docker

This is pretty much the same as the Vagrant story.

The relevant GitHub Docker builds, like the ORDS containers, have been updated to include ORDS and SQLcl 20.3.

They are on the latest release of Tomcat and Java from AdoptOpenJDK.

Conclusion

As always, this is made simple using automation! πŸ™‚

Cheers

Tim…

Data Pump : API for PL/SQL (DBMS_DATAPUMP) and REST (ORDS)

I’m on holiday, so it’s a lot easier to tick things off my giant list at the moment. πŸ™‚

Someone asked me about the DBMS_DATAPUMP package, which is a PL/SQL API for Data Pump. I knew I had written about it, but it turned out I only had one little example in my main Data Pump article here. I put it on my list to expand it a little with some other common examples. I finally got round to this here.

It’s by no means an exhaustive list of what the DBMS_DATAPUMP package can do, but if someone new is coming to it, it puts a bit more meat on the bones compared to what I had before.

One of the reasons that moved that up the list was I was thinking about looking at the ORDS Database API endpoints related to Data Pump, which resulted in this.

That was a rather odd experience for a few of reasons.

  • There were some issues with authentication. I expected it to work with both the default administrator and an ORDS enabled schema, but it only worked with the latter. That was confusing the hell out of me for a time. I’m told it will work with both in the future…
  • There are two ways to use Data Pump. There is a REST endpoint “/database/datapump/jobs/” that can perform all the actions, and there are two RPC-style endpoints “/database/datapump/export” and “/database/datapump/import”. I decided that RPC is not for me and chose to ignore it. I’m a lot happier with the REST endpoint.
  • The API has extremely limited functionality at the moment.

I guess I was expecting this to feel like a wrapper over the DBMS_DATAPUMP package, but it is not like that at all. In this version it’s an API that covers a few basic tasks and completely locks you out of everything else. I can’t imagine many DBAs being able to use it, but it might be ideal for some developers if you are OK with giving them DBA credentials to the database. I wrote some of my thoughts/concerns at the bottom of the article here.

Despite my reservations, this is effectively version 1.0 of this functionality. I would suggest people try it out and feedback their requirements to the ORDS folks, so it can be improved.

On a more general note, it’s clear the ORDS Database API needs a rethink in terms of access management. It seems to be a bit of an all or nothing at the moment. I think it’s safe to say that as the APIs expand, there will be a range of people needing to use them, each with very different security constraints. I might want someone to be able to perform table or schema-level exports/imports, but I will want to control what they can do, and by giving them the access to do this, it doesn’t mean I want them to be able to run DBCA commands to create new instances as well. I think you get my point. It would seem like the access to the APIs need to respond to the underlying roles and privileges for the user, not using a separate ORDS parameter mechanism. Just my thought…

Cheers

Tim…

Oracle REST Data Services (ORDS) : Database APIs – First Steps

In my never ending quest for automation, I finally got round to looking at the Oracle REST Data Services (ORDS) Database APIs.

These have been around for some time, but I was testing them for the first time using ORDS version 20.2, so I was basing my tests on that version of the documentation, and more importantly version 20 of the APIs.

The are several sets of APIs, and they don’t have the same dependencies or authentication methods. It’s not that big a deal once you know what’s going on, but it confused the hell out of me for a while, and the documentation doesn’t give you much of a steer for some of this.

PDB Lifecycle Management

My first tests were of the PDB Lifecycle Management endpoints. I enable all the relevant features in my normal installation, but there was one big road block. I always install ORDS in the PDB, and this feature only works if ORDS is installed in the root container. This makes sense as the management of PDBs is done at the root container level, but I prefer not to put anything in the root container if I can help it. I uninstalled and reinstalled ORDS so I could give it a go. This resulted in this article.

The PDB Lifecycle Management functionality seemed better suited to a self-contained article, as it is only available from a CDB installation, has its own authentication setup and only has a small number of endpoints. The available APIs are kind-of basic, but they could still be useful. It will be interesting to see if this expands to fit all the possible requirements for a PDB, which are now pretty large. I suspect not.

Most of the other stuff

Next up was “most of the other stuff”. There are too many endpoints to go into any level of detail in a single article, so I figured this should focus on the setup to use most of the other endpoints.

There are two methods of authentication discussed. The default administrator approach, which is good because it hides the database credentials from the user making the API calls. Instead they use application server credentials mapped to the “System Administrator” role. This is similar to that used by the PDB Lifecycle Management feature, except that uses the “SQL Administrator” role, and the ORDS properties are different..

The other approach is to use an ORDS enabled schema. This will be very familiar to people already using ORDS, but it comes with one big disadvantage compared to the previous method. For this functionality you have to expose the database credentials of the ORDS enabled schema to the person calling the API. Normally we would not expose these, instead using another form of authentication (Basic, OAUTH2 etc.) to allow the user to gain access. Even then the ORDS enabled schema would be a weak user that only has access to the specific objects we want it to interact with, but in this case it’s a DBA user, so it makes me nervous. Using the default administrator method the caller is constrained to some extent by the APIs, but with the database credentials they have everything if they have direct access to the database server. It’s probably insignificant when you consider the amount of damage someone could do with the APIs alone, but I feel myself wincing a little when putting DBA credentials into a HTTPS call.

For me as a DBA/Developer I would see myself as the person using these APIs to develop something, whether that was an automation, or an application. If this were to be handed over to a developer to do the work, these security questions may be a much bigger issue.

Having read that, you are probably thinking, just use the default administrator method then. I would, only some APIs don’t work with that method. Some seem to only work with the ORDS enabled schema method for authentication, while others only work with the default administrator method. What’s more, I don’t see any reference to this in the documentation. The API doc doesn’t even mention the default administrator approach, and the setup doc doesn’t mention the limitations on any of the approaches except the PDB lifecycle management. As a result, I think you will need to use a mix of the authentication methods if you plan to use a variety of functionality.

The good thing is they can all live side-by-side. At one point I was testing with a CDB installation of ORDS with credentials for PDB Lifecycle Management, default administrator and ORDS enabled schema authentication all configured at the same time. No problem. It’s just confusing when endpoints fail and you have to “trial and error” your way through them. It would be nice if there was a grid of which groups of endpoints need which type of authentication.

Now I am a noob, so maybe I’ve missed the point here, but I spent a long time trying out variations, and this seems like the way it is. If someone can educate me about why I am wrong I will willingly amend the articles, and this blog post. πŸ™‚

Thoughts and what next?

At this point I’ve just been finding my feet, and I’m not sure what I will do next. There are some endpoints that interest me, so I might do separate articles on those, and refer back to the setup in the above articles. Then again, it may feel like just regurgitating the API documentation, so I may not. It’s worth taking a look at the available endpoints, broken down into these main sections.

  • Clusterware CLIs
  • Data Dictionary
  • Environment
  • Fleet Patching and Provisioning
  • General
  • Monitoring
  • Performance
  • Pluggable Database Lifecycle Management

Some will require additional setup, but many will not.

From the look of it, the vast majority of the endpoints are for reporting purposes. There are far fewer that actually allow you to manipulate the contents of the database. You can always write your own services for that, or use REST Enabled SQL to do it I guess. The question will be, can I get enough value out of these APIs as they stand to warrant the investment in time? I’m not sure at this point.

Cheers

Tim…

PS. If you were watching my twitter feed over the weekend and wondered what bit of tech I gave up on. It was this. I’m very stubborn though, so I came back…

Video : Simple Oracle Document Access (SODA) for SQLcl

In today’s video we give a demonstration of Simple Oracle Document Access (SODA) for SQLcl. SODA is a feature of Oracle REST Data Services (ORDS),
but this allows to access the document store directly from SQLcl.

This video is based on the following article.

You might find these useful.

The star of today’s video is Kris Rice, who’s essentially singing “dot com” underwater. πŸ™‚

Cheers

Tim…

Video : Simple Oracle Document Access (SODA) for PL/SQL

In today’s video we give a demonstration of Simple Oracle Document Access (SODA) for PL/SQL. SODA is a feature of Oracle REST Data Services (ORDS),
but this PL/SQL interface for SODA was introduced in Oracle Database 18c.

The video is based on this article.

You might find these useful.

The star of today’s video is the son of Dan Iverson. Dan, not his son, is an Oracle ACE focusing on PeopleSoft and Oracle Cloud Architect. He’s also in Army National Guard, which means he’s already prepared for the zombie apocalypse! πŸ™‚

Cheers

Tim…

Video : Simple Oracle Document Access (SODA) for REST

In today’s video we’ll give a demonstration of Simple Oracle Document Access (SODA) for REST. This is a feature of Oracle REST Data Services (ORDS).

This video is based on the following article.

There are some other resources that might come in handy.

The star of today’s video is the son of Rodrigo Mufalani. Rodrigo is a fellow Oracle ACE and you can check out Rodrigo’s blog here.

Cheers

Tim…

Vagrant and Docker Builds : ORDS 20.2 and SQLcl 20.2 Updates

The recent Oracle REST Data Services (ORDS) 20.2 release prompted my usual reaction. I’ve gone through my Vagrant and Docker builds, and updated them to use ORDS 20.2 and SQLcl 20.2.

The Vagrant database builds, which include ORDS, can be found here.

The Docker ORDS builds can be found here.

There were also some small Tomcat mods.

  • Tomcat upgraded to 9.0.37.
  • HTTP/2 enabled.
  • Compression enabled.
  • Cache-Control enabled for images, CSS and Javascript.

All that went pretty well so as soon as I got to work yesterday I rolled ORDS 20.2 to all non-production environments, and a few “not yet production” environments. If you follow the blog you will know we use Docker for ORDS (similar to my Github builds). It makes rolling out a new version really simple. Just throw away all the containers and replace them with the spangly new ones.

If it’s looking OK after a few days we’ll push it out to the remaining production installations.

Cheers

Tim…

Video : Oracle REST Data Services (ORDS) : Including Hyperlinks in JSON Output

In today’s video we’ll demonstrate how to include hyperlinks in JSON output delivered by Oracle REST Data Services (ORDS).

This is based on this article, which includes some more complete examples.

Good API design is not as simple as you might think, and making sure you pass back relevant information, like URLs for navigating through the services and maybe even service documentation links can make things a lot clearer.

The star of today’s video is James Morle. One of the OGs of performance!

You can check out my ORDS articles and YouTube ORDS playlist here.

Cheers

Tim…