I’m 2% DevOps, 3% agile and 4% automated because of 3rd party apps…

I was having a discussion with my boss about the impact of 3rd party apps on the way we work, and how difficult things are when you have to deal with 3rd party apps, as opposed to just writing your own software.

It’s easier to do things well when you are in control of all the pieces. Most of the examples you see are people writing their own software, typically on new projects. That’s very different to dealing with old projects and 3rd party apps. I’ll give you some examples, without trashing the companies responsible for this.

Example 1

Our student system is provided by a 3rd party. The company in question has a really antiquated way of delivering applications. In recent years they’ve tried to resolve this by writing their own delivery mechanism, made up of some custom software and Jenkins. The problem is, this is just a wrapper over the old process, so it is not the most reliable tool in the world. Someone like me would describe it as putting lipstick on a pig.

In addition to that, you have to use a GUI to perform the operations. At this point there is no API to allow you to script operations, which makes building them into a bigger process really problematic. We have internal development which is gradually moving to something resembling CI/CD, but it will never truly meet that goal, because we have to include manual management of things because of the limitations of the 3rd party software.

I’m sure long-term customers see the new delivery mechanism as a great improvement, but it’s not something you would deliver for a new product. It’s less painful than it was, but not really good.

Example 2

We have a publishing system that is written in Java and runs on Tomcat. It is so close to being hands-off, but there are a couple of problems.

  • When you deploy a new version, it starts in maintenance mode and you need manual interaction to click an OK button a few times on a web-based maintenance screen. I’ve never “not clicked” the OK button, so I just want a “just do it” option, so I can let it get on with it.
  • When some features are enabled by the power users, the next restart of the application flips you into maintenance mode. We’ve had P1 incidents because a host failure has caused the VM to start on a new host, and because a user has enabled a new feature in the app, the automatic startup stalls, waiting for me to click the OK button a few times.

There are some other annoyances, which impact on availability and possible topology, as well. There is no way to resolve these because of limitations in the application. All we can do is raise enhancement requests with the vendor.

I could go on with more examples, but I think you get the message.

So what do you do?

It can be quite disheartening when you want to do things well, but you have to keep compromising because of factors outside your control. You have to try not to give up, and just keep plugging away.

  • Don’t make unrealistic comparisons between your environment and others. There’s no point comparing your mixed environment to a software house. I’ve worked in both. They are very different. Take what works. Ditch what doesn’t.
  • Semi-automated processes are better than processes that are 100% manual. Maybe you can use Robotic Process Automation (RPA) to automate what is essentially a manual process.
  • Try to make sure these considerations become part of your procurement process, or you will keep buying crap.
  • Try to be creative and find workarounds, don’t just bury your head in the sand. There’s always *something* you can do to improve things.
  • Even if something is terrible, that doesn’t stop you improving the processes around it.

I guess you should focus on the values, rather than trying to exactly match some prescriptive ideal.

Good luck!

I wrote a series of posts about automation here.



PS. I’m pretty sure my boss is reading this laughing, as I’m following none of this advice myself, but instead stomping round the place like a thirteen year old having a strop because, “Everything is crap!” 🙂

Do you even [ Agile | DevOps ] bruh?

It seems I can’t turn around without getting myself involved in some discussion about Agile or DevOps these days.


I agree with many of the concepts and the aims of Agile, DevOps, Continuous Delivery etc. I find it hard to believe anyone wouldn’t see value in what they are trying to promote. As always, it is how people interpret and implement them that makes all the difference.

It’s just like religion. They all seem to be pretty sound at heart, but let a few lunatics and fundamentalists loose on them and next thing you know…

Things like Agile and DevOps have arisen to address perceived problems. If your organisation doesn’t suffer from those problems, you may not need to consider them, or you may already be doing something like them without knowing you are. 🙂

Your company can be agile, without following Scrum or Kanban. You will inevitably have arrived at similar patterns I guess. Likewise, your streamlining of process, automation of testing and deployment, good communication between silos (if present) may leave you wondering what all the DevOps fuss is about.

I am both a fan and hater of Agile and DevOps. I’m a fan of what they are able to achieve when used correctly. I’m a hater of all the bullshit that surrounds them!

Rant over. 🙂



Agile Morning

You may have noticed some tweets from me on Friday morning on the subject of Agile methodologies. Work arranged for a couple of speakers to come in to speak to us on the subject.

Our company is made up of three types of people.

  • Those that are fully invested in agile and use it on a day-to-dat basis.
  • Those that think they know agile and perform some bastardised version of it.
  • Those that need a warm up before they can even say the word agile.

If I’m honest, I fit into the last group. 🙂 I understand the approach and know some of the terminology, but I’ve never worked in a truly agile team, just those that play at it. I’ve also seen terms like agile, RAD and extreme programming used as an excuse to avoid doing the job properly, so there is a slightly bitter taste in my mouth.

In the recent office reshuffle, the DBAs have been put into a development team. We are still responsible for all production stuff as well, but we share the office with a development team and as usual, will be involved in a number of projects. As a result, getting involved in the whole agile thing is quite important…

The first speaker up was Kay Johnson from IBM, who has been involved in the rollout of agile throughout IBM. She started off with an overview of agile, then talked about how to scale agile through a company as large as IBM. It’s interesting that even the CICS team at IBM now use agile methodologies. Kay is a really enthusiastic speaker and great at getting the crowd involved. I really enjoyed the session. If you get chance to see her speak, you really should make the effort.

Next up was Gavin Barton from the BBC, who was talking about how the BBC has implemented agile throughout the web team. Gavin was a very relaxed and confident speaker, very open and honest when answering questions. Once again, you should definitely make the effort to go and see him if you get the opportunity to hear him speak. As you would expect, there was a lot of common ground between what he said and what Kay said, but it was good to hear how two sets of people arrived at similar conclusions via different routes, and of course some of the differences between the two.

If I combine a few of the take-home messages from both speakers, it would go something like this:

  • Agile is not an excuse to avoid planning and documentation. If anything it requires more discipline than the traditional waterfall approach. The point is you don’t spend so long planning that the requirement has changed by the time you come to code. Likewise, you don’t spend months producing documentation nobody will ever read. Keep it all direct and to the point.
  • Agile is about being user/business driven. Work with the user/business to give them what they want. Don’t assume you know better.
  • Sprints should be no longer than 2 weeks. Gavin suggested for many things they do, 2-3 days is actually better. As Kay said, there is always a way to break stuff down to something smaller, so even massive projects can be broken down to 2 week chunks.
  • Sprints should produce something that is production ready, even if it hasn’t completed UAT and doesn’t make sense to release it on its own. That means the developer should have confidence in what they have produced. A sprint might be one function or one page that can’t be released on its own, but in itself is production ready.
  • Each sprint should include planning and a retrospective look at how things went so you can learn from previous successes and failures.

I’m going to stop here before I embarrass myself with my total lack of knowledge. 🙂 Suffice to say, it all sounds really great providing you invest in it. Do a half-hearted version of it and it’s going to suck. Kind-of like everything in IT I guess.

It was a great morning. I’d like to personally thank Kay and Gavin for taking the time to come and speak to us and of course Dawn for organising it. Everyone I’ve spoken to about it has come away with a really positive view of the event.

I’m not going to act like I’m Billy-Agile now, but over the coming weeks as I get agiled-up, I will no doubt mention it a little. Especially how it fits in with the DBA role. I think that is something that is not so obvious as it maybe is for the developers. Please bear with me. It’s a learning curve and I will make mistakes, but I’m more than happy for you to call me on them… 🙂