Family and Friends post: Day 1 in Perth…

Hello family and friends…

The flight to perth took a little longer than the planned 7 hours and 45 minutes. The last 2 hours were really tough. The lack of sleep meant I was really irritable. Everything was aggrivating my skin and my legs felt really stiff and bloated. I got into the hotel at about 19:00 and by about 19:30 I was in bed. I think it had been something like 40 hours without sleep and about 21 hours in a plane. Not good karma.

This morning I feel much better. I even did a bit of Yoga. The cold has graduated to a full on cough, but at least I’ve got a week off flying. It’s a holiday in Perth today, so everything is shut down. I had a quick look round this morning, but I still feel a bit shakey so I’ve come back to the hotel. I’m gonna do my washing and go back to bed for a bit. πŸ™‚

I’m thinking about taking a boat trip in the next couple of days. Stationary floors feel like they are shaking, so I hope a boat will feel a little more steady. πŸ™‚



Magic Pixies…

I just checked the time in San Francisco and I understand why I’m on the verge of seeing Magic Pixies. It’s about 36 hours since I last slept. I’m about to board a flight that will take 7 hours and 45 minutes and get me to Perth at about 17:00. I really hope I pass out on the plane. I don’t care if I snore or dribble. I just checked out the symtoms of sleep deprevation. I’m so there dude!

Next stop visions of flames and voices in my head. I just hope the voice don’t try and persuade me to eat any animal prducts… πŸ™‚



Tropic Thunder, Kungfu Panda and Deja Vu…

It’s difficult to understand how Tropic Thunder could possible become any more politically incorrect.

Ben Stiller plays a fading action star that tries to kickstart his serious acting career by playing “Simple Jack”, only to fail because he goes “full retard and you should never go full retard!”

Robert Downey Jr. plays an oscar winning method actor who undergoes a controversial skin pigmentation to play a black man in the film. Things go from bad to worse when he can’t get out of character.

Jack Black is an actor/junkie/bad boy in a franchise of films based totally around farting for laughs.

Brandon T. Jackson plays “Alpa Chino”, a rapper turned actor who endorses an energy drink called “Booty Sweat” and energy bars called “Busta Nut”.

You can’t imagine how painful it is watching Robert Downey Jr. “in character” talking to Brandon T. Jackson. It’s all so very wrong, which is why it works so well.

If you are easily offended stay well clear…

I watched Tropic Thunder in SF, but I’ve also seen a bunch of films on the plane, but I’m struggling to remember what… The hours kinda merge. I know I saw Kung Fu Panda, which was pretty cool. I also saw Deja Vu, which is a couple of years old, but it was quite good. I’ve seen at least two more, but I really can’t remember…

I’m sure this airport is built on a floating platform and the sea is really rough. I’m going to lie down on the floor again. Everything stops shaking then. The cleaners keep looking at me like I’m a tramp. You gotta laugh… πŸ™‚



OpenWorld 2008 Summary…

This is the 3rd OpenWorld I’ve attended and I think it was probably the most enjoyable for me so far.

The first year I was more than a little overwhelmed and intimidated by everything. The scale of the event is hard to imagine and being surrounded by people who are at the top of their game is quite daunting. I wouldn’t say I was suffering from hero worship, but I did get the odd “we are not worthy” moment.

The second year I had a bit of a “been there, done that” attitude and I don’t think I really appreciated it as much as I should have done.

This year I think I found my groove and relaxed into things. It’s really cool to reconnect with people from previous years and meet new people.

Massive thanks to Victoria Lira and Lillian Buziak who I bugged incessantly about my travel arrangements for this trip. You are both worth your weight in gold. Big thanks to Justin Kestelyn and the OTN crew for making sure everything ran smoothly.



PS. Doug, perhaps next year you will be 3rd on the list…

Just landed in Auckland…

Just landed in Auckland. I left San Francisco 13 hours ago and it was 08:00 on Friday. It’s now 05:00 on Sunday. It’s something like 21 hours different, plus I flew overnight, plus the clocks just switched to summer time here. It’s like someone is trying, very successfully, to mess with my mind.

I’m going to be in the airport for 7+ hours waiting for for my flight to Perth. I was hoping to crash at a mates house while I waited, but he’s away for the weekend… πŸ™

The lack of sleep, the planes and this pesky cold have all combined to leave me feeling like I’m on a ship. I can feel myself physically swaying. In a few hours I expect I’ll be blogging about being found sleeping in a toilet or something similar. I just hope I’ve got my clothes on. πŸ™‚

That reminds me, someone fainted on the flight over here. She was walking along and just went out cold. Flat out on the floor for a couple of minutes. They brought an oxygen mask and everything. A few minutes later she was up and back at her seat. Falling over in a plane is bad karma. There are so many things to hit your head on!

Anyway, I’ve got to get some food and find a quiet toilet to sleep in… πŸ™‚



Real World Performance, Exadata Storage & ACE Session…

Today was a very full day…

Real-World Database Performance Techniques and Methods
Real-World Database Performance Roundtable

I went to the primer session on Tuesday and both sessions today. I don’t think there was anything particularly new to me here, but listening to the experiences of a team like that helps you crystallize things in your mind. The three sessions were well presented and the last session, which was a Q&A session, through up a few interesting things.

Take-home messages in no particular order:

  • It all boils down to the SQL and the cardinality of the steps in the execution plan. If the optimizer gets the correct cardinality for each step it should make the correct decision. If it doesn’t it will make a bad decision. When you spot a problem SQL you need to find out why it make that bad decision.
  • Debug methodically. If only a fraction of your SQL performs badly, making global changes to the database is a very bad idea. Fix the problems, don’t tamper with the whole system.
  • You have to decide if your aim is to have predictable execution plans or to let execution plans evolve over time. Either choice comes with issues. To make plans predictable, use SQL Profiles, hints, check high/low values, avoid histograms and don’t be too quick to gather new stats. If you want plans to evolve, gather stats regularly and accept that plans will change over time. Identify poorly performing SQL when it arises and figure out why it is going wrong.
  • Tune 3rd party apps, and Oracle Apps the way the vendor recommends, even if it doesn’t make sense to you. Some applications are quite fussy and doing what you think is correct may be the wrong move.
  • Your aim should be to spread I/O evenly across all disks. ASM will do this for you.
  • In Oracle 11g, you should gather statistics with the auto sample size, unless you know of a specific reason not to. Don’t make random decisions on sample size.
  • The team always use 8K block sizes because that’s what the majority of the database testing is done against and what the majority of people use.
  • Partitioning: Global stats tend to give predictable execution plans, but that’s not always a good thing. Partition stats give a more accurate picture, but won’t always result in predictable execution plans. You have to decide which you want.
  • High/low values are important stats. Don’t default to extreme values that make high/low values unrepresentative as this affects the optimizer.

There’s a lot of information in the slides, so check them out when they come online. The main message is you should be methodical and consistent. Nothing new in that. πŸ™‚

Oracle’s New Database Accelerator: Query Processing Revolutionized
Oracle’s New Database Accelerator: A Technical Overview

These two sessions were renamed to include the Exadata name. I didn’t record the new names, but I guess that will be obvious when you see the list of presentations.

Take-home messages:

  • The exadata storage server is also known as a storage cell.
  • You can buy the storage cells on their own and use them to build your own database system, or buy the preconfigured database machine.
  • The system is only supported on Linux at the moment, but other platforms will follow. I assume this means databases on other platforms plugging into the same Linux storage appliances.
  • The storage cells are single purpose. They are for holding database files only. If you want a shared filesystem for any other file types you need to use something else. So you guys who want to use it to store MP3s and pr0n better store it in BLOBs in the database. πŸ™‚
  • The iDB protocol (like iSCSI) presents storage to ASM. No OS config (LUNs mount points) is necessary. At its simplest, you can consider each storage cell as a single disk when viewed from ASM.
  • Storage cells don’t talk to each other, only to the database. The cells are aware of the database state and the database knows about the cells because of ASM.
  • Multiple databases can use the same storage cell.
  • The storage is presented to the database as shared-everything, but it has elements of shared-nothing architecture. Because the presentation to the database is unchanged, it is suitable for use with any existing application without change.
  • There is an EM plugin and several tools, but for the most part it needs no management. It is essentially a storage appliance (their words), but it can do some clever stuff.
  • Because of the presentation of storage cells to ASM, adding new storage cells is like adding disks. It triggers an ASM rebalance.
  • Normal ASM mirroring (2 or 3-way) is present, but it makes sure it mirrors to different cells, not just different disks in a cell, so it can tolerate a disk or a cell failure.
  • The storage cells should not need a software upgrade to work with new versions of Oracle, but if new offload features require changes, you may be restricting the offload functionality by not upgrading the cell server software. This was likended to the ASM compatibility modes in 11g.
  • Resource management information is passed from the database to the storage cells so I/O is prioritized to match the database resource management.
  • When you offload processing to the storage cell, only the relevant column information is returned. As a result, this won’t be stored in the buffer cache, but it is possible to cache it in the SQL result cache.
  • Up to 6 of the prebuilt database machines can fit together by just cabling them. You only need to worry about extra infiniband switches if you want more than 6. I can see this being a big problem for me… Not! πŸ™‚
  • It is designed for intensive query work, like warehouses. The performance of insert, update and delete operations is just like any other block storage device. That’s what the guy said, but I guess it was a glib statement. I would assume it is still faster than most storage devices due to the wicked network bandwidth.

This list represents many of the points made in the two talks, but I’ve started to read some of the stuff on Kevin Closson‘s blog, and the related links and it’s making me a bit nervous about my random ramblings. I’m going to leave this list here to remind me of what I heard, but you really need to check out the white papers.

There were some neat slides explaining the cell server software and the way the project evolved.

The biggest surprise to me was how few people were actually at these sessions. I blogged previously to say I was underwhelmed by the announcement, but I was still curious. It seems not many other people were. πŸ™

After hearing a bit more about the kit I can see why some people are really excited by it. It is very clever, but I still think it is pretty irrelevant to grunt DBA/Developers like me. Of course, I reserve the right to get excited by it at a later date and back track on all my cynical rantings. When they are in the bargain section at PC World I will race to the shop to buy one, but until then I guess I’ll be stuck with crappy old disks. πŸ™‚

Oracle ACE Director Birds-of-a-Feather Tips and Techniques Panel

This was my only real job at OpenWorld and I was pretty nervous going in to it. When you present a formal session you are talking about a specific area and you have an idea of what questions might come up. Sitting on a panel without access to Google and no time to compose your thoughts is a whole different ball game. Luckily the verbal diahorea gene that runs in my family was expressed in full effect and I managed to do more than my fair share of talking. Thanks to Lewis for setting this up. Thanks to the other guys on the panel for answering the questions that would have stumped this chump. Most of all thanks to the attendees for turning up and asking questions. I hope you got something out of it. πŸ™‚

I think I’ll let the events of the last few days sink in, then write a summing up post tomorrow.



Oracle Keynote Aftermath…

I think I’m in the minority when I say I was a little underwhelmed by the keynote yesterday. I’m sure there are many positive points about Oracle Exadata Storage and HP Oracle Database Machine, but it all seems a little irrelevant to me. I hear what Kevin Closson is saying and it sounds cool, but for me this is like discussing the merrits of a Lamborghini when I’m actually going to buy a Renault Clio.

I guess only time will tell how this will affect a grunt DBA/Developer like myself…



Tuesday at OpenWorld…

Despite feeling under the weather I managed to get in a pretty full day…

Oracle VM Scalability and Manageability with HP

Takehome messages:

  • Paravirtualization (PVM) works at near base hardware speed.
  • Hardware virtualization (HVM) is significantly slower than paravirtualization.
  • HVM can have up to 8 LUNs, while PVM can have up to 26 LUNs.
  • Oracle VM scales well. In the test shown, 5 VMs give better throughput than 1 large VM.
  • Live migration speed is dependent on the size of the SGA, how quickly its contents are dirtied and the speed of your network.

Deploying High Availability in Virtualized Environments with Oracle VM

Takehome messages:

  • Do any NIC binding and multipath configuration in Dom0 so it is available for all VMs on the server.
  • Clustered active/passive with automatic failover can be configured using Oracle clusterware.
  • Deploy servers in pools, each pool having a master. The VMs are not affected by the VM Manager being down as the master servers can still communicate.
  • OVM 2.1.2 can now have VM Manger database on an external server, not on the OVM Server box.

Some of this talk passed me by as it was dealing with such large amounts of hardware I was starting to get server-envy. πŸ™‚

Current Trends in Real-World Database Performance

This session was an introduction to the two session I’m down to see on Thursday. Nothing new hear, really just a primer. Even so, it was easy listening.

Top 10 Things You Wanted to Know About Automatic Storage Management but Were Afraid to Ask

I don’t use ASM much, so I expected there to be some new stuff here for me. As it happened, the session was a basic primer. It seemed to go down quite well with the audience, many of who I expect are just venturing into the world of ASM. I suppose I should have written down the 10 things so I could post them, but I couldn’t be bothered. I don’t think anything was mentioned that was not already in my 10g and 11g ASM articles. There are also some whitepapers on OTN.

That was it for the talks…

I bumped into Slavik Markovich of Sentrigo towards the end of the day. He used his Jesus phone to get me directions to the Oracle ACE dinner. Nice one!

I was a bit late for the Oracle ACE dinner, but I was glad I made it. It was a good night and I got to talk to a few new people. Big thanks to the ACE program for making the effort!



Cancelled my Unconference session…

I’ve decided to pull out of my unconference session. The main reason for this is I started to feel a little unwell last night. I got back to the hotel at 19:00, threw up and went straight to bed. I was hoping it was just because I was over-tired, but this morning I have a runny nose and dodgy throat. I’m now on Theraflu, which is similar to Lemsip, and I’m going to play it by ear today. If I feel rough I will head back to the hotel early.

The events, or lack of, last night meant I missed the OTN Night and InstallFest. I’m not sure if I will be up to this evenings stuff. I guess we’ll see.

I place the blame firmly on the recycled air in the aeroplane and the number of people I’ve shaken hands with. πŸ™‚ I’m going to stop touching people… πŸ™‚



Oracle, Open Source and Virtualization…

Due to other commitments, I only made it to 2 presentations today…

Executive Panel: Oracle and Open Source:

The session was a seeded question and answer session to highlight Oracles commitment to “Free” software downloads, Open Source and Open Standards. If you follow the subject areas, I guess you know the score. The standout points for me were:

  • InnoDB and BerkeleyDB are have continued investment.
  • The linux kernel community is really happy with the work being done by Oracle. Oracle are not considered an evil empire by the inner circle.
  • The new Windows drivers for Oracle VM are currently in use within Oracle. No fixed release date, but they are on their way, so Windows performance under Oracle VM should improve when this is released. The were expected some time ago, but Microsoft have revised a bunch of APIs, so they had to be rewritten.
  • Oracle VM has supported Live Migration of VMs since it was released, but this is a manual process. Something similar to VMwares automated VMotion will be introduced at some point, but it will probably be under the control of Enterprise Manager/Grid Control, not part of the Oracle VM product itself. The impression I got was that this is a feature they want, not one that is in development, so don’t hold your breath.
  • Oracle are continuing to invest in a variety of Open Source products and Open Standards. Obviously they have their own interests at heart, but being open source means it is helping the Open Source community generally. If Linux get’s better, Oracle benefit, but so does everyone else.

Oracle Database in a server virtualization environment

A general run through current support/certification of virtualization by Oracle. Standout points include:

  • In addition to Oracle VM, Oracle support a variety of hardware partitioning and OS partioning methods provided by the big UNIX vendors. There was a URL of a certification matrix on OTN, but I can’t find it now. πŸ™
  • Oracle support dynamic resource configuration in single instance databases. So you can alter the number of CPU’s and amount of memory allocated to a VM and Oracle will notice and work with it. Some of this was possible since 9i, but there were issues. It’s now solid for (+ patches) and
  • Oracle support RAC on Oracle VM, but the configuration is quite limited at the moment. The RAC must be on OEL 5.1 or above and be running on OVM 2.1.2. In addiiton it must be paravirtualized and is only supported in static mode, so there are no dynamic changes to CPU or memory and no live migrations. I assume over time some of these restrictions will be lifted and 11g support will be added.

As far as best practices go, I guess most of it was pretty standard:

  • Don’t overcommit resources. You can’t expect good performance if you have 1 core running 5 VMs.
  • Standard CPU, memory, I/O and network sizing/thoughout considerations still apply.
  • RAC should use dedicated NICs, not shared ones.
  • NIC bonding best done at hypervisor level (Dom0), not at the VM level.
  • All VMs, especially RAC, need time synchronization (NTP).
  • Use native physical devices and multipath or NFS. This will perform much better than virtual disks.
  • Oracle still recommend consolidation of databases, rather than many small databases/instances. Only split things into multiple VMs, database or instances when it is necessary, otherwise you are wasting resources.
  • Expected performance will depend on platform, application type, workload type and resource allocation.

The licensing issue will no doubt catch many people out. Oracle doesn’t recognize software virtualizations ability to limit processing to specifc processors or cores, so by default you must license the Oracle software for all cores on the system when using software virtualization. Those platforms which support hard partitioning (big UNIX vendors) can use per-core licensing. If you plan on using a server just for VMs running the database, this is fine, but if you want to run half database and half app server you will be buying double the CPU licenses that you are using. Bummer. πŸ™

Update: A talk this morning says pinning the VMs to specifc cores is within the spirit of the licensing agreement. Nice to see a consistent message… Not! πŸ™‚