A Rational Response to a Tech CEO's Rant

A friend and co-worker of mine passed along this link http://bit.ly/1bUMi9N to me last week.    Even though he is a developer, not an (Oracle) DBA, as I am, he was miffed at the comments made by Mr. Kellogg in the last portion of his blog post.   I kind of laughed it off at first; I am used to people taking pot shots at Oracle (some well deserved, mind you), but after reading the post, I was miffed as well.  

Mr. Kellogg apparently has a flair for the dramatic.   He uses such words as "pillorying", "disruptive technology", and "oligopoly", in my opinion, to make his post more dramatic, but none of them are appropriate.

Mr. Kellogg makes some valid points in the middle of his blog concerning the problems with healthcare.gov.   However, his disdain for Oracle is clearly evident throughout the rest of the post and he even carries that disdain over to those persons that work with Oracle's database products.

How can he claim that the New York Times is "pillorying" the company at which he was formerly CEO, Marklogic?   He states:

And if you think that looking at 1% of the costs is the right way to diagnose a struggling $630M project, I’d beg to differ.

Yes, There are many other problems related to healthcare.gov besides a particular component of the technology.   However, if a key component of the system is causing a critical bottleneck, it is perfectly valid to point that out.   Also, since when is the amount spent on a particular component even anywhere near relevant regarding whether the component works properly or not?   From my reading of the New York Times article that he refers to, Marklogic had performance problems.   If healthcare.gov used free and open source software (FOSS), such as MySQL or PostgreSQL and it had performance problems, would it be less of an issue in his mind since they are free?  I think not.   

The title of the blog post refers to selling "disruptive technology" to the government.  Again, a flair for the dramatic.    One definition of a "disruptive technology", found at http://bit.ly/18kUC1P, is:

Such technologies surprise the market by generating a substantial improvement over existing technology, and this can be accomplished in a variety of ways. A disruptive technology may be cheaper than an existing technology, for example, or more basic in nature, attracting more potential users. When disruptive technologies expand the market by providing low cost, they are known as low-market disruptive technologies, while new-market technologies are entirely new innovations which replace existing ones.

Marklogic may be cheaper than Oracle, but it has not expanded the market of database technologies nor replaced existing ones.   Typically, disruptive technologies are such that they cause "disruption" in the existing markets.    Marklogic has not done that; Oracle, Microsoft, and other relational database vendors are not quaking in their boots at the prospect that Marklogic is stealing significant market share from them.   Is Marklogic an alternative tool in the "database" tool chest?   Sure, just like MongoDB, Cassandra, and others are as well.   But a "disruptive technology"?   

Mr. Kellogg refers to an "Oracle-led oligopoly".   An oligopoly is when a small number of firms controls a particular market.   Let me see, I can count at least four firms offering commercial relational database software: Oracle, Microsoft, IBM, and SAP, plus the community version of MySQL, and FOSS alternatives MariaDB, and PostgreSQL.   Other non-relational database offerings exist, such as MongoDB and Cassandra, as well as Hadoop for "Big Data" analytics.    While Oracle, indeed, controls a large portion of the relational database market, it is not the only alternative.   Indeed, MySQL is chosen by a large number of small startups due to it being a FOSS alternative.   PostgreSQL is also chosen for the same reasons, but to a lesser extent.   Companies do not have a gun to their head forcing them to purchase Oracle's database product.    They purchase it because it provides value in terms of features, technical maturity, tools, and instrumentation.   If it didn't provide value, why would they purchase it?

Mr. Kellogg makes a blanket statement about technologists that work with a particular product and indicts them because of it:

I met many Oracle-DBA-lifers during my time working with the government.  And I’m OK with their personal decision to stop learning, not refresh their skills, not stay current on technology, and to want to ride a deep expertise in the Oracle DMBS into a comfortable retirement.  I get it.  It’s not a choice I’d make, but I can understand.

What I cannot understand, however, is when someone takes a personal decision and tries to use it as a reason to not use a new technology.  Think:  I don’t know MarkLogic, it is new, ergo it is a threat to my personal career plan, and ergo I am opposed to using MarkLogic, prima facie, because it’s not aligned with my personal interests.  That’s not OK.

A confession - I am one of those "Oracle DBA-Lifers" that Mr. Kellogg refers to in his blog post.   Yes, I have twenty years of experience working with Oracle, which has been around since 1977.   I chose to focus on it because of it's leadership in market share; makes it a little bit easier to find a job than with a product that has a decade under it's belt and very little market share to show for it.   Plus, in my opinion, Marklogic is a proprietary technology.    Yes, there are other XML data stores out there.   However, understanding how to tune and optimize Marklogic is not simple.   Nor is XQuery easy to understand.  I can go from one relational database to another and pretty much be able to tune it fairly easily.   I speak from experience.    And THAT is what made Marklogic a BAD choice for healthcare.gov, a web site that was very high-profile and under significant schedule pressure.    Those persons that balked at using Marklogic knew that there was a significant risk using a technology with which relatively few persons have experience.   Of course, salesmen such as Mr. Kellogg want to make a sale and sold the government a bill of goods that their new fangled XML database would save the day.    Forget the fact that healthcare.gov isn't dealing with a ton of document based data that is more suited for storing in MarkLogic; it is structured data, which works very well (and has for decades) in a relational database.

Mr. Kellogg states at the end of his blog post:

Oracle was non-standard in 1983.  Thirty years later it’s too standard (i.e., part of an oligopoly) and not adapted to the new technical challenges at hand.  All because some bright group of people wanted to try something new, to meet a new challenge, that cost probably a fraction of what Oracle would have charged, the naysayers and Oracle lifers will challenge it endlessly saying it’s “different.”

He is correct that the problems with healthcare.gov are not solely to blame on one particular technology component.   It IS, however, to blame on a number of different, classic mistakes that are made with too many technology projects, one of which was to use an immature product with which the project team was unfamiliar on a project that was very high profile  and under significant schedule pressure.    What is NOT to blame, are technologists that make sound, well thought out decisions / recommendations to their customer based on years of experience, only to be ignored by their customer, and then lambasted by a technology CEO that believes he knows more than they do.   

Mr. Kellogg states in his post:

The trick with selling disruptive technology to the government is that you encounter two types of people.

• Those who look objectively at requirements and try to figure out which technology can best do the job.  Happily, our government contains many of these types of people.

• Those who look at their own skill sets and view any disruptive technology as a threat.

(Emphasis above added by me)  Interesting, in my opinion, the technologists that recommended against using MarkLogic for healthcare.gov did exactly what he states in his first bullet point, above.    A relational database is much better suited for the structured data being captured by healthcare.gov than is a document-based data store such as Marklogic.   Also, a relational database is a mature, well-known technology for which there is an abundance of technical talent available that has experience with the technology.

Finally, Mr. Kellogg concludes his post with "Follow the money".    I could be totally off base here, but could it be that Mr. Kellogg still has a financial interest in the privately held MarkLogic in the form of options or shares in the company?   And bad publicity could lower Marklogic's perceived value by other technology companies or by potential investors, right?    Yes, follow the money.....

Oracle, we have a (permissions) problem....

With apologies to NASA for the title of this post.... However, it seems fitting given several problems I have experienced due to file permissions problems with Oracle Grid Infrastructure 11gR2 (both 11.2.0.1 and 11.2.0.2).

Some background: I am setting up several RAC clusters using 11gR2 of Grid Infrastructure and taking advantage of the new "role separation" feature that allows you to install and run your databases under one or more separate OS accounts from the Grid Infrastructure OS user. I did this to provide a little more security between the different databases in the cluster as well as the grid infrastructure software. In addition, some of the databases will be 10gR2 rather than 11g databases.

The permissions problems have been manifested in a couple of ways.

1. 10gR2 database installer does not recognize cluster AFTER application of 11gR2 Grid Infrastructure patch 11.2.0.1 due to library permissions.
   
2. 11gR2 (11.2.0.2) database installer has problem due to permissions on gpnp file permissions
3. 11.2.0.2 Grid Infrastructure will not start in root.sh due to incorrect permissions caused by umask not being set properly before running root.sh.

This appears to be a common theme when running in a role separation environment under 11gR2 Grid Infrastructure. File permissions are set to be TOO restrictive for other OS accounts in the oinstall group to access and then problems occur during operations run under the OS accounts that will own the databases.

I have submitted one SR already for item #1 above and will be submitting another for the second item soon. This seems to be a pattern for Oracle with this new feature; I hope that they will implement more rigorous testing processes in the future to address these issues.  The third issue just cropped up recently and I will be submitting a bug report on that one as well.

Has anyone else out there had similar issues related to running in a role separation environment under 11gR2 of Grid Infrastructure?  Also, is anyone having problems in general with permissions during Oracle installs / upgrades.   While my general experience with Oracle is that after you get it installed and running it is rock solid, my experience with installing / patching Oracle, especially recently, is that there are TONS of problems related to insufficient permissions on files.   Oracle's testing of their installation and patching processes leaves alot to be desired....

A snowstorm, a traffic jam, technology, and social media

Last night's snowstorm in the Baltimore-Washington area gave me first hand experience with how technology and social media have radically changed our lives. We tend to take those changes for granted as we use these tools during our daily, routine lives. However, when something other than the routine happens, it makes you think about the impact of those tools.

This snowstorm started putting down snow at about 4:00 PM at a rate of 1-2 inches per hour and kept up at that pace for 4-5 hours. Always wanting to get as much done as possible, I got out of work a little later than I had hoped (4:15 PM vs my planned 3:30 PM). That 45 minutes may have turned my commute from 1 hour 15 minutes into over 9 hours.

I called my wife several times during the first two hours of my commute home, which would normally have taken me about 40 minutes. There's the first example of how technology has impacted our lives in situations that can cause concern for our loved ones. 20 years ago I didn't own a cell phone; if this would have happened to me as a newlywed, my wife would have been MUCH more worried than she was last night. While she was concerned, she and I could easily call one another on our cell phones. While we sometimes decry "constantly being connected", it was a blessing last night. During the remaining 7 hours (5 of which were spent sitting - not moving at all) on I-70 westbound about 1-2 miles east of the Mt. Airy, MD exit (exit 68), my wife and I spoke several times. Not only did it make her feel better knowing where I was and that I was OK, it helped me to pass the time and not go stir crazy. Hearing her voice made things "normal" in a not so normal situation.

The other example is social media. As many people do these days, I have a facebook account and check it on pretty much a daily basis. I don't post daily, but I do check it to see if any of my friends have anything to say. I do post more often when something "out of the routine" happens (e.g. the NFL playoffs, this snowstorm). Last night, I posted messages using my Droid smartphone indicating that I was stuck in traffic. Throughout the evening, some of my facebook friends commented on my posts, sending prayers and offering their support. In particular, one friend, after one of my posts later in the evening, asked me to please call her. We spoke for about 15-20 minutes about what was happening, if I had enough gas, and about several of her other friends that we also stuck in the same traffic jam. When the cars started moving in front of me, we cut our conversation short. We spoke several other times between then and when I finally arrived home, with her repeating her offer of a place to sleep if I couldn't make it to my home, which was 30 miles from where I was stuck in traffic.

Again, 20 years ago the world wide web and social media didn't exist. While I did have a book that I had brought home from work to read, that would have gotten old after an hour or so. Having a smart phone (something I didn't have 18 months ago) again made the time spent stuck along with thousands of other motorists go by much more quickly and lessened the stress that I might have otherwise felt if I wasn't able to check other posts on facebook and post about my own experiences. My posts enabled many of my friends to be able to share in my experience and send their prayers and good wishes, and even offer up a place to lay my head for the night. 20 years ago, they wouldn't have known where I was, and for some, probably wouldn't have cared; not that they are cold-hearted, but if a tree falls in the woods, and no one is there to hear it.... Well, you know how that ends. Again, we complain about being constantly connected with our smart phones, email, texts, twitter, facebook, etc., but this was a perfect example of how social media and technology is beneficial, especially in times where things are not so normal and our friends and loved ones would be feeling LESS connected if we didn't have these modern day tools in our lives.

Hello.... Is anybody out there?

Let's see, three posts over 3 years does not a blog make, does it? OK, so, I haven't had much to say. Sue me (JK!!!! Please don't; I can't afford the legal fees!) Oh, and contrary to my first ever blog post, I will be posting stuff technical in nature. I seem to have a knack for the stuff, so I might as well share my experiences, right?

Anyway, I hope to start posting more often, mostly about stuff related to my job, specifically, my joyful (NOT!) experiences with trying to configure an Oracle RAC Cluster on RHEL5, with Oracle Clusterware 11.2 and Oracle Database 10g. Oh, and throw in my wise (or not so) decision to also use the capability in 11g clusterware to use different OS accounts for the clusterware and each of the databases running in the cluster. Needless to say, it has been (and continues to be) and adventure.

So, if anyone happens across this blog, I hope that my experiences and posts will help you in some small way to avoid the pleasurable pain I have incurred these past several months.

Until next time....

He's baaaaack....

OK, so my last post on this blog was over 9 months ago; so, who's counting? Anyway, I hope to get back to posting more often. And, despite what I said in my opening blog post, I probably will be posting on technical issues, as well as general life issues as well. Anyway, so what happened with the Christian Music Festival? Here's a brief recap of events since my last post.

Well, I ended up being much more involved in the hands-on coordination of things leading up to the festival. The chairman of our festival committee, by his own admission (afterwards), is great with ideas, but a little less great with the logistics ends of things. So, I jumped in and helped with various logistical issues (working with vendors to get stages, handling health dept. permits for our food vendors, putting together the program, etc).

August 2nd finally rolled around. Of course, Mother Nature had it in for us from the start. Anyone in the Mid-Atlantic region of the U.S. knows what the weather can be like at that time of the year. It was warm (in the low to mid 90's) but, worse, we were looking at possible thunderstorms. As it turns out, we had two near misses from heavy thunderstorms and hail - resulting in two stoppages of performances. At around 5:00 PM, we finally got hit directly - heavy winds and a short downpour. This apparently was the last straw for our chairman as he announced (as the storm is happening) "That's it!". No waiting to see how long it would last - which ended up being about 30 minutes, at most.

In terms of attendance, we had about 120-130 paying attendees; another 50 or so were complimentary tickets for sponsors. So, not an overwhelming turnout. The nice thing was the feedback from the performers. Almost all were impressed with how organized everything was and indicated that they would love to come back. Our attendees were more critical. Most apparently do not attend "festivals" and understand that they would be sitting out in the open for a number of hours. Our biggest complaint was about providing shade for attendees. Point taken.

The chairman of last year's festival stepped down and I stepped up to run for Chairman. Of course, life has a way of throwing you curves. Right after taking over the chairman position, my wonderful wife began working full-time again (although it was supposed to be temporary, she is still working full-time as I write this post). So, the time I can devote to planning and coordination of the festival became much tighter. Ultimately, at the November meeting of our committee, I regretably announced my resignation as chairman. Our committee chair for performers also stepped down, so now three of the persons that were very much involved last year are no longer in key positions on the committee.

As a result of the resignations and the economy, a discussion ensued about how to proceed for the 2009 festival. We agreed that it might be best to scale it back to an indoor concert where there are less logistics to worry about. However, we did not get much past the general idea of a concert. Much more planning needs to be done to pull something off.

Well, that's where things stand with the festival. As thing change, I will try to provide updates via this blog.

Taneytown Christian Music Festival

Well, if anyone is actually out there reading this blog, please mark Saturday, August 2, 2008 on your calendar. That is when the first ever Taneytown Christian Music Festival will be taking place at Taneytown Memorial Park in Taneytown MD. The festival will run from 9:30 AM until 8:45 PM, rain or shine. We have 18 acts scheduled throughout the day that will feature mostly local and regional christian music from contemporary christian, christian rock, country, bluegrass, and gospel artists and groups. More details to come regarding the artists in upcoming blog postings.

The festival board is busy working on numerous items to make this festival a success. One thing that we are in great need of is volunteers; Many are needed to put on an event of this size. We need volunteers in many areas including fundraising, vendors, event day, parking, security, stage, etc. You can send an email to tcmf AT ricknco DOT com or allen DOT shatzer @ yahoo DOT com for additional information and to sign up to voluteer.

Again, mark Saturday, August 2nd on your calendars for this fantastic day of wholesome family entertainment!

Hello world....

No, this isn't a technical blog about programming (you have to be a C programmer to get that comment), although I am a technologist. I'm sure over time I will rant about what drives me mad in my profession, but I'll save that for another day. No, I'll just say Hi and hope that my posts will provoke a least a little thought, entertain, inform, or maybe, just give you a break from everyday life.

I don't profess to be an expert on anything, but I do have an opinion about many things. Over time, that may become evident. I invite your comments, but keep things in perspective - we don't need to waste our energy flaming one another. Let the discussions be civil.

Anyway, that's my first post. Nothing earth shattering, but, its a start....