The Death of the DBA

Its all predicated on the cost of the data connections (multiple paths required to the cloud for reliability). As long as reliable high speed connections cost big bucks the range of applications will be narrowed to those with low data rate requirements.

i have been hearing about "cloud computing" since the mid 1990's and how everything will be outsourced and no more jobs.

Hotmail was probably the original cloud computing solution. you used to get your email from your ISP and if you changed ISP's you had to contact everyone to tell them your new email address. then in 1997 hotmail came out and then MS bought it and now it's going the way of AOL.

On the corporate side a new type of company popped up in the 1990's called the ASP or Application Service Provider. They bought a lot of servers and SAN's and sold you terminal sessions you would log into and run your office apps and whatever. The tech media was saying the era of the IT department was over. My current employer bought a nice EMC SAN at that time for around $250,000 or more. 2 years later you could buy one on Ebay for $40,000 that was previously owned by one of these ASP's that went out of business.

Come 2002 the new buzzword was webservices. all your apps were going away and everything was going to be through a web browser.

2003 or 2004 Sun was a shadow of it's former self and about this time they had this bright idea of selling CPU time because no one was buying their over priced hardware. kind of like Ford "selling" cars to Hertz back in the day.

Flash forward to today. Now this CPU time thing that no one really knew how it would work but the tech media kept hyping it is now called "cloud computing" and supposedly everyone is going to use it in the future.

My prediction is that IT departments won't go away. Any savings will be eaten by the enourmous bandwidth costs and companies still will want to have their data to themselves. with all the recent incidents of employees losing data on laptops no one wants a lawsuit.

If Amazon and Google will want the big corporate business they will have to sign SLA's promising a certain amount of uptime. Will they do this? I doubt it. IBM has been doing corporate IT consulting for decades and they are probably the one to watch in this. IBM is even writing software to automate most of IT and they run central data centers for different clients

Amazon and Google are going to be the solution for small businesses which are 99% of all companies in the US. few years back i helped set up a few servers for a small doctor's office. Total cost was probably close to $20,000 for the hardware and software. Today they can buy a few cheapo PC's and "outsource" the servers to Amazon or Google or Microsoft who will keep their data.

Either way pesky little things like SLA's, HIPPA, SOX and some other laws come into play so any "cloud" provider will have to offer a solution that complies with them including the guarantee of back ups

and on the bright side, 2 months ago i had a cloud computing provider call me for an interview for a DBA job. they host a data center for Fortune 1000 clients and they needed a DBA. turns out they still have databases that need to be managed even when everything is in the cloud

and i get several emails a week for dba job interviews.

I see your points in this article. However I see major issues taking the database into the cloud. How does an organization handle securing the cloud to meet SOX/PCI/SAS70 and other regulatory requirements. Many of these require securing the entire system. I think there would have to be a lot more development in securing & hardening the database between applications. I think the cloud is going to be very successful for applications that don't have these security or regulatory concerns. However in the tech industry it can all change in a blink of an eye.

P.S. and you always hear about these supposedly revolutionary technologies in the first part of the year which is coincidentally the time i read about all these industry conventions. After the summer it dies down.

Reason of course is hype. Very few reporters are real journalists these days. most just reprint what the PR and marketing departments tell them to write. In tech this is probably at it's worst. everyone wants to be the first to break a new product and marketing departments play everyone with the NDA and "leaks". And the price is that you have to hype what the marketing people want to sell.

i remember this with IM. around 2002 or 2003 IM was in every tech rag and was the next big communications technology that was going to replace email and whatever.

Flash forward to 2008 and it's a feature in an appliance that is sold by a lot of vendors and email is still king.

all tech hype is not to be believed until proven true. just like 99% of the stock market stories and CNBC

I agree that cloud computing will be huge, but will in no way obviate the need for DBAs, especially those with a developer bend. Someone is still going to need to externally "manage" the database, take the role of data steward, and all of the other non-systems management stuff that we do as DBAs. Those "DBAs" who are really systems administrators might want to remove the "DBA" tag and focus on systems a bit more heavily (every company will still need computers). Those "DBAs" who do everything else can, in my opinion, sleep easily. And in either case, I don't think cloud computing will take over everything. Certain market segments simply can't use this kind of technology. Do you think a service like MySpace can possibly scale on SSDS? (Of course now that I've said that, next week there's going to be a press release from Microsoft and MySpace and I'm going to eat my words on that one )

Russell,
Gartner has released a report on cloud security. It is $195 but there is a synopsis here.
http://www.networkworld.com/news/2008/070208-cloud.html?page=1

jm

While many of the cloud systems have focused on make BBFM type database storage systems, they haven't yet gotten to the point of replacing database management systems, and this is a critical difference.

Since the relational model first came out, a number of technologies have come along to "kill the rdbms" (think OO db's, XML db's, ISAM systems, Hierarchical databases, and several other reincarnations along these themes) but the relational model is a tour de force that has outlasted all of these contenders, and imho will continue to do so as long as it's replacements focus on data storage as the problem, rather than data management.

Really this is what happened to Sybase; it isn't that some new technology came along and replaced it, it's that other people have made better versions of it's product than Sybase was able to do; from Oracle on the enterprise level, MSSQL as a more direct replacement, or something like Postgres as an open source alternative.

Yeah, new technology is fun, and cloud computing is what gets the VC's excited, but if want a foundation for technical skills, the relational model is where DBA's should spend thier time studying.

I don't think most companies will outsource their business-critical data, they would be in too much trouble from customers and shareholders if it got lost or disclosed by the hosting company. What will rather happen, I think, is that companies will set up their own internal clouds and ratched up number crunching and customer analysis by a factor of ten using these new technologies, and that database administrators will need to learn the configuration parameters of these new cloudy databases, and continue to do very nicely in their jobs. Could even be that the increasing use of data will require more DBAs than today.

It looks to me like de-ja-vie: in early 80s I’ve heard that computer would write programs itself and programmers will be out of job, in 90s I’ve heard that all jobs would be outsourced to low cost employees overseas, and now Jason reminds about Sybase DBAs. It was scary, it is scary and it would be even scarier.
However, I would like to be positive: I was developer and DBA, I am DBA, and I would be DBA at least for a while. And my life is busier and better. The key is in my behavior that is in line with IT evolution. I want to be good and I am evolving to be good.
I’ve started with languages like COBOL, Basic, and Pascal, I did assembler 8080 development in 80s, I did Progress, dbVista, Access, Oracle, Sybase database administration in 90s, I do MS SQL Server and Oracle administration today. I am sure there are only a few opportunities with Sybase and Powerbuilder, I am sure responsibilities for DBA will be different in 5, 10, 30 years from today, but I am sure every business will be in need of technology-focused persons as well as it needs accountants, data entry, and others.
IT in general and DBA in particular is a tool that gives opportunity to business to be effective and efficient. Every business person knows how to do business but good IT person/DBA/developer knows how to make it efficient. I think it gives me hope to be employed and retire as IT person.

One thing is certain: things are going to change some, and cloud computing is going to drive some of that change. I seriously doubt we're near the end of the need for DBAs any time soon, but that doesn't mean cloud computing will be a zero-sum.

Personally, I anticipate as many or more jobs will be created by opportunities for cloud computing than those lost due to it.

:{> Andy

To me, the problem with the death of the DBA idea isn't that some work won't be moved to the "cloud," it is that until we hit the saturation point of places that "could" use a database system versus those that do have one. As costs of hardware/software come down, companies still need some management assistance.

To me, the more serious problem is that we don't have nearly enough quality people around now to do the job, how on earth are we going to grow to meet the near-term needs, much less expand to meet longer term needs. And as more and more candidates that are barely qualified to use a DVR folks get DBA jobs, the worse and worse the situation will get. (How many interviews have you done with people who couldn't spell SQL that had years of experience and wanted more than you made?)

This problem does make me wonder if there Will be more jobs for talented people to clean up? Frankly clean up rarely means format DiskDrive: restart clean. It just means make it run "well enough" so we can move on to the next problem that should be completely redesigned instead of just cleaned up.

Eventually people will get fed up with poorly built relational databases and move somewhere else. Cloud? Perhaps. But frankly, sooner or later we are going to have to learn to design for the cloud too, right? It isn't magical, and will need tuned. And I figure that all of the qualified DBA's out there won't mind too much moving to a new technology... The unqualified...well, good for the rest of us if they go off on do stuff that they are qualified for and not what just what seemed like a good idea because the pay was better over here.

A lot of it also depends on how the cloud also pans out. Microsoft SSDS\Google BigTable\Amazon SimpleDB are basically presenting a web services that replaces the RDMS. There are other "cloud" offerings that offer full featured SQL Server either in a shared or virtualized environment.
For full disclosure, the company I work for does the virtualization. http://theenterprisecloud.com/

Comments from the following blog entry: The Death of the DBA Part 2, located at: http://statisticsio.com/Home/tabid/36/articleType/ArticleView/articleId/293/The-Death-of-the-DBA-Part-2.aspx

Comments from the following blog entry: http://mgarner.wordpress.com/2009/02/13/rdbms-doomed/

Comments from the following blog entry: http://209.251.180.21/site/archive/2008/11/the-death-of-the-dba-part-2/

Comments from the following blog entry: http://www.brentozar.com/archive/2008/08/long-live-the-dba/