Sunday, May 4, 2008

MySQL, Sun, and the Future of Open Source Databases

So what's it like now that Sun now owns MySQL? The executive summary: a little weird. I was at the MySQL User Conference a couple of weeks ago and had a chance to talk with a lot of people in the community as well as many MySQL folks. Marten Mickos is now the head of database products at Sun. It's not very hard to figure out what Sun will do with MySQL products for the near future--pretty much what MySQL was doing already.

The real question for a lot of people is what will happen with databases like PostgreSQL and Derby. Sun has invested heavily in both of them, and PostgreSQL in particular is now quite fast. With the MySQL acquisition, Sun has an opportunity to run the table with multiple offerings that cover both enterprise applications as well as web and embedded. However, that would mean cutting down the MySQL roadmap to concentrate, for example, on scale-out rather than scale-up. It would also require thinking big to combine with other vendors in order to disrupt the market leader Oracle. Done right, there's a chance to upend the industry in a way that has not occurred since Microsoft muscled into databases in the early 1990s using code bought from Sybase.

Based on talks from people like Rich Green and Marten Mickos, it's hard to see this happening. Sun is taking a hands-off approach to MySQL *and* giving MySQL management control of overall database strategy. A disruptive change therefore seems unlikely. In fact, the more likely result is stagnation, now that MySQL no longer has to fight for its existence. The MySQL roadmap is still pretty diffuse and there has been little product movement since the 2007 User Conference. MySQL 5.1 is still not out the door. Falcon is likely to show up ready for production use around the time the Boeing Dreamliner rolls out. MySQL is still working on multiple storage engines (2 new ones plus NDB and MyISAM, to name a couple.) There's not even a glimmer of a date for cool new replication features like a pluggable replication interface. In short, not much evidence for radical changes of any kind.

Also, there must be the awful temptation to focus on vertical scaling so that MySQL can work on Sun hardware with large numbers of cores. I asked Marten Mickos specifically about the choice between scaling up and scaling out but didn't get a very clear answer. Personally I think for MySQL to concentrate very hard on vertical scaling would be a strategic error. The community that made MySQL great is into commodity hardware and scale-out in a big way. First rate support for highly scaled SMP architectures is going to be a long slog that will compromise delivery of many other features.

Given all of this it's hard not to see innovation, particularly in problems like scale-out, shifting away from MySQL to other databases as well as middleware. This would be a great time for the PostgreSQL community to get really serious about data replication. MySQL won't fade--it's already a great database. But there's likely to be a crowd of people in the MySQL community eying other solutions. It's going to be interesting to see what they come up with.

3 comments:

Mark Callaghan said...

I enjoyed this. Have you submitted your blog feed to http://www.planetmysql.org/newfeed.php? If you haven't then many of us will miss out on your posts.

robert.hodges said...

Thanks! This blog is aggregated PlanetMySQL as you suggest but sometimes the posts don't appear in a very timely fashion, so things sometimes show up a couple pages back. I have been working with Jay Pipes (a very helpful guy) to work out why the posts appear so late.

Maybe they show up faster if you say things that are really nice for MySQL. :)

awksedgreep said...

The question of scaling horizontally versus scaling vertically is not one of community and what they're used to, but one of performance.

There's a very good reason why historically databases with >15% write load are scaled vertically. They are much more performant and have much less latency.

ORMs are becoming more popular and introduce many more queries to the db(n+1 problem, developer/SQL abstraction, etc). The queries are much more complex also, which doesn't favor MySQL's simplest of simple query optimizers.

I've been trying to grow a website database using an ORM with MySQL and have hit a brick wall. It seems PostgreSQL is the only option until MySQL 5.4 is GA(and acceptable in production environments, which is two separate events unfortunately).

Frankly I'm also tired of the warts. For every solution MySQL puts forward they introduce 3x as many bugs and workarounds. It seems as though no solution is complete or well thought out.

I've been administering MySQL for 12 years now and can't wait to get away from it. However, oddly enough there are plenty of available jobs for me???

Over time people will learn what I have about MySQL. PostgreSQL is a better option 95% of the time.

Scaling Databases Using Commodity Hardware and Shared-Nothing Design