How to Save $100,000 in Database Fees

Are you looking for ways to reduce the cost of database software and save money on database licensing and support fees? Can you really save 100,000 dollars in the cost of database software by reading this? I can't guarantee you will save that much money, but if you keep an open mind, forget any preconceived notions and old-wives' tales prevalent throughout the IT industry, then you will be well on your way to saving a tremendous amount of money on database licensing costs or reducing them entirely.

At a minimum, you will gain some insight from someone who's been working in the database field for decades.

What Type of Databases are You Talking About?

We're mainly referring to traditional relational database management systems (RDBMS) that most companies still use for storing their private proprietary information. These types of databases need to be secure, highly scalable and accurate. This would include Oracle, SQL Server (Microsoft), MySQL, MariaDB, and PostgreSQL and several other less popular platforms. The cost of database software for some of these platforms can be staggering, but for others, the entry price is free.

While we're not directly addressing the newer NoSQL and Big Data databases, the overall principles expressed here still apply.

Why Are You Writing This?

A few months ago I wrote a blog entry titled "The #1 Database Problem". I received some interesting feedback. Most comments I received were from people who were intrigued and wanted more details and solutions. On the other hand, some people were not too happy with my opinion, but I know that when you're getting some flak, you're usually right over the target.

You will get opinions from everyone, but opinions don't really matter in the database world. What matters are facts, experience and performance. The rest is just noise.

Why Should I Listen to You?

A Little Background about Me: I've spent the last 15+ years designing, building, installing and managing high-performance enterprise class database systems. I've worked with a variety of database software including Oracle, Informix, MariaDB, MySQL, SQL Server, Sybase, PostgreSQL and even Microsoft Access; however, my first and enduring love has been building and managing Oracle database systems. I have designed, built and maintained a variety of OLTP (online transaction processing), data warehouse, data mart and one-off reporting systems. Prior to starting Parthian Systems, my consulting/employment history includes Fortune 500 Companies, small startups, a credit card processor and a wide variety of nameless other firms which no longer exist. The experience has been invaluable.

The #1 Database Problem

Although my blog entry ignited a bit of a firestorm from some individuals, that doesn't make my arguments invalid. On the contrary.

No matter the type of database platform, in almost every situation where I've seen problems with a database implementation the situation is the same. The truth is that in roughly 90% of cases the #1 Database problem is the poorly written database code written by application developers. Following closely behind at #2 is having application developers or even inexperienced database developers design the database. Almost in every situation the application developers bury the database code within the application which is exactly where you don't want it to be. The database code belongs in the database. The application code belongs in the application. Compound this situation with dynamically-generated (built-on-the-fly) SQL code and you're well on your way to having a poorly performing database.

When this is allowed to happen, you will experience some or all of the following problems:

  • Redundant requests to the database from the application. This usually amounts to thousands of database requests (or calls) in a production database that will kill even the most powerful of servers.
  • Users will run long, resource-intensive queries on the database and bring everything to a standstill
  • Information will disappear. I've walked into companies where application developers would accidentally drop (delete) tables. Yes, they will do it. I have witnessed it and have experienced these situations more than once. For some reason it always seems to be financial information that they wipe out. Go figure.
  • Inconsistent data. The reasons are too many to list, but without an understanding of data flow, transactions, lost update rules, etc. you will get inconsistent data which then unleashes a torrent of problems from users unable to login to an application to business reports which instantly become inaccurate or completely useless.
  • Transaction Deadlocks
  • Transaction row locks, table locks
  • Lack of scalability
  • Database outages which bring your business to a halt

Database Independence is a Lie

This is a favorite one with ill informed software developers. "Database Independence" is a buzzword that's thrown around with reckless abandon, but it's indicative of a lack of understanding of databases. There is no such thing as database independence. Most software developers commit the same mistake over and over who use this approach. They take raw database code (SQL statements) and embed them directly into the application. The reason they do this is because it makes their job much easier. They don't have to work with a database administrator which would slow them down. While it makes their job easier, it is a horrible decision for the business, as it will cause database performance issues. Compounding this problem, in many cases a large majority of these statements are 'dynamic' or generated on the fly. A database (any database) cannot just be 'plugged in' in the hopes that it will work. Sure, in small, simple applications this can work fine, but for enterprise-scale applications it will never work to the scale and performance that is required.

If your software application is well designed, your logic and data layers will be clearly separated. In this case, you could have a separate database code layer for the particular database to be use allowing the software application to use a variety of databases on the back end. Unfortunately, I've yet to see but a handful of applications designed in this manner. The overwhelming majority of software applications have database code (SQL statements) embedded directly into the application code (Java,PHP, etc). The software engineers then assume they can just 'plug in' to any database. Things might work for a while, but you will hit a bottleneck at some point. Without an understanding of the inner workings of a database you will never be able to reach the full potential of the database itself. Each database has a different way of handling concurrency (some have none at all). Without this fundamental understanding of how a particular database works, your database will never be able to attain the performance that it was designed to deliver.

Don't Fall for Marketing Slogans

Despite the marketing slogans that you may hear, and the high cost of database software, there is no such thing as a self-managing database. There are some great features in the big name proprietary systems that are, in some cases, well worth the price or even a necessity. This does affect the cost of database software, of course. Unfortunately, in the real world, many times these features are not used, underutilized, or they are implemented improperly. Most software developers and even some database professionals do not use them or have any idea how to use them. In either case, it's more money down the drain. Without having the expertise to know the difference between a tuple and cluster, you are at the mercy of the database vendor's marketing department.

In my experience, many organizations, despite the cost of database software, purchase Enterprise versions of Oracle or SQL Server, but rarely use all of the features (or properly use them) that they provide. In reality, the standard edition of either platform would have been more than sufficient for the particular task at hand while also reducing the cost of database softare. Taking this even further, in many cases a properly designed open-source 'free' database would have sufficed, saving thousands, if not tens of thousands of dollars in database licensing fees and further reducing the cost of database software and the overall project.

Most, if not all, database software systems available today are extremely capable of delivering the performance needed for many organizations. Open-source databases such as MySQL, MariaDB and PostgreSQL have come a long way in the last few years and I expect this trend to continue. While they may lack some of the features and refinements of the big-name proprietary database systems, they are very capable systems. While reducing the cost of database software is obviously important, choosing the wrong database can be a costly mistake.

Don't Throw Expensive Hardware at the Problem

Planning and building a scalable database system requires knowledge of hardware that can handle the expected loads. Good servers and associated hardware are a necessity. However, what I've seen way too often is in an attempt to 'fix' performance problems with a database by throwing additional or more expensive hardware (servers, RAM/memory, CPUs, etc.) at the problem without addressing the real underlying issues. In certain cases this may buy you some time, but unless the underlying database issues are resolved, you will be throwing your money away. As much as I love working with fancy, new servers, they won't fix your problems. The high cost of database software (depending upon the platform) combined with the latest hardware can easily break your budget.

If you'd like details on this all-too-common problem, you might want to read my blog post titled, "Money To Burn" since this is a great example of spending lots of money on unnecessary hardware in an attempt to solve performance problems.

Solutions and Golden Rules

  • Have a database professional or team design your database correctly from the beginning
  • Always use database stored procedures and functions whenever possible - no matter what a software engineer tells you or how loud they whine
  • Have a dedicated database administrator manage the database
  • The Database Administrator should be the one writing database code, since ultimately he is the one who understands the inner workings of the database
  • If you have a Database Administrator who can't write database code, get one who can (DBAs should be the database developers as well. In most cases, the role of database developer needs to disappear)
  • Keep all database code in the database, not the application
  • Never listen to anyone who utters the phrase "Database Independence" unless they have built the system as outlined above
  • Never allow any software developers or anyone else to access a production database. The DBA can provide information requests or set up a reporting database for others to use.
  • Have a clear set of roles defined for application developers and database administrators. This actually makes working together easier as each team focuses on their strengths
  • When using a proprietary database platform, determine if you can choose a less expensive 'Standard' version to reduce the cost of database software

Too Many Cooks in the Kitchen and Dead Weight

Many consulting firms will bring in a raft of database 'consultants' from the beginning. Not only is this unnecessary, it is a tip-off that they are just trying to rack up billable hours with a high body count. One or two experienced database architects or DBAs is all you need to begin except in the most complicated of situations. Please keep in mind that not all database professionals are the same. Any good consulting firm will give you an honest initial estimate for little to no cost before any work begins. Beware of some 'foreign' outsourcing firms as many times their only prerequisite for placing a database professional on their team seems to be the ability to fog up a mirror.

Partnerships between some consulting firms and database vendors can oftentimes extert a tremendous influence on the database platform chosen for a project. Make sure that there are valid reasons for choosing a particular database system. Even if a 'deal' on a big-name database can reduce the cost of database software substantially, it may cost you more in consulting fees than is justified by the discounted sticker price.

For this reason, here at Parthian Systems we have instituted fixed-cost consulting and only charge clients by the project, not by the hour. While there are some very honest and capable database consulting firms, many are only concerned with billable hours. We have removed that concern from the equation.

How Does This Save Me Money?

If you keep in mind the 'Solutions and Golden Rules' above, you are well on your way to getting to the heart of what you really need in a database system.

You must have a dedicated, experienced and knowledgeable database professional(s) who will not only choose the correct database software, but also build and manage your database. If this prerequisite is followed, you will open up the possibilities of saving money in the following ways:

  1. Deploy a less expensive version of a proprietary database system (eg, Standard Version versus Enterprise Version, which would reduce the cost of database software in the range of thousands to tens of thousands of dollars)
  2. Purchase/Lease/Deploy less expensive servers and hardware saving thousands of dollars
  3. Instead of purchasing an expensive proprietary relational database management system, determine if you could possibly use an open-source (free database) to reduce or eliminate the cost of database software, and reducing your initial and recurring database licensing costs to zero.
  4. Hire a top-notch database team or outsource your database operations to a company that is responsive and experienced. This will save money, time, and sleepless nights.

If you follow this approach to building or improving your database systems you will gain any possible savings (which could be quite substantial) while still having a database that has the performance, security, and scalability that meets or exceeds your demands. The cost of database software should be taken into account, but should never be the primary factor when choosing the database platform for your organization. We have written a related breakdown of database licensing costs here which may help you get a rough estimate of some of the costs and/or savings you might expect.

We at Parthian Systems are dedicated to empowering our customers with knowledge and choices. We are ready to help answer any questions you have and are eager to assit you in building a database system that is an asset to your organization.