The popular fictional hero from a fantasy saga– Witcher – has two separate swords for two different kinds of monsters. A silver sword for fantastic monsters, and a steel sword for those monsters that walk on two legs and order a pint next to you in a tavern.
Steel is stronger and lasts longer, but is not able to hurt exotic monsters. Silver is less durable and impractical to use in all but the most special of situations – but when it cuts, it slices. The situation in the world of databases is the same as in life in general. To get the best result, you have to pick the right tool for the job.
Multiple databases in use
Let’s take a closer look at the steel sword of the database world, the one you use most often. The ACID (atomicity, consistency, isolation, durability) properties that make the traditional relational databases so reliable, also make them slow and hurt availability compared to the newer data storage technologies from the noSQL family of databases.
Most of the relational databases have been here from the beginning of IT and old enterprise times. Meanwhile, the connectivity of the world has changed so much that users expect IT systems to be available at all times. While achieving a very high availability is also possible in systems using relational databases, the related costs can grow astronomical fast and thus alternative solutions must be considered. That’s why Playtech has started to use multiple different types of databases based on the needs of the various services.
Playtech Infra team event some years ago
The second silver sword is the modern noSQL database in its many variations that perform well under specific constraints or conditions. The noSQL databases currently widely available have great horizonal scaling capabilities and good distribution capabilities, but at the cost of sacrificing one or more ACID principles.
The challenges involved
Like carrying around multiple swords on your back, having multiple database platforms also has its challenges. Even though relational databases seem similar, they behave differently in a lot of cases. They have different built-in functionalities, comply with the standards differently and even their abilities are different. Some systems are better for large object data handling, while others are better suited for metadata or even unstructured data.
Keeping up with the changes in this domain is a challenge all in itself and database administrators must constantly learn new technologies to keep up with the growing number of databases and the increasing velocity of changes. The knowledge we have today is not applicable tomorrow or even between two versions of the same database engine.
The growth of data and the number of databases in general is at a stage of exponential growth in the world The same trend is clearly recognizable at Playtech as well. More databases means less time available for managing and troubleshooting issues on each one. Many large environments require that DBAs move from the management of individual databases or queries on to inventory and capacity management of the entire environment all at once. Over time it becomes increasingly more difficult to find the balance between fine-tuning and inventory management. This often results in higher risks and thus potential
downtimes for the systems.
While here at Playtech, we mostly stick to the steel sword of ACID, we have been known to draw the silver sword of NoSQL from time to time as well. For example, recently we had a situation where the users needed to query similar data for the last 48 hours. As it was needed by a considerable number of parties and all at the same time, we started seeing enormous physical IO and CPU usages on a central Oracle database. Since this data was in its essence not changing, it was perfectly suited for caching. Playtech developed a solution on top of Elasticsearch that acted as a database cache for the past 2 days. The end result was that physical disk reads were reduced by a magnitude and the users were able to enjoy 10x faster response times for their queries.
Going forward, we see a lot of potential for brandishing the silver sword of noSQL. Especially for those cases that are not ACID-critical but need a thorough performance boost to cope with the increasing loads required by today's circumstance. ACID and noSQL are here to stay for the near future, complimenting each other.