MariaDB ColumnStore

In August of this year I was offered a change of career to work on a interesting project called MariaDB ColumnStore. The bulk of my career has been working on database software, especially MySQL based and this project intrigued me, so I took the offer.

Until a couple of years ago there was a product being developed called InfiniDB. This was an analytics database that stored using columns instead of rows and was built on top of MySQL 5.1. Unfortunately the company behind that no longer exists but MariaDB sees the value of this product. The MariaDB ColumnStore project was started as a continuation of the development of InfiniDB.

The initial goal is to take InfiniDB and modify it to work with MariaDB 10.1 whilst fixing bugs found along the way. I have only been a part of the project for a couple of months we have come a long way in that time. We have a great hard-working team behind the project which has helped us progress rapidly. I've had quite a few conversations with members of the community about the project and the feedback has been very positive. It definitely fills a gap in the MySQL/MariaDB world and I'm proud that MariaDB have invested resources into making this a successful product.

MariaDB ColumnStore is an Open Source project the source code server and engine are available on GitHub and bugs are tracked on Jira. We also have a public mailing list/forum to discuss the project and answer any questions. With the GitHub trees the master branch contains the latest release and develop contains the code going into the next release.

Today we are announcing the first beta release containing all our efforts so far, MariaDB ColumnStore 1.0.4. This release is our easiest ever to install, a lot cleaner and integrates better with the operating system than our previous alpha releases.

Beta Release

This first beta release contains a modified version of MariaDB 10.1.18. The modifications are required to support some of the more unique features of ColumnStore such as Window Functions. Queries are processed using external processes that can be run on multiple servers making this a distributed engine similar to map/reduce systems.

It does not yet support all the data types the MariaDB supports but for large analytics data sets it should support everything you need.

You can read about the beta here and download it from MariaDB's Download Page. There is also a follow up blog-post with more information.

The Future

As we head towards GA we are aiming to fix bugs found during the beta period and add a few utilities that are not yet present. We also plan to support more Linux distributions that we do at present.

From there we will be working on more enhancements and features including adding support for data types that are currently missing.

This is an exciting project with a complex codebase and it is a lot of fun to work on. I'm looking forward to the adventure ahead.

Other articles

Multi-Gigabit Networking on a Budget
Sat 02 Jan 2016, 14:10

Nginx

Nginx networking

I've recently acquired two "refurbished" Xeon workstations from eBay to do some NGINX testing with. These are:
- A Lenovo ThinkStation S30 - 6 core / 12 thread E5-1650v2 @3.5GHz, 4GB ECC RAM (upgraded to 12GB ECC - £10) - £650
- A Lenovo ThinkStation C20 - 2x 4 core / 8 thread (8 core / 16 ...
read more
Hitting Network Limits

Mon 06 Jul 2015, 18:10

Nginx

Nginx networking

I recently wrote a high-level blog post for NGINX outlining the new SO_REUSEPORT feature in NGINX. One of the problems I was having whilst doing the benchmarks for this was that I was hitting some kind of bottleneck before hitting the limit of what NGINX could process. This meant that ...
read more
The LinuxJedi at NGINX

Fri 26 Jun 2015, 21:40

Nginx

Nginx

Several months ago I was having a Sunday brunch with some friends in Seattle. One of the discussions that came up was the fact that NGINX needed to expand their developer relations team and we were thinking up names of likely candidates for the role. Fast forward to today and ...
read more
Why Dependency Testing is Important

Thu 23 Apr 2015, 22:27

General

Testing Parallels

Or...

Why I'm Ditching Parallels

I've been using Parallels quite successfully for several months now and I must admit it is a great way to run all my various Linux distros that I work with on my Mac.

One of my main VMs is the latest Fedora release ...
read more
Variable Length Arrays in C

Tue 24 Feb 2015, 13:25

Coding

HP Advanced Technology Group C

In a bid to aim this blog at multiple levels I'm going to talk today about Variable Length Arrays (VLAs) in C. I'm covering this topic in particular because a friend who is a Harvard CS50 student recently asked me about them.

As many people know, C is ...
read more
Live Kernel Patching - Why You Should NOT Use It

Sat 14 Feb 2015, 22:00

General

HP Advanced Technology Group Kernel

Just under a year ago on my old blog I discussed and even demoed the new Linux live kernel patching solutions. I was reviewing these technologies out of my own curiosity as well as HP's Advanced Technology Group having an interest. I think these technologies are great, I am ...
read more
Why The C10K Problem is Important

Sat 14 Feb 2015, 17:49

General

HP Advanced Technology Group Nginx libAttachSQL

For those who are not familiar with the concept the C10K problem is about trying to handle many simultaneous clients on a web server. Originally written about trying to handle 10,000 clients on a Gigabit Ethernet connection back in the late 90s, but this is still a problem today ...
read more
YubiKey NEO with GnuPG

Wed 28 Jan 2015, 10:24

Security

YubiKey Security HP Advanced Technology Group

Last week I published a blog post on using YubiKey for OS Logins. Since then I've had a request from inside HP to create a blog post on using the YubiKey NEO with GnuPG which is another thing I have done with my NEO.

The YubiKey NEO has a ...
read more
Working With Git

Sun 25 Jan 2015, 16:33

Coding

HP Advanced Technology Group Git

At a recent meeting HP's Advanced Technology Group has been agreeing on standards for working with git, covering mainly the collaboration and versioning aspects. Today I will share with you how I do this in my Open Source projects which has some crossover with the group's work.

Versioning ...
read more

The /dev/null ramblings of a Linux Jedi

Other articles

Multi-Gigabit Networking on a Budget

Hitting Network Limits

The LinuxJedi at NGINX

Why Dependency Testing is Important

Why I'm Ditching Parallels

Variable Length Arrays in C

Live Kernel Patching - Why You Should NOT Use It

Why The C10K Problem is Important

YubiKey NEO with GnuPG

Working With Git

Versioning ...

Pages