PostgreSQL-meetup-02-09-25
Russell Morris
- 4 minutes read - 777 wordsSo the current months PostgreSQL user meetup was tonight, subjects being discussed were very large PostgreSQL type solutions. These are more analytical (OLAP) then transactional (OLTP) and better suited to the Analytical processing then pure PostgreSQL. I am not saying that PostgreSQL can not do deal with analytical workloads but these solutions are simply better at it.
But lets start with where the meetup was set. Tonight we were in the Doggett’s Coat and Badge pub by Blackfriars bridge, for me a great destination as close enough to walk from work and also a short walk to Waterloo and onto the train home, and nice beers on tap.
So lets talk about what was looked at tonight:
Apache Cloudberry DB (2) - https://cloudberry.apache.org
So what is Cloudberry DB. Cloudberry was born out of an existing solution called GreenPlum.
Lets talk about GreenPlum first as its good to know where you have come from to know where you have got to. Greenplum was created in 2005, based on PostgreSQL but as an MPP (massively parallel processing), on an open source solution that was then passed from owner to owner to mow be owned by Broadcom - they have since taken it closed source. GreenPlum is based on an early version of PostgreSQL (but if its not broke don’t fix it). Did you know that AWS Redshift is built on PostgreSQL v8 and also utilises their own brand of MPP.
Cloudberry was forked from GreenPlum a few years ago, its kept all the good things and also are in the process of upgrading the PostgreSQL version used to 16 (currently its 14).
What makes this really fast for analytical workloads is the MPP, what does this mean you ask? Well it means that you are processing data over multiple shards. Even if you install this onto one machine you can still have multiple shards of the data resident here. This then means you can run the a query in parallel over the shards and its quicker then normal. The other thing it does (it does lots of really good things) is the automatic moving of data between the shards. So you do not have to worry about balancing the data.
This has been done not via building the solution directly on top of PostgreSQL but rebuilding this from the ground up.
It looks to be a really good solution and worth a try.
Ceder DB - https://cederdb.com
All I can really says is WOW. So this started as a university project in Munich back in 2018 (I think). The professor that started the work also worked on the Salesforce big data solution (sorry I have not got the name of the software).
So what they have done is built a DB from the ground up with a sliver of PostgreSQL, so at a command level it will interpret PostgreSQL commands from PSQL and you won’t really see a difference. But there’s a big difference.
I will try and explain, but I will be honest a lot of what was discussed was way above my head.
So similar to all db engines they have their query execution level, planner etc. But some of the query execution is merged. They are able to bring the some of the aggregation levels of the query into the joins part of the query. Depending on the query that’s being run the execution engine will treat different joins in a different order depending on what the query is doing.
Most DB engines will run a SQL query as SQL process, this DB will take it convert the query to assembler language and run this at a machine level. So what they have done is bypassed the C++ compiler and basically made their own copy. There is a lot of magic sauce in there but suffice to say it seems to work. There are some caveats that because of the translation from SQL to machine code is home brewed sometimes it may not work quite right - so there’s some creases to iron out there.
The benchmark test shown shows its very quick outstripping standard PostgreSQL a lot, but also beating Duckdb (not the pg_duckdb) the full version.
I would recommend looking at the website and seeing what they have done. It seems to be a very exciting project with the approach of if its not broken, I am still going to see if can be improved and they have done so.
Roundup
Tonight I saw two interesting and ready go databases that will easily handle big data, and potentially faster and better than other big players.
Next meetup subjects are performance monitoring and Cockroach db