Deconstructing Postgres into a cloud native platform

Feb 09, 2021 by melissa

[fusion_builder_container hundred_percent=”no” hundred_percent_height=”no” hundred_percent_height_scroll=”no” hundred_percent_height_center_content=”yes” equal_height_columns=”no” menu_anchor=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” status=”published” publish_date=”” class=”” id=”” link_color=”” link_hover_color=”” border_size=”” border_color=”” border_style=”solid” margin_top=”” margin_bottom=”” padding_top=”” padding_right=”” padding_bottom=”” padding_left=”” gradient_start_color=”” gradient_end_color=”” gradient_start_position=”0″ gradient_end_position=”100″ gradient_type=”linear” radial_direction=”center center” linear_angle=”180″ background_color=”” background_image=”” background_position=”center center” background_repeat=”no-repeat” fade=”no” background_parallax=”none” enable_mobile=”no” parallax_speed=”0.3″ background_blend_mode=”none” video_mp4=”” video_webm=”” video_ogv=”” video_url=”” video_aspect_ratio=”16:9″ video_loop=”yes” video_mute=”yes” video_preview_image=”” filter_hue=”0″ filter_saturation=”100″ filter_brightness=”100″ filter_contrast=”100″ filter_invert=”0″ filter_sepia=”0″ filter_opacity=”100″ filter_blur=”0″ filter_hue_hover=”0″ filter_saturation_hover=”100″ filter_brightness_hover=”100″ filter_contrast_hover=”100″ filter_invert_hover=”0″ filter_sepia_hover=”0″ filter_opacity_hover=”100″ filter_blur_hover=”0″][fusion_builder_row][fusion_builder_column type=”1_1″ type=”1_1″ layout=”1_1″ spacing=”” center_content=”no” link=”” target=”_self” min_height=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=”” hover_type=”none” border_size=”0″ border_color=”” border_style=”solid” border_position=”all” box_shadow=”no” box_shadow_blur=”0″ box_shadow_spread=”0″ box_shadow_color=”” box_shadow_style=”” background_type=”single” gradient_start_position=”0″ gradient_end_position=”100″ gradient_type=”linear” radial_direction=”center center” linear_angle=”180″ background_color=”” background_image=”” background_image_id=”” background_position=”left top” background_repeat=”no-repeat” background_blend_mode=”none” animation_type=”” animation_direction=”left” animation_speed=”0.3″ animation_offset=”” filter_type=”regular” filter_hue=”0″ filter_saturation=”100″ filter_brightness=”100″ filter_contrast=”100″ filter_invert=”0″ filter_sepia=”0″ filter_opacity=”100″ filter_blur=”0″ filter_hue_hover=”0″ filter_saturation_hover=”100″ filter_brightness_hover=”100″ filter_contrast_hover=”100″ filter_invert_hover=”0″ filter_sepia_hover=”0″ filter_opacity_hover=”100″ filter_blur_hover=”0″ first=”true” last=”true”][fusion_text columns=”” column_min_width=”” column_spacing=”” rule_style=”default” rule_size=”” rule_color=”” animation_type=”” animation_direction=”left” animation_speed=”0.3″ animation_offset=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=””]

Postgres has been around for a long time in the DB space. Known to be a very reliable and sturdy technology even considered boring to some. But we were lucky enough to have Álvaro Hernández on to talk to us about Postgres innovations that are coming our way. If you want to re-watch the original click here. If you don’t have time, give this article a read.

[/fusion_text][fusion_title title_type=”text” rotation_effect=”bounceIn” display_time=”1200″ highlight_effect=”circle” loop_animation=”off” highlight_width=”9″ highlight_top_margin=”0″ before_text=”” rotation_text=”” highlight_text=”” after_text=”” content_align=”left” size=”2″ font_size=”” animated_font_size=”” line_height=”” letter_spacing=”” text_color=”” animated_text_color=”” highlight_color=”” style_type=”default” sep_color=”” animation_type=”” animation_direction=”left” animation_speed=”0.3″ animation_offset=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=”” fusion_font_variant_title_font=””]

The protagonist:

Álvaro is the founder and CEO of OnGres which essentially means “on Postgres”. Apart from offering professional services like support and training one of their main focuses is R&D. They try to actively innovate on Postgres. Apart from community work of which he is an active member of. Álvaro has also founded a nonprofit in Spain to promote and develop Postgres, they organize a conference every year in Ibiza, don’t miss it. He’s also an AWS data hero.

How Álvaro got started with Kubernetes

Álvaro got started with Kubernetes the same way many people do, by trying to solve a problem. Postgres is very easy to download and run on your local machine, but it’s a very different story when we’re talking about running Postgres in production at the enterprise level. He quickly saw that there was a need for automation and finding a way to lower the time of installation and management of Postgres in different environments. Once Álvaro and his team discovered Kubernetes, this powerful orchestrating system that will offload the complexity of setting up, managing, configuring and implementing enterprise scale Postgres environments they’ve never looked back. And now they have developed a software stack StackGres, to run Postgres on Kubernetes the easy way.

Why postgres?

[/fusion_title][fusion_text columns=”1″ column_min_width=”” column_spacing=”” rule_style=”default” rule_size=”” rule_color=”” animation_type=”” animation_direction=”left” animation_speed=”0.3″ animation_offset=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=””]

Posgres is rock solid, reliable it just won’t let you down. Take it from Kelsey Hightower.

Even though Alvarado agrees with Kelsey Hightower he also wants to make clear that even though technology can be boring and reliable at its core, it can still innovate and that’s what he and his team work tirelessly to do.

Innovation opportunities

Postgres has grown in many areas, but there are some core areas that Postgres hasn’t such as deployment and automation. There should be ways that engineers that leverage Kubernetes to use full Postgres functionalities without knowing the in’s and out’s of the technology. This is how they plan on adjusting that.

Innovation agenda

Going about the idea of innovating Postgres in six key areas.

Envoy: offloading monitoring and SSL

There are many upsides to taking workloads like monitoring for example and running them elsewhere. You can free up CPU by offloading CPU-heavy workloads like SSL. Let Envoy take the monitoring and SSL task and let Postgres focus on the boring part.

How it would looks:

Distributed logs:

Much needed when you are troubleshooting a 16 node Postgres cluster.

[/fusion_text][fusion_table fusion_table_type=”1″ fusion_table_rows=”4″ fusion_table_columns=”2″ animation_type=”” animation_direction=”left” animation_speed=”0.3″ animation_offset=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=””]

Main features	Timescale usage
Postgres, Patroni logs (pgBouncer:WIP)	Open Source (Apache 2) version
Multi-tenant: all pods from one of more clusters	Partitions the logs space by timestamp
Query with SQL or via the web console	Provice support for partition pruning (log lifecycle)

[/fusion_table][fusion_title title_type=”text” rotation_effect=”bounceIn” display_time=”1200″ highlight_effect=”circle” loop_animation=”off” highlight_width=”9″ highlight_top_margin=”0″ before_text=”” rotation_text=”” highlight_text=”” after_text=”” content_align=”left” size=”2″ font_size=”” animated_font_size=”” line_height=”” letter_spacing=”” text_color=”” animated_text_color=”” highlight_color=”” style_type=”default” sep_color=”” animation_type=”” animation_direction=”left” animation_speed=”0.3″ animation_offset=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=”” fusion_font_variant_title_font=”” margin_top_mobile=”20px”]

Init-less containers. Sidecars. Anatomy of the pod. StackGress architecture

A container is not a slim vm, there is no need to take your Postgres vm put it into a 1 GB container and run it. There are better ways, this is a better way.

And this is what your architecture could look like

Monitoring Integration: Prometheus, Grafana

Using K8s RBAC for Web Console AuthN/AuthZ

There aren’t many Web UIs for Postgres cluster management. And instead of having an external auth pool. Postgres is going to use the existing k8s RBAC and alongside the StackGres web UI authenticate users that way.

Cloud-Nativization of Postgres DBA Operations

[/fusion_title][fusion_text columns=”” column_min_width=”” column_spacing=”” rule_style=”default” rule_size=”” rule_color=”” animation_type=”” animation_direction=”left” animation_speed=”0.3″ animation_offset=”” hide_on_mobile=”small-visibility,medium-visibility,large-visibility” class=”” id=””]One of the goals is to widen the Postgres user base, but not too many people are Postgres experts. To manage a production Postgres cluster you need a lot of expertise, but of course many of the tedious maintenance tasks could be automated. So here, there have tried to take the most frequent operations and automate them. Now all of your automated tasks can be a YAML file away.

Q&A

Do you integrate Patroni into the StackGres operator?

- Yes.

What could be the challenges for making a Postgres upgrade automation?

- It really depends on the automation that you are trying to make. The ones that they are working on now are the easily repeatable, and you can easily write and algorithm for. More complicated automations will come.

What is the performance overhear when running Postgres in Kubernetes?

- When running Postgres in containers the change in performance deviation is almost negligible.

Can you elaborate on the approach of the operator hierarchy, instead of having one operator to manage everything, how do you split your operators and what benefits do you see in that approach?

- The operator was initially built in a well known Kubernetes pattern (controller and one or more series). But they soon found out that there were certain operations that might better be performed by other components. The REST API needed to at some point connect to the Kubernetes API directly and a reconciliation cycle was used to create this connection.

The get your hand on the Art of Post:

The Art of PostgresQL

Enter promo code: dok21 to get 21% discount.

To get involved with the Postgres community you can find them at:

slack.stackgres.io
discord.stackgres.io

[/fusion_text][/fusion_builder_column][/fusion_builder_row][/fusion_builder_container]

Data on Kubernetes Day Europe 2024 talks are now available for streaming!

Deconstructing Postgres into a cloud native platform