Releases: postgres-ai/database-lab-engine
DBLab Engine v3.5.0
New features
- Switch license to Apache 2.0 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/817
- Simple installer for DBLab - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/816
- Enforce strong DB password values - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/813
- Rename DLE to DBLab - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/814, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/818
- Support PostgreSQL 16 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/820
Improvements and fixes
- Allow access to clones from an internal Docker network - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/802
- Adjust tags in the SE snippets - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/804
- Make snapshots request conditionally - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/819
- Correct disabled state for the DLE installation - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/803
- Use explicit tags for Postgres' dockerImage - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/808, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/824
- Verify for dumpLocation on the filesystem - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/801
- Adjust listen addresses in the foundation container - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/806
- Bump SE installer version - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/811
- Init provisioner after starting API server - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/809
- Update Generic Postgres tag in UI - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/821, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/831
- Stop bloating diagnostic logs - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/827
Internal
- Update README - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/800
- Prevent Cypress tests from failing - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/807
How to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Provide feedback, contribute
Any feedback will be greatly appreciated:
- Twitter: @Database_Lab
- Database Lab Community Slack: https://slack.postgres.ai/
If you want to help the project grow, consider various ways of contributing:
- Star the GitHub repo: https://github.com/postgres-ai/database-lab-engine
- Spread the word using Twitter (please mention @Database_Lab!) or your social network of choice
- Consider translating the README.md to one or several languages you know
- Consider implementing one of the features that will make DLE better: see CONTRIBUTING.md and the list of "good first issues" on GitLab.
Full diff
Diff between versions v3.4.0 and v3.5.0: v3.4.0...v3.5.0
Where to start / get help
- Tutorial
- DBLab 3.5 demo instance (token:
demo-token
) - Docs
- Postgres.ai Console
- Contact us
Provide feedback, contribute
We greatly value your feedback. Connect with us via:
- X/Twitter @Database_Lab
- DBLab Community Slack
Additional resources where you can get insights about DBLab and Postgres:
Interested in giving back to the project? Here's how you can make an impact:
- Give a star to our GitHub repository
- Help us reach more enthusiasts. Share about Database Lab on Twitter (don't forget to tag @Database_Lab) or any other platform you fancy
- Multilingual? Consider translating our README.md to share the knowledge in your language
- Are you a developer? Dive in and enhance the Database Lab Engine (DLE) experience; check out our CONTRIBUTING guidelines and explore the "good first issues" list on GitLab
DBLab Engine v3.4.0
New name: DBLab Engine
The new name for the Database Lab Engine is "DBLab Engine". Updates are currently underway across our materials to reflect this change. To align with this change, we have introduced specific domains for the product: dblab.dev
and dblab.sh
. For ease of access, we have established the following short URLs:
- https://cli.dblab.dev/ and https://dblab.sh – CLI setup script (works on macOS/Linux/Windows:
curl -sSL dblab.sh | bash
) - https://demo.dblab.dev/ – DBLab 3.4 demo (token:
demo-token
) - https://branching.dblab.dev/ – DBLab 4.0-alpha demo, implementing full-fledged DB branching and snapshots on demand (token:
demo-token
) - https://aws.dblab.dev/ – DBLab SE page in AWS Marketplace
- https://docs.dblab.dev/ – the docs
- https://api.dblab.dev/ – interactive API reference (powered by readme.io)
New contributors
🙇 Many thanks to our first-time contributors: @mbobin, @TheLiamGuy, @gromsterus, @ibash, @byx01, @davidsvenson.
By the numbers
- DBLab GitHub repository has crossed the 1700+ stars mark
- X/Twitter account @Database_Lab has more than 1300 followers
- Our LinkedIn page has amassed 1500+ followers
Thank you for the support!
New DBLab SE installer
We've expanded the installation options for DBLab SE. Besides its presence in the AWS Marketplace, you can now seamlessly install DBLab SE directly from the Postgres.ai Console.
This setup is entirely automated and can be used anywhere:
- For those who have existing machines, we support the "BYOM" (Bring Your Own Machine) method
- If you're utilizing AWS, GCP, DigitalOcean, or Hetzner Cloud, the installer handles resource provisioning—including VMs, disks, and more—automatically.
Check out the updated tutorial.
Migration notes
For those considering an upgrade, we recommend executing a fresh installation of DBLab. At present, we don't have automation in place for upgrades. If you need to update an existing DBLab installation, it's essential to stop and remove the current containers. Then, initiate a fresh start of the DBLab container using the 3.4.0
tag. More details can be found in the docs. Before initiation, ensure you've adjusted the DBLab configuration as outlined below.
Postgres images
There is a new method for running clones with customized port and socket directories. Please note that this is NOT backward compatible with previous versions of the Postgres images that DBLab utilized. It is crucial to ensure the dockerImage
options utilize tags with the -0.3.0
suffix or later, such as postgresai/extended-postgres:15-0.3.0
(details can be found in this MR).
Global telemetry
The global.telemetry
subsection has been deprecated and removed. The control to activate/deactivate telemetry has been relocated to the platform
section:
platform:
url: "https://postgres.ai/api/general"
enableTelemetry: true
Platform
From version v3.4.0
onward, the platform
section is enabled by default.
If either the accessToken
or orgKey
keys aren't empty, ensure the url
parameter is specified; otherwise, the DBLab instance will not initialize. By default, neither accessToken
nor orgKey
are defined when in case of self-managed installations.
platform:
url: "https://postgres.ai/api/general"
enableTelemetry: true
# orgKey: "org_key"
# accessToken: "platform_access_token"
Docker mounts
Please omit the following Docker mount directives during container startup, as they are no longer necessary:
--volume /sys/kernel/debug:/sys/kernel/debug:rw \
--volume /lib/modules:/lib/modules:ro \
--volume /proc:/host_proc:ro \
Changelog
New features
- Configuration: new option
ignoreErrors
in subsectionslogicalDump
andlogicalRestore
to allow not to interrupt the process of dump/restore in case of errors - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/699, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/700, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/730 - Configuration: new option
skipPolicies
in subsectionlogicalRestore
to allow to skip policies (CREATE POLICY
) during restore process, to avoid errors likepolicy "xxx" for table "yyy" already exists
: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/769 - Configuration: new option
cloneAccessAddresses
in sectionprovision
to control port publishing for clone containers; now limited to127.0.0.1
by default; support multiple addresses and IPv6: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/786 - UI: Improved Logs view with log filters - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/672, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/702
- CLI: ARM / Apple silicon (M1, M2) support for DBLab CLI binaries - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/719, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/733
- API: DBLab v3.4.0 API specification updated and is now published on api.dblab.dev: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/747
- Numerous updates to Postgres images
- supported Postgres major versions: 9.6-15
- image tags:
9.6-0.3.0
...15-0.3.0
;⚠️ older tags won't work with DBLab 3.4.x pgvector
extension added to- existing extension upgraded (docs)
- special images for DBLab SE/EE (with some extension being mocked): RDS Postgres, Aurora, GCP Cloud SQL, Supabase, Timescale, Heroku, PostGIS
- New installer for DBLab SE, available at Console.Postgres.ai, and simplified configuration for various managed PostgreSQL services such as AWS RDS, GCP Cloud SQL, Supabase, Heroku - various MRs
Improvements and fixes
- Enable debug mode by default - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/687
- UI: Instant view of configuration, not waiting for the end of retrieval (physical mode) - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/674
- UI: Collapse/expand the right-side panel - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/677, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/681
- UI: Prevent crash when project ID doesn't exist - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/684
- Add system stats (number of CPUs and size of RAM) to the EngineStarted telemetry event - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/688
- Add hint about Postgres logs - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/685
- UI: "Sign out" button - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/678
- Correctly detect clone activity / idleness for timezones different from UTC - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/692, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/734
- If full refresh fails, report it in
/status
with proper messages - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/696 - Mask sensitive data in logs - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/705
- UI: Extend and improve the Configuration tab - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/698
- Check in UI if retrieval mode is unknown - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/669
- Clean up data during full data refresh in logical mode - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/704
- Choose the last snapshot when reset to the latest snapshot is requested - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/708
- Remove timing estimator (experimental) - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/716
- Avoid panic with concurrent map read and map write: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/726
- Exclude timescaledb internal scheme from the restricted script: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/738
- UI: show Configuration tab if mode is unknown: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/735
- Change error message from JSON format to string: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/739
- UI: Various fixes: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/752, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/759
- Remove too detailed error message from the logs: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/777
- Exclude dump location volumes nested in the mount directory: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/778
- Ignore STDERR buffer of the process while reading exec command response (previously, DBLab considered a task failed if exit code was 0 but STDERR had messages; now only the exit code defines success/failure) https://gitlab.com/postgres-ai/database-l...
Database Lab Engine v3.3.0
New features
- Select docker images on the configuration page for Standard Edition: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/651, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/663, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/665, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/667
- Optionally disable configuration management via UI/API: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/622, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/664
- Add customOptions for pg_dump and pg_restore commands: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/660, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/661, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/666
Improvements and fixes
- Update demo instance: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/616
- Reading replication lag: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/618
- Fix errors on reporting empty activity: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/621
- Remove pg_basebackup environment variables for the promotion stage: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/617
- Enable logs using websockets for CloudFormation installations: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/623
- Build UI image in CI pipelines: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/592, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/625, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/626, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/628
- Adjust tests for PostgreSQL 15: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/633
- Use frameless and transparent SVG and PNG versions of Database Lab logo: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/634
- Update SwaggerUI specs and dist: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/632
- Refactor and fix issues on the Logs tab: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/636
- Hide Logs and Configuration tabs on the Platform: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/631
- Migrate UI components, assets and utils to Typescript: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/589, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/645, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/646
- Preserve collapsed state for left side: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/639
- Exclude Typescript files from linguist stats: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/643
- Display snackbar on arriving new data for Logs tab: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/640
- Improve message about skipping full-refresh: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/648
- Improve wording for connection/extensions/locales check: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/653
- Fix the checkup snippet: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/650
- Use extended Docker image with PostgreSQL 15 by default: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/655
- Fix RDS source type in configuration examples: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/659
How to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Provide feedback, contribute
Any feedback will be greatly appreciated:
- Twitter: @Database_Lab
- Database Lab Community Slack: https://slack.postgres.ai/
If you want to help the project grow, consider various ways of contributing:
- Star the GitHub repo: https://github.com/postgres-ai/database-lab-engine
- Spread the word using Twitter (please mention @Database_Lab!) or your social network of choice
- Consider translating the README.md to one or several languages you know
- Consider implementing one of the features that will make DLE better: see CONTRIBUTING.md and the list of "good first issues" on GitLab.
Diff between versions v3.2.0 and v3.3.0: v3.2.0...v3.3.0
Database Lab Engine v3.2.0
Release v3.2.0 delivers a good number of new features and improvements. DLE UI is extended with two new tabs, "Logs" and "Configuration", to improve the user experience when configuring or reconfiguring DLE for managed Postgres services such as AWS RDS and Google Cloud SQL:
This allows users to see PGDATA initialization errors when setting up DLE, then adjust the configuration and retry initialization right in the UI – without any need to connect to the DLE machine using SSH.
The demo DLE instance can be found here: https://demo.aws.postgres.ai:446/instance (token: demo_token
).
Postgres 15 is already supported!
PostgreSQL 15 is already supported by DLE and included in the list of default images, with many extensions shipped with it: https://postgres.ai/docs/database-lab/supported-databases.
AWS Marketplace – the easiest way to get DLE for any Postgres database
DLE in AWS Marketplace has now General Availability (GA) status, it has already been updated to version 3.2.0: https://aws.amazon.com/marketplace/pp/prodview-wlmm2satykuec, offering the Standard license, with support, embedded monitoring, simplified installation, and access to a wider variety of supported Postgres images (including extensions not available by default). An instance with DLE delivers "dev/test databases as a service" and levels up all dev&test processes. For example, on an r5.xlarge instance (4 vCPUs, 32 GiB RAM) with 200 GiB disk space, it is possible to run up to 30 database clones simultaneously (and even more, if shared_buffers
is adjusted to lower values) spending as low as $360 per month – less than for a single regular RDS clone of the same size.
New contributor
🙇 The development team greets a new contributor: @Adrinlol. This release wouldn't be possible without other contributors: @agneum, @vitabaks, @DmitryFomin1, @ane4ka, @NikolayS, @denis256, @yuridevx.
What's new
New features
- New tab in UI, "Configuration" and corresponding API to allow editing of certain configuration parameters (only in logical mode; secret values are masked) and viewing the current config (in any mode): !561, !575, !594, !601, !612, !613, !614 (@yuridevx, @Adrinlol), !608, !609, !612 (@Adrinlol), !580, !615
- New tab in UI, "Logs" to allow seeing DLE log messages to support troubleshooting (secret values are masked): !579 (@Adrinlol), !585, !593
- Manage logs of PostgreSQL containers to simplify diagnostics (previously, if an error happens inside Postgres container and the container is destroyed, it was impossible to find the details of the error): !558, !570, !598, https://gitlab.com/postgres-ai/docs/-/merge_requests/481 (@denis256)
- Support
queryPreprocessing
inlogicalRestore
!576 - Allow using inline SQL in
queryPreprocessing
(available in sectionslogicalRestore
,logicalSnapshot
,physicalSnapshot
) !576 - Simplify Postgres image customization – for initialization in logical mode, perform comprehensive testing of the source database and compare a container created using the Postgres image provided in configuration with the source DB: !573, !578 (:bow: contribution by @Adrinlol), !596
- Show details of continuous synchronization of PGDATA (API+CLI; currently only for physical mode): !562, !595 (@denis256)
- Show the progress of the full refresh process !574, !581, !584, !587, !588, !591, !610 (@Adrinlol)
- Configuration option
excludeTables
– allow excluding certain tables for data retrieval in logical mode: !536, https://gitlab.com/postgres-ai/database-lab/-/issues/348 (was also backpatched to 3.1.2) - Support PostgreSQL 15: !599, https://gitlab.com/postgres-ai/custom-images/-/merge_requests/42, https://gitlab.com/postgres-ai/custom-images/-/merge_requests/44, https://gitlab.com/postgres-ai/custom-images/-/merge_requests/45, https://gitlab.com/postgres-ai/docs/-/merge_requests/504
Improvements and fixes
- Redesign the Retrieval service: !549
- Mounting of socket directory for non-guest virtualization roles: !557 (@denis256)
- Add a link to the guide "PostgreSQL configuration in clones": !559
- Add session manager to CF template: !560
- Add
maintenance_work_mem
for Logical mode: !567 - Report the type of DLE subscription in UI: !563
- Detect if DLE is running in a Docker container (this resolves issues of running DLE on KVM): !571, !603
- Apply the configuration and then start Postgres on the restore stage, to reduce the number of restarts needed: !518, !547 (@yuridevx), !558, !572
- Fix the creation of a restricted user: !586
- Build CE UI Docker image after merging into the master branch: !590
- Adjust container readiness health checks: !597, !602
- Remove deprecated eslint analyzers: !605
- Increase the number of parallel jobs in logical mode from 2/2 to 4/4 – usually, many more CPU cores are available: !611
Diff showing all changes
Diff between versions v3.1.2 and v3.2.0: v3.1.2...v3.2.0.
Migration from DLE v3.1.x and configuration notes
There are no special actions that are required to migrate from DLE 3.1.x to 3.2.0: you need to stop the DLE container, change the image, and start a new one using the new image. All snapshots should remain available, and if, at stopping time, some clones were present, they should be automatically recreated by DLE.
Note: if you have an older version, check the previous release notes as well: https://gitlab.com/postgres-ai/database-lab/-/releases.
Configuration directory
The config directory, by default, is ~/.dblab/engine/configs
and the corresponding mounted volume must be writable:
...
--volume ~/.dblab/engine/configs:/home/dblab/configs
...
Logs directory
Make sure that ~/.dblab/engine/logs
is mounted as a volume to the DLE container:
...
--volume ~/.dblab/engine/configs:/home/dblab/configs \
--volume ~/.dblab/engine/meta:/home/dblab/meta \
--volume ~/.dblab/engine/logs:/home/dblab/logs \
...
Alternatively, you can mount the entire directory (including configs
and meta
– but make sure that these subdirectories exist, DLE doesn't create them yet):
...
--volume ~/.dblab/engin...
v3.1.2
Database Lab Engine v3.1.2
DLE v3.1.2 delivers multiple bug fixes and improvements.
🙇 The development team greets new contributors: @yuridevx, @denis256.
New features and logic adjustments
- Configuration option
excludeTables
– exclude certain tables for data retrieval in logical mode: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/536, https://gitlab.com/postgres-ai/database-lab/-/issues/348
Improvements and fixes
- Allow restricted users to access all databases in clones created with the
--restricted
flag https://gitlab.com/postgres-ai/database-lab/-/merge_requests/532, https://gitlab.com/postgres-ai/database-lab/-/issues/356 - Determine the LATEST backup correctly WAL-G – use creation time instead of modification time https://gitlab.com/postgres-ai/database-lab/-/merge_requests/540, https://gitlab.com/postgres-ai/database-lab/-/issues/365
- Detect and start existing UI container https://gitlab.com/postgres-ai/database-lab/-/merge_requests/538, https://gitlab.com/postgres-ai/database-lab/-/issues/371
- Avoid conflicts when CI checker sessions run at the same time https://gitlab.com/postgres-ai/database-lab/-/merge_requests/542, https://gitlab.com/postgres-ai/database-lab/-/issues/380
- Do not use a non-existing column in the logerrors stat query https://gitlab.com/postgres-ai/database-lab/-/merge_requests/543, https://gitlab.com/postgres-ai/database-lab/-/issues/381
- Keep the snapshot list with relevant values https://gitlab.com/postgres-ai/database-lab/-/merge_requests/539, https://gitlab.com/postgres-ai/database-lab/-/issues/382
- Fix README translation links https://gitlab.com/postgres-ai/database-lab/-/merge_requests/537
- Clarify how DLE works for managed Postgres services in clouds https://gitlab.com/postgres-ai/database-lab/-/merge_requests/482
- Avoid panicking with an empty scheduler context on physical restore https://gitlab.com/postgres-ai/database-lab/-/merge_requests/546, https://gitlab.com/postgres-ai/database-lab/-/issues/347
- Do not set "x" flag for the instance_id file https://gitlab.com/postgres-ai/database-lab/-/merge_requests/545
- Update retrieval flow to check and start existing control containers https://gitlab.com/postgres-ai/database-lab/-/merge_requests/544
- Wait for control containers to finish stopping when DLE shutdowns https://gitlab.com/postgres-ai/database-lab/-/merge_requests/544, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/550, https://gitlab.com/postgres-ai/database-lab/-/issues/387
- Implement a general approach to gracefully shutting down control containers with Postgres https://gitlab.com/postgres-ai/database-lab/-/merge_requests/547, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/551, https://gitlab.com/postgres-ai/database-lab/-/issues/350
- Change permissions on the instance_id file and its directory https://gitlab.com/postgres-ai/database-lab/-/merge_requests/555
Contributor's experience
- Make it possible to debug DLE locally in Docker container https://gitlab.com/postgres-ai/database-lab/-/merge_requests/548
How to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Provide feedback, contribute
Any feedback will be greatly appreciated:
- Twitter: @Database_Lab
- Database Lab Community Slack: https://slack.postgres.ai/
If you want to help the project grow, consider various ways of contributing:
- Star the GitHub repo: https://github.com/postgres-ai/database-lab-engine
- Spread the word using Twitter (please mention @Database_Lab!) or your social network of choice
- Consider translating the README.md to one or several languages you know
- Consider implementing one of the features that will make DLE better: see CONTRIBUTING.md and the list of "good first issues" on GitLab.
Diff between versions v3.1.1 and v3.1.2: v3.1.1...v3.1.2
v3.1.1
Database Lab Engine v3.1.1
DLE v3.1.1 delivers multiple bug fixes and improvements.
The development team greets a new contributor: @Alexand.
Improvements and fixes
- Use ZFS 2.1 in Docker images by default (DLE versions with ZFS 0.8 now shipped in separate images) - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/515
- Add pg_stat_kcache to the default value of
shared_preload_libraries
- https://gitlab.com/postgres-ai/database-lab/-/merge_requests/522 - Increase the number of health check attempts before considering a Postgres container as unhealthy - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/516
- Add pt-br translation to README - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/500 (:bow: contribution by @Alexand)
- Update translations and README - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/517, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/520, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/523, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/527
- Alter ownership for user-defined types, stored procedures, aggregate functions - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/528, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/521
- Various improvements to in CloudFourmation template to support publication in AWS Marketplace https://gitlab.com/postgres-ai/database-lab/-/merge_requests/526
- Fix how max query time is determined from pg_stat_statements for Postgres 13+ (DB Migration Checker) - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/530
- Update link to CLI installation - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/531
- Use default Postgres configuration in all Postgres containers managed by DLE - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/518
How to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Provide feedback, contribute
Any feedback will be greatly appreciated:
- Twitter: @Database_Lab
- Database Lab Community Slack: https://slack.postgres.ai/
If you want to help the project grow, consider various ways of contributing:
- Star the GitHub repo: https://github.com/postgres-ai/database-lab-engine
- Spread the word using Twitter (please mention @Database_Lab!) or your social network of choice
- Consider translating the README.md to one or several languages you know
- Consider implementing one of the features that will make DLE better: see CONTRIBUTING.md and the list of "good first issues" on GitLab.
Diff between versions v3.1.0 and v3.1.1: v3.1.0...v3.1.1
Database Lab Engine v3.1.0
Database Lab Engine v3.1.0
DLE 3.1 has many improvements and several new features, such as native support of pgBackRest and the ability to configure the timezone when using CLI.
In this release, the development team has focused on the Database Lab Engine community, making it easier to get help or contribute. The team greets all new contributors: @Nikolay Devxx, @asotolongo, @Tanya301, @denis-boost, @pietervincken, @ane4ka
Community news
- 🌠 DLE repository on GitHub now has 1,100+ stars; many thanks to everyone who supports the project in any way
- 💥 Pieter Vincken has published a blog post describing their experience of using DLE: "Testing with production data made easy"
- 📈 The Twitter account has reached 400 followers: @Database_Lab
- 🎉 DLE now has 15 contributors. More contributions are welcome! See "good first issues"
- 🥇 Please consider various ways to contribute – read CONTRIBUTING.md
The README.md has been significantly improved. It now has four translations, with more coming in the future. Many thanks to our README translation contributors! It helps engineers worldwide understand DLE concepts and improve the development experience by building powerful dev&test environments.
- Ukrainian / Українська (🙏 @denis-boost)
- Spanish / Español (🙏 @asotolongo)
- Russian / Pусский (🙏 @Tanya301)
- German / Deutsch (🙏 @ane4ka)
Migration notes
postgresai/dblab-server:3.1.0
) uses ZFS 2.1.2. If you need to upgrade DLE from a previous version that worked with a ZFS pool 0.8.x (default for previous DLE versions), consider the following options:
- Option 1: upgrade your system to use ZFS 2.1, optionally upgrade your pool (
zpool upgrade dblab_pool
), and then upgrade DLE to use the default imagepostgresai/dblab-server:3.1.0
- Option 2: postpone the ZFS upgrade, stay on ZFS 0.8, and upgrade DLE to version 3.1 using a special image,
postgresai/dblab-server:3.1.0-zfs0.8
Changelog
New features
- Native support for pgBackRest as a tool to restore data from archives (physical mode, including continuously updated state) https://gitlab.com/postgres-ai/database-lab/-/merge_requests/442, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/454 (:bow: contribution by @Nikolay Devxx)
- Allow configuring timezone in DLE CLI configuration to improve the experience of using DLE in CI/CD pipelines https://gitlab.com/postgres-ai/database-lab/-/merge_requests/486, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/502
- Automatically clean up failed clones with their sessions (for example, errors during the Postgres clone configuration – in the past, in such cases, DLE lost control over such clones and they could not be removed, even using an API request) https://gitlab.com/postgres-ai/database-lab/-/merge_requests/405, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/404
- For logical mode, during dump/restore, DLE now cleans up only those files or directories that are specified to be restored https://gitlab.com/postgres-ai/database-lab/-/merge_requests/410
- Improved support of multiple DLEs running on the same machine: Take the pool name into account when determining the clone existence https://gitlab.com/postgres-ai/database-lab/-/merge_requests/471
- Rework README, add translations https://gitlab.com/postgres-ai/database-lab/-/merge_requests/411, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/423, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/467, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/478, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/481, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/497
- CONTRIBUTING.md: Describe components and Makefile targets https://gitlab.com/postgres-ai/database-lab/-/merge_requests/413
Improvements and fixes
- Reset the snapshot state atomically to avoid the moments in time when the DLE does not have a snapshot https://gitlab.com/postgres-ai/database-lab/-/merge_requests/406
- Skip connection if the DLE container is already connected to an internal network. This solves the problem when in some cases, DLE cannot start because the DLE container is already connected to the bridge network https://gitlab.com/postgres-ai/database-lab/-/merge_requests/416
- Optimize snapshot list. This has to improve the performance of
/snapshot
API calls which could take many seconds in the past https://gitlab.com/postgres-ai/database-lab/-/merge_requests/426 - ZFS version 2.1.2 inside DLE container (optional, 0.8 remains the default in DLE 3.x) https://gitlab.com/postgres-ai/database-lab/-/merge_requests/343, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/477
- Do not specify the logerrors extension by default https://gitlab.com/postgres-ai/database-lab/-/merge_requests/485
- Replace the "standby_mode" parameter with "recovery_target" in the DLE configuration for the promotion stage https://gitlab.com/postgres-ai/database-lab/-/merge_requests/484
- Use the correct path to the data directory for the promote instance https://gitlab.com/postgres-ai/database-lab/-/merge_requests/503
- Fix wrong server version in the DLE response https://gitlab.com/postgres-ai/database-lab/-/merge_requests/508
- Avoid losing the snapshot list on configuration reload https://gitlab.com/postgres-ai/database-lab/-/merge_requests/506
- In logical mode, dump only databases to which user have access https://gitlab.com/postgres-ai/database-lab/-/merge_requests/501
- UI: Add clone port to clones list https://gitlab.com/postgres-ai/database-lab/-/merge_requests/447
- UI: Correct status detection https://gitlab.com/postgres-ai/database-lab/-/merge_requests/449
- UI: Add the total size of clones to instance page header https://gitlab.com/postgres-ai/database-lab/-/merge_requests/450
- UI: Fetch failed on connection refused calls https://gitlab.com/postgres-ai/database-lab/-/merge_requests/459
- UI: Adjust DLE stats labels https://gitlab.com/postgres-ai/database-lab/-/merge_requests/461
- UI (Platform only): Fix command page scroll https://gitlab.com/postgres-ai/database-lab/-/commit/992b0ecc4f43a578eeadd3818d8824e470b78009
- UI (Platform only): Fix markdown parser https://gitlab.com/postgres-ai/database-lab/-/merge_requests/496
Internal
- Set up GPG to have signed commits in GitPod https://gitlab.com/postgres-ai/database-lab/-/merge_requests/407
- Use GitHub CodeQL Analysis https://gitlab.com/postgres-ai/database-lab/-/merge_requests/409
- Define rules to reduce the running time of integration tests - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/414
- Upgrade the default Docker image - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/412, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/476
- Add .whitesource configuration file - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/420
- Rework LICENSE files - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/429
- Don't count .js in repo stats - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/430
- Rework CI - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/425, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/431, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/453, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/457, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/493, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/495, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/499
- Describe how
dataStateAt
is determined - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/356 - Isolate Database Lab Engine sources - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/443
- Update dependencies for security reasons - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/439, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/445, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/456, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/458, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/466, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/451, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/452, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/469, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/468, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/472, https://gitlab.com/postgres-ai/database-lab/-/merge_requests/504
- Make UI source code publicly available https://gitlab.com/postgres-ai/database-lab/-/merge_requests/421
- Update Go to 1.18 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/498
How to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and...
Database Lab Engine v3.0.3
DLE 3.0.3 is a backward-compatibility update to ensure that the DLE image is compatible with existing datasets (ZFS 0.8.*). In DLE 3.0.2, in the Docker image ZFS version was prematurely switched to 2.*. The release 3.0.3 mitigates it. There are plans to switch to ZFS 2.* in the future major versions of DLE.
The development team asks everyone to participate in testing and report any bugs or improvement ideas in the DLE repository on GitLab or GitHub:
Changelog
Improvements and fixes
- Explicitly set ZFS version in DLE – ZFS 0.8.4: https://gitlab.com/postgres-ai/database-lab/-/merge_requests/492
Internal
- Update the DLE version in the SwaggerUI spec: https://gitlab.com/postgres-ai/database-lab/-/commit/a59e5845dca510a2a3b929ba1834ce038e5a6936
Migration notes
Make sure the DLE running command contains the environment variable --env DOCKER_API_VERSION="1.41"
.
The maximum supported Docker API version is 1.41, but the client version (1.42) is too new.
How to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Request for feedback
Any feedback would be greatly appreciated:
- Database Lab Community Slack: https://slack.postgres.ai/
- DLE issue tracker: https://gitlab.com/postgres-ai/database-lab/-/issues
Compare versions
Diff between versions v3.0.2 and v3.0.3: https://gitlab.com/postgres-ai/database-lab/-/compare/v3.0.2...v3.0.3
Database Lab Engine v3.0.2
DLE 3.0.2 is a security update, mitigating multiple CVEs reported by various security inspection tools. There are no known critical vulnerabilities that are fixed or are still unfixed in this release but upgrading at the earliest convenience is recommended.
The development team asks everyone to participate in testing and report any bugs or improvement ideas in the DLE repository on GitLab or GitHub:
Changelog
Internal
- Fix old and indirect dependencies to avoid scanner warnings - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/466
- Upgrade engine libs to mitigate multiple CVEs - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/458
- Mitigate CVE scanner warnings related to SwaggerUI - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/472
- Update base Docker images to 20.10 - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/476
How to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Request for feedback
Any feedback would be greatly appreciated:
- Database Lab Community Slack: https://slack.postgres.ai/
- DLE issue tracker: https://gitlab.com/postgres-ai/database-lab/-/issues
Compare versions
Diff between versions v3.0.1 and v3.0.2: https://gitlab.com/postgres-ai/database-lab/-/compare/v3.0.1...v3.0.2
Database Lab Engine v3.0.1
The development team asks everyone to participate in testing and report any bugs or improvement ideas in the DLE repository: https://gitlab.com/postgres-ai/database-lab.
Changelog
Improvements and fixes
- Clean up only files or directories that are specified to be restored - https://gitlab.com/postgres-ai/database-lab/-/issues/313
- Remove clone with the FATAL status during idleness check - https://gitlab.com/postgres-ai/database-lab/-/issues/316
- Reload snapshots state atomically - https://gitlab.com/postgres-ai/database-lab/-/issues/317
- Do not connect the DLE container to an internal network if it is already connected - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/416
- Rework README, add CODE_OF_CONDUCT.md, CONTRIBUTING.md, SECURITY.md - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/411, https://gitlab.com/postgres-ai/database-lab/-/issues/319
Internal
- Set up GPG to have signed commits in Gitpod - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/407
- Try GitHub CodeQL Analysis - https://gitlab.com/postgres-ai/database-lab/-/merge_requests/409
How to get help
To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.
Request for feedback
Any feedback would be greatly appreciated:
- Database Lab Community Slack: https://slack.postgres.ai/
- DLE issue tracker: https://gitlab.com/postgres-ai/database-lab/-/issues
Compare versions
Diff between versions v3.0.0 and v3.0.1: https://gitlab.com/postgres-ai/database-lab/-/compare/v3.0.0...v3.0.1