Have you checked whether:

1) Your theoretical max memory usage exceeds ~90% of RAM?

Start with:

SELECT ((@@innodb_buffer_pool_size + @@innodb_log_buffer_size + @@key_buffer_size + @@query_cache_size + @@max_connections * (@@bulk_insert_buffer_size + @@join_buffer_size + @@read_buffer_size + @@read_rnd_buffer_size + @@sort_buffer_size + @@tmp_table_size)) / 1024 / 1024 / 1024) AS max_memory_GB;

2) Are you using /tmp/ for temporary tables and is your /tmp/ mounted as tmpfs?

On Thu, 7 Sep 2023, 14:17 Marco Dickert - evolver group via discuss, <discuss@lists.mariadb.org> wrote:
Hi Sergei,

a quick follow-up on this topic. We still experience the RAM usage problems.
After your suggestion to upgrade MariaDB to the latest 10.5, we switched to the
community repository for debian
(https://dlm.mariadb.com/repo/mariadb-server/...). This is what we did since:

  * upgrade MariaDB to 10.5.21
  * upgrade MariaDB to latest 10.6
  * switch to jemalloc2 library on 10.6
  * upgrade MariaDB to latest 10.11 (system malloc)
  * switch to jemalloc2 library on 10.11

We ran every setup for several days, to check if there is any improvement on the
RAM usage front, but nothing really changed. We noticed that when we use
jemalloc2, the gaps between the OOM-kills is a bit bigger, but the general
problem persists. Also, we couldn't determine a real "trigger" for the
behaviour.

Our application has a very regular usage pattern. There is a peak of activity in
the morning (08-10 a.m.), a steady baseline until about 10 p.m., and very low
activity at night (see the attached graph 1). The OOM incidents happen both in
the morning and in the evening. The second graph shows the RAM usage (RSS) of
the mariadbd process over the last three days. Third, I attached a text file
showing the timestamps of the OOM kills.

Unfortunately we are not quite sure which is the best way to debug this further.
We took a look at our queries, but couldn't determine a problem there. Our
frontend application does not use queries with JOINs. Every request triggers
only a few SELECTs and UPDATEs. Two tables we write to are relatively big (10
and 12 GB).  On average every frontend request triggers about 4-8 queries.

Are there any other metrics we could observe to get a hint to why this happens?
Or has anyone another idea on how to get a grip on what is going on?

Am 2023-08-04 19:13:44 schrieb Sergei Golubchik:
> Hi, Marco,
>
> Please, try the latest 10.5 release, there were few bugs with those
> symptoms fixed.
>
> On Aug 04, Marco Dickert - evolver group via discuss wrote:
> > Hi folks,
> >
> > we experience a RAM issue with MariaDB (version
> > 10.5.19-MariaDB-0+deb11u2-log) on a standard Debian bullseye system.
> > The problem is that over time MariaDB uses more and more RAM, until
> > the kernel's oom-killer terminates it.
> >
> Regards,
> Sergei
> VP of MariaDB Server Engineering
> and security@mariadb.org

--
Kind regards,
Marco Dickert
_______________________________________________
discuss mailing list -- discuss@lists.mariadb.org
To unsubscribe send an email to discuss-leave@lists.mariadb.org