[Maria-discuss] Clarification on Memory utilization of Mariadb
Hi Team, Need clarification on Memory utilization of Mariadb. I have a java application with MariaDB that runs on the same Rhel 7 server with physical memory(RAM) allocated as 12GB where I would see the normal utilization at the start as below. *sys> free -h* * total used free shared buff/cache available* *Mem: 11G 7.8G 1.6G 9.0M 2.2G 3.5G* *Swap: 0B 0B 0B* Over a period of time say >6month, I could see the RAM utilization is almost full, and available space is very less but still, the application continues to work fine without any issues. *sys> free -h* * total used free shared buff/cache available* *Mem: 11G 11G 134M 17M 411M 240M* *Swap: 0B 0B 0B* I don't see any OOM issue or high usage of RSS at the java application level but I could see MariaDB occupies huge memory.... apart from that overall application runs fine. To ensure this we observed memory utilization at the large-scale application servers where we deployed MariaDB and Java applications into different servers with 128GB each and could observe that the system with MariaDB uses full RAM size whereas the Java application consumption looks stable. *Server with Java application* sys> free total used free shared buff/cache available Mem: 128755 125171 606 118 2977 2694 Swap: 65535 9746 55789 *Server with MariaDB* sys> free total used free shared buff/cache available Mem: 128755 125171 606 118 2977 2694 Swap: 65535 9746 55789 *MariaDb conf:* innodb_buffer_pool_size=102G innodb_buffer_pool_instances=8 innodb_io_capacity=3000 innodb_log_buffer_size=8M innodb_log_file_size=2047MB innodb_log_files_in_group=2 Is this expected that MariaDB consumes overall memory and manages within RAM size.... regardless of buffer size configured? Do we need to optimize any configuration to control the MariaDB consumption apart from buffer pool size? Thanks and Regards, Ragul
The innodb_buffer_pool_size is just one memory setting of many. MariaDB will allocate additional memory for other things. Here's a good place to start… https://mariadb.com/kb/en/mariadb-memory-allocation/ From: Maria-discuss <maria-discuss-bounces+rhys.campbell=swisscom.com@lists.launchpad.net> On Behalf Of ragul rangarajan Sent: 09 November 2021 15:34 To: maria-discuss@lists.launchpad.net Subject: [Maria-discuss] Clarification on Memory utilization of Mariadb Hi Team, Need clarification on Memory utilization of Mariadb. I have a java application with MariaDB that runs on the same Rhel 7 server with physical memory(RAM) allocated as 12GB where I would see the normal utilization at the start as below. sys> free -h total used free shared buff/cache available Mem: 11G 7.8G 1.6G 9.0M 2.2G 3.5G Swap: 0B 0B 0B Over a period of time say >6month, I could see the RAM utilization is almost full, and available space is very less but still, the application continues to work fine without any issues. sys> free -h total used free shared buff/cache available Mem: 11G 11G 134M 17M 411M 240M Swap: 0B 0B 0B I don't see any OOM issue or high usage of RSS at the java application level but I could see MariaDB occupies huge memory.... apart from that overall application runs fine. To ensure this we observed memory utilization at the large-scale application servers where we deployed MariaDB and Java applications into different servers with 128GB each and could observe that the system with MariaDB uses full RAM size whereas the Java application consumption looks stable. Server with Java application sys> free total used free shared buff/cache available Mem: 128755 125171 606 118 2977 2694 Swap: 65535 9746 55789 Server with MariaDB sys> free total used free shared buff/cache available Mem: 128755 125171 606 118 2977 2694 Swap: 65535 9746 55789 MariaDb conf: innodb_buffer_pool_size=102G innodb_buffer_pool_instances=8 innodb_io_capacity=3000 innodb_log_buffer_size=8M innodb_log_file_size=2047MB innodb_log_files_in_group=2 Is this expected that MariaDB consumes overall memory and manages within RAM size.... regardless of buffer size configured? Do we need to optimize any configuration to control the MariaDB consumption apart from buffer pool size? Thanks and Regards, Ragul
Hi Ragul, Which version of MariaDB is this? The memory utilisation you show for 'Server with MariaDB' is identical to 'Server with Java application', so I think you pasted in the wrong 'free' output for one of them? Cheers, Karl On Tue, 9 Nov 2021 at 16:13, ragul rangarajan <ragulrangarajan@gmail.com> wrote:
Hi Team,
Need clarification on Memory utilization of Mariadb.
I have a java application with MariaDB that runs on the same Rhel 7 server with physical memory(RAM) allocated as 12GB where I would see the normal utilization at the start as below.
*sys> free -h*
* total used free shared buff/cache available*
*Mem: 11G 7.8G 1.6G 9.0M 2.2G 3.5G*
*Swap: 0B 0B 0B*
Over a period of time say >6month, I could see the RAM utilization is almost full, and available space is very less but still, the application continues to work fine without any issues.
*sys> free -h*
* total used free shared buff/cache available*
*Mem: 11G 11G 134M 17M 411M 240M*
*Swap: 0B 0B 0B*
I don't see any OOM issue or high usage of RSS at the java application level but I could see MariaDB occupies huge memory.... apart from that overall application runs fine.
To ensure this we observed memory utilization at the large-scale application servers where we deployed MariaDB and Java applications into different servers with 128GB each and could observe that the system with MariaDB uses full RAM size whereas the Java application consumption looks stable.
*Server with Java application*
sys> free
total used free shared buff/cache available
Mem: 128755 125171 606 118 2977 2694
Swap: 65535 9746 55789
*Server with MariaDB*
sys> free
total used free shared buff/cache available
Mem: 128755 125171 606 118 2977 2694
Swap: 65535 9746 55789
*MariaDb conf:*
innodb_buffer_pool_size=102G innodb_buffer_pool_instances=8 innodb_io_capacity=3000 innodb_log_buffer_size=8M innodb_log_file_size=2047MB innodb_log_files_in_group=2
Is this expected that MariaDB consumes overall memory and manages within RAM size.... regardless of buffer size configured?
Do we need to optimize any configuration to control the MariaDB consumption apart from buffer pool size?
Thanks and Regards,
Ragul _______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Thanks Karl, Yes wrongly pasted...these are below outputs. *MariaDB Version:* 10.2.39 *Server with Java application * sys> free total used free shared buff/cache available Mem: 128755 37957 632 944 90165 89082 Swap: 4095 28 4067 *Server with MariaDB * sys> free total used free shared buff/cache available Mem: 128755 125171 606 118 2977 2694 Swap: 65535 9746 55789 Regards, Ragul On Wed, Nov 10, 2021 at 3:43 PM Karl Levik <karl.levik@gmail.com> wrote:
Hi Ragul,
Which version of MariaDB is this?
The memory utilisation you show for 'Server with MariaDB' is identical to 'Server with Java application', so I think you pasted in the wrong 'free' output for one of them?
Cheers, Karl
On Tue, 9 Nov 2021 at 16:13, ragul rangarajan <ragulrangarajan@gmail.com> wrote:
Hi Team,
Need clarification on Memory utilization of Mariadb.
I have a java application with MariaDB that runs on the same Rhel 7 server with physical memory(RAM) allocated as 12GB where I would see the normal utilization at the start as below.
*sys> free -h*
* total used free shared buff/cache available*
*Mem: 11G 7.8G 1.6G 9.0M 2.2G 3.5G*
*Swap: 0B 0B 0B*
Over a period of time say >6month, I could see the RAM utilization is almost full, and available space is very less but still, the application continues to work fine without any issues.
*sys> free -h*
* total used free shared buff/cache available*
*Mem: 11G 11G 134M 17M 411M 240M*
*Swap: 0B 0B 0B*
I don't see any OOM issue or high usage of RSS at the java application level but I could see MariaDB occupies huge memory.... apart from that overall application runs fine.
To ensure this we observed memory utilization at the large-scale application servers where we deployed MariaDB and Java applications into different servers with 128GB each and could observe that the system with MariaDB uses full RAM size whereas the Java application consumption looks stable.
*Server with Java application*
sys> free
total used free shared buff/cache available
Mem: 128755 125171 606 118 2977 2694
Swap: 65535 9746 55789
*Server with MariaDB*
sys> free
total used free shared buff/cache available
Mem: 128755 125171 606 118 2977 2694
Swap: 65535 9746 55789
*MariaDb conf:*
innodb_buffer_pool_size=102G innodb_buffer_pool_instances=8 innodb_io_capacity=3000 innodb_log_buffer_size=8M innodb_log_file_size=2047MB innodb_log_files_in_group=2
Is this expected that MariaDB consumes overall memory and manages within RAM size.... regardless of buffer size configured?
Do we need to optimize any configuration to control the MariaDB consumption apart from buffer pool size?
Thanks and Regards,
Ragul _______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
participants (3)
-
Karl Levik -
ragul rangarajan -
Rhys.Campbell＠swisscom.com