[Maria-discuss] In memory DB, how to in MariaDB?
I have seen the next blog post which had a video at 2014: https://blogs.oracle.com/pcoe/post/the-future-of-the-database-begins The demo video was removed from YouTube for some reason but I had a copy of it at: https://cloud1.ngtech.co.il/static/oracle/Oracle%20Database%20In-Memory%20Po wering%20the%20Real-Time%20Enterprise%20.mp4 I am not a MariaDB or MySQL or DB in general expert and was wondering if there is something these days in MariaDB that can be used in comparison to what Oracle DB demo shows? I have used LevelDB as a backend in a customized DB and it worked very very fast and it is great. With batches it works even faster but in general it works faster then standard MariaDB setups I have implemented. So what are the options? I have millions of values and I would like to be able to insert and select as fast as possible on a single DB Server. Currently I am using a Caching tier with a specific TTL to lower the stress off the DB, but, is I possible to make the DB Faster like maybe with some writeback (leaving aside fault tolerance for speed). Thanks, Eliezer ---- Eliezer Croitoru NgTech, Tech Support Mobile: +972-5-28704261 Email: mailto:ngtech1ltd@gmail.com ngtech1ltd@gmail.com
If your buffer pool is big enough to store the entire data set, then
there is no need to use any kind of in-memory database engine, the
buffer pool will end up caching everything (once it is warmed up).
For faster writes if you don't particularly care about ACID
compliance, you can disable InnoDB transaction log flushing by setting
innodb_flush_log_at_trx_commit=0. You will lose transaction if there
is a dirty shutdown, but it will largely bypass any disk write I/O
bottleneck.
Over and above that, the usual performance tuning applies - make sure
your queries are well written and well indexed. In most cases that
makes far more difference than keeping all of your data in memory.
On Sun, May 22, 2022 at 3:01 PM Eliezer Croitoru
I have seen the next blog post which had a video at 2014:
https://blogs.oracle.com/pcoe/post/the-future-of-the-database-begins
The demo video was removed from YouTube for some reason but I had a copy of it at:
https://cloud1.ngtech.co.il/static/oracle/Oracle%20Database%20In-Memory%20Po...
I am not a MariaDB or MySQL or DB in general expert and was wondering if there is something these days in MariaDB that can be used in comparison to what Oracle DB demo shows?
I have used LevelDB as a backend in a customized DB and it worked very very fast and it is great. With batches it works even faster but in general it works faster then standard MariaDB setups I have implemented.
So what are the options?
I have millions of values and I would like to be able to insert and select as fast as possible on a single DB Server.
Currently I am using a Caching tier with a specific TTL to lower the stress off the DB, but, is I possible to make the DB Faster like maybe with some writeback (leaving aside fault tolerance for speed).
Thanks,
Eliezer
----
Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1ltd@gmail.com
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Thanks! I assumed so and was wondering how exactly Oracle claimed that the in-memory DB makes the speed fast compared between in memory to on disk and without indexes. What the demo shows is that a DB in memory can be fast and so fast that you wouldn't need indexes if and only if I understood right. Can you abd others please try to verify my assumption and clear out my doubts? Thanks, Eliezer בתאריך יום א׳, 22 במאי 2022, 15:07, מאת Gordan Bobic < gordan.bobic@gmail.com>:
If your buffer pool is big enough to store the entire data set, then there is no need to use any kind of in-memory database engine, the buffer pool will end up caching everything (once it is warmed up). For faster writes if you don't particularly care about ACID compliance, you can disable InnoDB transaction log flushing by setting innodb_flush_log_at_trx_commit=0. You will lose transaction if there is a dirty shutdown, but it will largely bypass any disk write I/O bottleneck.
Over and above that, the usual performance tuning applies - make sure your queries are well written and well indexed. In most cases that makes far more difference than keeping all of your data in memory.
On Sun, May 22, 2022 at 3:01 PM Eliezer Croitoru
wrote: I have seen the next blog post which had a video at 2014:
https://blogs.oracle.com/pcoe/post/the-future-of-the-database-begins
The demo video was removed from YouTube for some reason but I had a copy
of it at:
https://cloud1.ngtech.co.il/static/oracle/Oracle%20Database%20In-Memory%20Po...
I am not a MariaDB or MySQL or DB in general expert and was wondering if
there is something these days in MariaDB that can be used
in comparison to what Oracle DB demo shows?
I have used LevelDB as a backend in a customized DB and it worked very very fast and it is great. With batches it works even faster but in general it works faster then standard MariaDB setups I have implemented.
So what are the options?
I have millions of values and I would like to be able to insert and select as fast as possible on a single DB Server.
Currently I am using a Caching tier with a specific TTL to lower the stress off the DB, but, is I possible to make the DB Faster like maybe with some writeback (leaving aside fault tolerance for speed).
Thanks,
Eliezer
----
Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1ltd@gmail.com
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
It's called "marketing". They will pick a specific case and then
pretend that somehow it can solve generalised O(n) problems in
O(log(n)) time.
On Sun, May 22, 2022 at 3:14 PM NgTech LTD
Thanks! I assumed so and was wondering how exactly Oracle claimed that the in-memory DB makes the speed fast compared between in memory to on disk and without indexes. What the demo shows is that a DB in memory can be fast and so fast that you wouldn't need indexes if and only if I understood right.
Can you abd others please try to verify my assumption and clear out my doubts?
Thanks, Eliezer
בתאריך יום א׳, 22 במאי 2022, 15:07, מאת Gordan Bobic
: If your buffer pool is big enough to store the entire data set, then there is no need to use any kind of in-memory database engine, the buffer pool will end up caching everything (once it is warmed up). For faster writes if you don't particularly care about ACID compliance, you can disable InnoDB transaction log flushing by setting innodb_flush_log_at_trx_commit=0. You will lose transaction if there is a dirty shutdown, but it will largely bypass any disk write I/O bottleneck.
Over and above that, the usual performance tuning applies - make sure your queries are well written and well indexed. In most cases that makes far more difference than keeping all of your data in memory.
On Sun, May 22, 2022 at 3:01 PM Eliezer Croitoru
wrote: I have seen the next blog post which had a video at 2014:
https://blogs.oracle.com/pcoe/post/the-future-of-the-database-begins
The demo video was removed from YouTube for some reason but I had a copy of it at:
https://cloud1.ngtech.co.il/static/oracle/Oracle%20Database%20In-Memory%20Po...
I am not a MariaDB or MySQL or DB in general expert and was wondering if there is something these days in MariaDB that can be used in comparison to what Oracle DB demo shows?
I have used LevelDB as a backend in a customized DB and it worked very very fast and it is great. With batches it works even faster but in general it works faster then standard MariaDB setups I have implemented.
So what are the options?
I have millions of values and I would like to be able to insert and select as fast as possible on a single DB Server.
Currently I am using a Caching tier with a specific TTL to lower the stress off the DB, but, is I possible to make the DB Faster like maybe with some writeback (leaving aside fault tolerance for speed).
Thanks,
Eliezer
----
Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1ltd@gmail.com
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
Thanks Gordan,(not sure if this is your first name...)
It's very nice to get a honest answer about a technical subject.
My usual place is Squid-Cache and there we used to plain text ... but
marketing is a whole new world for me.
All The Bests,
Eliezer
* I'm here in the background
----
Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1ltd@gmail.com
-----Original Message-----
From: Gordan Bobic
Thanks! I assumed so and was wondering how exactly Oracle claimed that the in-memory DB makes the speed fast compared between in memory to on disk and without indexes. What the demo shows is that a DB in memory can be fast and so fast that you wouldn't need indexes if and only if I understood right.
Can you abd others please try to verify my assumption and clear out my doubts?
Thanks, Eliezer
בתאריך יום א׳, 22 במאי 2022, 15:07, מאת Gordan Bobic
: If your buffer pool is big enough to store the entire data set, then there is no need to use any kind of in-memory database engine, the buffer pool will end up caching everything (once it is warmed up). For faster writes if you don't particularly care about ACID compliance, you can disable InnoDB transaction log flushing by setting innodb_flush_log_at_trx_commit=0. You will lose transaction if there is a dirty shutdown, but it will largely bypass any disk write I/O bottleneck.
Over and above that, the usual performance tuning applies - make sure your queries are well written and well indexed. In most cases that makes far more difference than keeping all of your data in memory.
On Sun, May 22, 2022 at 3:01 PM Eliezer Croitoru
wrote: I have seen the next blog post which had a video at 2014:
https://blogs.oracle.com/pcoe/post/the-future-of-the-database-begins
The demo video was removed from YouTube for some reason but I had a copy of it at:
https://cloud1.ngtech.co.il/static/oracle/Oracle%20Database%20In-Memory%20Po...
I am not a MariaDB or MySQL or DB in general expert and was wondering if there is something these days in MariaDB that can be used in comparison to what Oracle DB demo shows?
I have used LevelDB as a backend in a customized DB and it worked very very fast and it is great. With batches it works even faster but in general it works faster then standard MariaDB setups I have implemented.
So what are the options?
I have millions of values and I would like to be able to insert and select as fast as possible on a single DB Server.
Currently I am using a Caching tier with a specific TTL to lower the stress off the DB, but, is I possible to make the DB Faster like maybe with some writeback (leaving aside fault tolerance for speed).
Thanks,
Eliezer
----
Eliezer Croitoru
NgTech, Tech Support
Mobile: +972-5-28704261
Email: ngtech1ltd@gmail.com
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp
participants (3)
-
Eliezer Croitoru
-
Gordan Bobic
-
NgTech LTD