http://www.codeproject.com/Articles/1107279/Writing-a-MySQL-storage-engine-f... is also a recent blog. Was written with MySQL rather than MariaDB in mind however there are probably lots of common concepts. On 19/07/16 20:00, Sergei Golubchik wrote:

Hi, Jonathan!

On Jul 18, Jonathan Ellithorpe wrote:

...

Hi All,

I searched around but couldn't find documentation / tutorials on creating a custom storage backend for MariaDB. Could anyone point me in the right direction?

Storage engine examples in the source tree. "MySQL 5.1 Plugin Development" book. https://mariadb.com/kb/en/mariadb/storage-engines-storage-engine-development...

Regards, Sergei Chief Architect MariaDB and security@mariadb.org

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

Typo: It does*n't* support indexes. On 07/19/2016 09:48 PM, Kim Ebert wrote:

I struggled to find good examples / blogs of creating a storage engine. I used the MySQL references the most for developing a storage engine.

https://dev.mysql.com/doc/internals/en/ https://dev.mysql.com/doc/internals/en/custom-engine.html

Here is a Python Engine I've worked on. It does support indexes and it is read only right now, but hey it is progress. My hope and goal was to make it so that I could prototype engines in Python before writing them in C. I may have some time in the next month or two to take it further, but that is still in the air.

https://github.com/reflectivedevelopment/mariadb-plugin-python-engine

-Kim

On 07/19/2016 07:17 PM, Daniel Black wrote:

...

http://www.codeproject.com/Articles/1107279/Writing-a-MySQL-storage-engine-f...

is also a recent blog. Was written with MySQL rather than MariaDB in mind however there are probably lots of common concepts.

On 19/07/16 20:00, Sergei Golubchik wrote:

...

Hi, Jonathan!

On Jul 18, Jonathan Ellithorpe wrote:

...

Hi All,

I searched around but couldn't find documentation / tutorials on creating a custom storage backend for MariaDB. Could anyone point me in the right direction? Storage engine examples in the source tree. "MySQL 5.1 Plugin Development" book. https://mariadb.com/kb/en/mariadb/storage-engines-storage-engine-development...

Regards, Sergei Chief Architect MariaDB and security@mariadb.org

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

Follow-up: I see in the blog pointed to by Daniel mentioned an SE for Cassandra.... so I take it the answer is yes? Does anyone have experience with this / know if anything "extra" must be done to make that work? Jonathan On Tue, Jul 19, 2016 at 9:23 PM Jonathan Ellithorpe wrote:

Thanks for the references! That helps a lot

One other question I have is: can MariaDB work in a distributed fashion? To be more clear, the storage backend I'm developing is distributed, like Cassandra, and I would like to deploy many MariaDB instances that all access the same storage cluster. Can this be done?

Jonathan

On Tue, Jul 19, 2016 at 8:48 PM Kim Ebert wrote:

...

I struggled to find good examples / blogs of creating a storage engine. I used the MySQL references the most for developing a storage engine.

https://dev.mysql.com/doc/internals/en/ https://dev.mysql.com/doc/internals/en/custom-engine.html

Here is a Python Engine I've worked on. It does support indexes and it is read only right now, but hey it is progress. My hope and goal was to make it so that I could prototype engines in Python before writing them in C. I may have some time in the next month or two to take it further, but that is still in the air.

https://github.com/reflectivedevelopment/mariadb-plugin-python-engine

-Kim

On 07/19/2016 07:17 PM, Daniel Black wrote:

...

http://www.codeproject.com/Articles/1107279/Writing-a-MySQL-storage-engine-f...

...

is also a recent blog. Was written with MySQL rather than MariaDB in mind however there are probably lots of common concepts.

On 19/07/16 20:00, Sergei Golubchik wrote:

...

Hi, Jonathan!

On Jul 18, Jonathan Ellithorpe wrote:

...

Hi All,

I searched around but couldn't find documentation / tutorials on creating a custom storage backend for MariaDB. Could anyone point me in the right direction? Storage engine examples in the source tree. "MySQL 5.1 Plugin Development" book.

https://mariadb.com/kb/en/mariadb/storage-engines-storage-engine-development...

...

Regards, Sergei Chief Architect MariaDB and security@mariadb.org

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

I see... so the MariaDB top-half (by "top-half" I mean everything that sits above the storage engine) doesn't do nasty things like caching? If it did, then writes at other MariaDB sites wouldn't show up in reads performed at the site caching the data. If the MariaDB top-half does nothing more than essentially translate user operations / queries into calls on the SE, then in theory nothing special must be done to deploy MariaDB at multiple sites accessing a shared storage engine. Jonathan On Tue, Jul 19, 2016 at 10:20 PM Daniel Black wrote:

On 20/07/16 14:31, Jonathan Ellithorpe wrote:

...

Follow-up: I see in the blog pointed to by Daniel mentioned an SE for Cassandra.... so I take it the answer is yes? Does anyone have experience with this / know if anything "extra" must be done to make that work?

It falls to the responsibility of the storage engine to ensure that the ACID guarantees are implemented.

...

Jonathan

On Tue, Jul 19, 2016 at 9:23 PM Jonathan Ellithorpe mailto:jde@cs.stanford.edu> wrote:

Thanks for the references! That helps a lot

One other question I have is: can MariaDB work in a distributed fashion? To be more clear, the storage backend I'm developing is distributed, like Cassandra, and I would like to deploy many MariaDB instances that all access the same storage cluster. Can this be done?

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

Hehe, yes, I took the shotgun approach. The MySQL lists don't seem to be quite as active, and the MySQL forum moderators seem to take about 1-2 days before they approve my posts. Hmm... thanks for pointing that out about system metadata caching. I'll have the solve the problem of MariaDB thinking something exists (table/index/...) when it doesn't, or thinking that something doesn't exist when it does.... Thanks for your help! Jonathan On Wed, Jul 20, 2016 at 1:21 PM MARK CALLAGHAN wrote:

This discussion is much more interesting than the one on the quiet MySQL lists. I assume you need to worry about the top half caching dictionary information about indexes & tables.

On Wed, Jul 20, 2016 at 12:50 PM, Jonathan Ellithorpe

...

wrote:

...

I see... so the MariaDB top-half (by "top-half" I mean everything that sits above the storage engine) doesn't do nasty things like caching? If it did, then writes at other MariaDB sites wouldn't show up in reads performed at the site caching the data.

If the MariaDB top-half does nothing more than essentially translate user operations / queries into calls on the SE, then in theory nothing special must be done to deploy MariaDB at multiple sites accessing a shared storage engine.

Jonathan

On Tue, Jul 19, 2016 at 10:20 PM Daniel Black wrote:

...

On 20/07/16 14:31, Jonathan Ellithorpe wrote:

...

Follow-up: I see in the blog pointed to by Daniel mentioned an SE for Cassandra.... so I take it the answer is yes? Does anyone have experience with this / know if anything "extra" must be done to make that work?

It falls to the responsibility of the storage engine to ensure that the ACID guarantees are implemented.

...

Jonathan

On Tue, Jul 19, 2016 at 9:23 PM Jonathan Ellithorpe < jde@cs.stanford.edu mailto:jde@cs.stanford.edu> wrote:

Thanks for the references! That helps a lot

One other question I have is: can MariaDB work in a distributed fashion? To be more clear, the storage backend I'm developing is distributed, like Cassandra, and I would like to deploy many MariaDB instances that all access the same storage cluster. Can this be done?

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

_______________________________________________ Mailing list: https://launchpad.net/~maria-discuss Post to : maria-discuss@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-discuss More help : https://help.launchpad.net/ListHelp

-- Mark Callaghan mdcallag@gmail.com