Hello,
JIRA - https://jira.mariadb.org/browse/MDEV-12213
Project Status -
https://docs.google.com/spreadsheets/d/1nE-qFXhwwhF0hpOX3wUlczPjlPZTK_UzBu_P...
WIP Code Branch - https://github.com/theGodlessLakra/server/tree/numa_one
Cumulative work -
https://github.com/theGodlessLakra/server/tree/NUMA_Support
On Sat, Sep 16, 2017 at 4:36 PM,
Hi, Sumit!
On Sep 16, Sumit Lakra wrote:
The project 'NUMA Support in MariaDB' is still incomplete and like I mentioned in my mails to my mentors, I am still willing to work on it and see it through to the end.
Great, thanks!
Most of the minor tasks along with some important tasks are already complete. The important part that remains is to work on the task-queue interface between SQL and InnoDB. I was unable to implement this for the NUMA case and I won't be able to complete it without some help. So, do you think you can connect me with someone from the MariaDB/InnoDB/MySQL team who is an expert of this interface? I will need someone who has a very good understanding of the present structure to help me come up with a way to implement this for the NUMA case. If you do find someone who can be of help, please see to it that since GSoC is over (and otherwise as well), it shouldn't be a Student-Mentor kind of thing, but more like two or more developers working together trying to come up with a solution to this problem.
Why wouldn't you ask the question on the maria-developers@ mailing list? I'll try to make sure that it'll be answered.
The Buffer Pool has been divided into as many instances as there are NUMA nodes. All user SQL threads accessing the same Database has been bound to the same NUMA node. [1] Now, they must be restricted to use the Buffer Pool Instance of the same NUMA node for all purpose henceforth. For example, if a user SQL thread accessing Database A is running on NUMA node 1, then all the bpages related to Database A must be loaded into the Buffer Pool Instance existing on that NUMA node, let's call that Buffer Pool Instance 1. [2] The InnoDB read/write io threads are in multiples of the number of available NUMA nodes. They have been bound to respective NUMA nodes as well. After the above task [1] is implemented, these background threads will need to be restricted to pick tasks from the queues associated with the same NUMA node as they are bound to run on. This will mean that the threads will now only be executing tasks that are associated with the Buffer Pool Instance on the same NUMA node. Implementing [1] and [2] above will complete the integral part of the project 'NUMA Support in MariaDB'. DEBUG and test cases will need to be written for the same afterwards. However, one need not know the details of NUMA Implementation in order to answer this. Just suppose that we had to re-structure the SQL-InnoDB interface such that all background reading/writing tasks associated with a particular Database is put on one queue, and the background threads are restricted to use only one of these queues. How to implement this ?
However, if you are unable to connect me with someone who has expertise in the handler interface, I am afraid I won't be able to complete this project on my own. In this case, would you like me to create a pull request of the work done so far ( https://github.com/theGodlessLakra/server/tree/NUMA_Support). Although NUMA Support is incomplete, merging this code to the MariaDB code base will allow other developers to continue working on this.
Is it sufficiently complete to be useful as is?
Regards, Sergei Chief Architect MariaDB and security@mariadb.org
Thanks, Sumit Lakra