Daniel, all I would like to avoid these discussions because they take me a lot of time. Anyway I have to participate. The author of the original patch Fredrik Nylander called the new key cache structure 'segmented key cache'. When I started working on the submitted patch I immediately realized that the author actually used partitioning technique. I remind you that the original patch just replaced the regular key cache structure with the structure of the partitioned (segmented key cache) and this new structure still was called KEY_CACHE in his code. The number of partitions/segments was the compilation parameter of the server (the same for all key caches). So the original patch just added a new property to the existing key caches. I added a possibility to partition any key cache. I really did not see any valid arguments not to call this new type of key caches 'partitioned key cache', or to call the new feature 'key cache partitioning'. I don't mind if we call the paragraph in the manual Segmented Key Cache in acknowledgment of the original patch submitted by Fredrik Nylander. Yet, I would insist that Segmented Key Cache is the result of partitioning of a regular key cache. Accordingly it consists of key cache partitions, not segments. Using widely the term Segmented Key Cache ultimately would require massive renaming in the patch and I would like not to do it because the term 'segment' would be really misleading for the implementation. When resolving this argument we could ask: 1. the author of the original patch 2. our community members (not only PeterZ!) 3. our developers what term they prefer 'Segmented Key Cache' or 'Partitioned Key Cache'. To be quite honest I also should mention that MichaelR from Oracle/MySQL introduced in 5.5 a possibility for MySQL partitions in MyISAM to use different key caches for different partitions. Before that all partitions could use only default key caches. This feature is 100% orthogonal to the discussed key cache partitioning. Daniel Bartholomew wrote:
Hello everyone,
I'm working on the documentation for the Segmented Key Cache, one of the new features in MariaDB 5.2.
The documentation page will have "about", "syntax", and "example" sections.
I've based the "About" section on the High-Level Description from the worklog ( http://askmonty.org/worklog/Server-Sprint/?tid=85 ) and from the description in the mysys/mf_keycache.c file in the source.
Here is my first draft. I've changed the wording a bit to help it flow better and I've changed "partition" to "segment" in keeping with the official name.
----------------------------------------------------------------------
== About Segmented Key Cache ==
A segmented key cache is a collection of structures for regular MyISAM key caches called key cache segments. Segmented key caches mitigate one of the major problem of the simple key cache: thread contention for key cache lock (mutex). With regular key caches, every call of a key cache interface function must acquire this lock. Threads compete for this lock even when the file and pages they have acquired shared locks for are in the key cache buffers.
When working with a segmented key cache any key cache interface function that needs only one page has to acquire the key cache lock only for the segment the page is assigned to. This makes the chances for threads not having to compete for the same key cache lock better.
Any page from a file can be placed into a buffer of only one segment. The number of the segment is calculated from the file number and the position of the page in the file, and it's always the same for the page. Pages are evenly distributed among segments.
The idea and the original code of the segmented key cache was provided by Fredrik Nylander from Stardoll.com. The code was extensively reworked, improved, and eventually merged into MariaDB by Igor Babaev from Monty Program. ----------------------------------------------------------------------
If there are any factual errors with the above, let me know. Improvements and suggestions are also welcome.
One thing I'm not happy with is this sentence: "Threads compete for this lock even when the file and pages they have acquired shared locks for are in the key cache buffers."
The original sentence from the worklog reads: So threads compete for this lock even in the case when they have acquired shared locks for the file and pages they want read from are in the key cache buffers.
Ok ,I will help with the reading of the original statement adding missing words: So threads compete for this lock even in the case when they have acquired shared locks for the file and the pages they want to read from are in the key cache buffers.
... which is very confusing to me. If anyone has a better or more accurate rewording, please send it my way.
Also, there's a note in the source code (lines 5018-5023 of mysys/mf_keycache.c) which states: Unfortunately if we use a partitioned key cache with N partitions for B-tree indexes we can't say that the chances becomes N times less. The fact is that any index lookup operation requires reading from the root page that, for any index, is always ascribed to the same partition. To resolve this problem we should have employed more sophisticated mechanisms of working with root pages.
Do any of you have any opinions on whether or not this should be mentioned somewhere in the documentation? The only issue I have with mentioning it is that if I do, the first question that comes to my mind is: "If it is not N-times less, how much less is it?" I don't have an answer to that question (and math isn't one of my strengths).
I don't think it should be mentioned in the manual (at least in the first variant of it). Regards, Igor.
Next up will be the "Syntax" section.
Thanks.