Hi Kristian,

Thanks for the explanation. Let me elaborate a bit about the questions:
* how many workers are used: I know how many workers I have *enabled* by @@slave_parallel_threads. But how many are really used? I see in mysql.gtid_slave_pos that the number of rows grows and shrinks while the workers are replicating, but this does not give me any useful information on the effectiveness of the operations. Just to give you an idea, in Tungsten replication, I can see for every thread how many requests were processed. This can help me fine tuning the process, and add or remove threads to reach optimal performance.
* which database they are running is important even if the parallel replication is not split by schema. As a DBA, I need to know at a glance where the action is occurring. SHOW PROCESSLIST gives me that information, but it doesn't tell me the GTID, which is the info that gives me the pulse of the replication progress. What bothers me is that I can get either the GTID list or where the action is happening. Having both in one place should not be difficult (a I_S or P_S view, for example). I see in MariaDB implementation some of the lack of integration that I have seen in MySQL 5.6.

I hope to see better integration and more tools in the future. As it is now, parallel replication seems to be a very powerful engine with brake and steering wheel but with a blackened out dashboard. Makes for some adventurous driving!

Cheers

Giuseppe

On November 15, 2013 at 08:49:29 , Kristian Nielsen (knielsen@knielsen-hq.org) wrote:

Giuseppe Maxia <g.maxia@gmail.com> writes:

> I am looking for:
> * how many workers are used;

The number of workers created is determined by the value of
@@slave_parallel_threads. That many worker threads are in the replication
worker thread pool, shared among all multi-source master connections.

Workers are scheduled by the SQL threads dynamically, depending on what
parallelism is available in the events being read from the relay logs at any
given point in time. Workers for which there is currently no work available
will appear as "Waiting for work from SQL thread" in SHOW PROCESSLIST.

> * what query they are running.

I think this should appear in SHOW PROCESSLIST, same as for the SQL thread in
non-parallel replication.

> * in which schema they are working;

That is not really meaningful? A worker thread can be working in multiple
schemas at the same time (multi-table update...).

The parallel replication feature works like this:

- On the master, binlog events are marked for whether they can be replicated
in parallel. Two events can be replication in parallel if 1) their GTID
events have the same group commit ID (cid=XXX in mysqlbinlog output), or 2)
if their GTID has different replication domain (first number in the
GTID=D-S-N tripple), and the slave is operating in GTID mode.

- On the slave, the SQL thread reads events from the relay log, and
dynamically schedules the same or a new worker to execute the events so
read, depending on whether parallelism is possible or not. If the pool of
threads is exhausted, the SQL thread waits for a worker thread to become
idle.

So unlike MySQL multi-threaded slave, worker threads in MariaDB have no
identity. Any worker thread can potentially be scheduled to work on any event
from any master connection.

> What can I use to see the status of parallel replication?

Nowhere, really :-(

The parallel replication feature has been basically rushed ruthlessly to
release, with no time taken for any polish like this or other similar
reasonable concerns...

I did briefly think of what could be made available, but I did not come up
with any immediate ideas. The scheduling of parallel replication work really
is very dynamic in nature, just like any other application load. Maybe the
performance schema or something like that could be used, but I am not familiar
with how it works. Suggestions definitely welcome.

Giuseppe, btw, thanks for once again taking the time to look and comment on my
work!

- Kristian.