[Maria-discuss] Fwd: New Question: Non-blocking calls from PHP
oops, didn't copy the list. ---------- Forwarded message ---------- From: Justin Swanhart <greenlion@gmail.com> Date: Wed, Feb 24, 2016 at 11:34 AM Subject: Re: [Maria-discuss] New Question: Non-blocking calls from PHP To: Federico Razzoli <federico_raz@yahoo.it> Hi, For now, if you want FIFO, just change the ORDER BY q_id DESC to ORDER BY q_id. The execution is parallel, it shouldn't starve queries. I will probably change this anyway by changing the structure of the `q` table to have a primary key that sorts by the completion status and a unique key for the auto_increment columns, then it can naturally be FIFO without causing lock contention due to locking scanned rows in InnoDB. The workaround for GET DIAGNOSTICS is to change it to not use that and just put a generic error in the table. You can't use conditional version syntax in stored procs, and I don't want to have two different versions of the script, since everyone really should be using 5.6+ or MariaDB 5.5+ IMHO. I will remove count_running_threads, it isn't really needed because the metadata locks held by the stored proc enforce that no more than 'thread_count' threads can run. I will remove that code shortly. I wasn't using metadata locks when I started writing the tool. This will remove the dependency on global variables and the processlist. It is version 1.0 and is still experimental :) Thank you for the feedback. --Justin On Wed, Feb 24, 2016 at 10:58 AM, Federico Razzoli <federico_raz@yahoo.it> wrote:
Yes, GET DIAGNOSTICS is in 5.5 (and anyway, a workaround for older versions should be possible, correct?).
Some comments:
1) The LIFO tables sounds to me a little dangerous. Of course "asynchronous" means "asynchronous", but under certain circumstances some queries could wait too much... don't you agree? Maybe the direction could depend from an option...
2) SELECT queries are "materialized", which is cool. But I don't understand how the user can associate an async query with the results table.
3) In MariaDB, the P_S is disabled by default, and a user can still disable it in MySQL. Also, in MariaDB global_status is in I_S, while MySQL has it in P_S (another funny incompatibility developed by Oracle). The first problem can probably be addressed with CONNECT (SRCDEF='show processlist') and the second with a view in async schema. Would you accept a patch?
Probably more comments/questions will come. In the meanwhile, thanks for your work (again).
bye Federico
-------------------------------------------- Mer 24/2/16, Justin Swanhart <greenlion@gmail.com> ha scritto:
Oggetto: Re: [Maria-discuss] New Question: Non-blocking calls from PHP A: "Federico Razzoli" <federico_raz@yahoo.it> Cc: "Maria Discuss" <maria-discuss@lists.launchpad.net> Data: Mercoledì 24 febbraio 2016, 16:04
Hi, It is completely native. It consists of a queue table, a thread state table, a settings table, a stored procedure worker, an event to execute the worker (more than one run in parallel) and procs to drop off, check on, and get the results of a SQL statement (which always waits if the SQL is not done yet). I don't know if MariaDB 5.5 or 10 adds GET DIAGNOSTICS, but it will work with any version that has it. You can comment that out and just put a generic error into the table if it isn't supported, and it will work with any version that has SIGNAL. --Justin On Wed, Feb 24, 2016 at 10:01 AM, Federico Razzoli <federico_raz@yahoo.it> wrote: Hi Justin!
The name reminds me a a fatal trap for developers and sys admins (async npm module), but the tools sounds useful.
Is it publicly available? Is it entirely implemented via stored procedures + event scheduler?
bye
Federico
--------------------------------------------
Mer 24/2/16, Justin Swanhart <greenlion@gmail.com> ha scritto:
Oggetto: Re: [Maria-discuss] New Question: Non-blocking calls from PHP
A: "AskMonty KB" <noreply@askmonty.org>
Cc: "Maria Discuss" <maria-discuss@lists.launchpad.net>
Data: Mercoledì 24 febbraio 2016, 15:32
Replying
here in addition to the web page (ugh Frankenstein's
monster)...
I just finished
the first version of 'async' which runs queries in
the background (in parallel!) using a queue table, a crafty
stored procedure for a worker thread, and an event which
executes the workers.
http://swanhart.livejournal.com/138754.html
On Sun, Feb 21, 2016 at
8:55 AM, AskMonty KB <noreply@askmonty.org>
wrote:
Hello,
A new question has been asked in " MariaDB Connector/C
API Functions" by shestero. Please answer it at http://mariadb.com/kb/en/non-blocking-calls-from-php/
as the person asking the question may not be subscribed to
the mailing list.
--------------------------------
I want to use non-blocking API ( https://mariadb.com/kb/en/mariadb/non-blocking-api-reference
) from PHP.
Is it possible now?
If no, can anybody give me directions how to implement my
own PHP-extension in C (using Connector/C) or update the
existing module? I don't think it's very complex, do
I need to add just wraper-like implementations using
blocking function as samples, don't I?
--------------------------------
To view or answer this question please visit: http://mariadb.com/kb/en/non-blocking-calls-from-php/
_______________________________________________
Mailing list: https://launchpad.net/~maria-discuss
Post to : maria-discuss@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-discuss
More help : https://help.launchpad.net/ListHelp
-----Segue allegato-----
_______________________________________________
Mailing list: https://launchpad.net/~maria-discuss
Post to : maria-discuss@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-discuss
More help : https://help.launchpad.net/ListHelp
Hi, I removed dependencies on the perfschema, I removed the `threads` table, the queue is now FIFO. The queue() proc now also populates @query_list, with queries in order as they were submitted. You can submit a bunch of queries then call wait_all() to wait for each submitted query, and return results, in the order that the queries were queued. --Justin On Wed, Feb 24, 2016 at 11:34 AM, Justin Swanhart <greenlion@gmail.com> wrote:
oops, didn't copy the list.
---------- Forwarded message ---------- From: Justin Swanhart <greenlion@gmail.com> Date: Wed, Feb 24, 2016 at 11:34 AM Subject: Re: [Maria-discuss] New Question: Non-blocking calls from PHP To: Federico Razzoli <federico_raz@yahoo.it>
Hi,
For now, if you want FIFO, just change the ORDER BY q_id DESC to ORDER BY q_id. The execution is parallel, it shouldn't starve queries. I will probably change this anyway by changing the structure of the `q` table to have a primary key that sorts by the completion status and a unique key for the auto_increment columns, then it can naturally be FIFO without causing lock contention due to locking scanned rows in InnoDB.
The workaround for GET DIAGNOSTICS is to change it to not use that and just put a generic error in the table. You can't use conditional version syntax in stored procs, and I don't want to have two different versions of the script, since everyone really should be using 5.6+ or MariaDB 5.5+ IMHO.
I will remove count_running_threads, it isn't really needed because the metadata locks held by the stored proc enforce that no more than 'thread_count' threads can run. I will remove that code shortly. I wasn't using metadata locks when I started writing the tool. This will remove the dependency on global variables and the processlist.
It is version 1.0 and is still experimental :) Thank you for the feedback.
--Justin
On Wed, Feb 24, 2016 at 10:58 AM, Federico Razzoli <federico_raz@yahoo.it> wrote:
Yes, GET DIAGNOSTICS is in 5.5 (and anyway, a workaround for older versions should be possible, correct?).
Some comments:
1) The LIFO tables sounds to me a little dangerous. Of course "asynchronous" means "asynchronous", but under certain circumstances some queries could wait too much... don't you agree? Maybe the direction could depend from an option...
2) SELECT queries are "materialized", which is cool. But I don't understand how the user can associate an async query with the results table.
3) In MariaDB, the P_S is disabled by default, and a user can still disable it in MySQL. Also, in MariaDB global_status is in I_S, while MySQL has it in P_S (another funny incompatibility developed by Oracle). The first problem can probably be addressed with CONNECT (SRCDEF='show processlist') and the second with a view in async schema. Would you accept a patch?
Probably more comments/questions will come. In the meanwhile, thanks for your work (again).
bye Federico
-------------------------------------------- Mer 24/2/16, Justin Swanhart <greenlion@gmail.com> ha scritto:
Oggetto: Re: [Maria-discuss] New Question: Non-blocking calls from PHP A: "Federico Razzoli" <federico_raz@yahoo.it> Cc: "Maria Discuss" <maria-discuss@lists.launchpad.net> Data: Mercoledì 24 febbraio 2016, 16:04
Hi, It is completely native. It consists of a queue table, a thread state table, a settings table, a stored procedure worker, an event to execute the worker (more than one run in parallel) and procs to drop off, check on, and get the results of a SQL statement (which always waits if the SQL is not done yet). I don't know if MariaDB 5.5 or 10 adds GET DIAGNOSTICS, but it will work with any version that has it. You can comment that out and just put a generic error into the table if it isn't supported, and it will work with any version that has SIGNAL. --Justin On Wed, Feb 24, 2016 at 10:01 AM, Federico Razzoli <federico_raz@yahoo.it> wrote: Hi Justin!
The name reminds me a a fatal trap for developers and sys admins (async npm module), but the tools sounds useful.
Is it publicly available? Is it entirely implemented via stored procedures + event scheduler?
bye
Federico
--------------------------------------------
Mer 24/2/16, Justin Swanhart <greenlion@gmail.com> ha scritto:
Oggetto: Re: [Maria-discuss] New Question: Non-blocking calls from PHP
A: "AskMonty KB" <noreply@askmonty.org>
Cc: "Maria Discuss" <maria-discuss@lists.launchpad.net>
Data: Mercoledì 24 febbraio 2016, 15:32
Replying
here in addition to the web page (ugh Frankenstein's
monster)...
I just finished
the first version of 'async' which runs queries in
the background (in parallel!) using a queue table, a crafty
stored procedure for a worker thread, and an event which
executes the workers.
http://swanhart.livejournal.com/138754.html
On Sun, Feb 21, 2016 at
8:55 AM, AskMonty KB <noreply@askmonty.org>
wrote:
Hello,
A new question has been asked in " MariaDB Connector/C
API Functions" by shestero. Please answer it at http://mariadb.com/kb/en/non-blocking-calls-from-php/
as the person asking the question may not be subscribed to
the mailing list.
--------------------------------
I want to use non-blocking API ( https://mariadb.com/kb/en/mariadb/non-blocking-api-reference
) from PHP.
Is it possible now?
If no, can anybody give me directions how to implement my
own PHP-extension in C (using Connector/C) or update the
existing module? I don't think it's very complex, do
I need to add just wraper-like implementations using
blocking function as samples, don't I?
--------------------------------
To view or answer this question please visit: http://mariadb.com/kb/en/non-blocking-calls-from-php/
_______________________________________________
Mailing list: https://launchpad.net/~maria-discuss
Post to : maria-discuss@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-discuss
More help : https://help.launchpad.net/ListHelp
-----Segue allegato-----
_______________________________________________
Mailing list: https://launchpad.net/~maria-discuss
Post to : maria-discuss@lists.launchpad.net
Unsubscribe : https://launchpad.net/~maria-discuss
More help : https://help.launchpad.net/ListHelp
participants (1)
-
Justin Swanhart