Hi, I had been busy with my exams (which got finished today). I'll come up with more detailed working plan and doubts in a couple of days. On Tue, Apr 29, 2014 at 11:28 PM, Sergei Golubchik wrote:

Hi, Anshu!

So, how do we start?

Till now, I have not done much. I had started exploring the code for optimizer but stopped mid-way and started to read the book "Inside the SQL Server Query Optimizer" for a better understanding of how the optimizer works. I see you're often present on irc, which is great.

(I go by nick igniting)

You've looked at the code, what do you think?

Do you understand how different parts of this feature fit together?

We can start from just one constant (global or per-engine) and see how it'll work.

I had started exploring tmptable_create_cost. In sql/sql_const.h: HEAP_TEMPTABLE_CREATE_COST is defined to be 2.0 and DISK_TEMPTABLE_CREATE_COST as 40.0. As discussed on irc with serg, I'll start to profile this constant (which I have not started yet).

And, by the way, when you start coding (May 19) or earlier, as you prefer, I would like to start seeing some kind of weekly updates from you. In email or in your blog - whatever you feel more comfortable with.

Blog updates should be fine. Regards,

Sergei

Regards Anshu

Hi, Anshu! How are you doing? Any progress so far? On Apr 30, Anshu Avinash wrote:

...

And, by the way, when you start coding (May 19) or earlier, as you prefer, I would like to start seeing some kind of weekly updates from you. In email or in your blog - whatever you feel more comfortable with.

Blog updates should be fine.

That's fine. Whatever you prefer. One blog post every week then, preferrably on Monday. Regards, Sergei

Hi all, Sorry for the irregular updates. I had been busy for last couple of days and might still be busy for 1-2 days more. I would be completely free starting next week, and would be updating my blog weekly on every Monday (so 1st update would be on May 12). I would also send the link of my post weekly on the mailing list. As discussed on irc, I started to explore the pair of constants: handler::scan_time() and handler::read_time(). I also started looking into sql_statistics.cc for writing the optimizer constants into a persistent db. Regards Anshu Avinash On Thu, May 8, 2014 at 11:08 PM, Colin Charles wrote:

Hi!

On 8 May 2014, at 22:33, Sergei Golubchik wrote:

...

Hi, Anshu!

How are you doing? Any progress so far?

On Apr 30, Anshu Avinash wrote:

...

...

And, by the way, when you start coding (May 19) or earlier, as you prefer, I would like to start seeing some kind of weekly updates from you. In email or in your blog - whatever you feel more comfortable

with.

...

Blog updates should be fine.

That's fine. Whatever you prefer. One blog post every week then, preferrably on Monday.

For the benefit of others Anshu, please also post your weekly reports to maria-developers@lists.launchpad.net - I think it will be really good for those that don't drop by your blog and you'll likely also get other feedback maybe

This goes to all those participating in GSoC.

Also for those with a blog + RSS feed, you should aim to get it on http://planetmariadb.org/ and http://planet.mysql.com/

cheers, -colin

-- Colin Charles, Chief Evangelist, SkySQL - The MariaDB Company blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

Hi all, This week's blog entry would get delayed by couple of days. I have started coding though and would like to give heads up on what I'm doing. I've looked at the diffs for "Cost model project" of mysql: http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 and http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . These give a pretty good idea about what are the hard-coded constants and where are they being used. The idea is to multiply "READ_TIME_FACTOR" and "SCAN_TIME_FACTOR" to the values returned by read_time() and scan_time() in handler.h, while returning. These values would be read from a table in mysql db. For that I've looked at sql_statistics.cc. After completing this, I'll first change the values of these constants manually and check if the better or worse query plans are being selected. I'll first do the last step manually, to check if everything is working as expected and later automate it. Regards Anshu On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash wrote:

Hi all,

You can find my blog entry for this week at http://igniting.in/gsoc2014/2014/05/11/first-steps/ .

Regards Anshu Avinash

On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash wrote:

...

Hi all,

Sorry for the irregular updates. I had been busy for last couple of days and might still be busy for 1-2 days more. I would be completely free starting next week, and would be updating my blog weekly on every Monday (so 1st update would be on May 12). I would also send the link of my post weekly on the mailing list.

As discussed on irc, I started to explore the pair of constants: handler::scan_time() and handler::read_time(). I also started looking into sql_statistics.cc for writing the optimizer constants into a persistent db.

Regards Anshu Avinash

On Thu, May 8, 2014 at 11:08 PM, Colin Charles wrote:

...

Hi!

On 8 May 2014, at 22:33, Sergei Golubchik wrote:

...

Hi, Anshu!

How are you doing? Any progress so far?

On Apr 30, Anshu Avinash wrote:

...

...

And, by the way, when you start coding (May 19) or earlier, as you prefer, I would like to start seeing some kind of weekly updates from you. In email or in your blog - whatever you feel more comfortable

with.

...

Blog updates should be fine.

That's fine. Whatever you prefer. One blog post every week then, preferrably on Monday.

For the benefit of others Anshu, please also post your weekly reports to maria-developers@lists.launchpad.net - I think it will be really good for those that don't drop by your blog and you'll likely also get other feedback maybe

This goes to all those participating in GSoC.

Also for those with a blog + RSS feed, you should aim to get it on http://planetmariadb.org/ and http://planet.mysql.com/

cheers, -colin

-- Colin Charles, Chief Evangelist, SkySQL - The MariaDB Company blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

Hi all, You can find my this week's blog entry at http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ . I have created a branch on launchpad for my work: http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 . You can give your suggestions/reviews either on this thread or as a comment on the blog itself. Regards Anshu Avinash On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim wrote:

wow a big work, congratulation guy, i will read part by part to better understand mariadb code

2014-05-19 16:33 GMT-03:00 Anshu Avinash :

...

Hi all,

This week's blog entry would get delayed by couple of days. I have started coding though and would like to give heads up on what I'm doing.

I've looked at the diffs for "Cost model project" of mysql: http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 and http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . These give a pretty good idea about what are the hard-coded constants and where are they being used.

The idea is to multiply "READ_TIME_FACTOR" and "SCAN_TIME_FACTOR" to the values returned by read_time() and scan_time() in handler.h, while returning. These values would be read from a table in mysql db. For that I've looked at sql_statistics.cc. After completing this, I'll first change the values of these constants manually and check if the better or worse query plans are being selected. I'll first do the last step manually, to check if everything is working as expected and later automate it.

Regards Anshu

On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash < anshu.avinash35@gmail.com> wrote:

...

Hi all,

You can find my blog entry for this week at http://igniting.in/gsoc2014/2014/05/11/first-steps/ .

Regards Anshu Avinash

On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash < anshu.avinash35@gmail.com> wrote:

...

Hi all,

Sorry for the irregular updates. I had been busy for last couple of days and might still be busy for 1-2 days more. I would be completely free starting next week, and would be updating my blog weekly on every Monday (so 1st update would be on May 12). I would also send the link of my post weekly on the mailing list.

As discussed on irc, I started to explore the pair of constants: handler::scan_time() and handler::read_time(). I also started looking into sql_statistics.cc for writing the optimizer constants into a persistent db.

Regards Anshu Avinash

On Thu, May 8, 2014 at 11:08 PM, Colin Charles wrote:

...

Hi!

On 8 May 2014, at 22:33, Sergei Golubchik wrote:

...

Hi, Anshu!

How are you doing? Any progress so far?

On Apr 30, Anshu Avinash wrote: > >> And, by the way, when you start coding (May 19) or earlier, as you >> prefer, I would like to start seeing some kind of weekly updates from >> you. In email or in your blog - whatever you feel more comfortable with. > > Blog updates should be fine.

That's fine. Whatever you prefer. One blog post every week then, preferrably on Monday.

For the benefit of others Anshu, please also post your weekly reports to maria-developers@lists.launchpad.net - I think it will be really good for those that don't drop by your blog and you'll likely also get other feedback maybe

This goes to all those participating in GSoC.

Also for those with a blog + RSS feed, you should aim to get it on http://planetmariadb.org/ and http://planet.mysql.com/

cheers, -colin

-- Colin Charles, Chief Evangelist, SkySQL - The MariaDB Company blog: http://bytebot.net/blog/| t: +6-012-204-3201 | Skype: colincharles

_______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

Hi Sergei! nice :) i'm reading about it just one doubt i didn't understand everything yet there's a read time and a scan time, what the difference? read = sequencial read and scan = non sequencial read? or something like table read cost, index read cost? 2014-05-27 5:41 GMT-03:00 Sergei Golubchik :

Hi, Roberto!

On May 26, Roberto Spadim wrote:

...

hi guys, one question, that maybe at the right time could be answered... with the new cost model, maybe at explain we could have a new information about how many time we will expend? and maybe something like, estimate time / executed time (via show warnings?) i will read the patch about this work, and check if i could help with something like this is it interesting?

First, it's not about creating a new cost model. Only about providing more adequate real-life values to certain parameters in the current cost model.

And yes, the work could, possibly, help to estimate the query execution time. Although this wasn't the goal, we only wanted to find correct relative values, not absolute.

Regards, Sergei

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

nice :) maybe part of msql? :) very old times nice, i will stay updated with this gson project, very interesting and a very good work :) 2014-05-27 15:35 GMT-03:00 Sergei Golubchik :

Hi, Roberto!

On May 27, Roberto Spadim wrote:

...

Hi Sergei! nice :) i'm reading about it just one doubt i didn't understand everything yet there's a read time and a scan time, what the difference? read = sequencial read and scan = non sequencial read? or something like table read cost, index read cost?

Yes, the handler methods are:

virtual double handler::scan_time() { return stats.data_file_length / IO_SIZE + 2; } virtual double handler::read_time(uint index, uint ranges, ha_rows rows) { return ranges+rows; }

these are the default implementations, in the base handler class. Method names are historical and *very* old.

The return values are kind of "number of disk seeks". You can see that scan_time() assumes a sequential disk read of blocks of IO_SIZE bytes.

While read_time() assumes random reads of 'rows' rows from the data file and 'ranges' ranges from the index.

These default implementations, I suspect, predate even MyISAM.

Regards, Sergei

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

Hi, Anshu! On Jun 08, Anshu Avinash wrote:

Hi all,

As per serg comments on the previous commit, I have modified the code to add an init function which will initialize all the cost factors at the start of the server. The latest code is at https://github.com/igniting/server/commits/selfTuningOptimizer. I have also attached the diff here.

However, I am still not able to come up with an example for my test case for READ_TIME_FACTOR and SCAN_TIME_FACTOR. I will go through the code again to figure it out. Currently the test case which I have written just gives the workflow: EXPLAIN SELECT ...; Update constant tables directly; Reconnect; EXPLAIN SELECT.

For a simple test case, try something like the following: create table t1 (a int auto_increment primary key, b int); insert t1 (b) values (1),(2),(3),... I think, about 20-30 rows should be enough. explain select t1 where a > 15; it doesn't have to be 15, try to experiment with different constants in the query. Pick the one where 'a > N' gives you a table scan (explain shows 'ALL'), but with 'a > N+1' explain shows 'range'. Now, modify one of your cost factors and try again. explain should change from 'ALL' to 'range'. I'll send a second email soon with my comments about your patch, thanks for attaching it! Regards, Sergei

Hi serg, I have attached the updated diff. I'm still unable to observe the effect introduced by the cost factors. Maybe I just need to study the usage of scan_time() and read_time() in greater detail. Regards Anshu On Tue, Jun 10, 2014 at 1:19 PM, Sergei Golubchik wrote:

Hi, Anshu!

On Jun 10, Anshu Avinash wrote:

...

Hi serg,

I have implemented your suggestions. In the test case I created a table with 25 rows. Explain for 'select * from t1 where a > 19' gave 'ALL' while explain for 'select * from t1 where a > 20' gives range. I have also written public methods ha_scan_time() and ha_read_time(). I should replace every occurrence of handler::scan_time() with ha_scan_time(), right? How do I make sure that I don't miss any place?

See other ha_xxx vs xxx methods, e.g. ha_rnd_init and rnd_init.

You make scan_time() protected - and the compiler won't allow it to be called outside of the handler class.

...

Also regarding making everything in the class Cost_factors static vs creating an object and using it everywhere: cann't we use a namespace Cost_factors? How is it done usually? I don't see much usage of namespaces in the code.

Yes, namespaces aren't used much in MariaDB, but feel free to use them, if you'd like.

Regards, Sergei

Hi, Anshu! Please, see the attached patch. It introduces a macro top_level_cond_is_satisfied() that now marks all places where a comparison (in the TIME_FOR_COMPARE sense) is done. So, all you need to do is to convert this macro into an (inline) function and increment your counter from there. As for TIME_FOR_COMPARE_ROWID, it's simpler. There's handler::cmp_ref(), a virtual handler method. Create handler::ha_cmp_ref(), non-virtual inline public method (make cmp_ref() protected, as usual). And increment your counter from ha_cmp_ref(). By the way, before you start writing the code to solve our equations, please write an email, explaining how you're going to solve them, that is, what method you'll be using. There are lots of them. Some are faster, some are slower. Some work better with sparse matrices (and ours will be very much sparse). Regards, Sergei

Hi, Anshu! On Jun 25, Anshu Avinash wrote:

Hi Sergei,

I have been looking into various methods that can meet our conditions. The Gauss-Seidel Method (http://en.wikipedia.org/wiki/Gauss-Seidel_method) might meet our requirements. I had just started writing the code.

So in a nutshell, Gauss-seidel method is an iterative method. In our case, the initial values for all variables would be obtained by (total_query_time)/total_queries if total_queries not equal to 0, 0 otherwise. After each query, we will get update the value of one variable (which should have a non-zero coefficient in the current equation). We would cycle through all constants query after query. The logic is that after many queries, the values would converge to the correct value. Also updating one variable after each query shouldn't be much of an overhead and we just need to store coefficients for the current query.

The code for this is very simple, I can give a patch in 1-2 days, if the overall idea seems ok.

I don't have much experience in here :( Are you sure that Gauss-Seidel will converge? It's not obvious to me that the convergence criteria are met on our data. Besides, we do have a sparse matrix, that's true. But we don't (and won't ever) have a large matrix with millions of unknowns. We might get a thousand in a few years (ten per-engine constants times MAX_HA is already 640). So, methods that are asymptotically faster may not be faster in our case. I'd suggest the following: in your method that should be solving the equation - instead of solving anything, just dump the data to a file (make sure that data from different threads aren't intermixed, or use different files per thread). Then you can use any existing tool or software package to play with the data, try different methods, even prototype in python, if you'd like. One of the issues to keep in mind - the system won't be completely defined. Basically never. MAX_HA = 64, but practically you can expect the data for one or two engines only, so no matter how many equations you'll collect, ~120 variables will always have zero coefficients. By the way, just a thought, in iterative methods you can use your current values of optimizer constants as initial values for the first iteration. Regards, Sergei

Well i wws reading your posts Do you need big data to test read and scan times? Em segunda-feira, 9 de junho de 2014, Anshu Avinash < anshu.avinash35@gmail.com> escreveu:

Hi all,

You can find this week's blog entry at http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm now maintaining the code only on github: https://github.com/igniting/server/tree/selfTuningOptimizer.

Regards Anshu Avinash

On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash wrote:

Hi all,

You can find my this week's blog entry at http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ . I have created a branch on launchpad for my work: http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 . You can give your suggestions/reviews either on this thread or as a comment on the blog itself.

Regards Anshu Avinash

On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim wrote:

wow a big work, congratulation guy, i will read part by part to better understand mariadb code

2014-05-19 16:33 GMT-03:00 Anshu Avinash :

Hi all,

This week's blog entry would get delayed by couple of days. I have started coding though and would like to give heads up on what I'm doing.

I've looked at the diffs for "Cost model project" of mysql: http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 and http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . These give a pretty good idea about what are the hard-coded constants and where are they being used.

The idea is to multiply "READ_TIME_FACTOR" and "SCAN_TIME_FACTOR" to the values returned by read_time() and scan_time() in handler.h, while returning. These values would be read from a table in mysql db. For that I've looked at sql_statistics.cc. After completing this, I'll first change the values of these constants manually and check if the better or worse query plans are being selected. I'll first do the last step manually, to check if everything is working as expected and later automate it.

Regards Anshu

On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash

...

wrote:

Hi all,

You can find my blog entry for this week at http://igniting.in/gsoc2014/2014/05/11/first-steps/ .

Regards Anshu Avinash

On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash wrote:

Hi all,

Sorry for the irregular updates. I had been busy for last couple of days and might still be busy for 1-2 days more. I would be completely free starting next week, and would be updating my blog weekly on every Monday (so 1st update would be on May 12). I would also send the link of my post weekly on the mailing list.

As discussed on irc, I started to explore the pair of constants: handler::scan_time() and handler::read_time().

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

"Sorry this page does not exist =(" 2014-06-23 8:07 GMT-03:00 Anshu Avinash :

Hi all,

You can find this week's blog entry at: http://igniting.in/2014/06/23/work-before-mid-term/ Suggestions/reviews are welcome.

Regards Anshu Avinash

On Mon, Jun 9, 2014 at 7:30 PM, Roberto Spadim wrote:

...

Well i wws reading your posts Do you need big data to test read and scan times?

Em segunda-feira, 9 de junho de 2014, Anshu Avinash escreveu:

...

Hi all,

You can find this week's blog entry at http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm now maintaining the code only on github: https://github.com/igniting/server/tree/selfTuningOptimizer.

Regards Anshu Avinash

On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash wrote:

Hi all,

You can find my this week's blog entry at http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ . I have created a branch on launchpad for my work: http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 . You can give your suggestions/reviews either on this thread or as a comment on the blog itself.

Regards Anshu Avinash

On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim wrote:

wow a big work, congratulation guy, i will read part by part to better understand mariadb code

2014-05-19 16:33 GMT-03:00 Anshu Avinash :

Hi all,

This week's blog entry would get delayed by couple of days. I have started coding though and would like to give heads up on what I'm doing.

I've looked at the diffs for "Cost model project" of mysql: http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 and http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . These give a pretty good idea about what are the hard-coded constants and where are they being used.

The idea is to multiply "READ_TIME_FACTOR" and "SCAN_TIME_FACTOR" to the values returned by read_time() and scan_time() in handler.h, while returning. These values would be read from a table in mysql db. For that I've looked at sql_statistics.cc. After completing this, I'll first change the values of these constants manually and check if the better or worse query plans are being selected. I'll first do the last step manually, to check if everything is working as expected and later automate it.

Regards Anshu

On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash wrote:

Hi all,

You can find my blog entry for this week at http://igniting.in/gsoc2014/2014/05/11/first-steps/ .

Regards Anshu Avinash

On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash wrote:

Hi all,

Sorry for the irregular updates. I had been busy for last couple of days and might still be busy for 1-2 days more. I would be completely free starting next week, and would be updating my blog weekly on every Monday (so 1st update would be on May 12). I would also send the link of my post weekly on the mailing list.

As discussed on irc, I started to explore the pair of constants: handler::scan_time() and handler::read_time().

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

" MDEV. " it's nice to put full name (MDEV-350), since google and others search engines help when someone try to find information about mdev 350 text is ok :) 2014-06-23 11:04 GMT-03:00 Anshu Avinash :

Hi,

Sorry for the confusion, this is the new link: http://igniting.in/gsoc2014/2014/06/23/work-before-mid-term/ Thanks for pointing out.

Regards Anshu

On Mon, Jun 23, 2014 at 7:32 PM, Roberto Spadim wrote:

...

"Sorry this page does not exist =("

2014-06-23 8:07 GMT-03:00 Anshu Avinash :

...

Hi all,

You can find this week's blog entry at: http://igniting.in/2014/06/23/work-before-mid-term/ Suggestions/reviews are welcome.

Regards Anshu Avinash

On Mon, Jun 9, 2014 at 7:30 PM, Roberto Spadim wrote:

...

Well i wws reading your posts Do you need big data to test read and scan times?

Em segunda-feira, 9 de junho de 2014, Anshu Avinash escreveu:

...

Hi all,

You can find this week's blog entry at http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm now maintaining the code only on github: https://github.com/igniting/server/tree/selfTuningOptimizer.

Regards Anshu Avinash

On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash wrote:

Hi all,

You can find my this week's blog entry at http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ . I have created a branch on launchpad for my work: http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 . You can give your suggestions/reviews either on this thread or as a comment on the blog itself.

Regards Anshu Avinash

On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim wrote:

wow a big work, congratulation guy, i will read part by part to better understand mariadb code

2014-05-19 16:33 GMT-03:00 Anshu Avinash :

Hi all,

This week's blog entry would get delayed by couple of days. I have started coding though and would like to give heads up on what I'm doing.

I've looked at the diffs for "Cost model project" of mysql: http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 and http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . These give a pretty good idea about what are the hard-coded constants and where are they being used.

The idea is to multiply "READ_TIME_FACTOR" and "SCAN_TIME_FACTOR" to the values returned by read_time() and scan_time() in handler.h, while returning. These values would be read from a table in mysql db. For that I've looked at sql_statistics.cc. After completing this, I'll first change the values of these constants manually and check if the better or worse query plans are being selected. I'll first do the last step manually, to check if everything is working as expected and later automate it.

Regards Anshu

On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash wrote:

Hi all,

You can find my blog entry for this week at http://igniting.in/gsoc2014/2014/05/11/first-steps/ .

Regards Anshu Avinash

On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash wrote:

Hi all,

Sorry for the irregular updates. I had been busy for last couple of days and might still be busy for 1-2 days more. I would be completely free starting next week, and would be updating my blog weekly on every Monday (so 1st update would be on May 12). I would also send the link of my post weekly on the mailing list.

As discussed on irc, I started to explore the pair of constants: handler::scan_time() and handler::read_time().

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

could you 'display' the dataset you used with octave? 2014-07-08 13:55 GMT-03:00 Anshu Avinash :

Hi all,

This week's blog post is at: http://igniting.in/gsoc2014/2014/07/08/solving-linear-equations/ . Sorry for the delay. Suggestions for an approach to solve the system of linear equations are welcome.

Regards Anshu Avinash

On Mon, Jun 23, 2014 at 7:39 PM, Roberto Spadim wrote:

...

" MDEV. " it's nice to put full name (MDEV-350), since google and others search engines help when someone try to find information about mdev 350

text is ok :)

2014-06-23 11:04 GMT-03:00 Anshu Avinash :

...

Hi,

Sorry for the confusion, this is the new link: http://igniting.in/gsoc2014/2014/06/23/work-before-mid-term/ Thanks for pointing out.

Regards Anshu

On Mon, Jun 23, 2014 at 7:32 PM, Roberto Spadim wrote:

...

"Sorry this page does not exist =("

2014-06-23 8:07 GMT-03:00 Anshu Avinash :

...

Hi all,

You can find this week's blog entry at: http://igniting.in/2014/06/23/work-before-mid-term/ Suggestions/reviews are welcome.

Regards Anshu Avinash

On Mon, Jun 9, 2014 at 7:30 PM, Roberto Spadim wrote:

...

Well i wws reading your posts Do you need big data to test read and scan times?

Em segunda-feira, 9 de junho de 2014, Anshu Avinash escreveu:

> Hi all, > > You can find this week's blog entry at > http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm now > maintaining the > code only on github: > https://github.com/igniting/server/tree/selfTuningOptimizer. > > Regards > Anshu Avinash > > > On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash > wrote: > > Hi all, > > You can find my this week's blog entry at > http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ . I have > created a > branch on launchpad for my work: > http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 . > You > can > give your suggestions/reviews either on this thread or as a comment > on > the > blog itself. > > Regards > Anshu Avinash > > > On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim > > wrote: > > wow a big work, congratulation guy, i will read part by part to > better > understand mariadb code > > > 2014-05-19 16:33 GMT-03:00 Anshu Avinash > : > > Hi all, > > This week's blog entry would get delayed by couple of days. I have > started coding though and would like to give heads up on what I'm > doing. > > I've looked at the diffs for "Cost model project" of mysql: > http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 > and > http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . > These > give a pretty good idea about what are the hard-coded constants and > where > are they being used. > > The idea is to multiply "READ_TIME_FACTOR" and "SCAN_TIME_FACTOR" > to > the > values returned by read_time() and scan_time() in handler.h, while > returning. These values would be read from a table in mysql db. For > that > I've looked at sql_statistics.cc. After completing this, I'll first > change > the values of these constants manually and check if the better or > worse > query plans are being selected. I'll first do the last step > manually, > to > check if everything is working as expected and later automate it. > > Regards > Anshu > > > On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash > wrote: > > Hi all, > > You can find my blog entry for this week at > http://igniting.in/gsoc2014/2014/05/11/first-steps/ . > > Regards > Anshu Avinash > > > On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash > wrote: > > Hi all, > > Sorry for the irregular updates. I had been busy for last couple of > days > and might still be busy for 1-2 days more. I would be completely > free > starting next week, and would be updating my blog weekly on every > Monday (so > 1st update would be on May 12). I would also send the link of my > post > weekly > on the mailing list. > > As discussed on irc, I started to explore the pair of constants: > handler::scan_time() and handler::read_time().

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

=] nice 2014-07-08 14:18 GMT-03:00 Anshu Avinash :

Hi all,

You can download it here (https://drive.google.com/file/d/0B7NiQb4EbbUVNVJFZ2xkRVR3Ylk/edit?usp=sharin...). It is around 26M. I have added the link on blog too.

Regards Anshu

On Tue, Jul 8, 2014 at 10:38 PM, Roberto Spadim wrote:

...

could you 'display' the dataset you used with octave?

2014-07-08 13:55 GMT-03:00 Anshu Avinash :

...

Hi all,

This week's blog post is at: http://igniting.in/gsoc2014/2014/07/08/solving-linear-equations/ . Sorry for the delay. Suggestions for an approach to solve the system of linear equations are welcome.

Regards Anshu Avinash

On Mon, Jun 23, 2014 at 7:39 PM, Roberto Spadim wrote:

...

" MDEV. " it's nice to put full name (MDEV-350), since google and others search engines help when someone try to find information about mdev 350

text is ok :)

2014-06-23 11:04 GMT-03:00 Anshu Avinash :

...

Hi,

Sorry for the confusion, this is the new link: http://igniting.in/gsoc2014/2014/06/23/work-before-mid-term/ Thanks for pointing out.

Regards Anshu

On Mon, Jun 23, 2014 at 7:32 PM, Roberto Spadim wrote:

...

"Sorry this page does not exist =("

2014-06-23 8:07 GMT-03:00 Anshu Avinash : > Hi all, > > You can find this week's blog entry at: > http://igniting.in/2014/06/23/work-before-mid-term/ > Suggestions/reviews are welcome. > > Regards > Anshu Avinash > > > On Mon, Jun 9, 2014 at 7:30 PM, Roberto Spadim > > wrote: >> >> Well i wws reading your posts >> Do you need big data to test read and scan times? >> >> Em segunda-feira, 9 de junho de 2014, Anshu Avinash >> escreveu: >> >>> Hi all, >>> >>> You can find this week's blog entry at >>> http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm now >>> maintaining the >>> code only on github: >>> https://github.com/igniting/server/tree/selfTuningOptimizer. >>> >>> Regards >>> Anshu Avinash >>> >>> >>> On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash >>> wrote: >>> >>> Hi all, >>> >>> You can find my this week's blog entry at >>> http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ . I >>> have >>> created a >>> branch on launchpad for my work: >>> http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 >>> . >>> You >>> can >>> give your suggestions/reviews either on this thread or as a >>> comment >>> on >>> the >>> blog itself. >>> >>> Regards >>> Anshu Avinash >>> >>> >>> On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim >>> >>> wrote: >>> >>> wow a big work, congratulation guy, i will read part by part to >>> better >>> understand mariadb code >>> >>> >>> 2014-05-19 16:33 GMT-03:00 Anshu Avinash >>> : >>> >>> Hi all, >>> >>> This week's blog entry would get delayed by couple of days. I >>> have >>> started coding though and would like to give heads up on what >>> I'm >>> doing. >>> >>> I've looked at the diffs for "Cost model project" of mysql: >>> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 >>> and >>> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . >>> These >>> give a pretty good idea about what are the hard-coded constants >>> and >>> where >>> are they being used. >>> >>> The idea is to multiply "READ_TIME_FACTOR" and >>> "SCAN_TIME_FACTOR" >>> to >>> the >>> values returned by read_time() and scan_time() in handler.h, >>> while >>> returning. These values would be read from a table in mysql db. >>> For >>> that >>> I've looked at sql_statistics.cc. After completing this, I'll >>> first >>> change >>> the values of these constants manually and check if the better >>> or >>> worse >>> query plans are being selected. I'll first do the last step >>> manually, >>> to >>> check if everything is working as expected and later automate >>> it. >>> >>> Regards >>> Anshu >>> >>> >>> On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash >>> wrote: >>> >>> Hi all, >>> >>> You can find my blog entry for this week at >>> http://igniting.in/gsoc2014/2014/05/11/first-steps/ . >>> >>> Regards >>> Anshu Avinash >>> >>> >>> On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash >>> wrote: >>> >>> Hi all, >>> >>> Sorry for the irregular updates. I had been busy for last couple >>> of >>> days >>> and might still be busy for 1-2 days more. I would be completely >>> free >>> starting next week, and would be updating my blog weekly on >>> every >>> Monday (so >>> 1st update would be on May 12). I would also send the link of my >>> post >>> weekly >>> on the mailing list. >>> >>> As discussed on irc, I started to explore the pair of constants: >>> handler::scan_time() and handler::read_time(). >> >> >> >> -- >> Roberto Spadim >> SPAEmpresarial >> Eng. Automação e Controle >> >

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

Hi, The idea is we know the total time the query took, and how many times an operation was performed. For example, consider the case of 'read_time'. We know how many times an index read took place, but don't know how much time does it take to do an index read. By solving these equations, we are trying to find out time for individual operations. coefficients[i].value is `how many time the operation i took place in a single query.` Hope this clears things up. Regards Anshu Avinash On Tue, Jul 8, 2014 at 10:57 PM, Roberto Spadim wrote:

just to understand... --- the solve_equation part, today only used to save information: std::ofstream datafile; char file_name[100]; my_snprintf(file_name, 100, "/tmp/mariadb_cost_coefficients_%lu.txt", thread_id); datafile.open(file_name, std::ios::app); for(int i=0; i < MAX_CONSTANTS; i++) datafile << coefficients[i].value << " "; datafile << total_time << "\n"; datafile.close(); ----

the idea is: given a query and some coefficients[i].value, you got total_time need to execute the query you want to "train" something to tell you how many time the same query should execute? or, what's the "x[i]" variables from your system (hardware/hard disk/etc), and extend this to others queries?

2014-07-08 14:20 GMT-03:00 Roberto Spadim :

...

=] nice

2014-07-08 14:18 GMT-03:00 Anshu Avinash :

...

Hi all,

You can download it here ( https://drive.google.com/file/d/0B7NiQb4EbbUVNVJFZ2xkRVR3Ylk/edit?usp=sharin... ). It is around 26M. I have added the link on blog too.

Regards Anshu

On Tue, Jul 8, 2014 at 10:38 PM, Roberto Spadim wrote:

...

could you 'display' the dataset you used with octave?

2014-07-08 13:55 GMT-03:00 Anshu Avinash :

...

Hi all,

This week's blog post is at: http://igniting.in/gsoc2014/2014/07/08/solving-linear-equations/ .

Sorry

...

...

for the delay. Suggestions for an approach to solve the system of linear equations are welcome.

Regards Anshu Avinash

On Mon, Jun 23, 2014 at 7:39 PM, Roberto Spadim < roberto@spadim.com.br> wrote:

...

" MDEV. " it's nice to put full name (MDEV-350), since google and others

search

...

engines help when someone try to find information about mdev 350

text is ok :)

2014-06-23 11:04 GMT-03:00 Anshu Avinash Hi, > > Sorry for the confusion, this is the new link: > http://igniting.in/gsoc2014/2014/06/23/work-before-mid-term/ > Thanks for pointing out. > > Regards > Anshu > > > On Mon, Jun 23, 2014 at 7:32 PM, Roberto Spadim > > wrote: >> >> "Sorry this page does not exist =(" >> >> 2014-06-23 8:07 GMT-03:00 Anshu Avinash < anshu.avinash35@gmail.com>: >> > Hi all, >> > >> > You can find this week's blog entry at: >> > http://igniting.in/2014/06/23/work-before-mid-term/ >> > Suggestions/reviews are welcome. >> > >> > Regards >> > Anshu Avinash >> > >> > >> > On Mon, Jun 9, 2014 at 7:30 PM, Roberto Spadim >> > >> > wrote: >> >> >> >> Well i wws reading your posts >> >> Do you need big data to test read and scan times? >> >> >> >> Em segunda-feira, 9 de junho de 2014, Anshu Avinash >> >> escreveu: >> >> >> >>> Hi all, >> >>> >> >>> You can find this week's blog entry at >> >>> http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm now >> >>> maintaining the >> >>> code only on github: >> >>> https://github.com/igniting/server/tree/selfTuningOptimizer. >> >>> >> >>> Regards >> >>> Anshu Avinash >> >>> >> >>> >> >>> On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash >> >>> wrote: >> >>> >> >>> Hi all, >> >>> >> >>> You can find my this week's blog entry at >> >>> http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ . I >> >>> have >> >>> created a >> >>> branch on launchpad for my work: >> >>> http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 >> >>> . >> >>> You >> >>> can >> >>> give your suggestions/reviews either on this thread or as a >> >>> comment >> >>> on >> >>> the >> >>> blog itself. >> >>> >> >>> Regards >> >>> Anshu Avinash >> >>> >> >>> >> >>> On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim >> >>> >> >>> wrote: >> >>> >> >>> wow a big work, congratulation guy, i will read part by part to >> >>> better >> >>> understand mariadb code >> >>> >> >>> >> >>> 2014-05-19 16:33 GMT-03:00 Anshu Avinash >> >>> : >> >>> >> >>> Hi all, >> >>> >> >>> This week's blog entry would get delayed by couple of days. I >> >>> have >> >>> started coding though and would like to give heads up on what >> >>> I'm >> >>> doing. >> >>> >> >>> I've looked at the diffs for "Cost model project" of mysql: >> >>> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 >> >>> and >> >>> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . >> >>> These >> >>> give a pretty good idea about what are the hard-coded constants >> >>> and >> >>> where >> >>> are they being used. >> >>> >> >>> The idea is to multiply "READ_TIME_FACTOR" and >> >>> "SCAN_TIME_FACTOR" >> >>> to >> >>> the >> >>> values returned by read_time() and scan_time() in handler.h, >> >>> while >> >>> returning. These values would be read from a table in mysql db. >> >>> For >> >>> that >> >>> I've looked at sql_statistics.cc. After completing this, I'll >> >>> first >> >>> change >> >>> the values of these constants manually and check if the better >> >>> or >> >>> worse >> >>> query plans are being selected. I'll first do the last step >> >>> manually, >> >>> to >> >>> check if everything is working as expected and later automate >> >>> it. >> >>> >> >>> Regards >> >>> Anshu >> >>> >> >>> >> >>> On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash >> >>> wrote: >> >>> >> >>> Hi all, >> >>> >> >>> You can find my blog entry for this week at >> >>> http://igniting.in/gsoc2014/2014/05/11/first-steps/ . >> >>> >> >>> Regards >> >>> Anshu Avinash >> >>> >> >>> >> >>> On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash >> >>> wrote: >> >>> >> >>> Hi all, >> >>> >> >>> Sorry for the irregular updates. I had been busy for last couple >> >>> of >> >>> days >> >>> and might still be busy for 1-2 days more. I would be completely >> >>> free >> >>> starting next week, and would be updating my blog weekly on >> >>> every >> >>> Monday (so >> >>> 1st update would be on May 12). I would also send the link of my >> >>> post >> >>> weekly >> >>> on the mailing list. >> >>> >> >>> As discussed on irc, I started to explore the pair of constants: >> >>> handler::scan_time() and handler::read_time(). >> >> >> >> >> >> >> >> -- >> >> Roberto Spadim >> >> SPAEmpresarial >> >> Eng. Automação e Controle >> >> >> > >> >> >> >> -- >> Roberto Spadim >> SPAEmpresarial >> Eng. Automação e Controle > >

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

ops, linear equation 2014-07-08 15:47 GMT-03:00 Roberto Spadim :

going back... a1t1 + a2t2 + … + a130t130= ttotal

a1, t1...

a1 is something you don't know t1 is the coefficients[i]?

it's a first order equation, right?

2014-07-08 15:20 GMT-03:00 Anshu Avinash :

...

Hi,

The idea is we know the total time the query took, and how many times an operation was performed. For example, consider the case of 'read_time'. We know how many times an index read took place, but don't know how much time does it take to do an index read. By solving these equations, we are trying to find out time for individual operations. coefficients[i].value is `how many time the operation i took place in a single query.`

Hope this clears things up.

Regards Anshu Avinash

On Tue, Jul 8, 2014 at 10:57 PM, Roberto Spadim wrote:

...

just to understand... --- the solve_equation part, today only used to save information: std::ofstream datafile; char file_name[100]; my_snprintf(file_name, 100, "/tmp/mariadb_cost_coefficients_%lu.txt", thread_id); datafile.open(file_name, std::ios::app); for(int i=0; i < MAX_CONSTANTS; i++) datafile << coefficients[i].value << " "; datafile << total_time << "\n"; datafile.close(); ----

the idea is: given a query and some coefficients[i].value, you got total_time need to execute the query you want to "train" something to tell you how many time the same query should execute? or, what's the "x[i]" variables from your system (hardware/hard disk/etc), and extend this to others queries?

2014-07-08 14:20 GMT-03:00 Roberto Spadim :

...

=] nice

2014-07-08 14:18 GMT-03:00 Anshu Avinash :

...

Hi all,

You can download it here

(https://drive.google.com/file/d/0B7NiQb4EbbUVNVJFZ2xkRVR3Ylk/edit?usp=sharin...). It is around 26M. I have added the link on blog too.

Regards Anshu

On Tue, Jul 8, 2014 at 10:38 PM, Roberto Spadim wrote:

...

could you 'display' the dataset you used with octave?

2014-07-08 13:55 GMT-03:00 Anshu Avinash : > Hi all, > > This week's blog post is at: > http://igniting.in/gsoc2014/2014/07/08/solving-linear-equations/ . > Sorry > for > the delay. > Suggestions for an approach to solve the system of linear equations > are > welcome. > > Regards > Anshu Avinash > > > On Mon, Jun 23, 2014 at 7:39 PM, Roberto Spadim > > wrote: >> >> " MDEV. " >> it's nice to put full name (MDEV-350), since google and others >> search >> engines help when someone try to find information about mdev 350 >> >> text is ok :) >> >> 2014-06-23 11:04 GMT-03:00 Anshu Avinash >> : >> > Hi, >> > >> > Sorry for the confusion, this is the new link: >> > http://igniting.in/gsoc2014/2014/06/23/work-before-mid-term/ >> > Thanks for pointing out. >> > >> > Regards >> > Anshu >> > >> > >> > On Mon, Jun 23, 2014 at 7:32 PM, Roberto Spadim >> > >> > wrote: >> >> >> >> "Sorry this page does not exist =(" >> >> >> >> 2014-06-23 8:07 GMT-03:00 Anshu Avinash >> >> : >> >> > Hi all, >> >> > >> >> > You can find this week's blog entry at: >> >> > http://igniting.in/2014/06/23/work-before-mid-term/ >> >> > Suggestions/reviews are welcome. >> >> > >> >> > Regards >> >> > Anshu Avinash >> >> > >> >> > >> >> > On Mon, Jun 9, 2014 at 7:30 PM, Roberto Spadim >> >> > >> >> > wrote: >> >> >> >> >> >> Well i wws reading your posts >> >> >> Do you need big data to test read and scan times? >> >> >> >> >> >> Em segunda-feira, 9 de junho de 2014, Anshu Avinash >> >> >> escreveu: >> >> >> >> >> >>> Hi all, >> >> >>> >> >> >>> You can find this week's blog entry at >> >> >>> http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm now >> >> >>> maintaining the >> >> >>> code only on github: >> >> >>> https://github.com/igniting/server/tree/selfTuningOptimizer. >> >> >>> >> >> >>> Regards >> >> >>> Anshu Avinash >> >> >>> >> >> >>> >> >> >>> On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash >> >> >>> wrote: >> >> >>> >> >> >>> Hi all, >> >> >>> >> >> >>> You can find my this week's blog entry at >> >> >>> http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ . I >> >> >>> have >> >> >>> created a >> >> >>> branch on launchpad for my work: >> >> >>> >> >> >>> http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 >> >> >>> . >> >> >>> You >> >> >>> can >> >> >>> give your suggestions/reviews either on this thread or as a >> >> >>> comment >> >> >>> on >> >> >>> the >> >> >>> blog itself. >> >> >>> >> >> >>> Regards >> >> >>> Anshu Avinash >> >> >>> >> >> >>> >> >> >>> On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim >> >> >>> >> >> >>> wrote: >> >> >>> >> >> >>> wow a big work, congratulation guy, i will read part by part >> >> >>> to >> >> >>> better >> >> >>> understand mariadb code >> >> >>> >> >> >>> >> >> >>> 2014-05-19 16:33 GMT-03:00 Anshu Avinash >> >> >>> : >> >> >>> >> >> >>> Hi all, >> >> >>> >> >> >>> This week's blog entry would get delayed by couple of days. >> >> >>> I >> >> >>> have >> >> >>> started coding though and would like to give heads up on >> >> >>> what >> >> >>> I'm >> >> >>> doing. >> >> >>> >> >> >>> I've looked at the diffs for "Cost model project" of mysql: >> >> >>> >> >> >>> >> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 >> >> >>> and >> >> >>> >> >> >>> >> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . >> >> >>> These >> >> >>> give a pretty good idea about what are the hard-coded >> >> >>> constants >> >> >>> and >> >> >>> where >> >> >>> are they being used. >> >> >>> >> >> >>> The idea is to multiply "READ_TIME_FACTOR" and >> >> >>> "SCAN_TIME_FACTOR" >> >> >>> to >> >> >>> the >> >> >>> values returned by read_time() and scan_time() in handler.h, >> >> >>> while >> >> >>> returning. These values would be read from a table in mysql >> >> >>> db. >> >> >>> For >> >> >>> that >> >> >>> I've looked at sql_statistics.cc. After completing this, >> >> >>> I'll >> >> >>> first >> >> >>> change >> >> >>> the values of these constants manually and check if the >> >> >>> better >> >> >>> or >> >> >>> worse >> >> >>> query plans are being selected. I'll first do the last step >> >> >>> manually, >> >> >>> to >> >> >>> check if everything is working as expected and later >> >> >>> automate >> >> >>> it. >> >> >>> >> >> >>> Regards >> >> >>> Anshu >> >> >>> >> >> >>> >> >> >>> On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash >> >> >>> wrote: >> >> >>> >> >> >>> Hi all, >> >> >>> >> >> >>> You can find my blog entry for this week at >> >> >>> http://igniting.in/gsoc2014/2014/05/11/first-steps/ . >> >> >>> >> >> >>> Regards >> >> >>> Anshu Avinash >> >> >>> >> >> >>> >> >> >>> On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash >> >> >>> wrote: >> >> >>> >> >> >>> Hi all, >> >> >>> >> >> >>> Sorry for the irregular updates. I had been busy for last >> >> >>> couple >> >> >>> of >> >> >>> days >> >> >>> and might still be busy for 1-2 days more. I would be >> >> >>> completely >> >> >>> free >> >> >>> starting next week, and would be updating my blog weekly on >> >> >>> every >> >> >>> Monday (so >> >> >>> 1st update would be on May 12). I would also send the link >> >> >>> of my >> >> >>> post >> >> >>> weekly >> >> >>> on the mailing list. >> >> >>> >> >> >>> As discussed on irc, I started to explore the pair of >> >> >>> constants: >> >> >>> handler::scan_time() and handler::read_time(). >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Roberto Spadim >> >> >> SPAEmpresarial >> >> >> Eng. Automação e Controle >> >> >> >> >> > >> >> >> >> >> >> >> >> -- >> >> Roberto Spadim >> >> SPAEmpresarial >> >> Eng. Automação e Controle >> > >> > >> >> >> >> -- >> Roberto Spadim >> SPAEmpresarial >> Eng. Automação e Controle > >

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

a1,a2 is what your solve equation function is saving? and you want know t1,t2,t3..t130, to understand how much time each 'read function' take, that's it? doing this, what's the next step? this is a start point to select what's better? index vs table scan? 2014-07-08 15:54 GMT-03:00 Anshu Avinash :

Hi,

a1, a2, ..., a130 are coefficients. t1, t2, .., t130 are unknowns. We need 130 linearly independent equations to solve for these variables. We can never get 130 linearly independent equations as some of the coefficients would be 0 every time. Hence, we get an approximate solution by forming an overdetermined system (http://en.wikipedia.org/wiki/Overdetermined_system). Let me know if you have any further doubts.

Regards Anshu Avinash

On Wed, Jul 9, 2014 at 12:18 AM, Roberto Spadim wrote:

...

ops, linear equation

2014-07-08 15:47 GMT-03:00 Roberto Spadim :

...

going back... a1t1 + a2t2 + … + a130t130= ttotal

a1, t1...

a1 is something you don't know t1 is the coefficients[i]?

it's a first order equation, right?

2014-07-08 15:20 GMT-03:00 Anshu Avinash :

...

Hi,

The idea is we know the total time the query took, and how many times an operation was performed. For example, consider the case of 'read_time'. We know how many times an index read took place, but don't know how much time does it take to do an index read. By solving these equations, we are trying to find out time for individual operations. coefficients[i].value is `how many time the operation i took place in a single query.`

Hope this clears things up.

Regards Anshu Avinash

On Tue, Jul 8, 2014 at 10:57 PM, Roberto Spadim wrote:

...

just to understand... --- the solve_equation part, today only used to save information: std::ofstream datafile; char file_name[100]; my_snprintf(file_name, 100, "/tmp/mariadb_cost_coefficients_%lu.txt", thread_id); datafile.open(file_name, std::ios::app); for(int i=0; i < MAX_CONSTANTS; i++) datafile << coefficients[i].value << " "; datafile << total_time << "\n"; datafile.close(); ----

the idea is: given a query and some coefficients[i].value, you got total_time need to execute the query you want to "train" something to tell you how many time the same query should execute? or, what's the "x[i]" variables from your system (hardware/hard disk/etc), and extend this to others queries?

2014-07-08 14:20 GMT-03:00 Roberto Spadim :

...

=] nice

2014-07-08 14:18 GMT-03:00 Anshu Avinash : > Hi all, > > You can download it here > > > (https://drive.google.com/file/d/0B7NiQb4EbbUVNVJFZ2xkRVR3Ylk/edit?usp=sharin...). > It is around 26M. I have added the link on blog too. > > Regards > Anshu > > > On Tue, Jul 8, 2014 at 10:38 PM, Roberto Spadim > > wrote: >> >> could you 'display' the dataset you used with octave? >> >> 2014-07-08 13:55 GMT-03:00 Anshu Avinash >> : >> > Hi all, >> > >> > This week's blog post is at: >> > http://igniting.in/gsoc2014/2014/07/08/solving-linear-equations/ >> > . >> > Sorry >> > for >> > the delay. >> > Suggestions for an approach to solve the system of linear >> > equations >> > are >> > welcome. >> > >> > Regards >> > Anshu Avinash >> > >> > >> > On Mon, Jun 23, 2014 at 7:39 PM, Roberto Spadim >> > >> > wrote: >> >> >> >> " MDEV. " >> >> it's nice to put full name (MDEV-350), since google and others >> >> search >> >> engines help when someone try to find information about mdev >> >> 350 >> >> >> >> text is ok :) >> >> >> >> 2014-06-23 11:04 GMT-03:00 Anshu Avinash >> >> : >> >> > Hi, >> >> > >> >> > Sorry for the confusion, this is the new link: >> >> > http://igniting.in/gsoc2014/2014/06/23/work-before-mid-term/ >> >> > Thanks for pointing out. >> >> > >> >> > Regards >> >> > Anshu >> >> > >> >> > >> >> > On Mon, Jun 23, 2014 at 7:32 PM, Roberto Spadim >> >> > >> >> > wrote: >> >> >> >> >> >> "Sorry this page does not exist =(" >> >> >> >> >> >> 2014-06-23 8:07 GMT-03:00 Anshu Avinash >> >> >> : >> >> >> > Hi all, >> >> >> > >> >> >> > You can find this week's blog entry at: >> >> >> > http://igniting.in/2014/06/23/work-before-mid-term/ >> >> >> > Suggestions/reviews are welcome. >> >> >> > >> >> >> > Regards >> >> >> > Anshu Avinash >> >> >> > >> >> >> > >> >> >> > On Mon, Jun 9, 2014 at 7:30 PM, Roberto Spadim >> >> >> > >> >> >> > wrote: >> >> >> >> >> >> >> >> Well i wws reading your posts >> >> >> >> Do you need big data to test read and scan times? >> >> >> >> >> >> >> >> Em segunda-feira, 9 de junho de 2014, Anshu Avinash >> >> >> >> escreveu: >> >> >> >> >> >> >> >>> Hi all, >> >> >> >>> >> >> >> >>> You can find this week's blog entry at >> >> >> >>> http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm >> >> >> >>> now >> >> >> >>> maintaining the >> >> >> >>> code only on github: >> >> >> >>> >> >> >> >>> https://github.com/igniting/server/tree/selfTuningOptimizer. >> >> >> >>> >> >> >> >>> Regards >> >> >> >>> Anshu Avinash >> >> >> >>> >> >> >> >>> >> >> >> >>> On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash >> >> >> >>> wrote: >> >> >> >>> >> >> >> >>> Hi all, >> >> >> >>> >> >> >> >>> You can find my this week's blog entry at >> >> >> >>> http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ >> >> >> >>> . I >> >> >> >>> have >> >> >> >>> created a >> >> >> >>> branch on launchpad for my work: >> >> >> >>> >> >> >> >>> >> >> >> >>> http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 >> >> >> >>> . >> >> >> >>> You >> >> >> >>> can >> >> >> >>> give your suggestions/reviews either on this thread or >> >> >> >>> as a >> >> >> >>> comment >> >> >> >>> on >> >> >> >>> the >> >> >> >>> blog itself. >> >> >> >>> >> >> >> >>> Regards >> >> >> >>> Anshu Avinash >> >> >> >>> >> >> >> >>> >> >> >> >>> On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim >> >> >> >>> >> >> >> >>> wrote: >> >> >> >>> >> >> >> >>> wow a big work, congratulation guy, i will read part by >> >> >> >>> part >> >> >> >>> to >> >> >> >>> better >> >> >> >>> understand mariadb code >> >> >> >>> >> >> >> >>> >> >> >> >>> 2014-05-19 16:33 GMT-03:00 Anshu Avinash >> >> >> >>> : >> >> >> >>> >> >> >> >>> Hi all, >> >> >> >>> >> >> >> >>> This week's blog entry would get delayed by couple of >> >> >> >>> days. >> >> >> >>> I >> >> >> >>> have >> >> >> >>> started coding though and would like to give heads up on >> >> >> >>> what >> >> >> >>> I'm >> >> >> >>> doing. >> >> >> >>> >> >> >> >>> I've looked at the diffs for "Cost model project" of >> >> >> >>> mysql: >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 >> >> >> >>> and >> >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . >> >> >> >>> These >> >> >> >>> give a pretty good idea about what are the hard-coded >> >> >> >>> constants >> >> >> >>> and >> >> >> >>> where >> >> >> >>> are they being used. >> >> >> >>> >> >> >> >>> The idea is to multiply "READ_TIME_FACTOR" and >> >> >> >>> "SCAN_TIME_FACTOR" >> >> >> >>> to >> >> >> >>> the >> >> >> >>> values returned by read_time() and scan_time() in >> >> >> >>> handler.h, >> >> >> >>> while >> >> >> >>> returning. These values would be read from a table in >> >> >> >>> mysql >> >> >> >>> db. >> >> >> >>> For >> >> >> >>> that >> >> >> >>> I've looked at sql_statistics.cc. After completing this, >> >> >> >>> I'll >> >> >> >>> first >> >> >> >>> change >> >> >> >>> the values of these constants manually and check if the >> >> >> >>> better >> >> >> >>> or >> >> >> >>> worse >> >> >> >>> query plans are being selected. I'll first do the last >> >> >> >>> step >> >> >> >>> manually, >> >> >> >>> to >> >> >> >>> check if everything is working as expected and later >> >> >> >>> automate >> >> >> >>> it. >> >> >> >>> >> >> >> >>> Regards >> >> >> >>> Anshu >> >> >> >>> >> >> >> >>> >> >> >> >>> On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash >> >> >> >>> wrote: >> >> >> >>> >> >> >> >>> Hi all, >> >> >> >>> >> >> >> >>> You can find my blog entry for this week at >> >> >> >>> http://igniting.in/gsoc2014/2014/05/11/first-steps/ . >> >> >> >>> >> >> >> >>> Regards >> >> >> >>> Anshu Avinash >> >> >> >>> >> >> >> >>> >> >> >> >>> On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash >> >> >> >>> wrote: >> >> >> >>> >> >> >> >>> Hi all, >> >> >> >>> >> >> >> >>> Sorry for the irregular updates. I had been busy for >> >> >> >>> last >> >> >> >>> couple >> >> >> >>> of >> >> >> >>> days >> >> >> >>> and might still be busy for 1-2 days more. I would be >> >> >> >>> completely >> >> >> >>> free >> >> >> >>> starting next week, and would be updating my blog weekly >> >> >> >>> on >> >> >> >>> every >> >> >> >>> Monday (so >> >> >> >>> 1st update would be on May 12). I would also send the >> >> >> >>> link >> >> >> >>> of my >> >> >> >>> post >> >> >> >>> weekly >> >> >> >>> on the mailing list. >> >> >> >>> >> >> >> >>> As discussed on irc, I started to explore the pair of >> >> >> >>> constants: >> >> >> >>> handler::scan_time() and handler::read_time(). >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> >> Roberto Spadim >> >> >> >> SPAEmpresarial >> >> >> >> Eng. Automação e Controle >> >> >> >> >> >> >> > >> >> >> >> >> >> >> >> >> >> >> >> -- >> >> >> Roberto Spadim >> >> >> SPAEmpresarial >> >> >> Eng. Automação e Controle >> >> > >> >> > >> >> >> >> >> >> >> >> -- >> >> Roberto Spadim >> >> SPAEmpresarial >> >> Eng. Automação e Controle >> > >> > >> >> >> >> -- >> Roberto Spadim >> SPAEmpresarial >> Eng. Automação e Controle > >

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

http://www.netlib.org/lapack/lapacke.html a C api to lapack 2014-07-08 16:23 GMT-03:00 Roberto Spadim :

check lapack lib, i used it some years ago, and it solve linear equations, at least you don't waste time with 'how to solve linear equations', if you want to study :) lapack was a nice lib, at least i used without problems

2014-07-08 16:04 GMT-03:00 Roberto Spadim :

...

a1,a2 is what your solve equation function is saving? and you want know t1,t2,t3..t130, to understand how much time each 'read function' take, that's it?

doing this, what's the next step? this is a start point to select what's better? index vs table scan?

2014-07-08 15:54 GMT-03:00 Anshu Avinash :

...

Hi,

a1, a2, ..., a130 are coefficients. t1, t2, .., t130 are unknowns. We need 130 linearly independent equations to solve for these variables. We can never get 130 linearly independent equations as some of the coefficients would be 0 every time. Hence, we get an approximate solution by forming an overdetermined system (http://en.wikipedia.org/wiki/Overdetermined_system). Let me know if you have any further doubts.

Regards Anshu Avinash

On Wed, Jul 9, 2014 at 12:18 AM, Roberto Spadim wrote:

...

ops, linear equation

2014-07-08 15:47 GMT-03:00 Roberto Spadim :

...

going back... a1t1 + a2t2 + … + a130t130= ttotal

a1, t1...

a1 is something you don't know t1 is the coefficients[i]?

it's a first order equation, right?

2014-07-08 15:20 GMT-03:00 Anshu Avinash :

...

Hi,

The idea is we know the total time the query took, and how many times an operation was performed. For example, consider the case of 'read_time'. We know how many times an index read took place, but don't know how much time does it take to do an index read. By solving these equations, we are trying to find out time for individual operations. coefficients[i].value is `how many time the operation i took place in a single query.`

Hope this clears things up.

Regards Anshu Avinash

On Tue, Jul 8, 2014 at 10:57 PM, Roberto Spadim wrote: > > just to understand... > --- the solve_equation part, today only used to save information: > std::ofstream datafile; > char file_name[100]; > my_snprintf(file_name, 100, > "/tmp/mariadb_cost_coefficients_%lu.txt", thread_id); > datafile.open(file_name, std::ios::app); > for(int i=0; i < MAX_CONSTANTS; i++) > datafile << coefficients[i].value << " "; > datafile << total_time << "\n"; > datafile.close(); > ---- > > the idea is: given a query and some coefficients[i].value, you got > total_time need to execute the query > you want to "train" something to tell you how many time the same query > should execute? > or, what's the "x[i]" variables from your system (hardware/hard > disk/etc), and extend this to others queries? > > > 2014-07-08 14:20 GMT-03:00 Roberto Spadim : > > =] nice > > > > 2014-07-08 14:18 GMT-03:00 Anshu Avinash > > : > >> Hi all, > >> > >> You can download it here > >> > >> > >> (https://drive.google.com/file/d/0B7NiQb4EbbUVNVJFZ2xkRVR3Ylk/edit?usp=sharin...). > >> It is around 26M. I have added the link on blog too. > >> > >> Regards > >> Anshu > >> > >> > >> On Tue, Jul 8, 2014 at 10:38 PM, Roberto Spadim > >> > >> wrote: > >>> > >>> could you 'display' the dataset you used with octave? > >>> > >>> 2014-07-08 13:55 GMT-03:00 Anshu Avinash > >>> : > >>> > Hi all, > >>> > > >>> > This week's blog post is at: > >>> > http://igniting.in/gsoc2014/2014/07/08/solving-linear-equations/ > >>> > . > >>> > Sorry > >>> > for > >>> > the delay. > >>> > Suggestions for an approach to solve the system of linear > >>> > equations > >>> > are > >>> > welcome. > >>> > > >>> > Regards > >>> > Anshu Avinash > >>> > > >>> > > >>> > On Mon, Jun 23, 2014 at 7:39 PM, Roberto Spadim > >>> > > >>> > wrote: > >>> >> > >>> >> " MDEV. " > >>> >> it's nice to put full name (MDEV-350), since google and others > >>> >> search > >>> >> engines help when someone try to find information about mdev > >>> >> 350 > >>> >> > >>> >> text is ok :) > >>> >> > >>> >> 2014-06-23 11:04 GMT-03:00 Anshu Avinash > >>> >> : > >>> >> > Hi, > >>> >> > > >>> >> > Sorry for the confusion, this is the new link: > >>> >> > http://igniting.in/gsoc2014/2014/06/23/work-before-mid-term/ > >>> >> > Thanks for pointing out. > >>> >> > > >>> >> > Regards > >>> >> > Anshu > >>> >> > > >>> >> > > >>> >> > On Mon, Jun 23, 2014 at 7:32 PM, Roberto Spadim > >>> >> > > >>> >> > wrote: > >>> >> >> > >>> >> >> "Sorry this page does not exist =(" > >>> >> >> > >>> >> >> 2014-06-23 8:07 GMT-03:00 Anshu Avinash > >>> >> >> : > >>> >> >> > Hi all, > >>> >> >> > > >>> >> >> > You can find this week's blog entry at: > >>> >> >> > http://igniting.in/2014/06/23/work-before-mid-term/ > >>> >> >> > Suggestions/reviews are welcome. > >>> >> >> > > >>> >> >> > Regards > >>> >> >> > Anshu Avinash > >>> >> >> > > >>> >> >> > > >>> >> >> > On Mon, Jun 9, 2014 at 7:30 PM, Roberto Spadim > >>> >> >> > > >>> >> >> > wrote: > >>> >> >> >> > >>> >> >> >> Well i wws reading your posts > >>> >> >> >> Do you need big data to test read and scan times? > >>> >> >> >> > >>> >> >> >> Em segunda-feira, 9 de junho de 2014, Anshu Avinash > >>> >> >> >> escreveu: > >>> >> >> >> > >>> >> >> >>> Hi all, > >>> >> >> >>> > >>> >> >> >>> You can find this week's blog entry at > >>> >> >> >>> http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm > >>> >> >> >>> now > >>> >> >> >>> maintaining the > >>> >> >> >>> code only on github: > >>> >> >> >>> > >>> >> >> >>> https://github.com/igniting/server/tree/selfTuningOptimizer. > >>> >> >> >>> > >>> >> >> >>> Regards > >>> >> >> >>> Anshu Avinash > >>> >> >> >>> > >>> >> >> >>> > >>> >> >> >>> On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash > >>> >> >> >>> wrote: > >>> >> >> >>> > >>> >> >> >>> Hi all, > >>> >> >> >>> > >>> >> >> >>> You can find my this week's blog entry at > >>> >> >> >>> http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ > >>> >> >> >>> . I > >>> >> >> >>> have > >>> >> >> >>> created a > >>> >> >> >>> branch on launchpad for my work: > >>> >> >> >>> > >>> >> >> >>> > >>> >> >> >>> http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 > >>> >> >> >>> . > >>> >> >> >>> You > >>> >> >> >>> can > >>> >> >> >>> give your suggestions/reviews either on this thread or > >>> >> >> >>> as a > >>> >> >> >>> comment > >>> >> >> >>> on > >>> >> >> >>> the > >>> >> >> >>> blog itself. > >>> >> >> >>> > >>> >> >> >>> Regards > >>> >> >> >>> Anshu Avinash > >>> >> >> >>> > >>> >> >> >>> > >>> >> >> >>> On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim > >>> >> >> >>> > >>> >> >> >>> wrote: > >>> >> >> >>> > >>> >> >> >>> wow a big work, congratulation guy, i will read part by > >>> >> >> >>> part > >>> >> >> >>> to > >>> >> >> >>> better > >>> >> >> >>> understand mariadb code > >>> >> >> >>> > >>> >> >> >>> > >>> >> >> >>> 2014-05-19 16:33 GMT-03:00 Anshu Avinash > >>> >> >> >>> : > >>> >> >> >>> > >>> >> >> >>> Hi all, > >>> >> >> >>> > >>> >> >> >>> This week's blog entry would get delayed by couple of > >>> >> >> >>> days. > >>> >> >> >>> I > >>> >> >> >>> have > >>> >> >> >>> started coding though and would like to give heads up on > >>> >> >> >>> what > >>> >> >> >>> I'm > >>> >> >> >>> doing. > >>> >> >> >>> > >>> >> >> >>> I've looked at the diffs for "Cost model project" of > >>> >> >> >>> mysql: > >>> >> >> >>> > >>> >> >> >>> > >>> >> >> >>> > >>> >> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 > >>> >> >> >>> and > >>> >> >> >>> > >>> >> >> >>> > >>> >> >> >>> > >>> >> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . > >>> >> >> >>> These > >>> >> >> >>> give a pretty good idea about what are the hard-coded > >>> >> >> >>> constants > >>> >> >> >>> and > >>> >> >> >>> where > >>> >> >> >>> are they being used. > >>> >> >> >>> > >>> >> >> >>> The idea is to multiply "READ_TIME_FACTOR" and > >>> >> >> >>> "SCAN_TIME_FACTOR" > >>> >> >> >>> to > >>> >> >> >>> the > >>> >> >> >>> values returned by read_time() and scan_time() in > >>> >> >> >>> handler.h, > >>> >> >> >>> while > >>> >> >> >>> returning. These values would be read from a table in > >>> >> >> >>> mysql > >>> >> >> >>> db. > >>> >> >> >>> For > >>> >> >> >>> that > >>> >> >> >>> I've looked at sql_statistics.cc. After completing this, > >>> >> >> >>> I'll > >>> >> >> >>> first > >>> >> >> >>> change > >>> >> >> >>> the values of these constants manually and check if the > >>> >> >> >>> better > >>> >> >> >>> or > >>> >> >> >>> worse > >>> >> >> >>> query plans are being selected. I'll first do the last > >>> >> >> >>> step > >>> >> >> >>> manually, > >>> >> >> >>> to > >>> >> >> >>> check if everything is working as expected and later > >>> >> >> >>> automate > >>> >> >> >>> it. > >>> >> >> >>> > >>> >> >> >>> Regards > >>> >> >> >>> Anshu > >>> >> >> >>> > >>> >> >> >>> > >>> >> >> >>> On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash > >>> >> >> >>> wrote: > >>> >> >> >>> > >>> >> >> >>> Hi all, > >>> >> >> >>> > >>> >> >> >>> You can find my blog entry for this week at > >>> >> >> >>> http://igniting.in/gsoc2014/2014/05/11/first-steps/ . > >>> >> >> >>> > >>> >> >> >>> Regards > >>> >> >> >>> Anshu Avinash > >>> >> >> >>> > >>> >> >> >>> > >>> >> >> >>> On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash > >>> >> >> >>> wrote: > >>> >> >> >>> > >>> >> >> >>> Hi all, > >>> >> >> >>> > >>> >> >> >>> Sorry for the irregular updates. I had been busy for > >>> >> >> >>> last > >>> >> >> >>> couple > >>> >> >> >>> of > >>> >> >> >>> days > >>> >> >> >>> and might still be busy for 1-2 days more. I would be > >>> >> >> >>> completely > >>> >> >> >>> free > >>> >> >> >>> starting next week, and would be updating my blog weekly > >>> >> >> >>> on > >>> >> >> >>> every > >>> >> >> >>> Monday (so > >>> >> >> >>> 1st update would be on May 12). I would also send the > >>> >> >> >>> link > >>> >> >> >>> of my > >>> >> >> >>> post > >>> >> >> >>> weekly > >>> >> >> >>> on the mailing list. > >>> >> >> >>> > >>> >> >> >>> As discussed on irc, I started to explore the pair of > >>> >> >> >>> constants: > >>> >> >> >>> handler::scan_time() and handler::read_time(). > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> > >>> >> >> >> -- > >>> >> >> >> Roberto Spadim > >>> >> >> >> SPAEmpresarial > >>> >> >> >> Eng. Automação e Controle > >>> >> >> >> > >>> >> >> > > >>> >> >> > >>> >> >> > >>> >> >> > >>> >> >> -- > >>> >> >> Roberto Spadim > >>> >> >> SPAEmpresarial > >>> >> >> Eng. Automação e Controle > >>> >> > > >>> >> > > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> Roberto Spadim > >>> >> SPAEmpresarial > >>> >> Eng. Automação e Controle > >>> > > >>> > > >>> > >>> > >>> > >>> -- > >>> Roberto Spadim > >>> SPAEmpresarial > >>> Eng. Automação e Controle > >> > >> > > > > > > > > -- > > Roberto Spadim > > SPAEmpresarial > > Eng. Automação e Controle > > > > -- > Roberto Spadim > SPAEmpresarial > Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

Hi all, Here is a blog post describing my progress so far: http://igniting.in/gsoc2014/2014/08/04/progress-so-far/ Comments and suggestions are welcome. Regards Anshu Avinash On Wed, Jul 9, 2014 at 1:14 AM, Roberto Spadim wrote:

please include the output (solutions.txt) link too, to check what happened

...

http://www.netlib.org/lapack/lapacke.html a C api to lapack

2014-07-08 16:23 GMT-03:00 Roberto Spadim :

...

check lapack lib, i used it some years ago, and it solve linear equations, at least you don't waste time with 'how to solve linear equations', if you want to study :) lapack was a nice lib, at least i used without problems

2014-07-08 16:04 GMT-03:00 Roberto Spadim :

...

a1,a2 is what your solve equation function is saving? and you want know t1,t2,t3..t130, to understand how much time each 'read function' take, that's it?

doing this, what's the next step? this is a start point to select what's better? index vs table scan?

2014-07-08 15:54 GMT-03:00 Anshu Avinash :

...

Hi,

a1, a2, ..., a130 are coefficients. t1, t2, .., t130 are unknowns. We need 130 linearly independent equations to solve for these variables. We can never get 130 linearly independent equations as some of the coefficients would be 0 every time. Hence, we get an approximate solution by

...

...

...

...

overdetermined system ( http://en.wikipedia.org/wiki/Overdetermined_system). Let me know if you have any further doubts.

Regards Anshu Avinash

On Wed, Jul 9, 2014 at 12:18 AM, Roberto Spadim < roberto@spadim.com.br> wrote:

...

ops, linear equation

2014-07-08 15:47 GMT-03:00 Roberto Spadim : > going back... > a1t1 + a2t2 + … + a130t130= ttotal > > a1, t1... > > a1 is something you don't know > t1 is the coefficients[i]? > > it's a first order equation, right? > > > > 2014-07-08 15:20 GMT-03:00 Anshu Avinash <

anshu.avinash35@gmail.com>:

...

>> Hi, >> >> The idea is we know the total time the query took, and how many times >> an >> operation was performed. For example, consider the case of 'read_time'. >> We >> know how many times an index read took place, but don't know how much >> time >> does it take to do an index read. By solving these equations, we are >> trying >> to find out time for individual operations. coefficients[i].value is >> `how >> many time the operation i took place in a single query.` >> >> Hope this clears things up. >> >> Regards >> Anshu Avinash >> >> >> On Tue, Jul 8, 2014 at 10:57 PM, Roberto Spadim < roberto@spadim.com.br> >> wrote: >>> >>> just to understand... >>> --- the solve_equation part, today only used to save information: >>> std::ofstream datafile; >>> char file_name[100]; >>> my_snprintf(file_name, 100, >>> "/tmp/mariadb_cost_coefficients_%lu.txt", thread_id); >>> datafile.open(file_name, std::ios::app); >>> for(int i=0; i < MAX_CONSTANTS; i++) >>> datafile << coefficients[i].value << " "; >>> datafile << total_time << "\n"; >>> datafile.close(); >>> ---- >>> >>> the idea is: given a query and some coefficients[i].value, you got >>> total_time need to execute the query >>> you want to "train" something to tell you how many time the same query >>> should execute? >>> or, what's the "x[i]" variables from your system (hardware/hard >>> disk/etc), and extend this to others queries? >>> >>> >>> 2014-07-08 14:20 GMT-03:00 Roberto Spadim >> > =] nice >>> > >>> > 2014-07-08 14:18 GMT-03:00 Anshu Avinash >>> > : >>> >> Hi all, >>> >> >>> >> You can download it here >>> >> >>> >> >>> >> ( https://drive.google.com/file/d/0B7NiQb4EbbUVNVJFZ2xkRVR3Ylk/edit?usp=sharin... ). >>> >> It is around 26M. I have added the link on blog too. >>> >> >>> >> Regards >>> >> Anshu >>> >> >>> >> >>> >> On Tue, Jul 8, 2014 at 10:38 PM, Roberto Spadim >>> >> >>> >> wrote: >>> >>> >>> >>> could you 'display' the dataset you used with octave? >>> >>> >>> >>> 2014-07-08 13:55 GMT-03:00 Anshu Avinash >>> >>> : >>> >>> > Hi all, >>> >>> > >>> >>> > This week's blog post is at: >>> >>> > http://igniting.in/gsoc2014/2014/07/08/solving-linear-equations/ >>> >>> > . >>> >>> > Sorry >>> >>> > for >>> >>> > the delay. >>> >>> > Suggestions for an approach to solve the system of linear >>> >>> > equations >>> >>> > are >>> >>> > welcome. >>> >>> > >>> >>> > Regards >>> >>> > Anshu Avinash >>> >>> > >>> >>> > >>> >>> > On Mon, Jun 23, 2014 at 7:39 PM, Roberto Spadim >>> >>> > >>> >>> > wrote: >>> >>> >> >>> >>> >> " MDEV. " >>> >>> >> it's nice to put full name (MDEV-350), since google and others >>> >>> >> search >>> >>> >> engines help when someone try to find information about mdev >>> >>> >> 350 >>> >>> >> >>> >>> >> text is ok :) >>> >>> >> >>> >>> >> 2014-06-23 11:04 GMT-03:00 Anshu Avinash >>> >>> >> : >>> >>> >> > Hi, >>> >>> >> > >>> >>> >> > Sorry for the confusion, this is the new link: >>> >>> >> > http://igniting.in/gsoc2014/2014/06/23/work-before-mid-term/ >>> >>> >> > Thanks for pointing out. >>> >>> >> > >>> >>> >> > Regards >>> >>> >> > Anshu >>> >>> >> > >>> >>> >> > >>> >>> >> > On Mon, Jun 23, 2014 at 7:32 PM, Roberto Spadim >>> >>> >> > >>> >>> >> > wrote: >>> >>> >> >> >>> >>> >> >> "Sorry this page does not exist =(" >>> >>> >> >> >>> >>> >> >> 2014-06-23 8:07 GMT-03:00 Anshu Avinash >>> >>> >> >> : >>> >>> >> >> > Hi all, >>> >>> >> >> > >>> >>> >> >> > You can find this week's blog entry at: >>> >>> >> >> > http://igniting.in/2014/06/23/work-before-mid-term/ >>> >>> >> >> > Suggestions/reviews are welcome. >>> >>> >> >> > >>> >>> >> >> > Regards >>> >>> >> >> > Anshu Avinash >>> >>> >> >> > >>> >>> >> >> > >>> >>> >> >> > On Mon, Jun 9, 2014 at 7:30 PM, Roberto Spadim >>> >>> >> >> > >>> >>> >> >> > wrote: >>> >>> >> >> >> >>> >>> >> >> >> Well i wws reading your posts >>> >>> >> >> >> Do you need big data to test read and scan times? >>> >>> >> >> >> >>> >>> >> >> >> Em segunda-feira, 9 de junho de 2014, Anshu Avinash >>> >>> >> >> >> escreveu: >>> >>> >> >> >> >>> >>> >> >> >>> Hi all, >>> >>> >> >> >>> >>> >>> >> >> >>> You can find this week's blog entry at >>> >>> >> >> >>> http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm >>> >>> >> >> >>> now >>> >>> >> >> >>> maintaining the >>> >>> >> >> >>> code only on github: >>> >>> >> >> >>> >>> >>> >> >> >>> https://github.com/igniting/server/tree/selfTuningOptimizer. >>> >>> >> >> >>> >>> >>> >> >> >>> Regards >>> >>> >> >> >>> Anshu Avinash >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash >>> >>> >> >> >>> wrote: >>> >>> >> >> >>> >>> >>> >> >> >>> Hi all, >>> >>> >> >> >>> >>> >>> >> >> >>> You can find my this week's blog entry at >>> >>> >> >> >>> http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ >>> >>> >> >> >>> . I >>> >>> >> >> >>> have >>> >>> >> >> >>> created a >>> >>> >> >> >>> branch on launchpad for my work: >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 >>> >>> >> >> >>> . >>> >>> >> >> >>> You >>> >>> >> >> >>> can >>> >>> >> >> >>> give your suggestions/reviews either on this

...

...

...

...

...

>>> >>> >> >> >>> as a >>> >>> >> >> >>> comment >>> >>> >> >> >>> on >>> >>> >> >> >>> the >>> >>> >> >> >>> blog itself. >>> >>> >> >> >>> >>> >>> >> >> >>> Regards >>> >>> >> >> >>> Anshu Avinash >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim >>> >>> >> >> >>> >>> >>> >> >> >>> wrote: >>> >>> >> >> >>> >>> >>> >> >> >>> wow a big work, congratulation guy, i will read

...

...

...

...

...

>>> >>> >> >> >>> part >>> >>> >> >> >>> to >>> >>> >> >> >>> better >>> >>> >> >> >>> understand mariadb code >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> 2014-05-19 16:33 GMT-03:00 Anshu Avinash >>> >>> >> >> >>> : >>> >>> >> >> >>> >>> >>> >> >> >>> Hi all, >>> >>> >> >> >>> >>> >>> >> >> >>> This week's blog entry would get delayed by couple of >>> >>> >> >> >>> days. >>> >>> >> >> >>> I >>> >>> >> >> >>> have >>> >>> >> >> >>> started coding though and would like to give heads up on >>> >>> >> >> >>> what >>> >>> >> >> >>> I'm >>> >>> >> >> >>> doing. >>> >>> >> >> >>> >>> >>> >> >> >>> I've looked at the diffs for "Cost model project" of >>> >>> >> >> >>> mysql: >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 >>> >>> >> >> >>> and >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . >>> >>> >> >> >>> These >>> >>> >> >> >>> give a pretty good idea about what are the hard-coded >>> >>> >> >> >>> constants >>> >>> >> >> >>> and >>> >>> >> >> >>> where >>> >>> >> >> >>> are they being used. >>> >>> >> >> >>> >>> >>> >> >> >>> The idea is to multiply "READ_TIME_FACTOR" and >>> >>> >> >> >>> "SCAN_TIME_FACTOR" >>> >>> >> >> >>> to >>> >>> >> >> >>> the >>> >>> >> >> >>> values returned by read_time() and scan_time() in >>> >>> >> >> >>> handler.h, >>> >>> >> >> >>> while >>> >>> >> >> >>> returning. These values would be read from a table in >>> >>> >> >> >>> mysql >>> >>> >> >> >>> db. >>> >>> >> >> >>> For >>> >>> >> >> >>> that >>> >>> >> >> >>> I've looked at sql_statistics.cc. After completing

...

...

...

...

...

>>> >>> >> >> >>> I'll >>> >>> >> >> >>> first >>> >>> >> >> >>> change >>> >>> >> >> >>> the values of these constants manually and check if the >>> >>> >> >> >>> better >>> >>> >> >> >>> or >>> >>> >> >> >>> worse >>> >>> >> >> >>> query plans are being selected. I'll first do the last >>> >>> >> >> >>> step >>> >>> >> >> >>> manually, >>> >>> >> >> >>> to >>> >>> >> >> >>> check if everything is working as expected and later >>> >>> >> >> >>> automate >>> >>> >> >> >>> it. >>> >>> >> >> >>> >>> >>> >> >> >>> Regards >>> >>> >> >> >>> Anshu >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash >>> >>> >> >> >>> wrote: >>> >>> >> >> >>> >>> >>> >> >> >>> Hi all, >>> >>> >> >> >>> >>> >>> >> >> >>> You can find my blog entry for this week at >>> >>> >> >> >>> http://igniting.in/gsoc2014/2014/05/11/first-steps/ . >>> >>> >> >> >>> >>> >>> >> >> >>> Regards >>> >>> >> >> >>> Anshu Avinash >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash >>> >>> >> >> >>> wrote: >>> >>> >> >> >>> >>> >>> >> >> >>> Hi all, >>> >>> >> >> >>> >>> >>> >> >> >>> Sorry for the irregular updates. I had been busy for >>> >>> >> >> >>> last >>> >>> >> >> >>> couple >>> >>> >> >> >>> of >>> >>> >> >> >>> days >>> >>> >> >> >>> and might still be busy for 1-2 days more. I would be >>> >>> >> >> >>> completely >>> >>> >> >> >>> free >>> >>> >> >> >>> starting next week, and would be updating my blog weekly >>> >>> >> >> >>> on >>> >>> >> >> >>> every >>> >>> >> >> >>> Monday (so >>> >>> >> >> >>> 1st update would be on May 12). I would also send

2014-07-08 16:24 GMT-03:00 Roberto Spadim : forming an thread or part by this, the

...

...

...

...

...

>>> >>> >> >> >>> link >>> >>> >> >> >>> of my >>> >>> >> >> >>> post >>> >>> >> >> >>> weekly >>> >>> >> >> >>> on the mailing list. >>> >>> >> >> >>> >>> >>> >> >> >>> As discussed on irc, I started to explore the pair of >>> >>> >> >> >>> constants: >>> >>> >> >> >>> handler::scan_time() and handler::read_time(). >>> >>> >> >> >> >>> >>> >> >> >> >>> >>> >> >> >> >>> >>> >> >> >> -- >>> >>> >> >> >> Roberto Spadim >>> >>> >> >> >> SPAEmpresarial >>> >>> >> >> >> Eng. Automação e Controle >>> >>> >> >> >> >>> >>> >> >> > >>> >>> >> >> >>> >>> >> >> >>> >>> >> >> >>> >>> >> >> -- >>> >>> >> >> Roberto Spadim >>> >>> >> >> SPAEmpresarial >>> >>> >> >> Eng. Automação e Controle >>> >>> >> > >>> >>> >> > >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> >> -- >>> >>> >> Roberto Spadim >>> >>> >> SPAEmpresarial >>> >>> >> Eng. Automação e Controle >>> >>> > >>> >>> > >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> Roberto Spadim >>> >>> SPAEmpresarial >>> >>> Eng. Automação e Controle >>> >> >>> >> >>> > >>> > >>> > >>> > -- >>> > Roberto Spadim >>> > SPAEmpresarial >>> > Eng. Automação e Controle >>> >>> >>> >>> -- >>> Roberto Spadim >>> SPAEmpresarial >>> Eng. Automação e Controle >> >> > > > > -- > Roberto Spadim > SPAEmpresarial > Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

if you have time, you could take a look at linear programming lpsolve is a well know library, i used it a lot some years ago, there's others libs but this one is well known, http://lpsolve.sourceforge.net/5.5/ maybe it give better results, instead of trying to solve a equation you execute a aproximation (there's many optional parameters and others flags about solving a linear problem with this lib) 2014-08-04 13:35 GMT-03:00 Roberto Spadim :

i didn't read the code yet, but some min/max problems can add restrictions about variables, for example you could set that variables are >=0 must check but well that's a nice step guy :) a nice work :)

2014-08-04 13:31 GMT-03:00 Anshu Avinash :

Hi all,

...

Here is a blog post describing my progress so far: http://igniting.in/gsoc2014/2014/08/04/progress-so-far/

Comments and suggestions are welcome.

Regards Anshu Avinash

On Wed, Jul 9, 2014 at 1:14 AM, Roberto Spadim wrote:

...

please include the output (solutions.txt) link too, to check what happened

...

http://www.netlib.org/lapack/lapacke.html a C api to lapack

2014-07-08 16:23 GMT-03:00 Roberto Spadim :

...

check lapack lib, i used it some years ago, and it solve linear equations, at least you don't waste time with 'how to solve linear equations', if you want to study :) lapack was a nice lib, at least i used without problems

2014-07-08 16:04 GMT-03:00 Roberto Spadim :

...

a1,a2 is what your solve equation function is saving? and you want know t1,t2,t3..t130, to understand how much time each 'read function' take, that's it?

doing this, what's the next step? this is a start point to select what's better? index vs table scan?

2014-07-08 15:54 GMT-03:00 Anshu Avinash Hi, > > a1, a2, ..., a130 are coefficients. t1, t2, .., t130 are unknowns. We need > 130 linearly independent equations to solve for these variables. We can > never get 130 linearly independent equations as some of the coefficients > would be 0 every time. Hence, we get an approximate solution by

...

...

...

> overdetermined system ( http://en.wikipedia.org/wiki/Overdetermined_system). > Let me know if you have any further doubts. > > Regards > Anshu Avinash > > > On Wed, Jul 9, 2014 at 12:18 AM, Roberto Spadim < roberto@spadim.com.br> > wrote: >> >> ops, linear equation >> >> 2014-07-08 15:47 GMT-03:00 Roberto Spadim : >> > going back... >> > a1t1 + a2t2 + … + a130t130= ttotal >> > >> > a1, t1... >> > >> > a1 is something you don't know >> > t1 is the coefficients[i]? >> > >> > it's a first order equation, right? >> > >> > >> > >> > 2014-07-08 15:20 GMT-03:00 Anshu Avinash < anshu.avinash35@gmail.com>: >> >> Hi, >> >> >> >> The idea is we know the total time the query took, and how many times >> >> an >> >> operation was performed. For example, consider the case of 'read_time'. >> >> We >> >> know how many times an index read took place, but don't know how much >> >> time >> >> does it take to do an index read. By solving these equations, we are >> >> trying >> >> to find out time for individual operations. coefficients[i].value is >> >> `how >> >> many time the operation i took place in a single query.` >> >> >> >> Hope this clears things up. >> >> >> >> Regards >> >> Anshu Avinash >> >> >> >> >> >> On Tue, Jul 8, 2014 at 10:57 PM, Roberto Spadim < roberto@spadim.com.br> >> >> wrote: >> >>> >> >>> just to understand... >> >>> --- the solve_equation part, today only used to save information: >> >>> std::ofstream datafile; >> >>> char file_name[100]; >> >>> my_snprintf(file_name, 100, >> >>> "/tmp/mariadb_cost_coefficients_%lu.txt", thread_id); >> >>> datafile.open(file_name, std::ios::app); >> >>> for(int i=0; i < MAX_CONSTANTS; i++) >> >>> datafile << coefficients[i].value << " "; >> >>> datafile << total_time << "\n"; >> >>> datafile.close(); >> >>> ---- >> >>> >> >>> the idea is: given a query and some coefficients[i].value, you got >> >>> total_time need to execute the query >> >>> you want to "train" something to tell you how many time the same query >> >>> should execute? >> >>> or, what's the "x[i]" variables from your system (hardware/hard >> >>> disk/etc), and extend this to others queries? >> >>> >> >>> >> >>> 2014-07-08 14:20 GMT-03:00 Roberto Spadim < roberto@spadim.com.br>: >> >>> > =] nice >> >>> > >> >>> > 2014-07-08 14:18 GMT-03:00 Anshu Avinash >> >>> > : >> >>> >> Hi all, >> >>> >> >> >>> >> You can download it here >> >>> >> >> >>> >> >> >>> >> ( https://drive.google.com/file/d/0B7NiQb4EbbUVNVJFZ2xkRVR3Ylk/edit?usp=sharin... ). >> >>> >> It is around 26M. I have added the link on blog too. >> >>> >> >> >>> >> Regards >> >>> >> Anshu >> >>> >> >> >>> >> >> >>> >> On Tue, Jul 8, 2014 at 10:38 PM, Roberto Spadim >> >>> >> >> >>> >> wrote: >> >>> >>> >> >>> >>> could you 'display' the dataset you used with octave? >> >>> >>> >> >>> >>> 2014-07-08 13:55 GMT-03:00 Anshu Avinash >> >>> >>> : >> >>> >>> > Hi all, >> >>> >>> > >> >>> >>> > This week's blog post is at: >> >>> >>> > http://igniting.in/gsoc2014/2014/07/08/solving-linear-equations/ >> >>> >>> > . >> >>> >>> > Sorry >> >>> >>> > for >> >>> >>> > the delay. >> >>> >>> > Suggestions for an approach to solve the system of linear >> >>> >>> > equations >> >>> >>> > are >> >>> >>> > welcome. >> >>> >>> > >> >>> >>> > Regards >> >>> >>> > Anshu Avinash >> >>> >>> > >> >>> >>> > >> >>> >>> > On Mon, Jun 23, 2014 at 7:39 PM, Roberto Spadim >> >>> >>> > >> >>> >>> > wrote: >> >>> >>> >> >> >>> >>> >> " MDEV. " >> >>> >>> >> it's nice to put full name (MDEV-350), since google and others >> >>> >>> >> search >> >>> >>> >> engines help when someone try to find information about mdev >> >>> >>> >> 350 >> >>> >>> >> >> >>> >>> >> text is ok :) >> >>> >>> >> >> >>> >>> >> 2014-06-23 11:04 GMT-03:00 Anshu Avinash >> >>> >>> >> : >> >>> >>> >> > Hi, >> >>> >>> >> > >> >>> >>> >> > Sorry for the confusion, this is the new link: >> >>> >>> >> > http://igniting.in/gsoc2014/2014/06/23/work-before-mid-term/ >> >>> >>> >> > Thanks for pointing out. >> >>> >>> >> > >> >>> >>> >> > Regards >> >>> >>> >> > Anshu >> >>> >>> >> > >> >>> >>> >> > >> >>> >>> >> > On Mon, Jun 23, 2014 at 7:32 PM, Roberto Spadim >> >>> >>> >> > >> >>> >>> >> > wrote: >> >>> >>> >> >> >> >>> >>> >> >> "Sorry this page does not exist =(" >> >>> >>> >> >> >> >>> >>> >> >> 2014-06-23 8:07 GMT-03:00 Anshu Avinash >> >>> >>> >> >> : >> >>> >>> >> >> > Hi all, >> >>> >>> >> >> > >> >>> >>> >> >> > You can find this week's blog entry at: >> >>> >>> >> >> > http://igniting.in/2014/06/23/work-before-mid-term/ >> >>> >>> >> >> > Suggestions/reviews are welcome. >> >>> >>> >> >> > >> >>> >>> >> >> > Regards >> >>> >>> >> >> > Anshu Avinash >> >>> >>> >> >> > >> >>> >>> >> >> > >> >>> >>> >> >> > On Mon, Jun 9, 2014 at 7:30 PM, Roberto Spadim >> >>> >>> >> >> > >> >>> >>> >> >> > wrote: >> >>> >>> >> >> >> >> >>> >>> >> >> >> Well i wws reading your posts >> >>> >>> >> >> >> Do you need big data to test read and scan times? >> >>> >>> >> >> >> >> >>> >>> >> >> >> Em segunda-feira, 9 de junho de 2014, Anshu Avinash >> >>> >>> >> >> >> escreveu: >> >>> >>> >> >> >> >> >>> >>> >> >> >>> Hi all, >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> You can find this week's blog entry at >> >>> >>> >> >> >>> http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm >> >>> >>> >> >> >>> now >> >>> >>> >> >> >>> maintaining the >> >>> >>> >> >> >>> code only on github: >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> https://github.com/igniting/server/tree/selfTuningOptimizer. >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> Regards >> >>> >>> >> >> >>> Anshu Avinash >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash >> >>> >>> >> >> >>> wrote: >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> Hi all, >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> You can find my this week's blog entry at >> >>> >>> >> >> >>> http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ >> >>> >>> >> >> >>> . I >> >>> >>> >> >> >>> have >> >>> >>> >> >> >>> created a >> >>> >>> >> >> >>> branch on launchpad for my work: >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 >> >>> >>> >> >> >>> . >> >>> >>> >> >> >>> You >> >>> >>> >> >> >>> can >> >>> >>> >> >> >>> give your suggestions/reviews either on this

...

...

...

>> >>> >>> >> >> >>> as a >> >>> >>> >> >> >>> comment >> >>> >>> >> >> >>> on >> >>> >>> >> >> >>> the >> >>> >>> >> >> >>> blog itself. >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> Regards >> >>> >>> >> >> >>> Anshu Avinash >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> wrote: >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> wow a big work, congratulation guy, i will read

...

...

...

>> >>> >>> >> >> >>> part >> >>> >>> >> >> >>> to >> >>> >>> >> >> >>> better >> >>> >>> >> >> >>> understand mariadb code >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> 2014-05-19 16:33 GMT-03:00 Anshu Avinash >> >>> >>> >> >> >>> : >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> Hi all, >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> This week's blog entry would get delayed by couple of >> >>> >>> >> >> >>> days. >> >>> >>> >> >> >>> I >> >>> >>> >> >> >>> have >> >>> >>> >> >> >>> started coding though and would like to give

...

...

...

>> >>> >>> >> >> >>> what >> >>> >>> >> >> >>> I'm >> >>> >>> >> >> >>> doing. >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> I've looked at the diffs for "Cost model

...

...

...

>> >>> >>> >> >> >>> mysql: >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 >> >>> >>> >> >> >>> and >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . >> >>> >>> >> >> >>> These >> >>> >>> >> >> >>> give a pretty good idea about what are the hard-coded >> >>> >>> >> >> >>> constants >> >>> >>> >> >> >>> and >> >>> >>> >> >> >>> where >> >>> >>> >> >> >>> are they being used. >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> The idea is to multiply "READ_TIME_FACTOR" and >> >>> >>> >> >> >>> "SCAN_TIME_FACTOR" >> >>> >>> >> >> >>> to >> >>> >>> >> >> >>> the >> >>> >>> >> >> >>> values returned by read_time() and scan_time() in >> >>> >>> >> >> >>> handler.h, >> >>> >>> >> >> >>> while >> >>> >>> >> >> >>> returning. These values would be read from a

...

...

...

>> >>> >>> >> >> >>> mysql >> >>> >>> >> >> >>> db. >> >>> >>> >> >> >>> For >> >>> >>> >> >> >>> that >> >>> >>> >> >> >>> I've looked at sql_statistics.cc. After completing this, >> >>> >>> >> >> >>> I'll >> >>> >>> >> >> >>> first >> >>> >>> >> >> >>> change >> >>> >>> >> >> >>> the values of these constants manually and check if the >> >>> >>> >> >> >>> better >> >>> >>> >> >> >>> or >> >>> >>> >> >> >>> worse >> >>> >>> >> >> >>> query plans are being selected. I'll first do

...

...

...

>> >>> >>> >> >> >>> step >> >>> >>> >> >> >>> manually, >> >>> >>> >> >> >>> to >> >>> >>> >> >> >>> check if everything is working as expected and later >> >>> >>> >> >> >>> automate >> >>> >>> >> >> >>> it. >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> Regards >> >>> >>> >> >> >>> Anshu >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash >> >>> >>> >> >> >>> wrote: >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> Hi all, >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> You can find my blog entry for this week at >> >>> >>> >> >> >>> http://igniting.in/gsoc2014/2014/05/11/first-steps/ . >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> Regards >> >>> >>> >> >> >>> Anshu Avinash >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash >> >>> >>> >> >> >>> wrote: >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> Hi all, >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> Sorry for the irregular updates. I had been busy for >> >>> >>> >> >> >>> last >> >>> >>> >> >> >>> couple >> >>> >>> >> >> >>> of >> >>> >>> >> >> >>> days >> >>> >>> >> >> >>> and might still be busy for 1-2 days more. I would be >> >>> >>> >> >> >>> completely >> >>> >>> >> >> >>> free >> >>> >>> >> >> >>> starting next week, and would be updating my blog weekly >> >>> >>> >> >> >>> on >> >>> >>> >> >> >>> every >> >>> >>> >> >> >>> Monday (so >> >>> >>> >> >> >>> 1st update would be on May 12). I would also send the >> >>> >>> >> >> >>> link >> >>> >>> >> >> >>> of my >> >>> >>> >> >> >>> post >> >>> >>> >> >> >>> weekly >> >>> >>> >> >> >>> on the mailing list. >> >>> >>> >> >> >>> >> >>> >>> >> >> >>> As discussed on irc, I started to explore the

2014-07-08 16:24 GMT-03:00 Roberto Spadim : forming an thread or part by heads up on project" of table in the last pair of

...

...

...

>> >>> >>> >> >> >>> constants: >> >>> >>> >> >> >>> handler::scan_time() and handler::read_time(). >> >>> >>> >> >> >> >> >>> >>> >> >> >> >> >>> >>> >> >> >> >> >>> >>> >> >> >> -- >> >>> >>> >> >> >> Roberto Spadim >> >>> >>> >> >> >> SPAEmpresarial >> >>> >>> >> >> >> Eng. Automação e Controle >> >>> >>> >> >> >> >> >>> >>> >> >> > >> >>> >>> >> >> >> >>> >>> >> >> >> >>> >>> >> >> >> >>> >>> >> >> -- >> >>> >>> >> >> Roberto Spadim >> >>> >>> >> >> SPAEmpresarial >> >>> >>> >> >> Eng. Automação e Controle >> >>> >>> >> > >> >>> >>> >> > >> >>> >>> >> >> >>> >>> >> >> >>> >>> >> >> >>> >>> >> -- >> >>> >>> >> Roberto Spadim >> >>> >>> >> SPAEmpresarial >> >>> >>> >> Eng. Automação e Controle >> >>> >>> > >> >>> >>> > >> >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> -- >> >>> >>> Roberto Spadim >> >>> >>> SPAEmpresarial >> >>> >>> Eng. Automação e Controle >> >>> >> >> >>> >> >> >>> > >> >>> > >> >>> > >> >>> > -- >> >>> > Roberto Spadim >> >>> > SPAEmpresarial >> >>> > Eng. Automação e Controle >> >>> >> >>> >> >>> >> >>> -- >> >>> Roberto Spadim >> >>> SPAEmpresarial >> >>> Eng. Automação e Controle >> >> >> >> >> > >> > >> > >> > -- >> > Roberto Spadim >> > SPAEmpresarial >> > Eng. Automação e Controle >> >> >> >> -- >> Roberto Spadim >> SPAEmpresarial >> Eng. Automação e Controle > >

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle

-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle