Hi all, Here is a blog post describing my progress so far: http://igniting.in/gsoc2014/2014/08/04/progress-so-far/ Comments and suggestions are welcome. Regards Anshu Avinash On Wed, Jul 9, 2014 at 1:14 AM, Roberto Spadim <roberto@spadim.com.br> wrote:
please include the output (solutions.txt) link too, to check what happened
http://www.netlib.org/lapack/lapacke.html a C api to lapack
2014-07-08 16:23 GMT-03:00 Roberto Spadim <roberto@spadim.com.br>:
check lapack lib, i used it some years ago, and it solve linear equations, at least you don't waste time with 'how to solve linear equations', if you want to study :) lapack was a nice lib, at least i used without problems
2014-07-08 16:04 GMT-03:00 Roberto Spadim <roberto@spadim.com.br>:
a1,a2 is what your solve equation function is saving? and you want know t1,t2,t3..t130, to understand how much time each 'read function' take, that's it?
doing this, what's the next step? this is a start point to select what's better? index vs table scan?
2014-07-08 15:54 GMT-03:00 Anshu Avinash <anshu.avinash35@gmail.com>:
Hi,
a1, a2, ..., a130 are coefficients. t1, t2, .., t130 are unknowns. We need 130 linearly independent equations to solve for these variables. We can never get 130 linearly independent equations as some of the coefficients would be 0 every time. Hence, we get an approximate solution by
overdetermined system ( http://en.wikipedia.org/wiki/Overdetermined_system). Let me know if you have any further doubts.
Regards Anshu Avinash
On Wed, Jul 9, 2014 at 12:18 AM, Roberto Spadim < roberto@spadim.com.br> wrote:
ops, linear equation
2014-07-08 15:47 GMT-03:00 Roberto Spadim <roberto@spadim.com.br>: > going back... > a1t1 + a2t2 + … + a130t130= ttotal > > a1, t1... > > a1 is something you don't know > t1 is the coefficients[i]? > > it's a first order equation, right? > > > > 2014-07-08 15:20 GMT-03:00 Anshu Avinash <
anshu.avinash35@gmail.com>:
>> Hi, >> >> The idea is we know the total time the query took, and how many times >> an >> operation was performed. For example, consider the case of 'read_time'. >> We >> know how many times an index read took place, but don't know how much >> time >> does it take to do an index read. By solving these equations, we are >> trying >> to find out time for individual operations. coefficients[i].value is >> `how >> many time the operation i took place in a single query.` >> >> Hope this clears things up. >> >> Regards >> Anshu Avinash >> >> >> On Tue, Jul 8, 2014 at 10:57 PM, Roberto Spadim < roberto@spadim.com.br> >> wrote: >>> >>> just to understand... >>> --- the solve_equation part, today only used to save information: >>> std::ofstream datafile; >>> char file_name[100]; >>> my_snprintf(file_name, 100, >>> "/tmp/mariadb_cost_coefficients_%lu.txt", thread_id); >>> datafile.open(file_name, std::ios::app); >>> for(int i=0; i < MAX_CONSTANTS; i++) >>> datafile << coefficients[i].value << " "; >>> datafile << total_time << "\n"; >>> datafile.close(); >>> ---- >>> >>> the idea is: given a query and some coefficients[i].value, you got >>> total_time need to execute the query >>> you want to "train" something to tell you how many time the same query >>> should execute? >>> or, what's the "x[i]" variables from your system (hardware/hard >>> disk/etc), and extend this to others queries? >>> >>> >>> 2014-07-08 14:20 GMT-03:00 Roberto Spadim <roberto@spadim.com.br : >>> > =] nice >>> > >>> > 2014-07-08 14:18 GMT-03:00 Anshu Avinash >>> > <anshu.avinash35@gmail.com>: >>> >> Hi all, >>> >> >>> >> You can download it here >>> >> >>> >> >>> >> ( https://drive.google.com/file/d/0B7NiQb4EbbUVNVJFZ2xkRVR3Ylk/edit?usp=sharin... ). >>> >> It is around 26M. I have added the link on blog too. >>> >> >>> >> Regards >>> >> Anshu >>> >> >>> >> >>> >> On Tue, Jul 8, 2014 at 10:38 PM, Roberto Spadim >>> >> <roberto@spadim.com.br> >>> >> wrote: >>> >>> >>> >>> could you 'display' the dataset you used with octave? >>> >>> >>> >>> 2014-07-08 13:55 GMT-03:00 Anshu Avinash >>> >>> <anshu.avinash35@gmail.com>: >>> >>> > Hi all, >>> >>> > >>> >>> > This week's blog post is at: >>> >>> > http://igniting.in/gsoc2014/2014/07/08/solving-linear-equations/ >>> >>> > . >>> >>> > Sorry >>> >>> > for >>> >>> > the delay. >>> >>> > Suggestions for an approach to solve the system of linear >>> >>> > equations >>> >>> > are >>> >>> > welcome. >>> >>> > >>> >>> > Regards >>> >>> > Anshu Avinash >>> >>> > >>> >>> > >>> >>> > On Mon, Jun 23, 2014 at 7:39 PM, Roberto Spadim >>> >>> > <roberto@spadim.com.br> >>> >>> > wrote: >>> >>> >> >>> >>> >> " MDEV. " >>> >>> >> it's nice to put full name (MDEV-350), since google and others >>> >>> >> search >>> >>> >> engines help when someone try to find information about mdev >>> >>> >> 350 >>> >>> >> >>> >>> >> text is ok :) >>> >>> >> >>> >>> >> 2014-06-23 11:04 GMT-03:00 Anshu Avinash >>> >>> >> <anshu.avinash35@gmail.com>: >>> >>> >> > Hi, >>> >>> >> > >>> >>> >> > Sorry for the confusion, this is the new link: >>> >>> >> > http://igniting.in/gsoc2014/2014/06/23/work-before-mid-term/ >>> >>> >> > Thanks for pointing out. >>> >>> >> > >>> >>> >> > Regards >>> >>> >> > Anshu >>> >>> >> > >>> >>> >> > >>> >>> >> > On Mon, Jun 23, 2014 at 7:32 PM, Roberto Spadim >>> >>> >> > <roberto@spadim.com.br> >>> >>> >> > wrote: >>> >>> >> >> >>> >>> >> >> "Sorry this page does not exist =(" >>> >>> >> >> >>> >>> >> >> 2014-06-23 8:07 GMT-03:00 Anshu Avinash >>> >>> >> >> <anshu.avinash35@gmail.com>: >>> >>> >> >> > Hi all, >>> >>> >> >> > >>> >>> >> >> > You can find this week's blog entry at: >>> >>> >> >> > http://igniting.in/2014/06/23/work-before-mid-term/ >>> >>> >> >> > Suggestions/reviews are welcome. >>> >>> >> >> > >>> >>> >> >> > Regards >>> >>> >> >> > Anshu Avinash >>> >>> >> >> > >>> >>> >> >> > >>> >>> >> >> > On Mon, Jun 9, 2014 at 7:30 PM, Roberto Spadim >>> >>> >> >> > <roberto@spadim.com.br> >>> >>> >> >> > wrote: >>> >>> >> >> >> >>> >>> >> >> >> Well i wws reading your posts >>> >>> >> >> >> Do you need big data to test read and scan times? >>> >>> >> >> >> >>> >>> >> >> >> Em segunda-feira, 9 de junho de 2014, Anshu Avinash >>> >>> >> >> >> <anshu.avinash35@gmail.com> escreveu: >>> >>> >> >> >> >>> >>> >> >> >>> Hi all, >>> >>> >> >> >>> >>> >>> >> >> >>> You can find this week's blog entry at >>> >>> >> >> >>> http://igniting.in/gsoc2014/2014/06/09/more-coding/. I'm >>> >>> >> >> >>> now >>> >>> >> >> >>> maintaining the >>> >>> >> >> >>> code only on github: >>> >>> >> >> >>> >>> >>> >> >> >>> https://github.com/igniting/server/tree/selfTuningOptimizer. >>> >>> >> >> >>> >>> >>> >> >> >>> Regards >>> >>> >> >> >>> Anshu Avinash >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> On Sun, May 25, 2014 at 3:27 PM, Anshu Avinash >>> >>> >> >> >>> <anshu.avinash35@gmail.com> wrote: >>> >>> >> >> >>> >>> >>> >> >> >>> Hi all, >>> >>> >> >> >>> >>> >>> >> >> >>> You can find my this week's blog entry at >>> >>> >> >> >>> http://igniting.in/gsoc2014/2014/05/25/coding-things-up/ >>> >>> >> >> >>> . I >>> >>> >> >> >>> have >>> >>> >> >> >>> created a >>> >>> >> >> >>> branch on launchpad for my work: >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> http://bazaar.launchpad.net/~igniting/maria/maria/revision/4211 >>> >>> >> >> >>> . >>> >>> >> >> >>> You >>> >>> >> >> >>> can >>> >>> >> >> >>> give your suggestions/reviews either on this
>>> >>> >> >> >>> as a >>> >>> >> >> >>> comment >>> >>> >> >> >>> on >>> >>> >> >> >>> the >>> >>> >> >> >>> blog itself. >>> >>> >> >> >>> >>> >>> >> >> >>> Regards >>> >>> >> >> >>> Anshu Avinash >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> On Tue, May 20, 2014 at 1:22 AM, Roberto Spadim >>> >>> >> >> >>> <roberto@spadim.com.br> >>> >>> >> >> >>> wrote: >>> >>> >> >> >>> >>> >>> >> >> >>> wow a big work, congratulation guy, i will read
>>> >>> >> >> >>> part >>> >>> >> >> >>> to >>> >>> >> >> >>> better >>> >>> >> >> >>> understand mariadb code >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> 2014-05-19 16:33 GMT-03:00 Anshu Avinash >>> >>> >> >> >>> <anshu.avinash35@gmail.com>: >>> >>> >> >> >>> >>> >>> >> >> >>> Hi all, >>> >>> >> >> >>> >>> >>> >> >> >>> This week's blog entry would get delayed by couple of >>> >>> >> >> >>> days. >>> >>> >> >> >>> I >>> >>> >> >> >>> have >>> >>> >> >> >>> started coding though and would like to give heads up on >>> >>> >> >> >>> what >>> >>> >> >> >>> I'm >>> >>> >> >> >>> doing. >>> >>> >> >> >>> >>> >>> >> >> >>> I've looked at the diffs for "Cost model project" of >>> >>> >> >> >>> mysql: >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7596 >>> >>> >> >> >>> and >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> http://bazaar.launchpad.net/~mysql/mysql-server/5.7/revision/7222 . >>> >>> >> >> >>> These >>> >>> >> >> >>> give a pretty good idea about what are the hard-coded >>> >>> >> >> >>> constants >>> >>> >> >> >>> and >>> >>> >> >> >>> where >>> >>> >> >> >>> are they being used. >>> >>> >> >> >>> >>> >>> >> >> >>> The idea is to multiply "READ_TIME_FACTOR" and >>> >>> >> >> >>> "SCAN_TIME_FACTOR" >>> >>> >> >> >>> to >>> >>> >> >> >>> the >>> >>> >> >> >>> values returned by read_time() and scan_time() in >>> >>> >> >> >>> handler.h, >>> >>> >> >> >>> while >>> >>> >> >> >>> returning. These values would be read from a table in >>> >>> >> >> >>> mysql >>> >>> >> >> >>> db. >>> >>> >> >> >>> For >>> >>> >> >> >>> that >>> >>> >> >> >>> I've looked at sql_statistics.cc. After completing
>>> >>> >> >> >>> I'll >>> >>> >> >> >>> first >>> >>> >> >> >>> change >>> >>> >> >> >>> the values of these constants manually and check if the >>> >>> >> >> >>> better >>> >>> >> >> >>> or >>> >>> >> >> >>> worse >>> >>> >> >> >>> query plans are being selected. I'll first do the last >>> >>> >> >> >>> step >>> >>> >> >> >>> manually, >>> >>> >> >> >>> to >>> >>> >> >> >>> check if everything is working as expected and later >>> >>> >> >> >>> automate >>> >>> >> >> >>> it. >>> >>> >> >> >>> >>> >>> >> >> >>> Regards >>> >>> >> >> >>> Anshu >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> On Mon, May 12, 2014 at 11:22 AM, Anshu Avinash >>> >>> >> >> >>> <anshu.avinash35@gmail.com> wrote: >>> >>> >> >> >>> >>> >>> >> >> >>> Hi all, >>> >>> >> >> >>> >>> >>> >> >> >>> You can find my blog entry for this week at >>> >>> >> >> >>> http://igniting.in/gsoc2014/2014/05/11/first-steps/ . >>> >>> >> >> >>> >>> >>> >> >> >>> Regards >>> >>> >> >> >>> Anshu Avinash >>> >>> >> >> >>> >>> >>> >> >> >>> >>> >>> >> >> >>> On Thu, May 8, 2014 at 11:46 PM, Anshu Avinash >>> >>> >> >> >>> <anshu.avinash35@gmail.com> wrote: >>> >>> >> >> >>> >>> >>> >> >> >>> Hi all, >>> >>> >> >> >>> >>> >>> >> >> >>> Sorry for the irregular updates. I had been busy for >>> >>> >> >> >>> last >>> >>> >> >> >>> couple >>> >>> >> >> >>> of >>> >>> >> >> >>> days >>> >>> >> >> >>> and might still be busy for 1-2 days more. I would be >>> >>> >> >> >>> completely >>> >>> >> >> >>> free >>> >>> >> >> >>> starting next week, and would be updating my blog weekly >>> >>> >> >> >>> on >>> >>> >> >> >>> every >>> >>> >> >> >>> Monday (so >>> >>> >> >> >>> 1st update would be on May 12). I would also send
2014-07-08 16:24 GMT-03:00 Roberto Spadim <roberto@spadim.com.br>: forming an thread or part by this, the
>>> >>> >> >> >>> link >>> >>> >> >> >>> of my >>> >>> >> >> >>> post >>> >>> >> >> >>> weekly >>> >>> >> >> >>> on the mailing list. >>> >>> >> >> >>> >>> >>> >> >> >>> As discussed on irc, I started to explore the pair of >>> >>> >> >> >>> constants: >>> >>> >> >> >>> handler::scan_time() and handler::read_time(). >>> >>> >> >> >> >>> >>> >> >> >> >>> >>> >> >> >> >>> >>> >> >> >> -- >>> >>> >> >> >> Roberto Spadim >>> >>> >> >> >> SPAEmpresarial >>> >>> >> >> >> Eng. Automação e Controle >>> >>> >> >> >> >>> >>> >> >> > >>> >>> >> >> >>> >>> >> >> >>> >>> >> >> >>> >>> >> >> -- >>> >>> >> >> Roberto Spadim >>> >>> >> >> SPAEmpresarial >>> >>> >> >> Eng. Automação e Controle >>> >>> >> > >>> >>> >> > >>> >>> >> >>> >>> >> >>> >>> >> >>> >>> >> -- >>> >>> >> Roberto Spadim >>> >>> >> SPAEmpresarial >>> >>> >> Eng. Automação e Controle >>> >>> > >>> >>> > >>> >>> >>> >>> >>> >>> >>> >>> -- >>> >>> Roberto Spadim >>> >>> SPAEmpresarial >>> >>> Eng. Automação e Controle >>> >> >>> >> >>> > >>> > >>> > >>> > -- >>> > Roberto Spadim >>> > SPAEmpresarial >>> > Eng. Automação e Controle >>> >>> >>> >>> -- >>> Roberto Spadim >>> SPAEmpresarial >>> Eng. Automação e Controle >> >> > > > > -- > Roberto Spadim > SPAEmpresarial > Eng. Automação e Controle
-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle
-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle
-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle
-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle
-- Roberto Spadim SPAEmpresarial Eng. Automação e Controle