Vadim Tkachenko <vadim@percona.com> writes:

I did yesterday commit to lp:maria where I replaced InnoDB by XtraDB, but I do not see message with commit to maria-developers@lists.launchpad.net.

Is there something special I need to do that commit is sent to mail-list ?

Yes. You need to install the mysql plugin for bzr. I think you just place the mysql_plugins/ directory under ~/.bazaar/plugins/ But where to get the mysql_plugins? They are GPL'ed (according to the license in the directory), but it does not appear to be publicly available anywhere at this point. I asked Monty Taylor (who wrote the plugins with Chad Miller), and he said he would check if it was possible to get them released publicly. After installing the plugins, configure them in ~/.bazaar/bazaar.conf [DEFAULT] launchpad_username = knielsen allow_remote_options = mysql email = knielsen@knielsen-hq.org per_file_commits = On post_commit_mailer = smtplib modified for personal circumstances. And finally, make sure that local mail sending works. That should be it. Hopefully, Monty Taylor will get the mysql plugins released. But otherwise, he told me that the commit mails are based on bzr-email, so we could probably switch to using that without too much trouble. - Kristian.

Vadim Tkachenko <vadim@percona.com> writes:

Ok, Got it. While mysql_plugin is not available is it OK if I send changeset for review manually ? Though it is rather big, as there a lot of changes InnoDB -> InnoDB plugin.

I think that is fine, please do. Though if it is bigger than 10 Mb or so it may be rejected by some mail systems.

I may push to another three and propose for merge, in this case you also will be able to review changeset.

Feel free to do this as well. We are not really using the Launchpad model of pushing to own branch and make merge request. But it would be good to get some experience with it in case it could work better for us. Though it seems to not produce diffs in the mails, which does not fit the current review model. It does make it much easier for people to try/use the code (as opposed to just the patch). If you upgrade your local bzr repository to version 1.9 (and use bzr >1.9 obviously), launchpad uses 'stacked branches', which means that when you push a new branch to launchpad it only has to transfer the new changeset. This makes creating new branches quite cheap (though xtradb may still take some bandwidth I assume ;-). - Kristian.

hi!

...

...

...

...

"Paul" == Paul McCullagh <paul.mccullagh@primebase.org> writes:

Paul> Hi All, Paul> I am not sure of the procedure myself. Paul> I have just proposed merge of the maria-pbxt tree into lp:maria. Paul> It resulted in me, and all other captains, getting a HUGE diff in an e- Paul> mail - SORRY! Mac Mail really struggled, Yikes! ouch :( (but things like that happens and bandwidth is reasonable good nowadays) Paul> Anyway, here is the what the change includes: Paul> This branch includes the current MariaDB revision (2689) and the PBXT Paul> storage engine. The only changes are the addition of a storage/pbxt Paul> and a mysql-test/suite/pbxt directory. Paul> The mysql-test/suite/pbxt directory contains a test suite for PBXT. Paul> The test includes 167 test files. All tests run through without error. Paul> To run the tests use: Paul> ./mysql-test-run --mysqld=--default-storage-engine=pbxt --suite=pbxt Paul> In other words, you have to set the default storage engine to PBXT for Paul> this test to run. Paul> PBXT only creates files when the first PBXT table is created. Paul> So when you start MariaDB you will only know that PBXT is installed Paul> because of the startup message it prints in the log. It does not Paul> create any files in the database root directory. Sounds good! Paul> Please let me know if this is OK for merge. Paul> Best regards, Ok. Thanks! Regards, Monty

Hi Kristian, On Apr 6, 2009, at 1:17 PM, Kristian Nielsen wrote:

...

It resulted in me, and all other captains, getting a HUGE diff in an e- mail - SORRY! Mac Mail really struggled, Yikes!

Don't worry. Anyway, I found a setting on Launchpad for limiting diff sizes in emails. I changed it to max 5000 lines (from unlimited). Not 100% sure it was the right place I changed though, Launchpad can be a bit confusing sometimes.

Ah, good to know. I'll give that a try as well.

...

This branch includes the current MariaDB revision (2689) and the PBXT storage engine. The only changes are the addition of a storage/pbxt and a mysql-test/suite/pbxt directory.

PBXT seems to be included in BUILD/compile-pentium64-debug-max, great!

...

The mysql-test/suite/pbxt directory contains a test suite for PBXT. The test includes 167 test files. All tests run through without error. To run the tests use:

./mysql-test-run --mysqld=--default-storage-engine=pbxt --suite=pbxt

In other words, you have to set the default storage engine to PBXT for this test to run.

Testsuite did not pass without errors for me in the MariaDB tree. I saw these failures:

pbxt.count_distinct2 (glibc detected double free error) pbxt.count_distinct3 (mysqld crash) pbxt.lowercase_table_grant pbxt.lowercase_table_qcache pbxt.lowercase_view pbxt.pbxt_bugs pbxt.udf

The count* tests did not fail everytime I ran the tests. The *lowercase* test failures seems related to case differences. Test pbxt_bugs is missing file suite/pbxt/t/load_unique_error1.inc. The test pbxt.udf complains about missing udf_example.so (not sure why).

Just let me know if you need more help/information on these failures.

Yes, thanks. I would like to get these fixed.

However, since the PBXT test suite is not run by default, I think we can merge it anyway now, and then work on getting these issues solved afterwards.

OK

I have set up Buildbot to automatically test the MariaDB code, and I need to find a build slave somewhere that could run tests of the PBXT code (feel free to offer to run one if you have a spare machine handy ;-). That would be helpful in getting a clean PBXT test run (and keeping it clean).

Yes, OK. I think we can work something out.

...

Please let me know if this is OK for merge.

This is ok to push as it is.

Since you are a member of maria-captains on Launchpad, you can go ahead and push the merge to lp:maria yourself. (But if you prefer, I can also pull from your tree and push the merge).

OK, I will push it.

[The general policy for pushing to MariaDB is that you can push yourself once you get an 'ok to push' from review. Small fixes to a broken build etc. are sometimes pushed immediately to avoid delaying other work, and then reviewed after the fact.]

Yup, sounds good.

Do you have a plan for merging in subsequent PBXT changes to MariaDB? (I guess you do, just don't know how it works). Would you be in charge of regular merging, or should the MariaDB team pull/merge from PBXT?

Plan is to merge regularly and, of course, just before we make a release. However, we have not worked out a schedule yet. Generally we will make sure all our tests run through with MySQL before we merge into MariaDB. However, you are free to pull the latest PBXT tree and merge it into MariaDB any time you like. If you make any changes or fixes I would be glad to accept a patch for PBXT. Our goal is to keep one set of source code for all versions of MySQL supported.

With PBXT both in MySQL, Drizzle, and MariaDB, the merge effort on your part could be non-trivial, so feel free to ask for help if needed.

Yes, thanks. I will let you know if we need some help. Currently it is only Vlad and myself working on PBXT.

Again, thanks a lot for your efforts on this!

np :) Best regards, Paul

- Kristian.

-- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com

Hi Kristian, On Apr 7, 2009, at 12:48 PM, Kristian Nielsen wrote:

Paul McCullagh <paul.mccullagh@primebase.org> writes:

...

On Apr 6, 2009, at 1:17 PM, Kristian Nielsen wrote:

...

...

Just let me know if you need more help/information on these failures.

Yes, thanks. I would like to get these fixed.

I have attached a log of running the test suite on my machine. This is an Ubuntu Hardy 64-bit OS running on an Intel Q6600 quad-core CPU. It was built with BUILD/compile-pentium64-debug-max from your maria-pbxt branch.

Let me know if you need further information. Also, I should be able to help more once I get a buildbot slave up running the pbxt tests.

We will look into this. Vlad: would it be possible to give the Maria buildbot access to dev8? Or will our firewall be a problem...?

...

OK, I will push it.

Great, thanks.

Is now pushed! :)

pbxt.lowercase_table_grant [ fail ] ... GRANT USAGE ON *.* TO 'mysqltest_1'@'localhost' -GRANT ALL PRIVILEGES ON `mysqltest`.* TO 'mysqltest_1'@'localhost' +GRANT ALL PRIVILEGES ON `MySQLtest`.* TO 'mysqltest_1'@'localhost' select * from db where user = 'mysqltest_1'; ...

Strange diff with 'mysqltest' and 'MySQLtest'. This may have to do with the fact that I test in my Mac with case-insensitive file system. Will check on Linux.

pbxt.pbxt_bugs [ fail ] Test ended at 2009-04-07 12:09:04

CURRENT_TEST: pbxt.pbxt_bugs mysqltest: At line 924: query 'LOAD DATA LOCAL INFILE 'suite/pbxt/t/ load_unique_error1.inc' REPLACE INTO TABLE t1 FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (@c1,c2) SET c1 = @c1 % 2' failed: 2: File 'suite/pbxt/t/load_unique_error1.inc' not found (Errcode: 2)

The result from queries just before the failure was: < snip >

Sorry, forgot: 'load_unique_error1.inc'. Now pushed.

pbxt.udf [ fail ] Test ended at 2009-04-07 12:11:59

CURRENT_TEST: pbxt.udf mysqltest: At line 18: query 'CREATE FUNCTION metaphon RETURNS STRING SONAME "$UDF_EXAMPLE_LIB"' failed: 1126: Can't open shared library 'udf_example.so' (errno: 22 (null))

The result from queries just before the failure was: drop table if exists t1;

Hmmm, not sure what the problem is here. Will investigate.... Best regards, Paul -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com

On Apr 7, 2009, at 1:19 PM, Vladimir Kolesnikov wrote:

>>>
>>> pbxt.udf                                 [ fail ]
>>>       Test ended at 2009-04-07 12:11:59
>>>
>>> CURRENT_TEST: pbxt.udf
>>> mysqltest: At line 18: query 'CREATE FUNCTION metaphon RETURNS  
>>> STRING SONAME "$UDF_EXAMPLE_LIB"' failed: 1126: Can't open shared  
>>> library 'udf_example.so' (errno: 22 (null))
>>>
>>> The result from queries just before the failure was:
>>> drop table if exists t1;
>>
>> Hmmm, not sure what the problem is here. Will investigate....
> this usually happens because udf_example.so is not in the plugin_dir  
> - which might be because it's not built. Another reason, if you  
> "make test" from pbxt root - I simply copy udf_example.so to  
> pbxt .libs directory and set up --plugin_dir to it, but this relies  
> on the proper --with-mysql pbxt config option.
>

PBXT is part of the MariaDB tree, and does not have its own config.

The problem is occurring when running the PBXT test suite from the  
Maria mysql-test directory.

So maybe the problem is that udf_example.so was not built.


--
Paul McCullagh
PrimeBase Technologies
www.primebase.org
www.blobstreaming.org
pbxt.blogspot.com

On Apr 7, 2009, at 2:21 PM, Kristian Nielsen wrote:

Paul McCullagh <paul.mccullagh@primebase.org> writes:

...

The problem is occurring when running the PBXT test suite from the Maria mysql-test directory.

So maybe the problem is that udf_example.so was not built.

That was my thought too, but I checked, and the udf_example.so was present. And beside, even if it was not, the testcase does have contain a statement to skip the test when the udf is not there.

Hm. Maybe the path is wrong, perhaps due to running in an different suite from main.udf? Anyway, if you cannot repeat at your end I can take a closer look.

OK, we will try this on Linux, and let you know if we cannot repeat. -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com

Hi Kristian, No real plan here... :( My idea was basically to do this manually, which is admittedly a bit of a pain. It requires at about 3 steps: 1. Get a patch for the changes to the PBXT tree in MariaDB since the last time PBXT was merged. 2. Apply the patch to the current PBXT tree, compile, test and push. 3. Copy the PBXT over to MariaBD, compile, test and push. As you mentioned, change histories are lost in the MariaDB tree with this method. I would welcome any suggestions for a better method... Best regards, Paul On May 7, 2009, at 2:18 PM, Kristian Nielsen wrote:

Paul McCullagh <paul.mccullagh@primebase.org> writes:

...

On Apr 6, 2009, at 1:17 PM, Kristian Nielsen wrote:

...

...

Do you have a plan for merging in subsequent PBXT changes to MariaDB? (I guess you do, just don't know how it works). Would you be in charge of regular merging, or should the MariaDB team pull/merge from PBXT?

Plan is to merge regularly and, of course, just before we make a release. However, we have not worked out a schedule yet.

Generally we will make sure all our tests run through with MySQL before we merge into MariaDB.

However, you are free to pull the latest PBXT tree and merge it into MariaDB any time you like. If you make any changes or fixes I would be glad to accept a patch for PBXT.

I thought a bit more about the issues of merging PBXT (and later XtraDB) changes into MariaDB. I would like to understand better how to do it.

The main bzr repositories of both PBXT and XtraDB contain just the storage engine, and have no common history. So as I understand it, I cannot just pull in the changes using bzr (I tried bzr merge, and got an error due to no common ancestor).

However, Arjen Lentz told me you had some technique you are using to control merging into mysql, drizzle, and mariadb. Care to share it? ;-)

As I see it, the way we can do it now is to import PBXT (and XtraDB) snapshots into MariaDB. So keep a separate maria clone which is never updated, except that from time to time the storage/pbxt directory is replaced with a snapshot of PBXT, followed by commit. We can then bzr merge from this into main MariaDB. So essentially ignoring the PBXT bzr history. Is this how you do / plan to do it? Or do you have a smarter way?

It would be great if we could do this some bzr-supported way, so that we could have the PBXT (and XtraDB) history available within the MariaDB repository (it is not yet). But as I understand, this is not possible at the moment (correct me if I'm wrong). There is some planned bzr feature of nested-trees, but it appears to not be ready yet.

Anyway, I am in no way trying to change how you are doing or planning things, just trying to understand what the plan is :-)

Thanks,

- Kristian.

-- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com

Hi Stewart & Kristian, I was not aware of this bit of magic! :) So to correct this I will have to redo the integration of PBXT is into MariaDB. I think it would also be best if I redid the integration of PBXT into the Drizzle/PBXT tree because some of the merges have been done manually, so the history is missing. Or would I get the same effect if I just followed Stewart's procedure and merged over the pbxt files that already exist (in the MariaDB and Drizzle/PBXT trees)? Best regards, Paul On May 11, 2009, at 7:17 AM, Stewart Smith wrote:

On Thu, May 07, 2009 at 02:18:02PM +0200, Kristian Nielsen wrote:

...

I thought a bit more about the issues of merging PBXT (and later XtraDB) changes into MariaDB. I would like to understand better how to do it.

The main bzr repositories of both PBXT and XtraDB contain just the storage engine, and have no common history. So as I understand it, I cannot just pull in the changes using bzr (I tried bzr merge, and got an error due to no common ancestor).

You can :)

With BZR you can merge two trees without a common ancestor. This is useful for things like storage engines, file systems, so when they're merged, you can bring the whole revision history in.

This is how I did it for PBXT and Drizzle:

bzr branch lp:drizzle drizzle-pbxt bzr branch lp:pbxt pbxt bzr branch pbxt pbxt-rename pushd pbxt-rename mkdir storage/pbxt

Is this correct, or should it be bzr mkdir storage/pbxt?

bzr mv * storage/pbxt/ bzr commit popd pushd drizzle-pbxt bzr merge ../pbxt-rename -r1..-1 (fix up a conflict on storage/ or something... pretty easy, this is just done so that auto merges happen on file ops in main pbxt) bzr commit

and then, when mainline pbxt is updated pushd drizzle-pbxt bzr merge lp:pbxt

and it magically works.

You cannot, however then merge back from drizzle-pbxt into pbxt - so either someone has to extract patches and deal with the no-op merge when coming back or all dev moves to drizzle-pbxt derived branches.

Since we have two sep trees (MariaDB and Drizzle) both merging in PBXT, it makes sense for PBXT development to continue in the PBXT tree and then we can both merge PBXT into our trees from it.

If we get patches and fixes back into the main PBXT tree, then I think we all live in some magic world of harmony :)

It should also be (relatively) easy for someone to whip up a script that automatically tests a PBXT commit against current Drizzle and MariaDB.

Hope this helps -- Stewart Smith

-- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com

Stewart, thanks for providing this procedure! Nice to know that this method with renaming to common paths and merging can work. Paul McCullagh <paul.mccullagh@primebase.org> writes:

So to correct this I will have to redo the integration of PBXT is into MariaDB.

Yes, I think we should re-do the merge with this tracking of history and merging. I spent some time today playing with this technique, as I would like to be really sure that we have a good base for working with the PBXT code in the future, and also I would like to use the same technique for merging XtraDB. So I created a MariaDB tree with the PBXT merge re-done using Stewart's technique, taking some care to get everything done correctly to smooth the future work. If you want to use it as-is, I write the exact commands I used below if you want to do something similar for Drizzle. If you want to re-do it yourself to tweak it or control things fully, that is perfectly fine with me as well. Whatever works best for you. Just let me know if you want me to push this tree to MariaDB, or prepare another tree for me to merge. I put my merged tree here: lp:~knielsen/maria/mariadb-5.1-pbxt-merge-new Note that this tree is identical source-wise with the existing lp:maria tree, just the history now includes all of the PBXT changesets so history and merges can be tracked as per Stewarts method. So this does not include newest PBXT yet, just PBXT up to revision 608 (revid:paul.mccullagh@primebase.org-20090402202852-wa2fbcmrdy7gda2f). I did a quick test merge of newest PBXT just to check things, and it seems to work well. Most changes automerged. For deleted/renamed/newly added files, a conflict is flagged that must be resolved manually. Changes to files in PBXT that are omitted from the MariaDB source (COPYING, README, etc) also are flagged as a conflict that needs to be resolved. This is to be expected. I did see some wierd stuff in some testing. Eg. the `bzr diff` of my merge changeset is not empty as expected, rather it shows one diff removing a file and another identical (but reverse) diff adding that file back. I also saw a strange bzr error message in one case of testing merge. I guess we are (ab)using some uncommon corners of bzr here, but hopefully any problems/bug in bzr can be fixed. One think I wonder is what to do with the PBXT test suite in mysql-test/suite/pbxt/. In my merge I merged it into place with `bzr mv` so that merge tracking and history is kept. However, I don't know how well this will work for .result files, as I guess they are going to differ between MariaDB, Drizzle, and MySQL? Not 100% sure how we will deal with this. Could be that .result files cannot be meaningfully merged automatically. In any case, I think this is a usable starting point, and we can refine the .result handling later as we understand better how to do it. Just let me know if you think otherwise. Here are the exact commands I did to make the merge: bzr branch -r'revid:paul.mccullagh@primebase.org-20090402202852-wa2fbcmrdy7gda2f' pbxt tmp-pbxt-rename cd tmp-pbxt-rename/ bzr co bzr mkdir storage bzr mkdir storage/pbxt bzr mv `ls|grep -v storage` storage/pbxt/ bzr commit -m"Move all PBXT files into storage/pbxt/ in preparation for merging into MariaDB." cd .. bzr branch -r'revid:paul.mccullagh@primebase.org-20090407105746-tolv5dita1d3eavm' mariadb-5.1 tmp-mariadb-pbxt-merge cd tmp-mariadb-pbxt-merge/ bzr co bzr merge ../tmp-pbxt-rename -r0..-1 bzr mv storage storage.new bzr mv storage.moved storage bzr rm storage/pbxt bzr rm mysql-test/suite/pbxt/t bzr rm mysql-test/suite/pbxt/r bzr mv storage.new/pbxt/test/mysql-test/t mysql-test/suite/pbxt/t bzr mv storage.new/pbxt/test/mysql-test/r mysql-test/suite/pbxt/r bzr mv storage.new/pbxt storage/pbxt bzr rm --force storage.new # bzr rm + manual patch. Applying same changes done manually in previous PBXT merge. bzr resolved storage bzr ci -m "Re-merge PBXT into MariaDB to get proper history and merge tracking." cd .. bzr branch mariadb-5.1 mariadb-5.1-pbxt-merge-new cd mariadb-5.1-pbxt-merge-new bzr co bzr merge ../tmp-mariadb-pbxt-merge bzr ci -m "Pull new/improved PBXT merge into latest MariaDB trunk." So I used the revisions of MariaDB and PBXT that were also used to do the original merge. And I manually applied the same changes in the merge that you apparently did in your original merge of PBXT into MariaDB, mainly deleting some files like COPYING, configure.in, removing a number of non-pbxt test cases, and fixing some .result files. So I hope this will be a good base for future merging, with this it should be possible for either PBXT or MariaDB people to pull latest from PBXT into MariaDB without either getting in the way of the other. And it should be quite useful to us to have PBXT history available if debugging etc. Any comments welcome.

On May 11, 2009, at 7:17 AM, Stewart Smith wrote:

...

With BZR you can merge two trees without a common ancestor. This is useful for things like storage engines, file systems, so when they're merged, you can bring the whole revision history in.

This is how I did it for PBXT and Drizzle:

bzr branch lp:drizzle drizzle-pbxt bzr branch lp:pbxt pbxt bzr branch pbxt pbxt-rename pushd pbxt-rename mkdir storage/pbxt

Is this correct, or should it be bzr mkdir storage/pbxt?

I used `bzr mkdir`, got an error from bzr otherwise.

...

bzr merge ../pbxt-rename -r1..-1 (fix up a conflict on storage/ or something... pretty easy, this is just done so that auto merges happen on file ops in main pbxt)

I used -r0..-1 (not sure if it makes a difference?). And above log shows how I resolved the conflicts.

...

bzr commit

and then, when mainline pbxt is updated pushd drizzle-pbxt bzr merge lp:pbxt

- Kristian.

Hi Kristian and Stewart, On May 11, 2009, at 4:24 PM, Kristian Nielsen wrote:

I did a quick test merge of newest PBXT just to check things, and it seems to work well. Most changes automerged. For deleted/renamed/newly added files, a conflict is flagged that must be resolved manually. Changes to files in PBXT that are omitted from the MariaDB source (COPYING, README, etc) also are flagged as a conflict that needs to be resolved. This is to be expected.

I did see some wierd stuff in some testing. Eg. the `bzr diff` of my merge changeset is not empty as expected, rather it shows one diff removing a file and another identical (but reverse) diff adding that file back. I also saw a strange bzr error message in one case of testing merge. I guess we are (ab)using some uncommon corners of bzr here, but hopefully any problems/bug in bzr can be fixed.

I am just trying to merge for the second time into mariadb-5.1-pbxt- merge-new, and get the following: pmc-2:Repositories/maria/mariadb-5.1-pbxt-merge-new] build% bzr merge lp:pbxt M storage/pbxt/src/cache_xt.cc M storage/pbxt/src/ha_pbxt.cc M storage/pbxt/src/index_xt.h M storage/pbxt/src/lock_xt.cc M storage/pbxt/src/lock_xt.h M storage/pbxt/src/table_xt.h /Library/Python/2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(0/2, msg='Apply phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) /Library/Python/2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(2/3, msg='Merge phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) bzr: ERROR: No final name for trans_id 'new-1' file-id: None root trans-id: 'new-0' The merge was aborted because, after this I have: [pmc-2:Repositories/maria/mariadb-5.1-pbxt-merge-new] build% bzr diff . [pmc-2:Repositories/maria/mariadb-5.1-pbxt-merge-new] build% No diff. Any ideas? -- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com

Hi Kristian, Looks like this bug was reported already: https://bugs.launchpad.net/bzr/+bug/364305 Maybe we can all go in and click the "This bug doesn't affect me" link to bump up the priority. The question is: is this a show stopper for us at the moment? On May 12, 2009, at 5:30 PM, Kristian Nielsen wrote:

Paul McCullagh <paul.mccullagh@primebase.org> writes:

...

I am just trying to merge for the second time into mariadb-5.1-pbxt- merge-new, and get the following:

pmc-2:Repositories/maria/mariadb-5.1-pbxt-merge-new] build% bzr merge lp:pbxt M storage/pbxt/src/cache_xt.cc M storage/pbxt/src/ha_pbxt.cc M storage/pbxt/src/index_xt.h M storage/pbxt/src/lock_xt.cc M storage/pbxt/src/lock_xt.h M storage/pbxt/src/table_xt.h /Library/Python/2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(0/2, msg='Apply phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) /Library/Python/2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(2/3, msg='Merge phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) bzr: ERROR: No final name for trans_id 'new-1' file-id: None root trans-id: 'new-0'

Yes, this looks similar to what I have seen.

If you can repeat it, we should report it to the bzr guys. Hopefully they can fix it.

(let me know if you will report it, else I will).

- Kristian.

-- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com

Kristian Nielsen <knielsen@knielsen-hq.org> writes:

One thing I might try is to re-do the merge in smaller steps. There is a _lot_ of add/move/delete/patch the same file in the same changeset, and it is clear that it is very hard to get all the special cases of this right in the bzr code (I would personally lean towards not allowing a single file to be both added, deleted, moved, and/or patched in the same commit).

So if I re-do the merge with seperate commits:

1. In pbxt repo, move needed files into pbxt-to-be-merged, delete rest. 2. In MariaDB repo, delete PBXT files from the old merge. 3. Pull renamed pbxt into mariadb without other changes. 4. Move files in pbxt-to-be-merged into place 5. Apply needed patches.

Then each individual step will be in a separate commit, which should make the history hopefully more sane.

Ok, I tried this, and it did not make any difference, same problem. After re-merging PBXT revision 608 (same as orginal merge), I tried to merge in just revision 609 from PBXT (from a `bzr branch -r609 pbxt` branch): $ bzr merge ../test-merge-source +N mysql-test/suite/pbxt/r/type_varchar.result.OTHER +N mysql-test/suite/pbxt/t/type_varchar.test.OTHER M storage/pbxt/src/ha_pbxt.cc Contents conflict in mysql-test/suite/pbxt/r/type_varchar.result Contents conflict in mysql-test/suite/pbxt/t/type_varchar.test /usr/lib/python2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(0/2, msg='Apply phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) /usr/lib/python2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(2/3, msg='Merge phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) bzr: ERROR: No final name for trans_id 'new-1' file-id: None root trans-id: 'new-0' So I think this means we will have to put applying Stewart's technique to the PBXT merge on hold :-(. I will report this bug to bzr along with steps to reproduce. Also, I will try if some of our people with contacts to the bzr developers can get them to look into this. In case of interest, below are the steps I used for the new merge (which I like much better than the previous even if it did not solve the problem, as there were no merge conflicts at all to be solved and it just seems much cleaner). Also pushed as lp:~knielsen/maria/mariadb-5.1-pbxt-merge-take2 - Kristian. mkdir pbxt-new-merge-take2 cd pbxt-new-merge-take2/ bzr branch -r'revid:paul.mccullagh@primebase.org-20090402202852-wa2fbcmrdy7gda2f' lp:pbxt pbxt-rename cd pbxt-rename/ bzr co bzr mkdir pbxt-to-be-renamed bzr rm aclocal.m4 config config.h.in configure configure-dzl.in configure.in COPYING Makefile-dzl.am Makefile.in plug-dzl.in bin src/Makefile-dzl.am src/Makefile.in (cd test/mysql-test/t && bzr rm --force <lots of files>) (cd test/mysql-test/r && bzr rm --force <lots of files>) bzr mv test/mysql-test/t pbxt-to-be-renamed/ bzr mv test/mysql-test/r pbxt-to-be-renamed/ bzr rm test bzr mv AUTHORS INSTALL NEWS plug.in src ChangeLog Makefile.am README TODO pbxt-to-be-renamed/ bzr commit -m$'Prepare for merging PBXT into MariaDB tree:\n - Delete not needed files.\n - Move needed files into subdirectory to avoid conflicts.' cd .. bzr branch -r'revid:paul.mccullagh@primebase.org-20090407105746-tolv5dita1d3eavm' lp:maria mariadb-pbxt-merge cd mariadb-pbxt-merge/ bzr co bzr rm storage/pbxt mysql-test/suite/pbxt/{t,r} bzr commit -m"Delete old PBXT files in preparation for re-merging PBXT with proper history and merge tracking." bzr merge ../pbxt-rename -r0..-1 bzr ci -m"Re-merge PBXT with proper history and merge tracking, into temporary directory for later renaming into place." bzr mv pbxt-to-be-renamed/{t,r} mysql-test/suite/pbxt/ bzr mkdir storage/pbxt/ bzr mv pbxt-to-be-renamed/* storage/pbxt/ bzr rm pbxt-to-be-renamed/ bzr ci -m"Re-merge PBXT into MariaDB with proper history and merge tracking: move files into correct place." patch -p0 -s < /tmp/1 (cd mysql-test/suite/pbxt/ && patch -p0 -s < /tmp/2) (cd mysql-test/suite/pbxt/ && patch -p0 -s < /tmp/3) bzr ci -m"After-merge fixes for re-merge of PBXT into MariaDB with proper history and merge tracking."

On 13 May 2009, at 01:52, Paul McCullagh wrote:

On May 13, 2009, at 10:37 AM, Kristian Nielsen wrote:

...

Kristian Nielsen <knielsen@knielsen-hq.org> writes:

...

One thing I might try is to re-do the merge in smaller steps. There is a _lot_ of add/move/delete/patch the same file in the same changeset, and it is clear that it is very hard to get all the special cases of this right in the bzr code (I would personally lean towards not allowing a single file to be both added, deleted, moved, and/or patched in the same commit).

So if I re-do the merge with seperate commits:

1. In pbxt repo, move needed files into pbxt-to-be-merged, delete rest. 2. In MariaDB repo, delete PBXT files from the old merge. 3. Pull renamed pbxt into mariadb without other changes. 4. Move files in pbxt-to-be-merged into place 5. Apply needed patches.

Then each individual step will be in a separate commit, which should make the history hopefully more sane.

Ok, I tried this, and it did not make any difference, same problem.

After re-merging PBXT revision 608 (same as orginal merge), I tried to merge in just revision 609 from PBXT (from a `bzr branch -r609 pbxt` branch):

$ bzr merge ../test-merge-source +N mysql-test/suite/pbxt/r/type_varchar.result.OTHER +N mysql-test/suite/pbxt/t/type_varchar.test.OTHER M storage/pbxt/src/ha_pbxt.cc Contents conflict in mysql-test/suite/pbxt/r/type_varchar.result Contents conflict in mysql-test/suite/pbxt/t/type_varchar.test /usr/lib/python2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(0/2, msg='Apply phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) /usr/lib/python2.5/site-packages/bzrlib/ui/__init__.py:91: UserWarning: ProgressTask(2/3, msg='Merge phase') is not the active task ProgressTask(None/None, msg='') % (task, self._task_stack[-1])) bzr: ERROR: No final name for trans_id 'new-1' file-id: None root trans-id: 'new-0'

So I think this means we will have to put applying Stewart's technique to the PBXT merge on hold :-(.

Yes, unfortunately we cannot proceed with this problem. :(

I encountered same issues with my maria-federated merged repository. I found that if I pull into the 'renamed' intermediate repository, I could then merge that into the maria repository but if I try to do it direct, I get the bzr crash as above. (no merge conflicts though). Regards, Antony