Hi Kristian, On May 11, 2009, at 4:24 PM, Kristian Nielsen wrote:
Stewart, thanks for providing this procedure!
Nice to know that this method with renaming to common paths and merging can work.
Paul McCullagh <paul.mccullagh@primebase.org> writes:
So to correct this I will have to redo the integration of PBXT is into MariaDB.
Yes, I think we should re-do the merge with this tracking of history and merging.
I spent some time today playing with this technique, as I would like to be really sure that we have a good base for working with the PBXT code in the future, and also I would like to use the same technique for merging XtraDB.
So I created a MariaDB tree with the PBXT merge re-done using Stewart's technique, taking some care to get everything done correctly to smooth the future work.
If you want to use it as-is, I write the exact commands I used below if you want to do something similar for Drizzle. If you want to re-do it yourself to tweak it or control things fully, that is perfectly fine with me as well. Whatever works best for you.
Wow, excellent! Thanks, you have saved me the work :)
Just let me know if you want me to push this tree to MariaDB, or prepare another tree for me to merge.
I put my merged tree here:
lp:~knielsen/maria/mariadb-5.1-pbxt-merge-new
I just built it and tested: bzr merge lp:pbxt And all looks good :)
Note that this tree is identical source-wise with the existing lp:maria tree, just the history now includes all of the PBXT changesets so history and merges can be tracked as per Stewarts method. So this does not include newest PBXT yet, just PBXT up to revision 608 (revid:paul.mccullagh@primebase.org-20090402202852-wa2fbcmrdy7gda2f).
OK. Great.
I did a quick test merge of newest PBXT just to check things, and it seems to work well. Most changes automerged. For deleted/renamed/newly added files, a conflict is flagged that must be resolved manually. Changes to files in PBXT that are omitted from the MariaDB source (COPYING, README, etc) also are flagged as a conflict that needs to be resolved. This is to be expected.
Yup. OK.
I did see some wierd stuff in some testing. Eg. the `bzr diff` of my merge changeset is not empty as expected, rather it shows one diff removing a file and another identical (but reverse) diff adding that file back. I also saw a strange bzr error message in one case of testing merge. I guess we are (ab)using some uncommon corners of bzr here, but hopefully any problems/bug in bzr can be fixed.
One think I wonder is what to do with the PBXT test suite in mysql-test/suite/pbxt/. In my merge I merged it into place with `bzr mv` so that merge tracking and history is kept.
So I am not entirely sure how I should handle changes in the future. Maybe you have a suggestion.
However, I don't know how well this will work for .result files, as I guess they are going to differ between MariaDB, Drizzle, and MySQL?
Yes, this is a problem. And some of the tests have also be changed for MariaDB.
Not 100% sure how we will deal with this. Could be that .result files cannot be meaningfully merged automatically. In any case, I think this is a usable starting point, and we can refine the .result handling later as we understand better how to do it. Just let me know if you think otherwise.
Yes, I think we should just leave it as is for the moment, until we have a better idea. My original idea with the PBXT test suite is that it will act as a basic verification for the engine within MariaDB. The main testing of PBXT will probably continue as a plug-in for MySQL. In the PBXT tree, and can do "make test", and PBXT is automatically tested using the MySQL server that it was compiled with. So I was thinking of PBXT test suite for MariaDB as a branch of the PBXT tests that would never be merged again. Instead any bugs that turn up in MariaDB with PBXT would be added to this test suite. And, the test suite will be continually updated for any changes in MariaDB itself. This is similar to how things are in Drizzle, where PBXT is also tested against a different set of tests. Here I was also not planning to merge tests between Drizzle and the PBXT tree. One good thing about the Drizzle tests though, is that the same tests run with both InnoDB and PBXT. Would be great to add something like that to MariaDB.
Here are the exact commands I did to make the merge:
bzr branch -r'revid:paul.mccullagh@primebase.org-20090402202852-wa2fbcmrdy7gda2f ' pbxt tmp-pbxt-rename cd tmp-pbxt-rename/ bzr co bzr mkdir storage bzr mkdir storage/pbxt bzr mv `ls|grep -v storage` storage/pbxt/ bzr commit -m"Move all PBXT files into storage/pbxt/ in preparation for merging into MariaDB." cd .. bzr branch -r'revid:paul.mccullagh@primebase.org-20090407105746-tolv5dita1d3eavm ' mariadb-5.1 tmp-mariadb-pbxt-merge cd tmp-mariadb-pbxt-merge/ bzr co bzr merge ../tmp-pbxt-rename -r0..-1 bzr mv storage storage.new bzr mv storage.moved storage bzr rm storage/pbxt bzr rm mysql-test/suite/pbxt/t bzr rm mysql-test/suite/pbxt/r bzr mv storage.new/pbxt/test/mysql-test/t mysql-test/suite/pbxt/t bzr mv storage.new/pbxt/test/mysql-test/r mysql-test/suite/pbxt/r bzr mv storage.new/pbxt storage/pbxt bzr rm --force storage.new # bzr rm + manual patch. Applying same changes done manually in previous PBXT merge. bzr resolved storage bzr ci -m "Re-merge PBXT into MariaDB to get proper history and merge tracking." cd .. bzr branch mariadb-5.1 mariadb-5.1-pbxt-merge-new cd mariadb-5.1-pbxt-merge-new bzr co bzr merge ../tmp-mariadb-pbxt-merge bzr ci -m "Pull new/improved PBXT merge into latest MariaDB trunk."
So I used the revisions of MariaDB and PBXT that were also used to do the original merge. And I manually applied the same changes in the merge that you apparently did in your original merge of PBXT into MariaDB, mainly deleting some files like COPYING, configure.in, removing a number of non-pbxt test cases, and fixing some .result files.
So I hope this will be a good base for future merging, with this it should be possible for either PBXT or MariaDB people to pull latest from PBXT into MariaDB without either getting in the way of the other. And it should be quite useful to us to have PBXT history available if debugging etc.
Yes. I agree. This works well (thanks Stewart). Please go ahead and merge to the MariaDB trunk. Thanks! Paul
On May 11, 2009, at 7:17 AM, Stewart Smith wrote:
With BZR you can merge two trees without a common ancestor. This is useful for things like storage engines, file systems, so when they're merged, you can bring the whole revision history in.
This is how I did it for PBXT and Drizzle:
bzr branch lp:drizzle drizzle-pbxt bzr branch lp:pbxt pbxt bzr branch pbxt pbxt-rename pushd pbxt-rename mkdir storage/pbxt
Is this correct, or should it be bzr mkdir storage/pbxt?
I used `bzr mkdir`, got an error from bzr otherwise.
bzr merge ../pbxt-rename -r1..-1 (fix up a conflict on storage/ or something... pretty easy, this is just done so that auto merges happen on file ops in main pbxt)
I used -r0..-1 (not sure if it makes a difference?). And above log shows how I resolved the conflicts.
bzr commit
and then, when mainline pbxt is updated pushd drizzle-pbxt bzr merge lp:pbxt
- Kristian.
-- Paul McCullagh PrimeBase Technologies www.primebase.org www.blobstreaming.org pbxt.blogspot.com