Hi, I agree with you. If we want to know, what Google has developed as encryption feature, we will have to wait for your source code to be published. In the meantime, you can find our concept for the encryption on our website: http://bit.ly/1slJyuI Feedback (negative and positive) from all of you is welcome - and needed! Best Regards Elmar Am 17.06.2014 um 12:50 schrieb Jonas Oreland <jonaso@google.com>:
Hi again,
by "interfaces" I was looking for the Maria DB place/ function / hook... where you are enhancing the MariaDB Code.
I'm not sure how to convey this in a digestible form, attaching diffstats below. Not sure if it's helps :-(
There are many aspects of it. And each of the sub-projects (innodb data, innodb log, maria, tempfiles, binlog) has "interesting" details.
/Jonas
storage/innodb has this diffstat: CMakeLists.txt | 2 btr/btr0cur.cc | 9 buf/buf0buf.cc | 213 +++++ buf/buf0checksum.cc | 8 buf/buf0dblwr.cc | 40 - buf/buf0flu.cc | 6 buf/buf0rea.cc | 7 dict/dict0load.cc | 8 fil/fil0crypt.cc | 1986 +++++++++++++++++++++++++++++++++++++++++++++++++++ fil/fil0fil.cc | 280 ++++++- fsp/fsp0fsp.cc | 36 handler/ha_innodb.cc | 110 ++ handler/i_s.cc | 292 +++++++ handler/i_s.h | 1 include/buf0buf.h | 60 + include/buf0buf.ic | 29 include/fil0fil.h | 266 ++++++ include/fsp0fsp.h | 9 include/log0crypt.h | 85 ++ include/log0log.h | 21 include/log0recv.h | 5 include/mtr0log.ic | 2 include/mtr0mtr.h | 8 include/srv0srv.h | 8 log/log0crypt.cc | 256 ++++++ log/log0log.cc | 93 ++ log/log0recv.cc | 35 mtr/mtr0log.cc | 4 row/row0import.cc | 3 srv/srv0srv.cc | 14 srv/srv0start.cc | 29 31 files changed, 3853 insertions(+), 72 deletions(-)
storage/maria has this diffstat: CMakeLists.txt | 12 ha_maria.cc | 12 ma_bitmap.c | 63 ++-- ma_blockrec.c | 222 ++++++++------ ma_blockrec.h | 26 + ma_check.c | 49 +-- ma_checkpoint.c | 4 ma_close.c | 2 ma_create.c | 56 +++ ma_crypt.c | 464 ++++++++++++++++++++++++++++++ ma_crypt.h | 26 + ma_delete.c | 2 ma_key_recover.c | 10 ma_loghandler.c | 63 +--- ma_open.c | 48 ++- ma_pagecache.c | 154 ++++++--- ma_pagecache.h | 34 +- ma_pagecrc.c | 118 ++++--- ma_static.c | 1 ma_write.c | 24 - maria_def.h | 81 ++--- unittest/ma_pagecache_consist.c | 28 - unittest/ma_pagecache_rwconsist.c | 27 - unittest/ma_pagecache_rwconsist2.c | 27 - unittest/ma_pagecache_single.c | 27 - unittest/ma_test_loghandler_pagecache-t.c | 29 - 26 files changed, 1102 insertions(+), 507 deletions(-)
A noticeable difference between innodb and maria is that we didn't implement encryption of the log for maria, as we only added support for temporary tables. For maria we also only added encryption support for BLOCK format but added all the features to this format so that it was usable for all temp-table scenarios. maria also doesn't have key-rotation feature like innodb has.
I couldn't (as) easily extract diffstats for binlog and tempfile encryption. You have to wait for the code to get published...
On Tue, Jun 17, 2014 at 7:29 AM, Elmar Eperiesi-Beck <elmar@eperiesi-beck.de> wrote: Hi, by "interfaces" I was looking for the Maria DB place/ function / hook... where you are enhancing the MariaDB Code. This would help me to understand what you are trying to do.
Elmar
Am 17.06.2014 um 07:02 schrieb Jonas Oreland <jonaso@google.com>:
Hi again,
What is the type of license of your code?
I asked internally about license, and it seems like we releasing dual gpl2/apache licensed code.
I would like to know, which interfaces from maria-DB you are using.
I don't 100% understand the question. We didn't write any actual encryption code, but used the one provided in openssl. Other than that, we didn't really "use interfaces", but rather added/modified functionality/interfaces here and there.
Can you be more specific ?
/Jonas
On Sat, Jun 7, 2014 at 11:20 PM, Elmar Eperiesi-Beck <elmar@eperiesi-beck.de> wrote: Hi! We (eperi) would be glad to do a joined work with Google. Our solution works with MS-SQL, Oracle and other DBs and we are currently porting it to MariaDB - and - as Monty said - its never to late to put some sources together and make the best for the open source community.
What is the type of license of your code?
Jonas, I am looking forward to connect to you directly.
Regards Elmar
Hi!
Hi Jonas, (same Jonas we know from NDBCLUSTER? :-) Good to see you again)
On 6 Jun 2014, at 02:31, Jonas Oreland <jonaso@google.com> wrote:
Hi there, I read this blog post http://monty-says.blogspot.com/2014/05/for-your-eyes-only-or-adding-better.h... and wanted to inform you that we at Google has developed on-disk/block-level encryption for Innodb, aria (as used by temporary tables), binlogs and temp-files. The code is not yet published, but we expect it to be within a few weeks or so. We (of course?) think that it would be better if you instead of developing new code spent the time testing/reviewing ours.
We are out course happy to do this!
I'm happy to answer questions on the topic, and will let you know once we've published it.
The main question I have about the Innodb encryption is if it based on the compression code we did for fusion-io? The idea we had on our side was that by using the new compression hooks we could add encryption with very little changes to the Innodb code. Looking forward to when you are ready to publish the code so we can discuss your changes in detail.
This is great news!
From what I gather, from Monty's blog post (and a 1:1 we had some time back), this is something done by a partner/external company that has a mostly OSS solution, that we should integrate into 10.1
Yes, that's correct. It I would have known that Google was working on encryption I would have included them in my discussions with eperi. Fortunately it's not yet too late to do this. I am sure eperi would like to work on the Google code as a base!
That said, Google's release of something that works for InnoDB, Aria, binlogs, temp files (and presumably not too hard to add for MyISAM) is something we should definitely review and target for 10.1
Yes!
Regards, Monty