One question to this: Why not simply use 'compressed protocol' for the connection? Alright, the question below is regarding A) binlog size and B) transaction log size as well as bandwidth, the compressed client protocol does not address either issue, but does make accessing the data transparent for compressed and uncompressed clients alike. One database access solution that I like would be in regards to the field's storage itself and I have infact experimented and/or used this in varying circumstances with little inconvenience even under significant load. I simply took my blob field and did this to it with either function based inserts or on insert/update triggers COMPRESS(NEW.blob_field) this does actually address the bloat of the transaction log the statement based binlog however (I could be wrong) but does feature this being executed on each server. The overhead for 65K and under text fields was significant; however for backwards compatibility a view may need to be constructed to allow plain text querying eg: SELECT field1,field2,UNCOMPRESS(blob_field) AS blob_field until code changes are reflected. However a 'compressed' flag on the blob field would be awesome which would allow backwards compliant clients to allow server side decompression to occur, however if you could with a session or connection parameter, elect to offload this transparently to the client side to save on server CPU it would be an impressive efficiency gain with flexibility.

Just a suggestion from my own experience, I would be interested to hear how feasible this is. Jakob Lorberblatt

Peter Webyog

On Fri, Sep 30, 2011 at 22:31, wrote:

...

----------------------------------------------------------------------- WORKLOG TASK -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- TASK...........: client-side compressed types CREATION DATE..: Fri, 30 Sep 2011, 20:31 SUPERVISOR.....: IMPLEMENTOR....: COPIES TO......: CATEGORY.......: Client-BackLog TASK ID........: 240 (http://askmonty.org/worklog/?tid=240) VERSION........: WorkLog-4.0 STATUS.........: Un-Assigned PRIORITY.......: 60 WORKED HOURS...: 0 ESTIMATE.......: 0 (hours remain) ORIG. ESTIMATE.: 0

PROGRESS NOTES:

DESCRIPTION:

Is it possible to add a variant of the blob types that is compressed on the client side? By this I mean that the client library handles compression and decompression so that the client code does not have to deal with it.

The benefit from this is reduced load. There will be less network traffic between client/server and master/slave. This uses less space in the binlog, relay log, innodb transaction log and database files. The alternative is to use compression between client/server and master/slave but that doesn't help with the size of the binlog file nor with the innodb datafiles or log file.

We have too many cases where huge text/blob values store uncompressed JSON and the bloat from that hurts performance.

ESTIMATED WORK TIME

ESTIMATED COMPLETION DATE ----------------------------------------------------------------------- WorkLog (v4.0.0)

_______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp

_______________________________________________ Mailing list: https://launchpad.net/~maria-developers Post to : maria-developers@lists.launchpad.net Unsubscribe : https://launchpad.net/~maria-developers More help : https://help.launchpad.net/ListHelp