Hi, Kristian! On Jan 13, Kristian Nielsen wrote:
Hi Serg, Daniel,
Some time ago there was discussion of slow Buildbot upload speed (sorry, don't have the original mail thread to reply to). I found some new information on this.
Turns out the real problem was that hasky:/media/backup/archive/pack was full. And apparently, the error handling in Buildbot upload step is broken, so the error is not caught properly but just causes the upload to hang.
Ouch
With respect to slow uploads, while trying various things, one thing I did was increase the upload blocksize in Buildbot config. Now I see tarbake upload (50MB) taking just 1 minute, which seems fair (probably it can be more if the network or buildbot is busy).
So I'm wondering if the slow upload that was seen was in fact a combination of too low default blocksize, and uploads occasionally getting _completely_ stuck?
No, I was fixing only downloads, they were never stuck, just slow. Perhaps because of too small blocksize, indeed.
It's of course still fine to move to using rsync, as Serg proposed earlier; just wanted to mention what I found.
Right, the benefit of rsync - the tarball is downloaded only once per build host, not once per VM. Regards, Sergei