Hello Daniel, 2014-02-21 17:24 GMT+02:00 Daniel Bartholomew :

I'll start work on the Sid VMs today.

How is it going with this? It is a prerequisite that needs to be in place before I can start the process towards merging my debian/* work into upstream. -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

2014-02-27 16:19 GMT+02:00 Daniel Bartholomew :

I'm having some trouble with the Debian VMs. The installs are failing midway through. Everything appears to be going fine up until the installer just sort of freezes and it never finishes the install. I'm downloading other iso files to see if I can find a set that work.

Hmm.. it seems the current unstable installation image is, well, very unstable. Try installing using testing installation image from http://cdimage.debian.org/cdimage/daily-builds/daily/arch-latest/amd64/iso-c... and when done run sudo -s sed -i s/testing/unstable/g /etc/apt/sources.list apt-get update apt-get upgrade apt-get dist-upgrade ..and the system has turned into the unstable version. -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

Hello! Thanks for your help! 2014-03-06 9:32 GMT+02:00 Daniel Bartholomew : ..

I was able at last to create a set of unstable sid VMs using a variant of what you suggested. I had trouble with installing from testing iso files, similar to the issues I experienced with unstable iso files. So what I ended up doing was starting with our existing wheezy VMs and then updating them first to jessie and then to sid. The builders in Buildbot are called: kvm-deb-sid-amd64 and kvm-deb-sid-i386

Good, now the virtual machines exist...

I ran into an issue with getting the thrift libs to compile on the amd64 version, so I know that at least that bit is not right (there may be other issues). But the VMs are there and in Buildbot which is better than yesterday when they weren't in there at all. :)

..but it seems you just duplicate the existing model of build bot VMs. It is of course good to have more VMs that build and test MariaDB bzr commits, but we need a little bit more here in preparation of the debian/* merge from Debian back to MariaDB trunk. Kristian wrote on Feb 4th:

...

And while you are at it, please also consider setting up an instance 'kvm-deb-sid-pbuilder-amd64' that provides a Debian unstable

Daniel, can you prepare new sets of VMs for Ubuntu trusty and Debian sid? Then I can try to setup Buildbot to do these builds.

I assumed this meant that Kristian would finish the setup. Kristian, are you following this? You you Daniel please check out the thread called "MariaDB" on maria-developers starting here: https://lists.launchpad.net/maria-developers/msg06747.html https://lists.launchpad.net/maria-developers/msg06765.html https://lists.launchpad.net/maria-developers/msg06767.html https://lists.launchpad.net/maria-developers/msg06771.html I guess the definition of what we want to achieve was vague and the instructions terse. I'll try to write a more detailed list of commands and setups that needs to be done, but I unfortunately don't have time to do that today or tomorrow, so it will be next week.. sorry

One question: Because these VMs are what they are, should I plan on refreshing them periodically so that they stay current? If so, how often should I do it?

Install unattended-upgrades and let the package do it's work automatically. For some installs that do not run automatically you might want to manually run apt-get upgrade / apt-get dist-upgrade every few months. Pbuilder upgrade could be run maybe weekly. -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

2014-03-06 12:20 GMT+02:00 Kristian Nielsen :

...

I assumed this meant that Kristian would finish the setup. Kristian, are you following this?

I am. Unfortunately, I am extremely busy at the moment with several tasks.

I will try to get something done within a week or so, if possible... meanwhile, if you can make a list as detailed as possible about what you want me to do, that would be a great help. At the moment I am not sure about what exactly I need to do... Stuff like file names, repository URLs, how to get buildbot notifications from github, ...

This was actually already pretty detailed: https://lists.launchpad.net/maria-developers/msg06767.html You can just copy-paste. And for Github integration - what options in the build bot is there for accepting notifications? Can build bot do something like this http://developer.github.com/webhooks/testing/ ? If not, then the commands I sent can just be but in a cron job and run daily. There is the 'git fetch -- if HEAD..master' check that will yield true only if there are new commits and then run the build commands. -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

Kristian Nielsen writes:

But that is not going to work in Buildbot, we do not have 9+ GB of memory available for builds. I think to fix this requires changes in your packaging files?

Or alternatively - do you have a machine available? Then we could set that up as a buildslave, to run these builds. Then it would also be easiler for you to access any files from the build for investigation in case of error. The machine would only be used for your pushes - but of course, it depends on whether you have a machine available... - Kristian.

Otto Kekäläinen writes:

Change the line 3902 time git-buildpackage --git-pristine-tar --git-notify=false --git-pbuilder --git-dist=trusty --git-arch=amd64 to 3902 time git-buildpackage --git-pristine-tar --git-notify=false

Ok, I did that, we will see how it goes. Buildbot is suffering a lot from insufficient resources, one of my test builds has been waiting for > 6 hours, and has not started yet :-/. So some patience is needed.

On the first time you should run manually apt-get install git-buildpackage gbp-clone --pristine-tar git://github.com/ottok/mariadb-5.5.git

For now, I opted to just do this from scratch in every build. It won't be run that often anyway, I suppose. Maybe it would make sense to do this in a permanent VM (one that is updated in each build and is not reset for each new build). We have a couple that do that already, but it will need some dedicated copies of the VM images. I think you were already discussing this with Daniel, related to keepting the VM up-to-date with Debian sid? The build could then just run apt-get dist-upgrade at the start.

(gbp is a wrapper around git, it just makes sure git pulls in all branches and tags that git-buildpackage needs)

Ok, I see, I'm not really much familiar with .deb build infrastructure.

About your second message: yes, I have machines available and I could try to setup a mariadb-10.0 build on my own slave. Can you send me a link that describes how slaves work and how I set one up?

Here is the template we use to send for new build slaves: ----------------------------------------------------------------------- From: XXX-from Subject: Buildbot slave account details To: XXX-to Thank you very much for offering to run a Buildbot slave builder for MariaDB! I have created a slave account with these details: host: hasky.askmonty.org port: 9989 slave name: XXX-name password: XXX-passwd build directory: XXX-dir You will need this information when setting up the Buildbot slave. Generic information about setting up can be found in the Buildbot manual: http://docs.buildbot.net/0.8.5/full.html Just grab me on IRC or send mail if you have questions. And thanks again for offering to help the MariaDB project in this way! ----------------------------------------------------------------------- You should be able to install buildbot with apt-get. If you inform Daniel about how many buildslaves you will be running, he should be able to set up the accounts inside buildbot and inform you about the values you need for the XXX'es. (I am not sure if this would be one buildslave doing both trusty and sid? Or two buildslaves (on the same machine, using VMs or chroots), one for each? Or just one buildslave doing one of them only? Anyway, either could work, whatever makes most sense). - Kristian.

2014-03-17 9:24 GMT+02:00 Kristian Nielsen :

...

...

3902 time git-buildpackage --git-pristine-tar --git-notify=false

Ok, I did that, we will see how it goes.

Seems to work now, except that it fails with this:

gpg: /tmp/debsign.vybBjDWq/mariadb-5.5_5.5.36-1.dsc: clearsign failed: secret key not available

I suppose we need to add an option to skip the signing? But I didn't spot any obvious way to do this in `man git-buildpackage`, maybe you know what to do?

Yes, they are well hidden. The dpkg-buildpackage needs to be run with 'dpkg-buildpackage -us -uc' and to pass those do like this: time git-buildpackage --git-pristine-tar --git-notify=false -us -uc Although it would be nice to have a MariaDB.org key that would sign all deb and rpm binaries and also the source tar.gz releases, but that is another story for another day to implement.. Btw, 'time' could be dropped from the beginning as buildbot already tracks the build time in other ways. Tak Kristian! -- Check out our blog at http://seravo.fi/blog and follow @ottokekalainen

Otto Kekäläinen writes:

2014-03-14 12:59 GMT+02:00 Kristian Nielsen :

...

Create a new hook ("Add webhook" button), Make it post to this url:

https://buildbot.askmonty.org/buildbot/change_hook/github?project=MariaDB

These are in place, and I just pushed. Let's see what happens in the following hours.

Hm, it doesn't seem to work :-( No "pending builds" show up on the buildbot pages, and I see this error in the buildbot log: 2014-03-17 21:01:29+0200 [HTTPChannel,167,127.0.0.1] Attempting to load module buildbot.status.web.hooks.github 2014-03-17 21:01:29+0200 [HTTPChannel,167,127.0.0.1] in process_change 2014-03-17 21:01:29+0200 [HTTPChannel,167,127.0.0.1] ERROR:root:Encountered an exception: 2014-03-17 21:01:29+0200 [HTTPChannel,167,127.0.0.1] ERROR:root:Traceback (most recent call last): 2014-03-17 21:01:29+0200 [HTTPChannel,167,127.0.0.1] ERROR:root:File "/usr/lib/python2.7/dist-packages/buildbot/status/web/hooks/github.py", line 100, in getChanges 2014-03-17 21:01:29+0200 [HTTPChannel,167,127.0.0.1] changes = process_change(payload, user, repo, repo_url, project) 2014-03-17 21:01:29+0200 [HTTPChannel,167,127.0.0.1] ERROR:root:File "/usr/lib/python2.7/dist-packages/buildbot/status/web/hooks/github.py", line 139, in process_change 2014-03-17 21:01:29+0200 [HTTPChannel,167,127.0.0.1] when = convertTime( commit['timestamp']) 2014-03-17 21:01:29+0200 [HTTPChannel,167,127.0.0.1] ERROR:root:File "/usr/lib/python2.7/dist-packages/buildbot/status/web/hooks/github.py", line 63, in convertTime 2014-03-17 21:01:29+0200 [HTTPChannel,167,127.0.0.1] result.groups() 2014-03-17 21:01:29+0200 [HTTPChannel,167,127.0.0.1] ERROR:root:AttributeError: 'NoneType' object has no attribute 'groups' 2014-03-17 21:01:29+0200 [HTTPChannel,167,127.0.0.1] Unhandled Error Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/twisted/web/http.py", line 1349, in dataReceived finishCallback(data[contentLength:]) File "/usr/lib/python2.7/dist-packages/twisted/web/http.py", line 1563, in _finishRequestBody self.allContentReceived() File "/usr/lib/python2.7/dist-packages/twisted/web/http.py", line 1618, in allContentReceived req.requestReceived(command, path, version) File "/usr/lib/python2.7/dist-packages/twisted/web/http.py", line 773, in requestReceived self.process() --- <exception caught here> --- File "/usr/lib/python2.7/dist-packages/twisted/web/server.py", line 132, in process self.render(resrc) File "/usr/lib/python2.7/dist-packages/twisted/web/server.py", line 167, in render body = resrc.render(self) File "/usr/lib/python2.7/dist-packages/twisted/web/resource.py", line 216, in render return m(request) File "/usr/lib/python2.7/dist-packages/buildbot/status/web/change_hook.py", line 62, in render_POST changes, src = self.getChanges( request ) File "/usr/lib/python2.7/dist-packages/buildbot/status/web/change_hook.py", line 108, in getChanges changes, src = tempModule.getChanges(request,self.dialects[dialect]) exceptions.TypeError: 'NoneType' object is not iterable I tried pulling your repo into mine and pushing that, and then I get the same error. A quick google shows this as the likely culprit: http://trac.buildbot.net/ticket/2054 Looks like something changed recently with the data format somewhere, which buildbot does not like. Indeed, in the last delivery to the buildbot hook from github, it has this: "timestamp": "2014-03-13T11:08:10Z", Whereas my successful pushes had this: "timestamp": "2014-03-18T09:25:12+01:00", Note how the time zone is specified differently, the "Z" is not parsable by our current Buildbot, apparently :-( Maybe there is a way to avoid that kind of timestamp when committing, as a workaround? (I am not sure whether this is something that git does, or something that github does, though the fact that it works for my commits and not for others suggests git). Otherwise, I suppose we need to ask Daniel to upgrade buildbot to something that fixes this issue. There is a patch in the Buildbot track ticket, but it might be some work to upgrade...

The current error at http://buildbot.askmonty.org/buildbot/builders/debpkg-trusty/builds/3/steps/... will go away of you add to the build command '-us -uc' at the end as I wrote earlier today:

git-buildpackage --git-pristine-tar --git-notify=false -us -uc

Yes, should be fixed.

For http://buildbot.askmonty.org/buildbot/builders/debpkg-sid/builds/0/steps/com...

Should be fixed also (it's just an old build). But we'll need to sort out the problem with the github integration... - Kristian.

Otto Kekäläinen writes:

I assume Github.com changed how they query dates from git, as there is an option for time format output in git. I can't find any time input format options, so that is not anything we can change AFAIK.

The thing is, just today I pushed a test commit to my own repo, and it worked with respect to buildbot, no error. Then I pulled your repo and pushed that, and I got the error. I just tried pushing a test commit again, and no error. So somehow the "Z" as timezone comes in depending on setup. Maybe it depends on which time zone one is in, or has configured locally? I am not sure, I just know that it works for some commits and not for others...

...

Otherwise, I suppose we need to ask Daniel to upgrade buildbot to something that fixes this issue. There is a patch in the Buildbot track ticket, but it might be some work to upgrade... [..] Should be fixed also (it's just an old build).

But we'll need to sort out the problem with the github integration...

Can you launch the builds manually now once and then we'll wait until the github integration is fixed with a later release of Buildbot?

Done. - Kristian.

Otto Kekäläinen writes:

You're right. This thing seems to be that in this time format there is no +0000 time zone, instead it's called just 'Z'. So all commits done by Danes and Finns will work, but commits by James Page from England will fail. We can live with that and don't need to do anything further about this for now.

I see. Bummer, but I suppose an upgrade will fix it eventually.

Just one check: does build bot run a build for each commit individually, or does it skip commits and just build the latest commit each time a push comes in? From my point of view it does not need to build each commit. The latest commit in each push would be just fine.

It should be just the last commit. At least, that's how it works for bzr/launchpad, I think it's the same for git. - Kristian.