Richard Purdie
2018-11-23 17:02:50 UTC
I've take this quiet period as an opportunity to make some significant
autobuilder changes:
The targets have all been renamed and split up. "nightly-arm" became
"qemuarm" and "beaglebone". The "nightly-" prefixes were dropped.
The nightly target was dropped and replaced by "a-quick" and "a-full".
The difference are a number of targets which don't often find problems.
We're also ramping down on the amount of testing qemumips-lsb/qemuppc-
lsb get for example as they're more minor architectures as things
stand. The "a-" prefix is for sorting in the UI and to separate them
out from the mass of other builds.
There are now selftest targets per "distro" which can replace the
testing that Intel QA have been doing. This is only done for "a-full"
and the selftest on a distro at random is done by "a-quick".
Buildhistory is only being generated for the qemu* targets.
The "template" used for testing qemu or "real-hardware" targets was
standardised as there were weird differences for now good reason.
The structure of "non-release" builds on
https://autobuilder.yocto.io/pub/ was cleaned up and now appears under
a single date+number under the non-release directory which was how it
was meant to work. All the other spurious output that was there has
been cleaned up.
XML test results are now being published for all builds, e.g.:
https://autobuilder.yocto.io/pub/non-release/20181122-11/qemuarm64/
Ptest was added to "a-full" builds for arm and x86 although the arm
builds currently time out. The plan will be to enable arm when we have
an arm build server with KVM support in our rack (next week?). ptest
execution for x86 takes around 3 hours and some sample result output
is:
https://autobuilder.yocto.io/pub/non-release/20181122-11/qemux86-64-ptest/
"meta-oe" and "meta-virt" test targets were added for experimentation,
testing meta-openembedded and meta-virtualization respectively. These
are work in progress currently just doing a "world -k" build for
qemux86-64 which fails (they take around 2.5 hours).
We now run "a-quick" on master at 1am every day which will make
buildhistory more up to date and useful for the master branch and give
us better comparison data for -next and mut.
wine is installed onto the ubuntu1804 workers and the mingw testing was
split into a meta-mingw target, now with 32 and 64 bit build tests and
64 bit runtime tests. With master-next of meta-mingw this appears to
run the test. Huge kudos to Joshua Watt for putting that testing
together. If we install more onto the workers we could probably test 32
bit as well, I'm torn on that right now as its a lot more dependencies
on the workers.
Building older releases is now broken until I sort out the
corresponding helper changes for those branches but one thing at a
time!
If anyone would like to help with xml results file analysis tooling,
that is where we need work next to be able to summarise the ptest
results and do results comparison. I know Ee Peng has some work in this
area which I will be looking at next.
Cheers,
Richard
--
autobuilder changes:
The targets have all been renamed and split up. "nightly-arm" became
"qemuarm" and "beaglebone". The "nightly-" prefixes were dropped.
The nightly target was dropped and replaced by "a-quick" and "a-full".
The difference are a number of targets which don't often find problems.
We're also ramping down on the amount of testing qemumips-lsb/qemuppc-
lsb get for example as they're more minor architectures as things
stand. The "a-" prefix is for sorting in the UI and to separate them
out from the mass of other builds.
There are now selftest targets per "distro" which can replace the
testing that Intel QA have been doing. This is only done for "a-full"
and the selftest on a distro at random is done by "a-quick".
Buildhistory is only being generated for the qemu* targets.
The "template" used for testing qemu or "real-hardware" targets was
standardised as there were weird differences for now good reason.
The structure of "non-release" builds on
https://autobuilder.yocto.io/pub/ was cleaned up and now appears under
a single date+number under the non-release directory which was how it
was meant to work. All the other spurious output that was there has
been cleaned up.
XML test results are now being published for all builds, e.g.:
https://autobuilder.yocto.io/pub/non-release/20181122-11/qemuarm64/
Ptest was added to "a-full" builds for arm and x86 although the arm
builds currently time out. The plan will be to enable arm when we have
an arm build server with KVM support in our rack (next week?). ptest
execution for x86 takes around 3 hours and some sample result output
is:
https://autobuilder.yocto.io/pub/non-release/20181122-11/qemux86-64-ptest/
"meta-oe" and "meta-virt" test targets were added for experimentation,
testing meta-openembedded and meta-virtualization respectively. These
are work in progress currently just doing a "world -k" build for
qemux86-64 which fails (they take around 2.5 hours).
We now run "a-quick" on master at 1am every day which will make
buildhistory more up to date and useful for the master branch and give
us better comparison data for -next and mut.
wine is installed onto the ubuntu1804 workers and the mingw testing was
split into a meta-mingw target, now with 32 and 64 bit build tests and
64 bit runtime tests. With master-next of meta-mingw this appears to
run the test. Huge kudos to Joshua Watt for putting that testing
together. If we install more onto the workers we could probably test 32
bit as well, I'm torn on that right now as its a lot more dependencies
on the workers.
Building older releases is now broken until I sort out the
corresponding helper changes for those branches but one thing at a
time!
If anyone would like to help with xml results file analysis tooling,
that is where we need work next to be able to summarise the ptest
results and do results comparison. I know Ee Peng has some work in this
area which I will be looking at next.
Cheers,
Richard
--