Commit Graph

255 Commits

Author SHA1 Message Date
Ross Burton 8dee33872b lib/oeqa/runtime: output more logging from systemd when services have failed
If some services have failed to start, get the status of them and some of their
log to help debug the problem.

(From OE-Core rev: e371765431be25b81dbdb385233e3db5851e59d0)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05 14:25:30 +00:00
Ross Burton 5c58ba9fca lib/oeqa/runtime: print connman status if connman failed to start
If connman isn't running and we're running under systemd, use systemctl to get
the state according to systemd and the end of the connman log.

(From OE-Core rev: e77bcc8e4baf11701b105d071a979a21a2a303d5)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05 14:25:30 +00:00
Corneliu Stoicescu a67654e06e lib/oeqa/selftest: add tests for PR service
(From OE-Core rev: 162cfffb32d909f9cac88b4f872173d21524d8ed)

Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05 14:24:43 +00:00
Stefan Stanacar 347cc15ebd lib/oeqa/selftest: buildoptions: small fixes for some tests
While harmless, we should overwrite the config not append to it,
and use m4 as target, otherwise the WARN check will
build an entire image and we are not interested in that.
Also add an output check for the WARN_QA test.

(From OE-Core rev: 83b0a9231691aba9a7753350d131b0ab9196815e)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05 14:24:43 +00:00
Stefan Stanacar f17a6937ab lib/oeqa/selftest/base, scripts/oe-selftest: fix wrong remove path and do a complete cleanup at the end
The script should clean-up all the .inc files that might
have been created by tests regardless of the outcome or if
the script is interrupted. (currently the
last test will leave a conf/selftest.inc around, even
if it's not included anywhere)
Also fix delete_recipeinc to actually delete what's supposed to.

(From OE-Core rev: 6008745c56800e0f5f01a756be0701cebd9de4ae)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-05 14:24:43 +00:00
Stefan Stanacar 17e3dc2cb5 lib/oeqa: targetcontrol.py: add abstraction for running tests on different targets
Add a new module which abstracts the target object used by testimage.bbclass

The purpose of this module is to move the deployment of a target from testimage.bbclass,
basically abstracting different implementations of how we setup a target and how it runs commands.
It allows to select one implementation or another by setting TEST_TARGET (currently to: "qemu" and "simpleremote").

QemuTarget is used to start a qemu instance (as it's currently done in testimage.bbclass)
SimpleRemoteTarget is meant for a remote machine (by setting TEST_TARGET_IP) that's already up and running
with network and ssh.
Simply put, it opens the door for running the tests on different types of targets by adding new classes
(maybe qemu-nfsroot or remote-special etc.). One could also override BaseTarget which currently uses
the existing SSHControl module and add a serial implementation.

[ YOCTO #5554 ]

(From OE-Core rev: c1bfd4017f6f6502a68ceb3edf7d2027d02a309d)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:52 +00:00
Stefan Stanacar b4d9b4208b testimage: use the new targetcontrol.py module for running tests
This patch makes the necessary changes for using the targetcontrol.py module
so that one can run the same tests on a qemu instance or a remote machine
based on the value of TEST_TARGET variable: "qemu" or "simpleremote".
The default value is "qemu" which starts a qemu instance and it's the
with what we currently have.

With "simpleremote", the remote machine must be up with network and ssh
and you need to set TEST_TARGET_IP with the IP address of the remote machine
(it can still be a qemu instance that was manually started).

Basically testimage.bbclass now does something along the lines of:
 - load tests -> deploy (prepare) / start target -> run tests.
There were a couple of changes necessary for tests and
also some cleanups/renames that were needed to adjust this change. (use
ip everywhere when refering to target and server_ip when refering to host/build machine)
Also two unnecessary and unsed methods were dropped from sshcontrol.

[ YOCTO #5554 ]

(From OE-Core rev: a7820350fa3271d78ed7476e02f4aef593be1125)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:51 +00:00
Stefan Stanacar d4398d41be lib/oeqa: use the new manifest file for determining if a package is installed
Use the new manifest file instead of ${WORKDIR}/installed_pkgs.txt for determining
if an image has a certain package, because installed_pkgs.txt goes away with rm_work
enabled.
We can't use the IMAGE_MANIFEST var for the file path because that relies on IMAGE_NAME which
changes at every run (because of date), so we use the link which points to the last
one built.

[ YOCTO #5072 ]

(From OE-Core rev: f57c83fc33583c140f668946f3f3e79b960aa9ee)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:51 +00:00
Corneliu Stoicescu 1611b474b6 lib/oeqa/selftest: add test modules for expected bitbake output and bitbake-layers
Tests for bitbake-layers and expected output for some bitbake options.

(From OE-Core rev: 8408a7700cd9cab4559ddae0bbe57f0d7fae5c37)

Signed-off-by: Corneliu Stoicescu <corneliux.stoicescu@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:51 +00:00
Alexandru Palalau f8ee10865d lib/oeqa/selftest: buildoptions.py: add simple image build tests
Build images and tests different build options like RM_OLD_IMAGE
and for WARN_QA/ERROR_QA behaviour.

(From OE-Core rev: 75d2a1a37a18a22b0da7ab5b30cf005c78bc313f)

Signed-off-by: Alexandru Palalau <alexandrux.palalau@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:51 +00:00
Stefan Stanacar 645dd61cd2 scripts/oe-selftest: script to run builds as unittest against bitbake or various scripts
The purpose of oe-selftest is to run unittest modules added from meta/lib/oeqa/selftest,
which are tests against bitbake tools.

Right now the script it's useful for simple tests like:
  - "bitbake --someoption, change some metadata, bitbake X, check something" type scenarios (PR service, error output, etc)
  - or "bitbake-layers <...>" type scripts and yocto-bsp tools.

This commit also adds some helper modules that the tests will use and a base class.
Also, most of the tests will have a dependency on a meta-selftest layer
which contains specially modified recipes/bbappends/include files for the purpose of the tests.
The tests themselves will usually write to ".inc" files from the layer or in conf/selftest.inc
(which is added as an include in local.conf at the start and removed at the end)

It's a simple matter or sourcing the enviroment, adding the meta-selftest layer to bblayers.conf
and running: oe-selftest to get some results. It would finish faster if at least a core-image-minimal
was built before.

[ YOCTO #4740 ]

(From OE-Core rev: 41a4f8fb005328d3a631a9036ceb6dcf75754410)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:50 +00:00
Paul Eggleton 0cb38ee34a classes/buildhistory: add additional variables to image information
Add PACKAGE_EXCLUDE and NO_RECOMMENDATIONS to the info we track for
images, since these can change what ends up in the image.

(From OE-Core rev: a10189366f180b87f5be20b66834b7e7a9bb8c12)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-12-03 17:45:50 +00:00
Richard Purdie 80d522a5bc sstatesig.py: Fix image regeneration issue
With the "ABI safe" recipes, we've been excluding those from signatures. This
is fine in the general case but in the specific case of image recipes it breaks.

A good test case is the interfaces file. Editting this causes init-ifupdown
to rebuild but not an image containing it (e.g. core-image-minimal).

We need to ensure the checksums are added to the image recipes and this change
does that.

(From OE-Core rev: fd085f15e7cd093953f974f69277e130174d551d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-29 10:49:25 +00:00
Ross Burton afe82ef3c4 oeqa/connman: disable unique test
The connman unique test starts another instance of connmand and then does a ps to
verify that there's only one of these running, on the assumption that the new
one has quit because there's already one running (started by init).

However, connmand is forking into the background straight away so there's a race
between running ps and the second connmand discovering the first and exiting.
This race can be seen because the test displays the output of ps, and by the
time that second ps has been executed the new connmand has exited.

This is a classic race condition and on a heavily loaded autobuilder inserting
an arbitrary sleep isn't wise.  In the scheme of things this test isn't very
useful, so delete it.

(From OE-Core rev: 80ef721140c79e29430d0a5692a5c176db0061e6)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-22 12:03:02 +00:00
Paul Eggleton 36255d4c23 buildhistory_analysis: fix error when comparing image contents
OE-Core commit b7de1eaac9eed559b2d68058f5de67de74a6cb58 added an extra
argument to the compare_dict_blobs() function but missed adding the
argument to one call to compare two versions of the image-info.txt file.

(From OE-Core rev: 24a45d752c3e3d0d8b59c040355e4fe7de22b041)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-18 12:44:04 +00:00
Richard Purdie c393536210 lib/oe/path: Fixup handling of .* files in top level paths for copyhardlinktree()
Files named .* in the top level of directories handled by this function
were getting lost after the directory copying command was fixed. Rather
than complicate the function further, use cpio instead.

(From OE-Core rev: 59612905e9bb865762667aa7b3cb06e53c3a4071)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-12 10:34:09 +00:00
Richard Purdie 8a07a176d7 lib/oe/path: Fix performance issue got copyhardlinktree()
With the directory copy was added to avoid race issues, it wasn't noticed that
tar was recursing the directories and copying files too. This is completely
crazy when we hardlink those files in the next command.

Resolve the issue by telling tar not to recurse. This gives a significant
performance boost to various parts of the system (do_package for linux-yocto
256s -> 178s for example).

(From OE-Core rev: 8b90ed084d59b4e07aa547255d327b25cfb2ee2b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-12 10:34:09 +00:00
Stefan Stanacar 1e16634af6 lib/oeqa/utils: sshcontrol: make timeout depend on output
Instead of running the commands with a fixed timeout,
we should kill the command if there is no output
for timeout seconds.
Also changed some strings/comments.

(From OE-Core rev: beea86fa9637fd629719980e14beea758847b8f3)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-08 17:25:36 +00:00
Richard Purdie 26559c5816 lib/oe/sstatesig: Ensure packagegroups don't continually rebuild
packagegroups are allarch and shouldn't change depending on the target
or machine selected. In general they should have good stable namespaces
for their dependencies. As such we can exclude them from rebuilding when
dependency checksums change.

(From OE-Core rev: 80b065ff46322ec0cad039dfd9eb2d010168dba6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-08 17:25:35 +00:00
Stefan Stanacar 19aeca1cac lib/oeqa: add oeTest superclass
Add oeTest superclass, make oeRuntimeTest inherit that and
make the necesarry adjustments. Tests in lib/oeqa/runtime don't
change, they still inherit oeRuntimeTest.
We should do this because oetest.py in the future can be a base module
for more stuff than oeRuntimeTest.

(From OE-Core rev: cd4ed41a070bd52c446ac3df8337f17ab9421145)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-11-05 22:04:28 +00:00
Paul Eggleton f995febd2e buildhistory-diff: add ability to report version changes
Add a -v/--report-ver option to report changes in PKGE/PKGV/PKGR even
if the value is the same as the default from PE/PV/PR.

Also add a -a/--report-all option to report all changes instead of just
the default significant ones.

Addresses [YOCTO #5263].

(From OE-Core rev: b7de1eaac9eed559b2d68058f5de67de74a6cb58)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-26 15:59:13 +01:00
Stefan Stanacar d3cb369365 lib/oeqa/runtime: build kernel module on target test
Builds a simple Hello World module on target.
Added to the defaults for core-image-sato-sdk.

(From OE-Core rev: c61c3dee162aa1f5bf31b2a09d8b916dc1712056)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-18 15:58:16 +01:00
Richard Purdie bf19a8617d lib/oe/path: Fix copytree functions for tar 1.27
tar version 1.27 returns:

tar: --same-order option cannot be used with -c

with the commandlines we have been using. We can remove the -s option (which
is --same-order) to remove the error.

(From OE-Core rev: 69c26e795c117aabfaf313abbfd10e70ede633d9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-10-14 16:55:21 +01:00
Stefan Stanacar 41f88ee1db lib/oeqa/runtime: ping: wait for 5 echo replies
Instead of considering that ping test passed after 1 reply,
wait for at least 5 consecutive replies in 60 seconds (which should
be enough time for connman to reconfigure the interface in systemd
images and help with the fake ssh/tests fails.)

(From OE-Core rev: 6fd19a9df0ad25b2822f12e2c3a97f1b71068d4e)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-26 16:37:55 +01:00
Stefan Stanacar ad77d10386 lib/oeqa/runtime: cleanup and improve output readability for some tests
- move everything in the same test. setUp/tearDown aren't quite
the right thing here, everything it's part of the same test. (and
it get's confusing when ssh fails)

ldd:
   - change test name and add output to error message

vnc:
   - remove unnecessary check as there is no point in doing
both ps and netstat. Also improve error output a bit.

(From OE-Core rev: 2f91bb438cfcdd0a40daed5902b6e98fc0aee67f)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:33 +01:00
Stefan Stanacar f3ab2fc731 lib/oeqa/utils: sshcontrol: log how long the last command take
It might be useful for debugging to have in the ssh log
the number of seconds a command has run.

(From OE-Core rev: 48b9e45b9716130b015ae2ab7398d6aa243933dc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-22 12:19:33 +01:00
Ross Burton 4a224d31c8 qemurunner: disable grabs in automated testing
Use the new QEMU_DONT_GRAB environment variable to disable grabs,
finally/hopefully solving the random hangs that the autobuilder has been hitting
for a while.

[ YOCTO #5131 ]

(From OE-Core rev: 32f9575a565f350649264c11eceba8311584b0fd)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-18 17:52:04 +01:00
Saul Wold 14fee32867 oeqa/runtime/smart: Increase timeout to 1500 test
(From OE-Core rev: a99edb5552839fd50326dd3aa4ee2f36f6026882)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-17 19:34:55 +01:00
Stefan Stanacar 214bb6828e lib/oeqa/runtime: timeout increases
Increase the timeout for smart commands as under load for qemumips
it's still to small. Also give ping more time fixing a potential
timeout for sato systemd.

(From OE-Core rev: daa3ad5807f6fc0d15b9310937d07a16edac6d22)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:21:00 +01:00
Richard Purdie 8ebe7be3d9 bitbake.conf/package: Collapse PKGDATA_DIR into a single machine specific directory
Currently we have a hierarchy of pkgdata directories and the code has to put together
a search path and look through each in turn until it finds the data it needs.

This has lead to a number of hardcoded paths and file globing which
is unpredictable and undesirable. Worse, certain tricks that should be
easy like a GL specific package architecture become problematic with the
curretn search paths.

With the modern sstate code, we can do better and construct a single pkgdata
directory for each machine in just the same way as we do for the sysroot. This
is already tried and well tested. With such a single directory, all the code that
iterated through multiple pkgdata directories and simply be removed and give
a significant simplification of the code. Even existing build directories adapt
to the change well since the package contents doesn't change, just the location
they're installed to and the stamp for them.

The only complication is the we need a different shlibs directory for each
multilib. These are only used by package.bbclass and the simple fix is to
add MLPREFIX to the shlib directory name. This means the multilib packages will
repackage and the sstate checksum will change but an existing build directory
will adapt to the changes safely.

It is close to release however I believe the benefits this patch give us
are worth consideration for inclusion and give us more options for dealing
with problems like the GL one. It also sets the ground work well for
shlibs improvements in 1.6.

(From OE-Core rev: 1b8e4abd2d9c0901d38d89d0f944fe1ffd019379)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:21:00 +01:00
Paul Eggleton 6670be71f7 bitbake.conf: include machine name in DEPLOY_DIR_IMAGE
This allows a clean seperation between image outputs from different
machines, and makes it possible to have convenience symlinks to make
the output ready to deploy.

This did require some surgery in runqemu; if explicit paths to the image
and kernel are not supplied then DEPLOY_DIR_IMAGE needs to be determined
from bitbake or set in the environment. However the script does try to
avoid requiring it unless it really is needed. Corresponding changes
were made in the automated testing code as well.

Based on an RFC patch by Koen Kooi <koen@dominion.thruhere.net>

(From OE-Core rev: 7e90261aec61f79680b5eaeaf5b18c7b795412a4)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-14 08:21:00 +01:00
Stefan Stanacar 01135ee6f2 lib/oeqa/runtime: smart: limit channel add to useful ones
Don't add inappropiate channels on the target.
This happens when building two different machines in the
same dir and then running the tests for each machine.

(From OE-Core rev: 9e4e475b598f6b9bd2b653c72a1c1d5bd5c0b8c9)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-11 23:31:00 +01:00
Richard Purdie 9b84fc7cc2 lib/oeqa/qemurunner: Use a timeout in select() call
A 0 value to select.select() immediately returns with no timeout. This was
pegging the cpu at 100% for the python process which was bad and may be
contributing to some of the timeout problems.

Profile from -P of a core-image-minimal before:

    97526792 function calls (97525652 primitive calls) in 45.189 seconds

and after:

       50204 function calls (49064 primitive calls) in 17.318 seconds

Saving 97.5 million function calls has to be good :)

(From OE-Core rev: c0551436974d179df23418567f18a082830380f6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 23:04:51 +01:00
Stefan Stanacar 41d1c2d630 lib/oeqa/runtime: add basic scanelf test
This uses scanelf from the pax-utils package and scans the binaries in PATH
for TEXTREL and RPATH information. For a sato image with pax-utils installed
it shows no output (which is good).

(From OE-Core rev: 629099ad66f5fa2814e5f7908b426149e8978e43)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 23:04:50 +01:00
Mihai Lindner faed110e66 oeqa/runtime/scp: replace dd call
Use a file object to generate a our test file instead of calling `dd`;
removes dd's output from testimage.log, keeps unittest output clean.
Also remove unused imports.

(From OE-Core rev: 6ac48ffbab29a37b0eada533191878aeae3c91f0)

Signed-off-by: Mihai Lindner <mihaix.lindner@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 23:03:23 +01:00
Mihai Lindner fbb3e5e11f oeqa/utils/sshcontrol: tweak ssh options
Add ssh_options to be used, the same, by ssh and scp:
Decrease LogLevel to ERROR, to suppress warnings (e.g. ssh host
verifications, two warnings in case of having openssh with hpn patches);
We no longer presume that the first line is a warning.

(From OE-Core rev: 1f18d04eec03e586134b6d77ca1c6151c22353dd)

Signed-off-by: Mihai Lindner <mihaix.lindner@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 23:03:23 +01:00
Mihai Lindner f6fed84380 oeqa/oetest: oeRuntimeTest: enable long messages
Set longMessage to True for all tests derived from oeRuntimeTest, in
order to have somewhat info on assertions with cryptic or no messages.

(From OE-Core rev: e5196ebd6c652431a3b80cdc9f60b752401314f4)

Signed-off-by: Mihai Lindner <mihaix.lindner@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-06 23:03:23 +01:00
Stefan Stanacar e3bc93c2eb lib/oeqa/runtime: smart: serve repo on host ip only and increase timeout
Don't start the http server on 0.0.0.0, listen on host ip (end of tap interface) only.
Also use the timeout option (default is 300s for ssh commands) for all the commands
run in this module (mostly because smart update timeouts on mips).

(From OE-Core rev: 8c272641ef3e8410f331ca4133d28dea8f36e4f4)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-03 19:57:38 +01:00
Richard Purdie 554c892ccf meta: Don't use deprecated bitbake API
These have been deprecated for a long time, convert the remaining
references to the correct modules and prepare for removal of the
compatibility support from bitbake.

(From OE-Core rev: 6a39835af2b2b3c7797fe05479341d71a3f3aaf6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-09-01 15:51:42 +01:00
Stefan Stanacar efff0d5504 lib/oeqa/runtime: ping: fix ping false fail
We run the ping test as soon as we reach the login prompt.
But sometimes (seen in sato systemd) we end up with link down/link up stuff like:

    qemux86-64 login: IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
    e1000: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
    IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

The logic behind ping -w 30 -c 1 was to wait at most 30 seconds
for at least one reply,  but there is a catch: reply doesn't seems
to be echo reply but any reply (non-reply means loss not network error)
ping's man page:
    -w deadline
              Specify  a  timeout, in seconds, before ping exits regardless of
              how many packets have been sent or received. In this  case  ping
              does  not  stop after count packet are sent, it waits either for
              deadline expire or until count probes are answered or  for  some
              error notification from network.

Just when the link up/link down happens ping returns:
    From 192.168.7.1 icmp_seq=1 Destination Host Unreachable
    --- 192.168.7.2 ping statistics ---
    1 packets transmitted, 0 received, +1 errors, 100% packet loss, time 0ms

and exits sooner than the 30 seconds timeout.

This patch should do what was originally intended (wait at most
30 seconds for at least one reply).

(From OE-Core rev: 56d144fd22d37189e49cdf3032afb00f0be469c6)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-30 18:08:05 +01:00
Stefan Stanacar 8f32af94c0 lib/oeqa/runtime: syslog: fix test for logger in case of systemd
Recently syslog behaviour changed for systemd images (log
it's in a buffer not in /var/log/messages), account
for the new stuff.

(From OE-Core rev: 32576c4cc1621fa3013eac66c7caaa1e1fd14995)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-30 16:23:47 +01:00
Stefan Stanacar 249dc825a0 lib/oeqa/utils: qemurunner: fix when runqemu errors out
When qemu doesn't start or runqemu errors out
(some wrong option passed or sudo needs a password for setting tap) we
want to display the output but oe-core commit 9de7fe11967576f4a8b24e653c6b9a02e5f6d85b/
poky commit 51588936d4
changed the kill method and broke this code, so let's fix it.

(From OE-Core rev: 923d4bd548911caa0b419c10905895af1e8e7026)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-30 16:23:47 +01:00
Stefan Stanacar c52981c9bc lib/oeqa: move skeletoninit.py where it belongs
OE-core commit fcc59cbcdb1550489d372edf9f465efa7165245f /
poky commit 748ddc39e5 added a new test, but
in the wrong location.
I took the patch from Alex's branch but renamed it from meta/lib/oeqa/runtime/skeleton.py to meta/lib/oeqa/skeletoninit.py before sending. This was
unintentional, it should have been under meta/lib/oeqa/runtime.

(From OE-Core rev: f12c346ef48cb44be2e356e4cf4f28d015c3f507)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-30 16:23:44 +01:00
Stefan Stanacar 5fba9d8c6c classes/testimage: add support for finding tests in other layers
A layer can add tests in lib/oeqa/runtime (provided it extends BBPATH as
normal) and enable them with TEST_SUITES_append = " testname". Test
module names shouldn't collide though.

(From OE-Core rev: e1e347a2d509303e1c566450b0f2b485d3d6629f)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-28 23:36:08 +01:00
Stefan Stanacar 4e62e9cb76 lib/oeqa/runtime: remove some unnecessary checks from setUpModule
These checks are unnecessary.
setUpModule is run when a module is loaded and we
shouldn't run commands on the target here, (plus if
ssh doesn't work we error out in setup multiple times, instead
of skipping the real test, which might depend on test_ssh).

(From OE-Core rev: 188acd0a75e188fd7c0d2979acaf13fd18b12106)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-28 23:36:08 +01:00
Stefan Stanacar 44c3f72684 lib/oeqa/runtime: smart: add checks for smart output
Sometimes smart throws:

    Committing transaction...
    Preparing...                    ######################################## [  0%]
       1:Removing psplash-default   ######################################## [100%]
    error: Couldn't fork %postun: Cannot allocate memory

and returns a 0 exit code (it thinks it succesfully removed the package,
when in reality it didn't), so we need to catch those specifically.

Also, sometimes output from download command is:
    Saving cache...http://192.168.7.1:49456/rpm/x86_64_x32/psplash-default-0.1+git0+afd4e228c6-r15.x86_64_x32.rpm
and that tricks our smart download test, so use a regex there.

(From OE-Core rev: 2ac7783e04f5e8e6005f967e1a6dd65d2fc6a19a)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 16:29:18 +01:00
Stefan Stanacar a855180fe6 lib/oeqa/runtime: multilib: fix typo
The check was obviously wrong and it surfaced
with the recent change in behaviour for skipping tests.

(From OE-Core rev: 4a14535cd493cb2bdd46b2a5f2a1cd2b38161f0a)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 16:29:18 +01:00
Stefan Stanacar 2dfc0e1cc7 lib/oeqa: change behaviour for unskippable tests
When a test module wants to be skipped because it doesn't
apply to the image but it was nevertheless a required
test (one in TEST_SUITES), we issued an warning that it
was a required test and went on with running the module.
Usually all tests in the module failed (e.g gcc tests on a non-sdk image),
but this allowed us to know that something went wrong with the image
(some package/feature didn't make it).

However, instead of just issuing an warning and running the tests
it's better to throw an exception. The traceback will tell us what's wrong,
and we don't run every single test method.
Output will look like this:
--snip--
| NOTE: Test modules  ['oeqa.runtime.ping', 'oeqa.runtime.ssh', 'oeqa.runtime.gcc']
| NOTE: Found 5 tests
| test_ping (oeqa.runtime.ping.PingTest) ... ok
| test_ssh (oeqa.runtime.ssh.SshTest) ... ok
| ERROR
|
| ======================================================================
| ERROR: setUpModule (oeqa.runtime.gcc)
| ----------------------------------------------------------------------
| Traceback (most recent call last):
|   File "/mnt/back/yocto/poky/meta/lib/oeqa/runtime/gcc.py", line 8, in setUpModule
|     skipModule("Image doesn't have tools-sdk in IMAGE_FEATURES")
|   File "/mnt/back/yocto/poky/meta/lib/oeqa/oetest.py", line 108, in skipModule
|     "\nor the image really doesn't have the requred feature/package when it should." % (modname, reason))
| Exception:
| Test gcc wants to be skipped.
| Reason is: Image doesn't have tools-sdk in IMAGE_FEATURES
| Test was required in TEST_SUITES, so either the condition for skipping is wrong
| or the image really doesn't have the requred feature/package when it should.
|
| ----------------------------------------------------------------------
| Ran 2 tests in 1.036s
|
| FAILED (errors=1)
| NOTE: Sending SIGTERM to runqemu
--snip--

(From OE-Core rev: fd51cecf8b258d9f839a0ecebde69d09f75dc468)

Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:21 +01:00
Mihai Prica d7077bc8e5 lib/oeqa/runtime: add iptables, cvs and sudoku projects build tests on target
Downloads iptables/cvs/sudoku-savant sources and builds them on target.

(From OE-Core rev: df4568205c3a7e0b20c6299e29f96bd30560146b)

Signed-off-by: Mihai Prica <mihai.prica@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:21 +01:00
Mihai Prica c70bf6531c lib/oeqa/utils: targetbuild: Add helper class for building packages on target
This class can be used for test cases that configure
and build packages on target.

(From OE-Core rev: 4b15e82c4fcb0c40b0e316ef2050944eee4418ef)

Signed-off-by: Mihai Prica <mihai.prica@intel.com>
Signed-off-by: Stefan Stanacar <stefanx.stanacar@intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2013-08-26 11:47:21 +01:00