Commit Graph

83 Commits

Author SHA1 Message Date
Ross Burton e629e05a1f qemurunner: fix bad indentation in serial login
(cherry picked from commit c4f57aed7a29000067c63a2821fddf18a88a23ce)
(From OE-Core rev: 2de7ffd9f0656ffd5b6fa002213e5f619480aba8)

(From OE-Core rev: 1f15fb106f791fe0e7136578ca6f5501324eb826)

Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-04 11:11:58 +00:00
Leonardo Sandoval 9e036b011e qemurunner: print tail qemu log in case bootlog is empty
There are cases where the 'while loop' waiting for login prompt fails
and the bootlog variable does not get populated, thus use the the new
qemurunner member (self.msg) which stores all output coming from the qemu
process.

[YOCTO #12113]

(From OE-Core rev: 39ffa0f3779305c5e8ef86fe4572e961c5912021)

(From OE-Core rev: 40f9f0358184660f23ac7b5fc3e477e2c54e21bc)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-04 11:11:58 +00:00
Juro Bystricky dd79b50980 qemurunner.py: wait for PID to appear in procfs
We need QEMU PID in order to access "/proc/<qemupid>/cmdline"
Having a valid QEMU PID does not mean we can access the proc entry
immediately, we need to wait for the /proc/<qemupid> to appear
before we can access it.

(From OE-Core rev: d2d069fa9910d1c7a94c898355a63fca03ec5ad8)

(From OE-Core rev: bf05abc7931a9a9e0823b24c6f4df4f93913da82)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-04 11:11:58 +00:00
Juro Bystricky 8100e7c44e qemurunner.py: refactor searching for QEMU PID
Ask QEMU to store the QEMU process PID in a file, this way we don't
have to parse running processes and analyze descendents.
This is done via QEMU command line argument "-pidfile".

[YOCTO #12001]

(From OE-Core rev: 67612dcd2a8a1aa1d683dddb0bd2f592886ff020)

(From OE-Core rev: 9fe7aa1824666d3f7ceb085bef59026e245cb752)

Signed-off-by: Juro Bystricky <juro.bystricky@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-04 11:11:58 +00:00
Richard Purdie 50750e1572 oeqa/qemurunner: Improve logging
Python unittest intercepts stdout and stderr however qemurunner sets
up a streamhandler before that interception occurs, hence the messages
spam the unittest output.

By moving the logging init to the class init time, we use the unittest
stdout/stderr and this means unittest can only show the log output upon
failure. This cleans up the selftest and testimage output whilst still
showing logging upon failure.

(From OE-Core rev: 9099cecc727fe0ae5f1559582426d30ba7a9f4d3)

(From OE-Core rev: ee13216284c4d49b762eaab488f21de573d2079d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-04 11:11:58 +00:00
Richard Purdie 8f35c388b4 qemurunner: Tweak qemu pid starting timeout code
We're seeing timeouts in the autobuilder testing code. Increase these
timeouts to 120, print the length of time we're having to wait, change
the error messages to really be errors and don't print empty logs, its
not helpful, print a message about the empty log instead.

(From OE-Core rev: b87a33d9c86cc1d2ea196259020e1d3dc712ccef)

(From OE-Core rev: 4234ea8aae52347e49f848292deb0936ffd905ae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Armin Kuster <akuster808@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2018-03-04 11:11:58 +00:00
Patrick Ohly c76abc379e oeqa: allow persistent image writes in runqemu()
By default, QemuRunner avoids modifying the image files that it boots
into by enabling the qemu snapshot mode. However, some tests may want
to test changes that must persists across reboots, so this mode
should be optional.

This can be combined by copying the image file to a temporary location
first and then booting with that copy. It's also useful when testing
with additional drives attached to a virtual machine.

QemuTinyRunner doesn't use the snapshot parameter and therefore ignores
the new parameter.

Long term, a better way of passing these various configuration
parameters should be used, and perhaps QemuRunner and QemuTinyRunner
can be merged into one again to avoid code duplication. But for now
the patch follows the exiting style.

Also beware that QemuTarget.start() now acts in two different modes
(with or without explicit launch command), and depending on that mode
parameters like discard_writes must be ignored, i.e. not get passed to
launch().

(From OE-Core rev: 969d079a33a57f5a8f7af86d7bab04d35ab07584)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-28 10:34:37 +01:00
Patrick Ohly 7f6b1cd2c1 oeqa: tolerate interrupted select() while waiting for qemu
Sometimes, the OEQA utility code aborts with:

   ...
   File ".../meta/lib/oeqa/utils/qemurunner.py", line 131, in start
     return self.launch(launch_cmd, qemuparams=qemuparams, get_ip=get_ip, extra_bootparams=extra_bootparams)
   File ".../meta/lib/oeqa/utils/qemurunner.py", line 259, in launch
     sread, swrite, serror = select.select(socklist, [], [], 5)
  InterruptedError: [Errno 4] Interrupted system call

strace shows that this is because of a SIGWINCH:

   Connection from 127.0.0.1:52668
   select(21, [20], [], [], {5, 0})        = ? ERESTARTNOHAND (To be restarted if no handler)
   --- SIGWINCH {si_signo=SIGWINCH, si_code=SI_KERNEL} ---

This is related to some special conditions:
 * whether qemu opens a graphical console window (enabled in Poky by default)
 * where that window gets opened
 * whether the window manager changes the size of the shell window (mine
   is a tiling window manager and reorders and resizes windows automatically)

Ignoring the interrupted system calls avoids the problem. Code elsewhere (for example,
run() in ssh.py) already does the same thing.

(From OE-Core rev: 44fe106baf5fd5aebe26c5f28004e2b18d839b7c)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-28 08:43:13 +01:00
Chen Qi a5a1d4f431 qemurunner.py: fix handling of runqemuparams
The launch() doesn't need runqemuparams, we need handle it in start().

(From OE-Core rev: 16400f2b8bffc4cae0263bb597522071299b46ca)

Signed-off-by: Chen Qi <Qi.Chen@windriver.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 08:15:06 +01:00
Robert Yang b742fd023e oeqa/targetcontrol.py: modify it to test runqemu
Modify the following files to test runqemu:
    targetcontrol.py
    utils/commands.py
    utils/qemurunner.py

We need simulate how "runqemu" works in command line, so when test
"runqemu", the targetcontrol.py, utils/commands.py and
utils/qemurunner.py don't have to find the rootfs or set env vars.

[YOCTO #10249]

(From OE-Core rev: 9305d816bdf8837ea3a407091cb7f24a9a3ae8dc)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 08:15:06 +01:00
Patrick Ohly 58e6e7c204 qemurunner: configurable timeout for run_serial()
Some commands might need to run longer than the default timeout of
five seconds. If that occurred, run_serial() returned with a status
code of zero (sic!) and no other indication of what went wrong.

Now the timeout is configurable (with five still the default) and
an explicit warning ("<<< run_serial(): command timed out after 5 seconds without output >>>")
gets appended at the end of the data returned to the caller.

While at it, the logic for checking for the timeout was updated a bit
because both implementations could overshoot the timeout when entering
select() right before the final deadline.

(From OE-Core rev: accf0362f964cc9d6330b6e52e83d748d890521f)

Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-27 08:15:06 +01:00
Ed Bartosh 0e4ccf1734 qemurunner: configure guest networking
Configured guest network interface through serial connection
when kernel is not run by qemu.

This should make it possible to test wic images with testimage.

[YOCTO #10833]

(From OE-Core rev: 2032d9be26b539bf867622c0090fb4696209eba9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:19 +00:00
Ed Bartosh 62dc9d964f qemurunner: get network params from runqemu output
Parsed runqemu output to get guest network configuration
if it's not present in runqemu command line.

[YOCTO #10833]

(From OE-Core rev: d4d7ed48c1cff1351ddc2f60bcfa153c373a8ab8)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-22 11:35:19 +00:00
Ed Bartosh ac462e464f qemurunner.py: ignore decode errors
qemu output can contain control characters. This cause qemurunner
API to crash when decoding the output to utf-8:

Traceback (most recent call last):
  File "/usr/lib64/python3.4/threading.py", line 911, in _bootstrap_inner
    self.run()
  File "meta/lib/oeqa/utils/qemurunner.py", line 472, in run
    threading.Thread.run(self)
  File "/usr/lib64/python3.4/threading.py", line 859, in run
    self._target(*self._args, **self._kwargs)
  File "meta/lib/oeqa/utils/qemurunner.py", line 465, in threadtarget
    self.eventloop()
  File "meta/lib/oeqa/utils/qemurunner.py", line 526, in eventloop
    self.logfunc(data)
  File "meta/lib/oeqa/utils/qemurunner.py", line 77, in log
    msg = msg.decode("utf-8")
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xda in position 0:
unexpected end of data

Added errors='ignore' to decode call to fix this.

(From OE-Core rev: 4a46dd5190d97fdcb6297a0c1d8c824d425c4c51)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-08 11:52:56 +00:00
Ed Bartosh 383cd20828 qemurunner: add runqemuparams argument to commands.runqemu
Added possibility to pass additional runqemu parameters
down the stack of APIs:
 commands.runqemu -> QemuTarget.start -> QemuRunner.start

This will be used to pass ovmf parameter in testing of
efi wic images under qemu.

(From OE-Core rev: 5aa4b5a10fb8191cd3453d09701c8beeff9a952f)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-03-08 11:52:56 +00:00
Mariano Lopez 81c3d4824a oeqa/utils/qemurunner.py: Be sure to stop qemu-system
When runqemu fails, qemu-system process would keep running
and won't be killed, setpgrp() was used when runqemu was
a shell script but it seems it doesn't work always with python.

This would kill qemu-system explicity and to avoid leaving
it behind.

(From OE-Core rev: 9d2b1aa1bcfb2f1933a8eeb9470b4174d5da2f0d)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Mariano Lopez de17891185 oeqa/utils/qemurunner.py: Add missing sys module
This adds the missing sys module used by the child process
to exit. It seems the exception was cached in testimage and
selftest. It seems nobody noticed this because the module
is only used for sys.exit().

(From OE-Core rev: 66f66d1d763ff7bbaab9e8fcdf7fc882f2dfbb13)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2017-01-19 22:47:22 +00:00
Francisco Pedraza 7ae326c76b oeqa/utils: Add StreamHandler to logger
StreamHandler was added due missing log information on the console in
oe-selftest with Qemu Runner

(From OE-Core rev: a4e2df151af781edbcb6b0e17b51b5ed226bf77f)

Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-10-01 21:45:57 +01:00
bavery 92862c678f qemurunner: print out the runqemu command
This logs the launch command that was used for runqemu while running -c
testimage.  This way, if I'd like to easily launch qemu manually in
order to debug a failed test, I know what commmand was run to create
the qemu instance.

(From OE-Core rev: 34aa20c6f323bbf7ad53beb643126e4e03634708)

Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-16 15:24:02 +01:00
Richard Purdie 8c46605d16 oeqa: Use snapshot instead of copying the rootfs image
Rather than copying images, use the snapshot option to qemu. This fixes a regression
caused by the recent runqemu changes where the wrong images were being testes since
the image is copied without the qemuboot.conf file. This means the latest image is
found by runqemu rather than the specified one, leading to various confused testing
results.

It could be fixed by copying more files but use snapshot mode instead.

(From OE-Core rev: eab91997d415b0e690b3482749a32087e6a8b00a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Robert Yang 95331c6d4e qemurunner.py/qemutinyrunner.py: remove runqemu-internal
There is no runqemu-internal any more.

(From OE-Core rev: 14bacf7203ab7a638b67eb143225d8c75bbb703d)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-09-09 12:07:32 +01:00
Bill Randle f479e3866d testimage: allow using kvm when running qemux86* machines
Using kvm can provide significant speedups when running qemux86* machines
on an x86* host. Enabled by using the new QEMU_USE_KVM variable.

[YOCTO #9298]

(From OE-Core rev: ebac2c8d1fcd09ebce0659a4abb445e4f1c18571)

Signed-off-by: Bill Randle <william.c.randle@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-08-18 09:27:53 +01:00
Aníbal Limón ad6aaae4bf utils/qemurunner.py: QemuRunner.start() add support for specify extra kernel cmdline
Add ability to specify extra_bootargs (kernel cmdline) in order to enable systemd
debug log in images that enables systemd init.

[YOCTO #9299]

(From OE-Core rev: 09d62551c289b5607341a4f9c46eecd6390ad774)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-20 10:28:47 +01:00
Richard Purdie 440c681646 qemurunner: Use surrogateescape decoding
Since the stream can contain invalid binary characters (e.g. from
ppc's bootloader) use surrogateescape decoding to ensure we do process
the character stream, else it can hang/timeout.

(From OE-Core rev: 28a0030430d4cfcaf5dfc3e71bda07cdbfbbf4a7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:02 +01:00
Richard Purdie 3b39971748 classes/lib: Complete transition to python3
This patch contains all the other misc pieces of the transition to
python3 which didn't make sense to be broken into individual patches.

(From OE-Core rev: fcd6b38bab8517d83e1ed48eef1bca9a9a190f57)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Ed Bartosh 63404baadb qemurunner: convert data when working with socket
Converted str to bytes before sending to socket.
Converted bytes to str after receiving from socket.

This should fix TypeError: 'str' does not support the buffer interface
for qemurunner.run_serial method.

(From OE-Core rev: 210e290c9251839dc74e3aabdcea3655dd707a50)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:01 +01:00
Richard Purdie 44e9a0d2fa classes/lib: Update to explictly create lists where needed
Iterators now return views, not lists in python3. Where we need
lists, handle this explicitly.

(From OE-Core rev: caebd862bac7eed725e0f0321bf50793671b5312)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:00 +01:00
Richard Purdie a7309d5790 classes/lib: Update to use python3 command pipeline decoding
In python3, strings are unicode by default. We need to encode/decode
from command pipelines and other places where we interface with the
real world using the correct locales. This patch updates various
call sites to use the correct encoding/decodings.

(From OE-Core rev: bb4685af1bffe17b3aa92a6d21398f38a44ea874)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:00 +01:00
Richard Purdie 3ddde5f32a meta: Update to modern exception syntax
Update older exception syntax to modern one required by python 3.
Compatible with python 2.7.

(From OE-Core rev: d13f0ac614f1d1e2ef2c8ddc71cbfcf76a8dc3f2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-21 22:26:33 +01:00
Richard Purdie 9094ea9fdd classes/oeqa: Update for print statements and file() -> open() for python3
Found some more syntax cleanups needed for python3.

(From OE-Core rev: 1181d86e8707c5b8e8d43d5e785d7d9cf01fa491)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-16 09:31:23 +01:00
Ed Bartosh 0ade65834b qemurunner: add parameter to method 'start'
QemuRunner requires pair of ip addresses provided through kernel
commandline for method 'start' to work. These ip addresses are
used to connect to the image using ssh and run tests there.
However, this functionality should not be mandatory as testing
doesn't always require ssh connection. Some tests can be run using
serial console.

Added new parameter 'get_ip' to QemuRunner.start to make it possible
to skip getting pair of ip addresses from kernel command line. This
should allow oe-selftest to test images without modifying kernel
command line.

[YOCTO #8498]

(From OE-Core rev: 3f8b734ebb81d035849288091bb0b97b9c4fba34)

(From OE-Core rev: 4c90daaeb946f1adf58b2f71f1af8eb7f5906474)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:28:44 +00:00
Ross Burton b3641838d6 oeqa/qemurunner: pass nographic to runqemu if DISPLAY isn't set
Not everyone wants to run the tests with a qemu that has a graphical output, so
allow display to be None and pass nographic to runqemu in that case.

(From OE-Core rev: b1d85f13e79f8a493b9849551357a1a5cf0d0fec)

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-12-01 21:32:01 +00:00
Mariano Lopez 3b95964bc1 qemurunner: Remove the timeout in run_serial
Sometmes when there is high load in the server the
commands executed in the target take a lot of time
to complete and this lead to incorrect dump files or
empty files. This is caused because run_serial has a
timeout of five seconds when running the commands in
the target.

This change removes the timeout and just keep reading
the socket until it finds the prompt from the target
or when the socket is not ready to be read.

[YOCTO #8510]

(From OE-Core rev: d1a97475b4e6c7066a3161cb9cec1d4b27644518)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:39:36 +00:00
Leonardo Sandoval e63e1918ff qemurunner: Enable timestamps on kernel boot-up
Having the timestamps in the log is useful when debugging those cases when
the booting time takes longer than expected. printk.times=1 enables stamping
into every printk Kernel call.

(From OE-Core rev: 8f8db89a91142bf71d009df881107b4c1fc61237)

Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-11-16 11:39:34 +00:00
Mariano Lopez b13c0be4b0 qemurunner: Show the output of runqemu script
This change will show runqemu output in the logs
when running the runtime tests. This is helpful
for debugging testimage errors.

[YOCTO #8550]

(From OE-Core rev: 751a3fec83792037aff23c8cca1bc60664f7b581)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-27 07:24:28 +00:00
Mariano Lopez 48e5579c1e oeqa/utils/qemurunner: Add support for Unicode from qemu
The current state of qemurunner will drop the Unicode
characters received from qemu, this is because error
report web had problems with Unicode characters; now
that the server support Unicode, it is possible to
log all the output from qemu. So far the only Unicode
character seen is the copyright symbol.

This patch allows to get Unicode characters from the qemu
target and save the log in an UTF-8 file for latter use.

[YOCTO #8225]

(From OE-Core rev: 4708a55879e1d8fe830d230b0621029cc40de9c3)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-07 00:11:20 +01:00
Mariano Lopez cb93670840 qemurunner: Sanitize output from qemu and qemu pid
Currently the output from qemu could contain control
or Unicode characters; having such characters in the
log will cause an internal server error when sending
the report to error reporting web. Control characters
can be found in the command line used to run quemu too.

This change sanitize the output from qemu and the
command line used to run qemu, this way the logs doesn't
contain control or Unicode characters and this will solve
the issue of error reporting web and generate better
logs.

The only Unicode character found in the qemu output
is the copyright symbol, it really doesn't interfer
removing Unicode characters with debugging.

[YOCTO #8225]

(From OE-Core rev: 9f25f723828d6709aad581267ccc63d2ade1ff5c)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-10-01 07:43:33 +01:00
Richard Purdie 3408d0dc87 qemurunner: Handle qemu start failure correctly
If qemu didn't start correctly, we may not have registered the child
signal. This results in a nasty traceback which confuses the underlying
issue. Cleanup this code and make the handler cleanup conditional.

(From OE-Core rev: 0ead936925c61aaaa7a819f6c5b999ba3a4daeda)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-28 12:00:18 +01:00
Aníbal Limón 524ddd86c3 oeqa/utils/qemurunner.py: Remove duplicate message on LoggingThread start
The Starting logging thread message is also executed on run() inside
LoggingThread class.

(From OE-Core rev: 2ad7308ee7166641eff99f3b9fe6794de143f6bc)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 09:53:21 +01:00
Aníbal Limón 376ce717bd oeqa/utils/qemurunner.py: Fix HIGH CPU usage on LoggingThread
LoggingThread is used for receive console output from QEMU
over TCP, so add filter to only wake poll on read events,
also change the event mask variable name to be more descriptive.

This fixes HIGH CPU consume caused by wake on ready to write
events.

(From OE-Core rev: 7b6c9b6327638c9731cea80a52d30d4a8fb6e081)

Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-23 09:53:21 +01:00
Richard Purdie a14c363409 oeqa: Test failure/cleanup improvements
Currently, if qemu segfaults, the tests merrily continue trying to execute
which takes time for them to timeout and is a bit silly. Worse, no logs about
the segfault are shown to the user, its silent!

This patch tries to unravel the tangled web of issues and ensures that we:

* install a SIGCHLD handler which tells the user qemu exited
* check if qemu is running, if it isn't fail the test outright
* don't leave processes behind in sshcontrol which would hold
  bitbake.lock and block shutdown

(From OE-Core rev: 77e9363feba53b72429154be5713c46b007ae0a4)

(From OE-Core rev: c77b57e2fa8a4cf90543038fe1939f6ca9b12bd1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-12 22:48:33 +01:00
Richard Purdie e87c182b6c qemurunner: Fix killpg pid usage
killpg should be being called with a pgid, not a pid.

(From OE-Core rev: d23bf86f305a04a47d19236d72979c1027a31620)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09 14:27:47 +01:00
Richard Purdie 335260cbfc qemurunner: Ensure runqemu doesn't survive SIGKILL
Currently, we see runqemu and qemu-system-* processes left behind when
bitbake is killed by buildbot. This is due to the use of setpgrp() in
the runqemu subprocess call.

We need the setpgrp call so that all runqemu processes can easily be
killed (by killing their process group). This presents a problem if this
controlling process itself is killed however since those processes don't
notice the death of the parent and merrily continue on.

Rather than hack runqemu to deal with this, we add something to
qemurunner, at least for now to resolve the issue. Basically we fork off
another process which holds an open pipe to the parent and also is
setpgrp. If/when the pipe sees EOF from the parent dieing, it kills the
process group. This is like pctrl's PDEATHSIG but for a process group
rather than a single process.

(From OE-Core rev: 99428eafb5352bd39bc4329bdba07c6d6f17b03f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09 14:27:46 +01:00
Richard Purdie e3b5d8f42c qemurunner: Drop error to a warning to improve user feedback
If a task shows an error, the full log is surpressed since bitbake assumes
the user has been shown what is wrong. In this code path that isn't the
case and its much more helpful to show the user the full error. Therefore
show a warning instead to aid usability.

(From OE-Core rev: 4ccef5543649262a1630bff586ef9048fe164016)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-09 14:27:44 +01:00
Richard Purdie ef0fe3193e oeqa: Test failure/cleanup improvements
Currently, if qemu segfaults, the tests merrily continue trying to execute
which takes time for them to timeout and is a bit silly. Worse, no logs about
the segfault are shown to the user, its silent!

This patch tries to unravel the tangled web of issues and ensures that we:

* install a SIGCHLD handler which tells the user qemu exited
* check if qemu is running, if it isn't fail the test outright
* don't leave processes behind in sshcontrol which would hold
  bitbake.lock and block shutdown

(From OE-Core rev: 5c04b1ca1e989f569d5755a646734d01a0c56cae)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-06 15:26:18 +01:00
Mariano Lopez 097f36f636 qemurunner: Handle lack of data on run serial gracefully
This changes the behavior when data was not received over
the serial console when a command is run. With this the
socket is no longer closed but it throws and exception that
can handled in upper layers. With this the test can continue
without throwing errors for not having the socket anymore.

[YOCTO #8118]

(From OE-Core rev: 4770a766389b94ddd5639d7a92e196abac38da22)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-03 12:43:31 +01:00
Mariano Lopez 53ab41a5f6 qemurunner: Added host dumps when there are errors
This adds an instance of HostDumper to qemurunner,
with this instance now is possible to get dumps
from the host when there is an error.

This adds dump points in the next cases:
    - runqemu exits before seeing qemu pid
    - Fail to get qemu process arguments
    - Not reach login banner before timeout
    - qemu pid never appears

This also modifies the constructors of BaseDumper,
HostDumper and TargetDumper, they don't require
the datastore anymore, but the feature to replace
datastore variables has been lost (never used)

[YOCTO #8118]

(From OE-Core rev: b0af40fb76cd5035696e9d8a44f815f64214d23a)

Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-09-03 12:43:16 +01:00
Randy Witt 1e8d03827d qemurunner: Don't loop on EWOULDBLOCK in logging thread.
EAGAIN/EWOULDBLOCK can be followed by no data. So don't tight loop
waiting for data.

(From OE-Core rev: 3aad1f489f38e999914ee6ccbf87367b9a75ee5e)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-29 13:38:03 +01:00
Randy Witt 315fb4c9e2 qemurunner: In the logging thread retry on EAGAIN, EWOULDBLOCK
On a nonblocking socket an exception can be generated for the EAGAIN
and EWOULDBLOCK errnos. Since these aren't actually errors make sure to
retry rather than bailing out.

(From OE-Core rev: 2f5cbfee0ab1189fbb83f0e785d79c8d123fccc2)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-26 08:28:06 +01:00
Randy Witt 55077eac1e qemurunner: Shut down logging thread successfully when test fails
Before this change on a test failure an exception would be generated due
to runqemu being killed before the logging thread which was on the other
end of the socket.

The exception was actually correct saying there was no data on a socket
marked readable, but this was because the qemu process was killed before
the listener thread.

(From OE-Core rev: 6497fddf7f6c4a59e16dab4a9daeb3614a61a8dc)

Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2015-08-25 23:23:53 +01:00