ref-manual, dev-manual: Applied review comments for test section
The section and variables for performing automated runtime tests on images was reviewed by Paul Eggleton. The suggested changes were made. (From yocto-docs rev: c2f84ea3c162892e4da3df30fb833f88bab3d3cc) Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
parent
56c4de93b9
commit
e44538f4e4
|
@ -4252,24 +4252,22 @@
|
|||
</section>
|
||||
</section>
|
||||
|
||||
<section id="running-and-writing-tests-for-a-qemu-image">
|
||||
<title>Running and Writing Tests for a QEMU Image</title>
|
||||
<section id="performing-automated-runtime-testing">
|
||||
<title>Performing Automated Runtime Testing</title>
|
||||
|
||||
<para>
|
||||
The OpenEmbedded build system makes available a series of automated
|
||||
tests for QEMU images.
|
||||
These tests are commands that run on the target system over
|
||||
<filename>ssh</filename> and are written in Python, thus making
|
||||
use of the <filename>unittest</filename> module.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
tests for images.
|
||||
<note>
|
||||
The current release of Yocto Project supports these tests
|
||||
for QEMU images only.
|
||||
</note>
|
||||
These tests are written in Python making use of the
|
||||
<filename>unittest</filename> module, and the majority of them
|
||||
run commands on the target system over
|
||||
<filename>ssh</filename>.
|
||||
This section describes how you set up the environment to use these
|
||||
tests, run available tests, and write and add your own tests.
|
||||
You can the same information in a different form and with example
|
||||
log files of test runs on the
|
||||
<ulink url='https://wiki.yoctoproject.org/wiki/Image_tests'>Image Tests</ulink>
|
||||
Wiki page.
|
||||
</para>
|
||||
|
||||
<section id="qemu-image-enabling-tests">
|
||||
|
@ -4278,9 +4276,8 @@
|
|||
<para>
|
||||
In order to run tests, you need to do the following:
|
||||
<itemizedlist>
|
||||
<listitem><para><emphasis>Ensure you run the test
|
||||
automatically with no interaction and under
|
||||
<filename>sudo</filename>:</emphasis>
|
||||
<listitem><para><emphasis>Set up to avoid interaction
|
||||
with <filename>sudo</filename> for networking:</emphasis>
|
||||
To accomplish this, you must do one of the
|
||||
following:
|
||||
<itemizedlist>
|
||||
|
@ -4300,12 +4297,13 @@
|
|||
<listitem><para>Manually configure a tap interface
|
||||
for your system.</para></listitem>
|
||||
<listitem><para>Run as root the script in
|
||||
<filename>scripts/runqemu-gen-tapdev</filename>,
|
||||
<filename>scripts/runqemu-gen-tapdevs</filename>,
|
||||
which should generate a list of tap devices.
|
||||
List generation is usually done in environments
|
||||
similar to AutoBuilder.</para></listitem>
|
||||
This is the option typically chosen for
|
||||
Autobuilder-type environments.
|
||||
</para></listitem>
|
||||
</itemizedlist></para></listitem>
|
||||
<listitem><para><emphasis>Set up the
|
||||
<listitem><para><emphasis>Set the
|
||||
<filename>DISPLAY</filename> variable:</emphasis>
|
||||
You need to set this variable so that you have an X
|
||||
server available (e.g. start
|
||||
|
@ -4321,11 +4319,10 @@
|
|||
<filename>${DEPLOY_DIR}/rpm</filename> so it can run
|
||||
smart channel commands. That means your host's firewall
|
||||
must accept incoming connections from 192.168.7.0/24,
|
||||
which is the default class used for
|
||||
<filename>tap0</filename> devices by
|
||||
<filename>runqemu</filename>.</para></listitem>
|
||||
<listitem><para><emphasis>Have your QEMU image built:</emphasis>
|
||||
The QEMU image on which you want to run tests needs to
|
||||
which is the default IP range used for tap devices
|
||||
by <filename>runqemu</filename>.</para></listitem>
|
||||
<listitem><para><emphasis>Have your image built:</emphasis>
|
||||
The image on which you want to run tests needs to
|
||||
be built.
|
||||
For example, the following command would build the
|
||||
<filename>core-image-sato</filename> image when
|
||||
|
@ -4333,18 +4330,34 @@
|
|||
<literallayout class='monospaced'>
|
||||
bitbake core-image-sato
|
||||
</literallayout></para></listitem>
|
||||
<listitem><para><emphasis>Globally inherit
|
||||
<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage'><filename>testimage.class</filename>:</ulink></emphasis>
|
||||
Edit your <filename>local.conf</filename> file as
|
||||
<listitem><para><emphasis>Run the tests:</emphasis>
|
||||
You can start the tests automatically or manually.
|
||||
To have the tests start automatically after the
|
||||
OpenEmbedded build system successfully creates a QEMU
|
||||
image, set the
|
||||
<ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_IMAGE'><filename>TEST_IMAGE</filename></ulink>
|
||||
variable to "1" in your
|
||||
<filename>local.conf</filename> file in the
|
||||
<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
|
||||
To manually run the tests, globally inherit
|
||||
<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage'><filename>testimage.class</filename>:</ulink>
|
||||
by editing your <filename>local.conf</filename> file as
|
||||
follows:
|
||||
<literallayout class='monospaced'>
|
||||
INHERIT += "testimage"
|
||||
</literallayout>
|
||||
After editing your
|
||||
<filename>local.conf</filename> file, use BitBake to
|
||||
run the tests:
|
||||
<literallayout class='monospaced'>
|
||||
bitbake -c testimage <qemu-image>
|
||||
</literallayout></para></listitem>
|
||||
</itemizedlist>
|
||||
</para>
|
||||
|
||||
<note>
|
||||
If you built your QEMU image using <filename>rm_work</filename>,
|
||||
Regardless of how you initiate the tests, if you built your
|
||||
image using <filename>rm_work</filename>,
|
||||
most of the tests will fail with errors because they rely on
|
||||
<filename>${WORKDIR}/installed_pkgs.txt</filename>.
|
||||
</note>
|
||||
|
@ -4355,14 +4368,16 @@
|
|||
|
||||
<para>
|
||||
After setting up to run the tests, use BitBake to start
|
||||
the standard suite of tests:
|
||||
the standard suite of tests manually or, if you are running
|
||||
them automatically, build your image:
|
||||
<literallayout class='monospaced'>
|
||||
bitbake core-image-sato -c testimage
|
||||
bitbake core-image-sato
|
||||
</literallayout>
|
||||
</para>
|
||||
|
||||
<para>
|
||||
Once you start the tests, the following happens:
|
||||
Once the tests start, the following happens:
|
||||
<itemizedlist>
|
||||
<listitem><para>A copy of the root filesystem is written
|
||||
to <filename>${WORKDIR}/testimage</filename>.
|
||||
|
@ -4400,10 +4415,10 @@
|
|||
All test files reside in
|
||||
<filename>meta/lib/oeqa/runtime</filename> in the
|
||||
<link linkend='source-directory'>Source Directory</link>.
|
||||
Tests (also referred to as modules) have a one-to-one
|
||||
relationship with the filenames in <filename>runtime</filename>.
|
||||
Modules can have multiple classes and test methods and are
|
||||
usually grouped together by the area tested (e.g tests for
|
||||
A test name (sometimes referred to as a module) maps to a
|
||||
method name within a class within a module.
|
||||
Tests can be used within multiple classes and the tests
|
||||
are usually grouped together by the area tested (e.g tests for
|
||||
<filename>systemd</filename> reside in
|
||||
<filename>meta/lib/oeqa/runtime/systemd.py</filename>).
|
||||
</para>
|
||||
|
@ -4412,12 +4427,13 @@
|
|||
You can add tests to any layer provided you place them in the
|
||||
proper area and you extend
|
||||
<ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
|
||||
in the <filename>local.conf</filename> file.
|
||||
in the <filename>local.conf</filename> file as normal.
|
||||
Be sure that tests reside in
|
||||
<filename><layer>/lib/oeqa/runtime</filename>.
|
||||
<note>
|
||||
Be sure that module names do not collide with module names
|
||||
provided by the Yocto Project.
|
||||
used in the default set of test modules in
|
||||
<filename>meta/lib/oeqa/runtime</filename>.
|
||||
</note>
|
||||
</para>
|
||||
|
||||
|
@ -4458,12 +4474,12 @@
|
|||
<listitem><para>Run the default set of tests simply by
|
||||
running the following:
|
||||
<literallayout class='monospaced'>
|
||||
bitbake <qemu_image> -c testimage
|
||||
bitbake <image> -c testimage
|
||||
</literallayout></para></listitem>
|
||||
<listitem><para>The default tests for the image are defined
|
||||
as:
|
||||
<literallayout class='monospaced'>
|
||||
DEFAULT_TEST_SUITES_pn-<qemu_image> = "ping ssh df connman syslog xorg scp vnc date rpm smart dmesg"
|
||||
DEFAULT_TEST_SUITES_pn-<image> = "ping ssh df connman syslog xorg scp vnc date rpm smart dmesg"
|
||||
</literallayout></para></listitem>
|
||||
<listitem><para>Add your own test to the list of the
|
||||
by using the following:
|
||||
|
@ -4487,15 +4503,16 @@
|
|||
<para>
|
||||
As mentioned previously, all new test files need to be in the
|
||||
proper place for the build system to find them.
|
||||
New tests can go in the
|
||||
<link linkend='source-directory'>Source Directory</link> at
|
||||
<filename>meta/lib/oeqa/runtime</filename>.
|
||||
Alternatively, a layer can add its own tests in
|
||||
<filename><layer>/lib/oeqa/runtime</filename> as long
|
||||
as you extend <filename>BBPATH</filename>.
|
||||
Just remember that filenames need to map directory to test
|
||||
(module) names and you not use module names that collide with
|
||||
existing core tests.
|
||||
New tests for additional functionality outside of the core
|
||||
should be added to the layer that adds the functionality, in
|
||||
<filename><layer>/lib/oeqa/runtime</filename> (as
|
||||
long as
|
||||
<ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
|
||||
is extended in the layer's
|
||||
<filename>layer.conf</filename> file as normal.
|
||||
Just remember that filenames need to map directly to test
|
||||
(module) names and that you do not use module names that
|
||||
collide with existing core tests.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
|
|
|
@ -940,8 +940,12 @@
|
|||
|
||||
<para>
|
||||
You can use this class to enable running a series of automated tests
|
||||
for QEMU images.
|
||||
The class handles loading the tests and starting the QEMU image.
|
||||
for images.
|
||||
The class handles loading the tests and starting the image.
|
||||
<note>
|
||||
The current release of Yocto Project supports these tests
|
||||
for QEMU images only.
|
||||
</note>
|
||||
</para>
|
||||
|
||||
<para>
|
||||
|
@ -950,15 +954,13 @@
|
|||
The tests are commands that run on the target system over
|
||||
<filename>ssh</filename>.
|
||||
they are written in Python and make use of the
|
||||
<filename>unittest modulebuild</filename> software.
|
||||
<filename>unittest</filename> module.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
For information on how to enable, run, and create new tests, see the
|
||||
"<ulink url='running-and-writing-tests-for-a-qemu-image'>Running and Writing Tests for a QEMU Image</ulink>"
|
||||
section and the
|
||||
<ulink url='https://wiki.yoctoproject.org/wiki/Image_tests'>Image Tests</ulink>
|
||||
Wiki page.
|
||||
"<ulink url='&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing'>Performing Automated Runtime Testing</ulink>"
|
||||
section.
|
||||
</para>
|
||||
</section>
|
||||
|
||||
|
|
|
@ -5220,21 +5220,27 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
|
|||
<glossentry id='var-TEST_IMAGE'><glossterm>TEST_IMAGE</glossterm>
|
||||
<glossdef>
|
||||
<para>
|
||||
Automatically runs the series of automated tests for QEMU
|
||||
images when a QEMU image is successfully built.
|
||||
These tests are commands that run on the target system over
|
||||
<filename>ssh</filename> and are written in Python.
|
||||
Automatically runs the series of automated tests for
|
||||
images when an image is successfully built.
|
||||
<note>
|
||||
The current release of Yocto Project supports these tests
|
||||
for QEMU images only.
|
||||
</note>
|
||||
These tests are written in Python making use of the
|
||||
<filename>unittest</filename> module, and the majority of
|
||||
them run commands on the target system over
|
||||
<filename>ssh</filename>.
|
||||
You can set this variable to "1" in your
|
||||
<filename>local.conf</filename> file in the
|
||||
<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>
|
||||
to have the OpenEmbedded build system automatically run
|
||||
these tests after a QEMU image successfully builds:
|
||||
these tests after an image successfully builds:
|
||||
<literallayout class='monospaced'>
|
||||
TEST_IMAGE = "1"
|
||||
</literallayout>
|
||||
For more information on enabling, running, and writing
|
||||
these tests, see the
|
||||
"<ulink url='&YOCTO_DOCS_DEV_URL;#running-and-writing-tests-for-a-qemu-image'>Running and Writing Tests for a QEMU Image</ulink>"
|
||||
"<ulink url='&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing'>Performing Automated Runtime Testing</ulink>"
|
||||
section in the Yocto Project Development Manual and the
|
||||
"<link linkend='ref-classes-testimage'><filename>testimage.bbclass</filename></link>"
|
||||
section.
|
||||
|
@ -5245,24 +5251,18 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
|
|||
<glossentry id='var-TEST_QEMUBOOT_TIMEOUT'><glossterm>TEST_QEMUBOOT_TIMEOUT</glossterm>
|
||||
<glossdef>
|
||||
<para>
|
||||
The time in seconds allowed for a QEMU image to boot before
|
||||
tests begin to run against the image using the following
|
||||
command:
|
||||
<literallayout class='monospaced'>
|
||||
bitbake <qemu_image> -c testimage
|
||||
</literallayout>
|
||||
</para>
|
||||
|
||||
<para>
|
||||
The time in seconds allowed for an image to boot before
|
||||
automated runtime tests begin to run against an
|
||||
image.
|
||||
The default timeout period to allow the boot process to
|
||||
reach the login prompt is 500 seconds.
|
||||
You can edit this default value in the
|
||||
You can specify a different value in the
|
||||
<filename>local.conf</filename> file.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
For more information on testing QEMU images, see the
|
||||
"<ulink url='running-and-writing-tests-for-a-qemu-image'>Running and Writing Tests for a QEMU Image</ulink>"
|
||||
For more information on testing images, see the
|
||||
"<ulink url='&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing'>Performing Automated Runtime Testing</ulink>"
|
||||
section in the Yocto Project Development Manual.
|
||||
</para>
|
||||
</glossdef>
|
||||
|
@ -5272,15 +5272,16 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
|
|||
<glossdef>
|
||||
<para>
|
||||
An ordered list of tests (modules) to run against
|
||||
a QEMU image using the following command:
|
||||
<literallayout class='monospaced'>
|
||||
bitbake <qemu_image> -c testimage
|
||||
</literallayout>
|
||||
an image when performing automated runtime testing.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
The Yocto Project provides a core set of tests that can
|
||||
be used against QEMU images.
|
||||
The OpenEmbedded build system provides a core set of tests
|
||||
that can be used against images.
|
||||
<note>
|
||||
The current release of Yocto Project supports these
|
||||
tests for QEMU images only.
|
||||
</note>
|
||||
Tests include <filename>ping</filename>,
|
||||
<filename>ssh</filename>, <filename>df</filename> among
|
||||
others.
|
||||
|
@ -5289,6 +5290,14 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
|
|||
<literallayout class='monospaced'>
|
||||
TEST_SUITES_append = " mytest"
|
||||
</literallayout>
|
||||
Alternatively, you can provide the "auto" option to
|
||||
have all applicable tests run against the image.
|
||||
<literallayout class='monospaced'>
|
||||
TEST_SUITES_append = " auto"
|
||||
</literallayout>
|
||||
Using this option causes the build system to automatically
|
||||
run tests that are applicable to the image.
|
||||
Tests that are not applicable are skipped.
|
||||
</para>
|
||||
|
||||
<para>
|
||||
|
@ -5307,8 +5316,8 @@ recipes-graphics/xorg-font/font-alias_1.0.3.bb:PR = "${INC_PR}.3"
|
|||
</para>
|
||||
|
||||
<para>
|
||||
For more information on testing QEMU images, see the
|
||||
"<ulink url='running-and-writing-tests-for-a-qemu-image'>Running and Writing Tests for a QEMU Image</ulink>"
|
||||
For more information on testing images, see the
|
||||
"<ulink url='&YOCTO_DOCS_DEV_URL;#performing-automated-runtime-testing'>Performing Automated Runtime Testing</ulink>"
|
||||
section in the Yocto Project Development Manual.
|
||||
</para>
|
||||
</glossdef>
|
||||
|
|
Loading…
Reference in New Issue