sysmo-bts
/
generic-poky
9
0
Fork 0
Code Pull Requests Releases Activity

ref-manual, dev-manual: Edits to running automated tests section. 

Applied a second round of review edits from Paul Eggleton
for the new "Performing Automated Runtime Testing" section.
I did some reorganization and some minor wording changes.

(From yocto-docs rev: fa8f8e5f0f6c1377a4fcafcd3d933af15ac01ff3)

Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
Scott Rifenbark 2013-09-23 16:35:26 -07:00 committed by Richard Purdie
parent b8bdd92ae6
commit eb36aaa07f
3 changed files with 52 additions and 63 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

103
documentation/dev-manual/dev-manual-common-tasks.xml
View File

@ -4257,10 +4257,10 @@
<para> <para>
The OpenEmbedded build system makes available a series of automated The OpenEmbedded build system makes available a series of automated
tests for images. tests for images to verify runtime functionality.
<note> <note>
The current release of Yocto Project supports these tests Currently, there is only support for running these tests
for QEMU images only. under QEMU.
</note> </note>
These tests are written in Python making use of the These tests are written in Python making use of the
<filename>unittest</filename> module, and the majority of them <filename>unittest</filename> module, and the majority of them
@ -4321,37 +4321,6 @@
must accept incoming connections from 192.168.7.0/24, must accept incoming connections from 192.168.7.0/24,
which is the default IP range used for tap devices which is the default IP range used for tap devices
by <filename>runqemu</filename>.</para></listitem> by <filename>runqemu</filename>.</para></listitem>
<listitem><para><emphasis>Have your image built:</emphasis>
The image on which you want to run tests needs to
be built.
For example, the following command would build the
<filename>core-image-sato</filename> image when
<filename>MACHINE</filename> is set to "qemu".
<literallayout class='monospaced'>
bitbake core-image-sato
</literallayout></para></listitem>
<listitem><para><emphasis>Run the tests:</emphasis>
You can start the tests automatically or manually.
To have the tests start automatically after the
OpenEmbedded build system successfully creates a QEMU
image, set the
<ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_IMAGE'><filename>TEST_IMAGE</filename></ulink>
variable to "1" in your
<filename>local.conf</filename> file in the
<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>.
To manually run the tests, globally inherit
<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage'><filename>testimage.class</filename>:</ulink>
by editing your <filename>local.conf</filename> file as
follows:
<literallayout class='monospaced'>
INHERIT += "testimage"
</literallayout>
After editing your
<filename>local.conf</filename> file, use BitBake to
run the tests:
<literallayout class='monospaced'>
bitbake -c testimage &lt;qemu-image&gt;
</literallayout></para></listitem>
</itemizedlist> </itemizedlist>
</para> </para>
@ -4367,17 +4336,42 @@
<title>Running Tests</title> <title>Running Tests</title>
<para> <para>
After setting up to run the tests, use BitBake to start You can start the tests automatically or manually:
the standard suite of tests manually or, if you are running <itemizedlist>
them automatically, build your image: <listitem><para><emphasis>Automatically Running Tests:</emphasis>
<literallayout class='monospaced'> To run the tests automatically after the
bitbake core-image-sato -c testimage OpenEmbedded build system successfully creates an image,
first set the
<ulink url='&YOCTO_DOCS_REF_URL;#var-TEST_IMAGE'><filename>TEST_IMAGE</filename></ulink>
variable to "1" in your <filename>local.conf</filename>
file in the
<ulink url='&YOCTO_DOCS_DEV_URL;#build-directory'>Build Directory</ulink>:
<literallayout class='monospaced'>
TEST_IMAGE = "1"
</literallayout>
Next, simply build your image.
If the image successfully builds, the tests will be
run:
<literallayout class='monospaced'>
bitbake core-image-sato bitbake core-image-sato
</literallayout> </literallayout></para></listitem>
<listitem><para><emphasis>Manually Running Tests:</emphasis>
To manually run the tests, first globally inherit
<ulink url='&YOCTO_DOCS_REF_URL;#ref-classes-testimage'><filename>testimage.class</filename></ulink>
by editing your <filename>local.conf</filename> file:
<literallayout class='monospaced'>
INHERIT += "testimage"
</literallayout>
Next, use BitBake to run the tests:
<literallayout class='monospaced'>
bitbake -c testimage &lt;image&gt;
</literallayout></para></listitem>
</itemizedlist>
</para> </para>
<para> <para>
Once the tests start, the following happens: Regardless of how you run the tests, once they start, the
following happens:
<itemizedlist> <itemizedlist>
<listitem><para>A copy of the root filesystem is written <listitem><para>A copy of the root filesystem is written
to <filename>${WORKDIR}/testimage</filename>. to <filename>${WORKDIR}/testimage</filename>.
@ -4415,11 +4409,10 @@
All test files reside in All test files reside in
<filename>meta/lib/oeqa/runtime</filename> in the <filename>meta/lib/oeqa/runtime</filename> in the
<link linkend='source-directory'>Source Directory</link>. <link linkend='source-directory'>Source Directory</link>.
A test name (sometimes referred to as a module) maps to a A test name maps directly to a Python module.
method name within a class within a module. Each test module may contain a number of individual tests.
Tests can be used within multiple classes and the tests Tests are usually grouped together by the area
are usually grouped together by the area tested (e.g tests for tested (e.g tests for <filename>systemd</filename> reside in
<filename>systemd</filename> reside in
<filename>meta/lib/oeqa/runtime/systemd.py</filename>). <filename>meta/lib/oeqa/runtime/systemd.py</filename>).
</para> </para>
@ -4443,12 +4436,13 @@
variable in <filename>local.conf</filename>. variable in <filename>local.conf</filename>.
Each name in <filename>TEST_SUITES</filename> represents a Each name in <filename>TEST_SUITES</filename> represents a
required test for the image. required test for the image.
Module skipping is not allowed even if a test is not suitable Test modules named within <filename>TEST_SUITES</filename>
for an image (e.g. running the <filename>rpm</filename> tests on cannot be skipped even if a test is not suitable for an image
an image without <filename>rpm</filename>). (e.g. running the rpm tests on an image without
<filename>rpm</filename>).
Appending "auto" to <filename>TEST_SUITES</filename> causes the Appending "auto" to <filename>TEST_SUITES</filename> causes the
build system to try to run all tests that are suitable for the build system to try to run all tests that are suitable for the
image (i.e. the build system decides whether to run each test). image (i.e. each test module may elect to skip itself).
</para> </para>
<para> <para>
@ -4469,13 +4463,8 @@
</para> </para>
<para> <para>
The following list summarizes how to run the tests: Here are some things to keep in mind when running tests:
<itemizedlist> <itemizedlist>
<listitem><para>Run the default set of tests simply by
running the following:
<literallayout class='monospaced'>
bitbake &lt;image&gt; -c testimage
</literallayout></para></listitem>
<listitem><para>The default tests for the image are defined <listitem><para>The default tests for the image are defined
as: as:
<literallayout class='monospaced'> <literallayout class='monospaced'>
@ -4509,7 +4498,7 @@
long as long as
<ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink> <ulink url='&YOCTO_DOCS_REF_URL;#var-BBPATH'><filename>BBPATH</filename></ulink>
is extended in the layer's is extended in the layer's
<filename>layer.conf</filename> file as normal. <filename>layer.conf</filename> file as normal).
Just remember that filenames need to map directly to test Just remember that filenames need to map directly to test
(module) names and that you do not use module names that (module) names and that you do not use module names that
collide with existing core tests. collide with existing core tests.

4
documentation/ref-manual/ref-classes.xml
View File

@ -943,8 +943,8 @@
for images. for images.
The class handles loading the tests and starting the image. The class handles loading the tests and starting the image.
<note> <note>
The current release of Yocto Project supports these tests Currently, there is only support for running these tests
for QEMU images only. under QEMU.
</note> </note>
</para> </para>

8
documentation/ref-manual/ref-variables.xml
View File

@ -5266,8 +5266,8 @@ PARALLEL_MAKEINST with the description ".
Automatically runs the series of automated tests for Automatically runs the series of automated tests for
images when an image is successfully built. images when an image is successfully built.
<note> <note>
The current release of Yocto Project supports these tests Currently, there is only support for running these tests
for QEMU images only. under QEMU.
</note> </note>
These tests are written in Python making use of the These tests are written in Python making use of the
<filename>unittest</filename> module, and the majority of <filename>unittest</filename> module, and the majority of
@ -5322,8 +5322,8 @@ PARALLEL_MAKEINST with the description ".
The OpenEmbedded build system provides a core set of tests The OpenEmbedded build system provides a core set of tests
that can be used against images. that can be used against images.
<note> <note>
The current release of Yocto Project supports these Currently, there is only support for running these tests
tests for QEMU images only. under QEMU.
</note> </note>
Tests include <filename>ping</filename>, Tests include <filename>ping</filename>,
<filename>ssh</filename>, <filename>df</filename> among <filename>ssh</filename>, <filename>df</filename> among