From eb36aaa07f07ecdcd9a6fd6a60b37b352a7cf5d6 Mon Sep 17 00:00:00 2001 From: Scott Rifenbark Date: Mon, 23 Sep 2013 16:35:26 -0700 Subject: [PATCH] ref-manual, dev-manual: Edits to running automated tests section. Applied a second round of review edits from Paul Eggleton for the new "Performing Automated Runtime Testing" section. I did some reorganization and some minor wording changes. (From yocto-docs rev: fa8f8e5f0f6c1377a4fcafcd3d933af15ac01ff3) Signed-off-by: Scott Rifenbark Signed-off-by: Richard Purdie --- .../dev-manual/dev-manual-common-tasks.xml | 103 ++++++++---------- documentation/ref-manual/ref-classes.xml | 4 +- documentation/ref-manual/ref-variables.xml | 8 +- 3 files changed, 52 insertions(+), 63 deletions(-) diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml index 4d1b731fa3..6eadcef022 100644 --- a/documentation/dev-manual/dev-manual-common-tasks.xml +++ b/documentation/dev-manual/dev-manual-common-tasks.xml @@ -4257,10 +4257,10 @@ The OpenEmbedded build system makes available a series of automated - tests for images. + tests for images to verify runtime functionality. - The current release of Yocto Project supports these tests - for QEMU images only. + Currently, there is only support for running these tests + under QEMU. These tests are written in Python making use of the unittest module, and the majority of them @@ -4321,37 +4321,6 @@ must accept incoming connections from 192.168.7.0/24, which is the default IP range used for tap devices by runqemu. - Have your image built: - The image on which you want to run tests needs to - be built. - For example, the following command would build the - core-image-sato image when - MACHINE is set to "qemu". - - bitbake core-image-sato - - Run the tests: - You can start the tests automatically or manually. - To have the tests start automatically after the - OpenEmbedded build system successfully creates a QEMU - image, set the - TEST_IMAGE - variable to "1" in your - local.conf file in the - Build Directory. - To manually run the tests, globally inherit - testimage.class: - by editing your local.conf file as - follows: - - INHERIT += "testimage" - - After editing your - local.conf file, use BitBake to - run the tests: - - bitbake -c testimage <qemu-image> - @@ -4367,17 +4336,42 @@ Running Tests - After setting up to run the tests, use BitBake to start - the standard suite of tests manually or, if you are running - them automatically, build your image: - - bitbake core-image-sato -c testimage + You can start the tests automatically or manually: + + Automatically Running Tests: + To run the tests automatically after the + OpenEmbedded build system successfully creates an image, + first set the + TEST_IMAGE + variable to "1" in your local.conf + file in the + Build Directory: + + TEST_IMAGE = "1" + + Next, simply build your image. + If the image successfully builds, the tests will be + run: + bitbake core-image-sato - + + Manually Running Tests: + To manually run the tests, first globally inherit + testimage.class + by editing your local.conf file: + + INHERIT += "testimage" + + Next, use BitBake to run the tests: + + bitbake -c testimage <image> + + - Once the tests start, the following happens: + Regardless of how you run the tests, once they start, the + following happens: A copy of the root filesystem is written to ${WORKDIR}/testimage. @@ -4415,11 +4409,10 @@ All test files reside in meta/lib/oeqa/runtime in the Source Directory. - A test name (sometimes referred to as a module) maps to a - method name within a class within a module. - Tests can be used within multiple classes and the tests - are usually grouped together by the area tested (e.g tests for - systemd reside in + A test name maps directly to a Python module. + Each test module may contain a number of individual tests. + Tests are usually grouped together by the area + tested (e.g tests for systemd reside in meta/lib/oeqa/runtime/systemd.py). @@ -4443,12 +4436,13 @@ variable in local.conf. Each name in TEST_SUITES represents a required test for the image. - Module skipping is not allowed even if a test is not suitable - for an image (e.g. running the rpm tests on - an image without rpm). + Test modules named within TEST_SUITES + cannot be skipped even if a test is not suitable for an image + (e.g. running the rpm tests on an image without + rpm). Appending "auto" to TEST_SUITES causes the build system to try to run all tests that are suitable for the - image (i.e. the build system decides whether to run each test). + image (i.e. each test module may elect to skip itself). @@ -4469,13 +4463,8 @@ - The following list summarizes how to run the tests: + Here are some things to keep in mind when running tests: - Run the default set of tests simply by - running the following: - - bitbake <image> -c testimage - The default tests for the image are defined as: @@ -4509,7 +4498,7 @@ long as BBPATH is extended in the layer's - layer.conf file as normal. + layer.conf file as normal). Just remember that filenames need to map directly to test (module) names and that you do not use module names that collide with existing core tests. diff --git a/documentation/ref-manual/ref-classes.xml b/documentation/ref-manual/ref-classes.xml index f08b7fefc8..6ed8fe5013 100644 --- a/documentation/ref-manual/ref-classes.xml +++ b/documentation/ref-manual/ref-classes.xml @@ -943,8 +943,8 @@ for images. The class handles loading the tests and starting the image. - The current release of Yocto Project supports these tests - for QEMU images only. + Currently, there is only support for running these tests + under QEMU. diff --git a/documentation/ref-manual/ref-variables.xml b/documentation/ref-manual/ref-variables.xml index 6689ef085f..c7fb036c3f 100644 --- a/documentation/ref-manual/ref-variables.xml +++ b/documentation/ref-manual/ref-variables.xml @@ -5266,8 +5266,8 @@ PARALLEL_MAKEINST with the description ". Automatically runs the series of automated tests for images when an image is successfully built. - The current release of Yocto Project supports these tests - for QEMU images only. + Currently, there is only support for running these tests + under QEMU. These tests are written in Python making use of the unittest module, and the majority of @@ -5322,8 +5322,8 @@ PARALLEL_MAKEINST with the description ". The OpenEmbedded build system provides a core set of tests that can be used against images. - The current release of Yocto Project supports these - tests for QEMU images only. + Currently, there is only support for running these tests + under QEMU. Tests include ping, ssh, df among