Commit Graph

962 Commits

Author SHA1 Message Date
Paul Eggleton ac5e720575 bitbake: lib: implement basic task progress support
For long-running tasks where we have some output from the task that
gives us some idea of the progress of the task (such as a percentage
complete), provide the means to scrape the output for that progress
information and show it to the user in the default knotty terminal
output in the form of a progress bar. This is implemented using a new
TaskProgress event as well as some code we can insert to do output
scanning/filtering.

Any task can fire TaskProgress events; however, if you have a shell task
whose output you wish to scan for progress information, you just need to
set the "progress" varflag on the task. This can be set to:
 * "percent" to just look for a number followed by a % sign
 * "percent:<regex>" to specify your own regex matching a percentage
   value (must have a single group which matches the percentage number)
 * "outof:<regex>" to look for the specified regex matching x out of y
   items completed (must have two groups - first group needs to be x,
   second y).
We can potentially extend this in future but this should be a good
start.

Part of the implementation for [YOCTO #5383].

(Bitbake rev: 0d275fc5b6531957a6189069b04074065bb718a0)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:26 +01:00
Paul Eggleton 1cf6e14a6c bitbake: knotty: import latest python-progressbar
Since we're going to make some minor extensions to it, it makes sense to
bring in the latest version of python-progressbar. Its structure has
changed a little but the API hasn't; however we do need to ensure our
overridden _needs_update() function's signature in BBProgress() matches
properly.

(Bitbake rev: c3e51d71b36cbc9e9ed1b35fb93d0978e24bc98a)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:26 +01:00
Paul Eggleton 481048cd2a bitbake: knotty: provide a symlink to the latest console log
If you're looking to find the latest console log repeatedly it can be a bit
tedious - let's just create a symlink just as we do with other logs to
make it easy to find.

(Bitbake rev: e9f41c0507a6527bf2ed86506813d4d4a89f8ebf)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-07-08 09:57:26 +01:00
Jussi Kukkonen 90d7b09af2 bitbake: depexp.py: port to gtk+3
The code is still a bit icky (and should be refactored to not use
Gdk.threads_enter/leave) but it should work about as reliably as
it did with Gtk+2.

Based on earlier patches by Maxin and Joshua.

(Bitbake rev: 8eee64a64144e27b5b8c2aca88e138882c3deab7)

Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-20 17:23:56 +01:00
Ed Bartosh 772e9b4eab bitbake: bitbake: fix wrong usage of format_exc
First parameter of traceback.format_exc is a 'limit' - a number
of stracktraces to format.

Passing exception object to format_exc is incorrect, but it works in
Python 2 as this code from traceback module works:
    while tb is not None and (limit is None or n < limit):
Comparing integer counter n with the exception object in Python 2
always results in True. However, in Python 3 it throws exception:
    TypeError: unorderable types: int() < <Exception type>()

As format_exc is used in except block of handling another
exception this can cause hard to find and debug bugs.

(Bitbake rev: a9509949d7e2adba6e3cd89f97daa19a955855b5)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-16 11:51:26 +01:00
Ed Bartosh f0740b038f bitbake: toaster: replace viewkeys() -> keys()
Python 3 doesn't have dict.viewkeys method, renaming
to keys().

[YOCTO #9584]

(Bitbake rev: 8ae2fc26fc7c4f42817dece62b9a59fbda27c0b0)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-15 08:35:05 +01:00
Ed Bartosh 9434eee39a bitbake: Revert "buildinfohelper: work around unicode exceptions"
This commit causes buildinfohelper to crash when run on python 3
as python 3 doesn't have unicode builtin function.

This reverts commit 7a309d964a.

(Bitbake rev: 750ca5c8d5a25fc519b75c56352dec7823c7e240)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:04 +01:00
Richard Purdie 8a93f5f32e bitbake: goggle/image-writer: Drop since bitrotting and no longer used
The upgrade to python3 is the final nail in the coffin for image-writer
and the goggle UI. Neither seem used or recieve patches and are based
on old versions of GTK+ so drop them, and the remaining crumbs support
pieces.

(Bitbake rev: ee7df1ca00c76f755057c157c093294efb9078d8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:03 +01:00
Richard Purdie 685628ebc4 bitbake: image-writer/goggle: Disable pygtkcompat problems
Disable the problematic gtk usage for use with pygtkcompat. The following
commit removes these tools/UIs entirely but we may as well leave this
piece in the history in case anyone does want a starting point for reusing
them.

(Bitbake rev: c53c7418d392452450352ca2175667dbdbd92401)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:03 +01:00
Richard Purdie 0f2c59367a bitbake: bitbake: Convert to python 3
Various misc changes to convert bitbake to python3 which don't warrant
separation into separate commits.

(Bitbake rev: d0f904d407f57998419bd9c305ce53e5eaa36b24)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-06-02 08:24:02 +01:00
Michael Wood db84307055 bitbake: toaster: Remove DATABASE_URL being passed around as an environment var
We don't need to pass the DATABASE_URL around and read it back if we
setup the django framework in the correct way.
We make the default sqlite database path a full path so that the
database isn't being assumed to be in CWD.

Also add some more useful comments on the database settings.

This is preparation work to migrate the build tests and be able to
trigger builds on differently configured databases.

(Bitbake rev: 973c740404ca6a09feea250d3433075995067fe0)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-20 10:09:02 +01:00
Richard Purdie 07eebc6689 bitbake: knotty: Fix output buffering issues
We need to flush the footer removal, else it may not be outputted until
the buffer is flushed as part of StreamHandler and this would lead to
it removing the ERROR output just printed which is extremely confusing.

Also ensure the footer is cleared before printing a summary as in
some cases it wasn't being removed, also leading to user confusion.

(Bitbake rev: 0e030c4d074c41859608dab5f3ad26b05f56b306)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-17 14:43:31 +01:00
Elliot Smith d9d715b9f9 bitbake: toasterui: capture keyboard interrupts the same way as knotty
knotty captures two levels of keyboard interrupt: a single interrupt
or two interrupts in a row. These then trigger stateShutdown
and stateForceShutdown respectively.

toasterui doesn't have an equivalent way of capturing interrupts and
using them to shut down bitbake. Now that we are no longer using
knotty + XMLRPCServer for our command line builds (since switching to
per-project build directories), we see some odd side effects of this,
such as builds continuing after they have been interrupted on the
command line.

Bring toasterui in line with knotty (copy-paste most of the code
in knotty.py which deals with interrupts) so that a keyboard
interrupt actually shuts down the bitbake server (if not in
observe only mode).

Additionally use the cancel_cli_build() method to set the Build
status to CANCELLED in Toaster's db when we get keyboard interrupts.
This means that builds interrupted on the command line show as
cancelled (same as if they'd been cancelled from the Toaster UI),
as specified in the UI designs.

[YOCTO #8515]

(Bitbake rev: d39d2edca95900da433074ee95a192d7bfe7090d)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-17 14:43:30 +01:00
Elliot Smith df0fc2d908 bitbake: buildinfohelper: add method to set current build as CANCELLED
This will be used from toasterui to cancel the current command-line
build when a keyboard interrupt is captured.

[YOCTO #8515]

(Bitbake rev: 1486c770327b53bb5e04baa5f3ea26d8154aed63)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-17 14:43:30 +01:00
Richard Purdie 471310e5e7 bitbake: cooker/toasterui: Drop SEND_DEPENDS_TREE UI feature
Now the event is sent unconditionally we can drop this feature
as its no longer needed.

(Bitbake rev: 473deeb0fc6065693e1fcfcbb8b79753103db537)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-14 23:05:14 +01:00
Richard Purdie 7d06ec193e bitbake: knotty: Ensure consolelog file handle is closed
If we don't close the console log file handle, python prints a warning
about unclosed file handles upon exit which is annoying.

(Bitbake rev: 624dd92952b2fc736fd86abe5f2390b87b3a7dd3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-13 13:41:32 +01:00
Richard Purdie 654eadfa30 bitbake: bitbake: Update logger.warn() -> logger.warning()
python deprecated logger.warn() in favour of logger.warning(). This is only
used in bitbake code so we may as well just translate everything to avoid
warnings under python 3. Its safe for python 2.7.

(Bitbake rev: 676a5f592e8507e81b8f748d58acfea7572f8796)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-05-11 10:34:30 +01:00
Richard Purdie 9f374c4e85 bitbake: providers: Add PREFERRED_RPROVIDER support
Sometimes you can end up in a situation where you need to specify that
a specific runtime entity should be provided by a specific entry.

An example of this is bluez where you could end up in a situation where
for example:

NOTE: multiple providers are available for runtime libasound-module-bluez (bluez4, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match libasound-module-bluez
NOTE: multiple providers are available for runtime bluez-hcidump (bluez-hcidump, bluez5)
NOTE: consider defining a PREFERRED_PROVIDER entry to match bluez-hcidump

The only option here is to set something like PREFERRED_PROVIDER_bluez4 = "bluez4"
which is clearly not very informative.

I've actually held off adding RPROVIDER support for a long while as this
does have sigificant potential for misuse. It doesn't for example allow
multiple runtime providers of the same name to coexist, that simply isn't
supported. It therefore doesn't replace some of the name mappings such
as busybox verses coreutils that OE-Core faces as that is a different
problem with different constraints. This mechanism is simply to provide
bitbake with a hint to decide what the dependency tree should look like.

Also, this allows us to stop printing a confusing message telling the user
to set PREFERRED_PROVIDER when the setting needed would be rather ambiguous.

[YOCTO #5044]

(Bitbake rev: 62eb39d1474d024b204634689071700605c6095c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-15 06:59:44 +01:00
Richard Purdie 8e7282c032 bitbake: cooker/knotty: Prefix parse logs with filename being parsed
We now prefix log messages coming from worker task context with the
PF and task info, however parsing messages all have to be manually
prefixed which is ugly and error prone. This change modifies the log
handler filter so this happens automatically, meaning we don't have
to change every message to include that information. This makes error
messages longer but more usable.

(Bitbake rev: 1af0ccaac81e182c4ca520037dda362d180e5605)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-15 06:59:44 +01:00
Joshua Lock 7a309d964a bitbake: buildinfohelper: work around unicode exceptions
We have been seeing UnicodeDecodeErrors when handling the
ImagePkgList MetadataEvent in ORMWrapper's
save_target_file_information() if the event includes filenames
that include non-ASCII characters.

In the short term work around this by converting paths to the
unicode type when passing them to Django's ORM. This is a bit of
a hack but it's too late in the cycle to do anything more invasive.

[YOCTO #9142]

(Bitbake rev: f50fff03b3de02e73a3cc2eb9935f7c345dbddc4)

Signed-off-by: Joshua Lock <joshua.g.lock@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-06 23:10:30 +01:00
Ed Bartosh 860cba85cf bitbake: toasterui: update build in internal state
buildinfohelper stores current Build object in its internal
state. Any changes to Build object will be lost if internal
state is not updated as current buildinfohelper code
saves Build object from internal state when build is
completed.

This bug causes incorrect build state when build is cancelled.
Updating internal state should fix it.

Note, that this commit updates internal state after status of
the build is changed to Build.CANCELLED. There are several other
places in the code where Build object is updated without updating
internal state. They should be carefully analyzed and fixed.

(Bitbake rev: d056cf40fc55530cb1736aedfb9a3c355884991e)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-06 23:10:30 +01:00
Ed Bartosh acb94078c4 bitbake: buildinfohelper: fix KeyError
When bitbake doesn't need to build anything it still sends
ImagePkgList event with empty 'pkgdata', 'imgdata' and 'filedata'
fields. This causes crash in buildinfohelper code as it's assumed
that above mentioned fields always have data keyed by build target:

ERROR: u'core-image-minimal'
Traceback (most recent call last):
  File "toasterui.py", line 423, in main
    buildinfohelper.store_target_package_data(event)
  File "buildinfohelper.py", line 1218, in store_target_package_data
    imgdata = BuildInfoHelper._get_data_from_event(event)['imgdata'][target.target]
KeyError: u'core-image-minimal'

Fixed this by using dict.get method with empty dictionary as default
return value instead of trying to get value without checking if target
key is in the data.

(Bitbake rev: c39cc463e6d9594bf2c5ac8bb74e834f6f2cf7c8)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-06 23:10:30 +01:00
Michael Wood 22242ae221 bitbake: buildinfohelper: Add handler for cancelling a build
When a build is cancelled the build (action) is complete if it has been
caused the request being cancelled then update the build outcome
accordingly.

(Bitbake rev: d94d12914d351bf560b06d6f4e45c294b04ecaa3)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-06 23:10:29 +01:00
Ed Bartosh d6992a8d69 bitbake: toasterui: shutdown on BuildCompleted event
toasterui exits event loop on one of the following events:
CommandCompleted, CommandFailed or CommandExit.
Unfortunately none of them come from bitbake when build fails.
This is normai if toasterui runs in observer mode. However, if it's
in build mode this causes toasterui to stuck in the infinite loop
waiting for new events.

The only event we can rely on is BuildCompleted as it always
comes from bitbake unlike 3 above mentioned events.

Modified the code to always shutdown toasterui in build mode
on BuildCompleted event.

(Bitbake rev: 9cd60f98b13cf7b1c518851a51e1cbaa596d8f81)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-06 23:10:28 +01:00
Ed Bartosh 4adddfde44 bitbake: toaster: fix jethro build
The keys 'started', 'ended', 'cpu_time_user', 'disk_io_read' and
'disk_io_write' were added to the event recently, so they don't
exist in the events generated by bitbake server from older releases.

Checking if task_to_update structure has these keys before using
them should fix build of older releases.

(Bitbake rev: 79611d0ea742263074fbb0bf5f1e39df75fd9f55)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-06 23:10:28 +01:00
Ed Bartosh 9d4c62d33e bitbake: toasterui: fix brbe reporting
buildinfohelper.brbe is lost when buildinfohelper is closed.
This causes incorrect report of brbe when build is done.

Saved brbe attribute before closing buildinfohelper and used
it to report correct brbe.

Got rid of useless and confusing 'ToasterUI build done 1'
log message.

(Bitbake rev: 5d7cce0d0ed70f6b3ebd6cbad300d86964a13398)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-06 23:10:27 +01:00
Ed Bartosh 5bcce68ea3 bitbake: buildinfohelper: improve handling of providermap
DepTreeGenerated event doesn't contain 'providermap' data in jethro.
Modified buildinfohelper to handle events without this data. This
should make it possible to handle jethro events coming from jethro
bitbake server by the latest buildinfohelper.

(Bitbake rev: f6dcb1c9967f042beae024146781cb8235a9e1f2)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-06 23:10:27 +01:00
Ed Bartosh 61b6b98f40 bitbake: uievent: improve BBUIEventQueue code
Return value of self.BBServer.registerEventHandler differs between
jethro and master. To be able to build jethro toaster should be
able to communicate with jethro bitbake server i.e. it must work
with both old and new registerEventHandler call.

(Bitbake rev: f356c154016c428a3b53af61a075de6f14d9d1d9)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-06 23:10:27 +01:00
Ed Bartosh 0b0d754950 bitbake: toasterui: add brbe parameter to buildinfohelper
In current toaster code BRBE(build request:build environment) value
is passed from toaster to buildinfohelper through the 'SetBRBE' event.

Passing it through environment variable is easier as it doesn't
involve rpc communication between toaster and bitbake server.
It also eliminates the need in running bitbake observer process.

Added parameter 'brbe' to BuildInfoHelper.__init__
Used environment variable TOASTER_BRBE to set brbe for
buildinfohelper object.

(Bitbake rev: a0c8e2b309055e5927a8ff729d292ccaa69d0575)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-06 23:10:27 +01:00
Michael Wood c9ad1e6703 bitbake: toaster: buildinfohelper Add additional metadata to the built layer
Add additional metadata to the layer created for build history to be
able to identify the layer and recipe later on. Specifically this is the
branch and release to which the recipe and layer are associated with
enabling differentiation of two recipes which are local release and
master and 'master' release.

[YOCTO #8528]
[YOCTO #8545]

(Bitbake rev: 3deebd887bddbbd02fd9829a180aab494b1af7c4)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-04-03 15:53:43 +01:00
Ed Bartosh 2918b50361 bitbake: toasterui: remove ParseStarted from the event list
bb.event.ParseStarted event is not processed by toasterui, but
present in event list. This causes the following error:
   WARNING: Unknown event: <bb.event.ParseStarted object at ...
and non-zero return code:
   WARNING: Return value is 1

(Bitbake rev: 1cc102f3d83d9467a3a3c422254333796ba95605)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23 17:56:16 +00:00
Michael Wood ab2abd46f6 bitbake: toasterui: Remove the excessive exception logging
Remove the very verbose log dump from toasterui. This generates several
megabytes of not that useful debug information and actually hinders
finding the original exception.

(Bitbake rev: a21dc134bdce2c9eb5e47c770094660f0c45c398)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23 17:56:15 +00:00
Michael Wood 3025092142 bitbake: toaster: orm Add a constant for the CustomImageRecipe's layer name
Use a constant to define the name for the toaster custom images layer;
this constant is then used to identify this layer in various places.

(Bitbake rev: 2540969ec71612af7f9041cadcc401513e9b357b)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-23 17:53:55 +00:00
Elliot Smith 0dcab0258e bitbake: toaster: rework task buildstats storage and display
The data available from buildstats is now more fine grained than
previously, so take advantage of that to enrich the data we save
against tasks:

* Store the CPU usage for user and system separately, and display
them separately.
* Disk IO is now measured in bytes, not ms. Also store the
read/write bytes separately.
* Store started and ended times, as well as elapsed_time. This
will enable future features such as showing which tasks were
running at a particular point in the build.

There was also a problem with how we were looking up the Task
object, which meant that the buildstats were being added to
new tasks which weren't correctly associated with the build. Fix
how we look up the Task (only looking for tasks which match the
build, and the task and recipe names in the build stats data) so
the build stats are associated with the correct task.

[YOCTO #8842]

(Bitbake rev: efa6f915566b979bdbad233ae195b413cef1b8da)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-09 22:45:16 +00:00
Ed Bartosh 3322fa7670 bitbake: toasterui: fix warning 'Unknown event'
Continue after processing BuildStarted event to fix
WARNING: Unknown event: <bb.event.BuildStarted object at 0x2554150>

(Bitbake rev: 12f1fb8c9b70fea0c9145f881bcceb8af32df6af)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
Ed Bartosh 621cbc86c9 bitbake: toasterui: exit on final events
Toasterui exits only if bitbake observer shuts down.
In build mode it should exit when build is done.

Made toasterui exit on bb.command.CommandCompleted,
bb.command.CommandFailed and bb.command.CommandExit events
when it's running in build mode.

(Bitbake rev: b11f9d6d3c2eb615335901e1dcea699daf3afb4c)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
Ed Bartosh 8e138b7f2a bitbake: toasterui: make toasterui to work in build mode
Currently toasterui works only in observer mode. This is
artificial limitation which was made to support current toaster
design. As we decided to stop using bitbake server we'll
need to run toasterui also in build mode.

[YOCTO #7880]

(Bitbake rev: d4b5796899c3ca5c7becd7322291afd8afb35a31)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
Ed Bartosh 0a6130697e bitbake: toasterui: check if setEventMask succeeded
Currently toasterui ignores return value of setEventMask
command, which created confusing difference between set of
events set by this command and the real set used in the code.

Checked if setEventMask succeeded. Print error message and
exit if it's not.

(Bitbake rev: 6e3f13ffb47102b5df2da91fbc3f5da3179245b2)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
Ed Bartosh dd3da9aa47 bitbake: toasterui: update list of events
Removed events not used in the code from the list.
Added events that are used in the code.

(Bitbake rev: 16b14ec16049cc2040a60ad5fc95f6e19dda91a6)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
Ed Bartosh f56fa5dfd4 bitbake: toasterui: reformat list of events
Reformatted and reordered list of events to make changes
easily and see them clearly in the diffs.

(Bitbake rev: 42a2d1115f2b23dc063a3172285ca3be73cf70bb)

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-07 17:23:04 +00:00
brian avery a7d1b95c96 bitbake: ui: remove the puccho ui
This ui does not work in master, nor has it been updated for several years.

[YOCTO #9178]

(Bitbake rev: 9fad1d13eed1f725971e6d12d3977cd31e07019a)

Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 22:41:23 +00:00
bavery a9dc72f08b bitbake: hob: removal of hob ui and associated ui files
We've been gearing up the Toaster web UI to replace the Hob (GTK+ based) UI
for some time now; Hob has basically been on life support for the past few
releases. As of late last month in master, Toaster has the capability to
select the packages in an image, removing the last thing that Hob could do
that Toaster couldn't.

To recap, the reasons why Hob is being removed include:

- The code is tightly woven into BitBake, making it fragile. This means it
needs significant QA and maintenance on an ongoing basis.

- Some of the implementation is not ideal; we'll be able to remove some cruft
from BitBake and OE-Core at the same time.

- It's GTK+ 2 based, not the current GTK+ 3.

- Toaster is now a much more capable UI and is being actively maintained

The discussion about removing hob can be found at:
http://lists.openembedded.org/pipermail/openembedded-architecture/2016-February/000082.html

(Bitbake rev: be2cceea159c6ca9111eff3df87b98513eab6d72)

Signed-off-by: bavery <brian.avery@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-03-02 22:41:23 +00:00
Mirela Rabulea 5a87d8c477 bitbake: Allow Hob to run images on a custom simulator, other than qemu
The current behavior of Hob is that there is a "Run Image" button which becomes visible only for qemu images.

My suggested change is:
- if an image is selected and it is qemu-compatible, let the "Run image" button be named "Run qemu image"
- if an image is selected and it is not qemu-compatible, let the same button show up with the name "Run custom image", and besides that, an option shows-up to allow the selection of the custom script (by default it points out to runqemu script) to be used for launching this custom image

Note: in case there is more than one toggled image (qemu runnable or deployable), when the user clicks the "Run custom image" button, a dialog will be presented, allowing to choose between any of the existing images.

[YOCTO #8940]

(Bitbake rev: cc4cfc2370297b8feb2dc39d4262e73adf06c09a)

Signed-off-by: Mirela Rabulea <mirela.rabulea@nxp.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-18 07:41:15 +00:00
Christopher Larson a25858982a bitbake: bb.ui.knotty: prefix task messages with recipe/task
An example prefix: `perl-5.22.1-r0 do_compile:`

(Bitbake rev: 792b759e59e31d2e43d525a6e50d866b4f51f072)

Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-16 09:04:23 +00:00
Richard Purdie 8b217208cf bitbake: knotty: Set exit failure code on runQueueTaskFailed events
If the worker segfaults, we may never see a TaskFailed event from it, only
a runQueueTaskFailed event. In this case, return_value isn't getting set
leading to an incorrect exit code from bitbake. Fix by setting return_value
in both places.

(Bitbake rev: e5dd50e0d95d532fe31dde61f8c6b1a7a72321e9)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-15 16:48:12 +00:00
Elliot Smith 1f7f077466 bitbake: buildinfohelper: unset brbe variable when build finishes
The brbe variable is set on the bitbake server when the Toaster
UI starts. This enables Toaster to associate events with the
correct build and build environment.

However, the brbe variable is also used when a build starts to
identify whether a new build needs to be created, or an existing
one looked up. This causes a bug for command-line builds which
happen after a Toaster-triggered build: because the brbe variable
is never unset on the server or the buildinfohelper, the new
command-line build events are treated as originating from the
previous build.

Ensure the brbe variable is reset when the buildinfohelper "closes"
a build, so that each build then either sets the brbe variable
(Toaster-triggered builds) or leaves it blank (command-line builds).

Also modify the localhostbecontroller so that the brbe variable
is not set on the server and not looked up from the server. This
ensures that it is only set when the triggerBuild() method is
called, and that it remains as None for command-line builds.

[YOCTO #9021]

(Bitbake rev: 4a6a8d0074f62208d843b06344be31ae73d9b745)

Signed-off-by: Elliot Smith <elliot.smith@intel.com>
Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-11 22:34:19 +00:00
Michael Wood a8ab1c6b25 bitbake: toaster: buildinfohelper CustomImagePackage update dependency info
Instead of keeping the original dependency information for the pool of
CustomImagePackage reset it with each new build.

(Bitbake rev: a0b97ffc7a468bad081ce3276c74728bf6830250)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:21 +00:00
Michael Wood a6e4f94b01 bitbake: toaster: buildinfohelper Add the concept of CustomImagePackage
This adds the concept of CustomImagePackage this is similar to the way
layers and recipes work in that we have a set of data which is part of
the build history and a set of data which is part of the configuration
data that toaster uses to guide people in configuring their project. We
create a set of  built_packages for every build but only create a package
for configuration purposes if we don't already have one, so that the
CustomImagePackage only ever contains a unique list of packages that are
available to be added and removed from a CustomImageRecipe.

(Bitbake rev: f81bb65883baa6c0f8a4d48a4de3291a10543992)

Signed-off-by: Michael Wood <michael.g.wood@intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-10 13:29:17 +00:00
Richard Purdie 602da7c921 bitbake: knotty: Don't show errors for universe provider issues
When running universe builds, we don't expect an error exit code for
provider warnings. Change the error messages to warnings in this case.

This deals with errors causing problems on our autobuilders amongst
other issues.

(Bitbake rev: d4989fb0355476de172169f0698757f7360e9a1f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-02-01 11:20:42 +00:00
Sujith H 3072361f86 bitbake: bitbake: BBUIHelper: Remove function findServerDetails
findServerDetails function can be removed safely
from the source tree. Couldn't find any files
calling this function.

(Bitbake rev: 46871f769db13ccd36deedc5b6f3dbc0a3d31c4b)

Signed-off-by: Sujith H <sujith.h@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-01-30 11:43:56 +00:00