The --help text was rather inconsistent in style and plain incorrect in places,
using confusing terminology in others. I guess most people know what the options
do and don't read this but its confusing to new users.
This updates it to use the terms recipe and task consistently, remove
the references to stage, bbread and generally try and make the output
more useful.
[YOCTO #4856]
(Bitbake rev: 516311946c7bd14c84947dc44c3bb0563e5a9667)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implementing feature set selection that allows a client
to enable specific features in the server at connection time.
Only enabling of features is supported, as there is
no way to safely remove data loaded into the cooker.
Once enabled, a feature will remain enabled for the
life of the cooker.
Client-server connection now supports specifying the feature
set required by the client. This is implemented in the Process
server using a managed proxy list, so the server cooker
will now load dynamically needed features based on what client
connects to it.
In the XMLRPC server the feature set is requested by
using a parameter for registerUIHandler function.
This allows observer-only clients to also specify features
for the server.
The server code configuration now is completly separated
from the client code. All hardcoding of client knowledge is
removed from the server.
The extra_caches is removed as the client can now specify
the caches it needs using the feature. The UI modules
now need to specify the desired featureSet. HOB is modified
to conform to the featureSet specification.
The only feature available is CookerFeatures.HOB_EXTRA_CACHES
which forces loading the bb.cache_extra:HobRecipeInfo class.
(Bitbake rev: 98e594837aab89ea042cfa9f3740d20a661b14e2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change allows bitbake to connect to a remote server
by reading the BBSERVER environment variable and effecting
the config. Basically a shortcut to the command line parameters.
(Bitbake rev: 81929f86c57ed0a4ad0cda7aaa820fceabaa61e9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I add an option to terminate a remote server gracefully
as not to need a kill command.
(Bitbake rev: 7495f835666a9561c2c7d84da7aaa74e4df55b9a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I add an observer only mode for the knotty UI and
the XMLRPC server that will allow the UI to register
a callback with a server in order to receive events.
The observer-UI is able to send read-only commands to the
server, and also is able to register as an event handler.
Read-only commands are the commands that do not change
the state of the server and have been marked as such in
the command module.
The observer can switch to a full client if it calls addClient
at any time, and the server has no other client running.
(Bitbake rev: 4de9ee21f1fa4d04937cc7430fb1fc8b7a8f61e2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Two fixes in bitbake related to running remote servers -
* can now specify correctly the bind port
* the information in print conforms to common server infrastructure
(Bitbake rev: b657208ee15ae065e5fcc2dd6e0051e03d246727)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added code in XMLRPC server that creates a stub local server
for a client-only connection and is able to connect to
a remote server, and receive events from the remote server.
Added the option to start a client with a remote server in
bitbake.
Original code by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: 25b2af76104d5aaf6435de8c158e0407512f97ce)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a code sanitization targeted at making further
server-related changes easier (launch a server
separately or creating a mockup-server) to do.
(Bitbake rev: eac00258d213137ef73aed255c92b7981e2f1c75)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The collection of the extra caching data should not
be performed by the cooker, but supplied to it.
This patch will also streamline the code for launching servers
without a UI attached.
Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: f0b54280a6bce522508e4741e5f507bc284113a8)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Slight change in bitbake as to retrieve the entire UI module
for further processing instead of just the main function.
Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: f49341a9599d971829ef65b2b02732543740a3c9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the removal of the none server type, we can remove the launchUI
method and simplify the code slightly.
(Bitbake rev: 9bef2f2dd0bcaa59528ebcb3c1ce053b7dff1ec6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The process server backend has been serving well as the default for a long
time now and the UI model is much better thought out that it used to be. With
the move to make bitbake a memory resident process, the none server is now
looking rather pointless and complicates the code needlessly. Lets therefore
now remove it.
(Bitbake rev: 9af03a89605e3db9bce3cea1e0f2d0b6cfaa6fe1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As the code stands today its hard to know which configuration variables
are used by which parts of the system. Some are used by the UIs, some
by bin/bitbake itself, some by cooker.
This patch changes the configuration to just contain the variables cooker
uses, and changes bin/bitbake to access the variables it needs directly
which hopefully lets us start to untangle this mess.
(Bitbake rev: e57497a24b6157c92519a34accd66035a39ad1f8)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building up a set of actions for the server is tricky since we depend upon the
commandline but fall back to values from the datastore. We should be able to build
a datastore without a commandline and vice versa. Ultimately the UI should send
the commands to the server.
This patch amounts to code rearranging, moving the heavy lifting to the UI, though
a helper in the configuration option. This will need further cleanup/tweaking but
this should be the only update needed to the UIs. The code now queries the server
for any missing data should it need to.
This code allows various knowledge of configuration variables to move to the UI side
only, partcularly pkgs_to_build but also all the command specifiers. It should also
be possible to move cmd eventually, I'm just unsure if any callers call the commands
expecting this to default to something sane right now.
(Bitbake rev: 2dbbb1d51dafd4451fef8fe16f095bcd4b8f1177)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the UI and server configuration is one big incestuous mess. To
start to untangle this we creater cookerdata, a new module which contains
various confiuration modules and the code for building the base datastore.
To start with we add a ConfigParameters() class which contains information
about both the commandline configuration and the original environment.
The CookerConfiguration class is created to contain the cooker.configuration
options. This means we can transfer new paramters to the server over something
like XMLRPC and then build a new configuration from these on the server.
Based on a patch from Alexandru Damian <alexandru.damian@intel.com>
(Bitbake rev: 35bd5997e8d8e74bc36019030cc10c560a8134f9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to the internal event processing, this excepting handler usually raises an
Empty error, masking the underlying failure. Ensure the original exception is
raised.
(Bitbake rev: 7d548568a55adfe84a976f2a549995e42da1afef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python multiprocessing needs /dev/shm, and if it doesn't exist (e.g. in
a chroot environment) or is not writable, you'll get
"OSError: [Errno 38] Function not implemented" or
"OSError: [Errno 13] Permission denied" with a traceback, which doesn't
really help to explain what is wrong.
Implements [YOCTO #4407].
(Bitbake rev: e7460d01f281f913a94192e1f4bd20688164bdd4)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if errors occur early in the init process, the errors may
not be shown to the user. This change ensures that if a failure does
occur, the messages are flushed from the queue and shown to the user.
(Bitbake rev: fda84c4285e0bc31c249b6dd5464aeb6ad622a9a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In case of an error in bitbake, the stack trace is limited to 5
items. This is an endless source of confusion and it makes bugs
reports impractical, since a full stack trace can't be included
in the bug report. This patch simply removes the depth limit.
(Bitbake rev: 02629c42fb09413d9da16cfe43e03338ce7db3ff)
Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We need to empty out the environment whilst we build the cooker but
we need the environment for the UIs since hob uses DISPLAY and other
session variables.
This patch adapts the utils functions to return removed environment
components so we can reinject them for use by the UI, allowing hob
to work again.
(Bitbake rev: fc330d810099c57fefd4e706159a73ad8401d97c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The method this older code was accessing doesn't exist. A library shouldn't
be changing behaviour based on the environment anyway when sepcific function
calls exist to correctly setup the logging. Fix the crash if BBDEBUG was used
by moving the code to bin/bitbake.
[YOCTO #3319]
(Bitbake rev: 3a1b75037fe1e5282ffd40b037e93353405faa45)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This code is dead and doesn't do anything so lets remove it.
(Bitbake rev: 8d45739f49618757a5d7d79782deda355e3981ec)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is possible to lose critical log data when python exits in an
unorderly fashion via segmentation fault or certain types of crashes.
This is because the buffer characteristics are inherited from the top
level stdout, which should be set to unbuffered, for the purpose of
all the forked children.
This pushes the buffering to the OS, instead of having python managing
the buffers in its stream handler class.
This change is also to provide the ability to tail logs written from
processes in "real time" because they would be written in an orderly
fashion depending upon the OS characteristics for the file I/O.
(Bitbake rev: c6a367bc3224adafca698a4ffc5414ad83842c16)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mainly intended for the purpose of debugging or forcing builds
from source, the --no-setscene will prevent any setscene
tasks from running.
(Bitbake rev: 440e479f3e248482c38c149643403c6907ac7034)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The '-s' option shows (input) recipes, not (built/output) packages. Clarify
the help wording for this option so it is consistent with how bitbake defines
recipes and packages.
(Bitbake rev: 0cac6d647c58ae449323959220775fc1afd5bfb0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is for giving the user a clear list to show which pkg would be
built, we have the "bitbake -g", but it is not easy to read for people,
it is for "dot". Improve the "bitbake -g" to also save a pn-buildlist:
$ bitbake -g core-image-sato
...
NOTE: PN build list saved to 'pn-buildlist'
[snip]
The contents of pn-buildlist:
busybox
shadow-native
pth
sysfsutils
qemu-helper-native
curl-native
ncurses-native
gdbm
xserver-xorg
linux-libc-headers
[snip]
[YOCTO #2404]
(Bitbake rev: 18aff925aece774d0172894e25584353519ca03f)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
What is specified is a command (or "cmd" as in the help text for -f)
rather than a task - i.e. you specify compile, not do_compile, so change
the sentence to reflect that and reorder it slightly so it makes sense.
(Bitbake rev: 9ab269410ef5cd4753fe11ec74759f421685bb7b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This new command line option forces the specified task and all dependent
tasks up to the default task to re-run. This means that the following
single step:
bitbake -C compile somerecipe
is equivalent to the following two steps (with the recent change to -f):
bitbake -c compile -f somerecipe
bitbake somerecipe
Note that to work this option needs full hashing enabled (i.e.
BB_SIGNATURE_HANDLER must be set to a signature handler that inherits
from BasicHash). If this is not the case, -C effectively does nothing.
Based on a previous implementation of this option by Jason Wessel
<jason.wessel@windriver.com>.
Implements [YOCTO #2615].
(Bitbake rev: 2530e0faada5775897cfd1b93aba6925826dca73)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patch adds support for a $BITBAKE_UI environment variable which allows
to configure the preferred user interface. Although an '-u' option
(which will override the environment variable) exists already, it was
required to specify this option on every invocation of bitbake.
Because user interface is instanciated very early in the program it is
not possible to use bitbake.conf for setting up a default. An
environment variable (which acts in a similar category like $PAGER or
$EDITOR) is a simple way for configuring the default.
(Bitbake rev: e3c213015953d1a0afb5ef4be59e1264990e5cee)
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On terminals which support it, add summary information to the end of the
build output about the number of tasks currently running and how many tasks
we've run so far.
This provides a summary at a glace of what the current state of the build is
and what the build is currently doing which is lacking in the current UI.
Also disable echo of characters on stdin since this corrupts the disable,
particularly Crtl+C.
The "waiting for X tasks" code can be merged into this code too since
that is only useful on interactive terminals and this improves the
readability of that output too.
Improvements since v0:
* The tasks are ordered in execution order.
* The display is only updated when the list of tasks changes or there
is output above the footer.
* Running task x oy y and package messages are supressed from the console
This UI can be accessed with "bitbake -u knotty2".
(From Poky rev: e38b4569648f2916c4370871c79e6a6090eb8bc1)
(Bitbake rev: 156189c799d2bb1f69bdaa04b5cd718fe7881425)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When start bitbake as a server only process, we need to assign certain
interface to it.
(Bitbake rev: 95b97d2dc6466ea3d99371f5b5bd68f6f3c99074)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a new option "--server-only" for bitbake command, which allows
bitbake runs as a server, and let frontend connect the server itself.
"--server-only" should work with "-t xmlrpc", or bitbake will exit.
bitbake --server-only -t xmlrpc will print out the server address and
port information.
(Bitbake rev: 2677254cf9792cee083605267570d93b425cc2db)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
At the moment it bugs me a lot that we only have one effective logging
level for bitbake, despite the logging module having provision to do
more advanced things. This patch:
* Changes the core log level to the lowest level we have messages of
(DEBUG-2) so messages always flow through the core logger
* Allows build.py's task logging code to log all the output regardless
of what output is on the console and sets this so log files now
always contain debug level messages even if these don't appear
on the console
* Moves the verbose/debug/debug-domains code to be a UI side setting
* Adds a filter to the UI to only print the user requested output.
The result is more complete logfiles on disk but the usual output to the
console.
There are some behaviour changes intentionally made by this patch:
a) the -v option now controls whether output is tee'd to the console.
Ultimately, we likely want to output a message to the user about where the
log file is and avoid placing output directly onto the console for every
executing task.
b) The functions get_debug_levels, the debug_levels variable, the
set_debug_levels, the set_verbosity and set_debug_domains functions are
removed from bb.msg.
c) The "logging" init function changes format.
d) All messages get fired to all handlers all the time leading to an
increase in inter-process traffic. This could likely be hacked around
short term with a function for a UI to only request events greater than
level X. Longer term, having masks for event handlers would be better.
e) logger.getEffectiveLevel() is no longer a reliable guide to what
will/won't get logged so for now we look at the default log levels instead.
[YOCTO #304]
(Bitbake rev: 45aad2f9647df14bcfa5e755b57e1ddab377939a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a data store in the cooker containing the values of the environment
from when BitBake is launched such that child processes can replicate
(and/or use values from) the host environment, rather than the cleaned up
environment that the main BitBake process uses.
(Bitbake rev: 54c7206165c0e7cfe5f7b243c80461baf5e7dfb1)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Default server is process, adjust comments and messages accordingly.
(Bitbake rev: 8ba4d0e98401cdb808f727703913ad8ba87f8e71)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie
Alexandru DAMIAN
Paul Eggleton
Bogdan Marinescu
Jason Wessel
Trevor Woerner
Robert Yang
Enrico Scholz
Dongxiao Xu
Joshua Lock