Currently if errors occur when starting the PR service, there is a race that
occurs since the UI runs various commands including starting builds before
processing the CookerExit(). By adding the error state and refusing to run
async commands in this mode, builds are prevented from starting and the
UI reaches the exit code with the system shutting down cleanly.
(Bitbake master rev: 42fa34142ea685f91115a551e74416ca28ef1c91)
(Bitbake rev: bc2e0796c1846d1567db6343b24b85fd7dba9163)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In Hob settings, there is a tab to add/remove extra settings. This
patch implements a way to "remove" variables from conf files, through
bitbake. But, to keep the history assigment of the variables synchronized,
instead of removing, it replaces the lines with blank lines.
[YOCTO #5284]
(Bitbake rev: bd720fb63cef6b399619b8fbcaeb8d7710f2d6df)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding the 'getAllKeysWithFlags' read-only command that will
return a dump of the global data state, together with specified
flags for each key. The flag list is passed in as the first
parameter to the command.
This will be used by UI clients to get the build configuration.
(Bitbake rev: 3e094da513e1220319288806cb76ddf804772afd)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Empty messages should trigger CommandFailed, not CommandCompleted as
otherwise the exit code will be incorrect.
(Bitbake rev: 70a8ead31f9ffc987d9c6db61a926f7a9af8f8b1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After running a command on the server, it needs to reset to the initial
state. This ensures that subsequent clients start from a known state
and notice any configuration changes.
Ultimately we may want to do more than this buts a good start and better
than nothing.
(Bitbake rev: dd15648fc2654b8d7c3e00ea7ab3dbf04f24f24b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The shutdown state causes the server to finish what its doing, stop was
them meant to completely stop it. It doesn't mean the server is stopped
though. Renaming the current stop event for forceshutdown gives more
meaning to what it actually does. The stopped namespace then becomes
available to indicate a completely stopped server.
(Bitbake rev: 12e9d33bfae5294e3870dfd1202f63383ad05e92)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
updateCache handles the logic for shutting down the parsing so we need
to call it for all cases when we're not running.
This fixes hangs if Ctrl+C is pressed during parsing.
(Bitbake rev: 552b8935dd2f9f11e8d5c08a597a7e966b891480)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add functionality to allow UIs to update and change the types of events they
recieve. To do this we need to add a new command and also need to be able
to obtain the current event hander ID. In the case of xmlrpc, this is
straightforward, in the case of the process server we need to save the result
in a multiprocessing.Value() so we can retrive it. An excplit command
was added to the server API to facilitate this.
The same function can also be used to mask or unmask specific log messages,
allowing the UI to optionally differ from the standard set of message
filtering.
Based upon work by Cristiana Voicu <cristiana.voicu@intel.com>
(Bitbake rev: ba5a6c88785d9889d4172ec79937ac2a5555327e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was necessary to append ${TOPDIR}/recipes/images to BBFILES.
Implemented the mechanism to append a value to a variable: a command and
the method in cooker.
[YOCTO #4193]
(Bitbake rev: 4aedbee90bd92395c2460a68702e6ede00e256c9)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When an new image is saved, the dialog for this action has
a field for the description. Changed how an image is saved, by
appending the DESCRIPTION variable at the end of the .bb file.
[YOCTO #4193]
(Bitbake rev: 5629007f2b984005e3a8ac5d9b71422cbc2f1409)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modified generateNewImage function from cooker, in order to be used to
save a template in Hob.
Created a command to ensure that some dirs are created. The templates
(recipes) will be saved in {TOPDIR}/recipes/images folder.
Called these methods from Hob.
[YOCTO #4193]
(Bitbake rev: 96ffa00945c7eb09a0132fa47159aef3ef20fb3e)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As it is implemented now, when parsing the configuration files is
needed, it also needs an initialization.
Also, implemented a function to create an extra configuration files
which can be used as postfile/prefiles when parsing.
(Bitbake rev: 5fe23c1b51da14f58f8c483f43f30b48766a0913)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The command for reparse does nothing, so I have managed to remove it,
because it can be misleading. When a reparse is needed, it can be used
the parse function.
(Bitbake rev: 188eaba121789112ffeb1188f0984d23dfe8df4f)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds a new command to create an extra configuration
file, in order to create it from Hob.
Also, it calls the enable/disable data tracking when needed.
Modified the prefiles and postfiles variables because the cooker
expects some lists.
(Bitbake rev: 92393701adcfa181a1090e3632bc6df7ee7852e6)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I add an observer only mode for the knotty UI and
the XMLRPC server that will allow the UI to register
a callback with a server in order to receive events.
The observer-UI is able to send read-only commands to the
server, and also is able to register as an event handler.
Read-only commands are the commands that do not change
the state of the server and have been marked as such in
the command module.
The observer can switch to a full client if it calls addClient
at any time, and the server has no other client running.
(Bitbake rev: 4de9ee21f1fa4d04937cc7430fb1fc8b7a8f61e2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The reasons for combining these objects is ancient history, it makes
sense to clean things up and separate them out now. This follows on
logically from the configuration cleansups and leads well into the
bitbake-worker changes.
(Bitbake rev: 89ffd62661ebcf2a97ce0c8dfd5e4d5bfbe27de7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Originally it seemed like a good idea to keep the parameters around. Having
seen this in real life use, its incorrect, we should pull all the data we need
into the cooker's configuguration and then use this to build the datastore.
Being able to just build the datastore from the parameters seemed like a good
idea but having a dummy cooker configuration object is now looking like
the better option.
This also fixes failures in hob since the parseFiles command can call
into cooker directly now and reset the configuration prefiles and postfiles
at will, rather than the indirect calls before which were breaking the datastore
(e.g. BBPATH wasn't set).
The cleanup this allows in tinfoil illustrates how this change makes more sense.
(Bitbake rev: f50df5b891bf318f12fc61c74adfcc626cc6f836)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building up a set of actions for the server is tricky since we depend upon the
commandline but fall back to values from the datastore. We should be able to build
a datastore without a commandline and vice versa. Ultimately the UI should send
the commands to the server.
This patch amounts to code rearranging, moving the heavy lifting to the UI, though
a helper in the configuration option. This will need further cleanup/tweaking but
this should be the only update needed to the UIs. The code now queries the server
for any missing data should it need to.
This code allows various knowledge of configuration variables to move to the UI side
only, partcularly pkgs_to_build but also all the command specifiers. It should also
be possible to move cmd eventually, I'm just unsure if any callers call the commands
expecting this to default to something sane right now.
(Bitbake rev: 2dbbb1d51dafd4451fef8fe16f095bcd4b8f1177)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the UI and server configuration is one big incestuous mess. To
start to untangle this we creater cookerdata, a new module which contains
various confiuration modules and the code for building the base datastore.
To start with we add a ConfigParameters() class which contains information
about both the commandline configuration and the original environment.
The CookerConfiguration class is created to contain the cooker.configuration
options. This means we can transfer new paramters to the server over something
like XMLRPC and then build a new configuration from these on the server.
Based on a patch from Alexandru Damian <alexandru.damian@intel.com>
(Bitbake rev: 35bd5997e8d8e74bc36019030cc10c560a8134f9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some settings in Hob were not saved in Bitbake and they were not displayed
correcty in Hob, because Hob used some other variables with the extension "_HOB".
I have removed this, in order to have a consistency for the variables.
ALso, I called a function to load the extra variables saved in .hob/template_default.hob.
[YOCTO #3957]
(Bitbake rev: 0f62a12ef3fa4daf8f3a289c984115302b75c23d)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Added a new command in bitbake to save a variable in a file; added a function
in cooker which is called by this command.
Added new command in bitbake to enable/disable data tracking.
The function saveConfigurationVar from cooker.py saves a variable in the file that
is received by argument. It checks all the operations made on that variable, using the history.
If it's the first time when it does some changes on a variable,it comments the lines where
an operation is made on it, and it sets it in a line to the end of file. If it's not
the first time(it has a comment before), it replaces the line.
Made some changes in hob to save the variables from bblayers.conf and local.conf
using the bitbake command.
[YOCTO #2934]
(Bitbake rev: 55b814ccfa413d461d12956896364ab63eed70a8)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Till now, a custom image made in Hob was using only the packages from
the base image. Now it is using everything declared in the base image.
Also next to hob-image.bb, it creates another .bb file which is used
in building process. Those images are ignored by git.
[YOCTO #2601]
(Bitbake rev: 22007cf71a00fbb03b40f97201a6eb06c9aebd16)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Executing "bitbake" doesn't get a sane message since the None return value
wasn't being handled correctly. Also fix msg -> cmd_action['msg'] as
otherwise an invalid variable is accessed which then crashes the server
due to the previous bug.
(Bitbake rev: c6211291ae07410832031a5274690437cc2b09a6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this, if an exception occurs the server will silently crash
with no feedback to the user about why (since traceback isn't imported).
(Bitbake rev: e637a635bf7b5a9a2e9dc20afc18aceec98d578f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, command.py can return an error message from runCommand, due to
being unable to run the command, yet few of our UIs (just hob) can handle it
today. This can result in seeing a TypeError with traceback in certain rare
circumstances.
To resolve this, we need a clean way to get errors back from runCommand,
without having to isinstance() the return value. This implements such a thing
by making runCommand also return an error (or None if no error occurred).
As runCommand now has a method of returning errors, we can also alter the
getCmdLineAction bits such that the returned value is just the action, not an
additional message. If a sync command wants to return an error, it raises
CommandError(message), and the message will be passed to the caller
appropriately.
Example Usage:
result, error = server.runCommand(...)
if error:
log.error('Unable to run command: %s' % error)
return 1
(Bitbake rev: 717831b8315cb3904d9b590e633000bc897e8fb6)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is the interface Hob uses to set variable values in many instances,
and at the moment it is possible that some of the values it passes are
not strings. If a non-string value gets into the datastore it can
trigger exceptions during parsing when we attempt to expand the variable
and substitute in the non-string value.
This fixes using the meta-ti layer within Hob - it currently has a
reference to BB_NUMBER_THREADS within a shell function and since this
is a variable that Hob was setting from its configuration as an integer,
due to the above this was triggering an ExpansionError.
Fixes [YOCTO #2875].
(Bitbake rev: 855b71d8a8e468bfeff9e1a6699d79d68ab27aa1)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To make the UI settings take effect, we need to hook at the end of each
config file parsing and set UI specific values.
(Bitbake rev: f54e733c7863110896f43900d9e4e791602f9d65)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The parameters in triggerEvent function is an event object, and it may
cost some time to pass this object through pipe, causing the pipe's
poll() function timeout. Change it to async mode.
(Bitbake rev: 3b5909ebc86a12dedfb30e5446aa81eb58921760)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Remove the "resolve" parameter since the original resolve=False
option is no longer be used.
(Bitbake rev: dadce609149cfb09ecdc53bfe1f416a3f57a5033)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we have generateTargetsTree API, which is used to get
dependency information. However in that tree, there will be
"virtual/xxx" in depends fields. Therefore we add the resolve option
to replace it with its real providers.
Besides, for packages that provided by multiple recipes, we will find
their preverred provider.
(Bitbake rev: 28501612efdfc6ee47576cc90deb6e897883e7f5)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This functions enables the client to request triggering specific event
from bitbake server.
(Bitbake rev: 45da6d709a69697158fae92e1c0c0a6ac8f30831)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new API in command.py to get the cpu count in order to set the appropriate default BB_NUMBER_THREADS and PARALLEL_MAKE variables.
(Bitbake rev: 335047b2e440e65713e88fabb24b47a9c82f939b)
Signed-off-by: Shane Wang <shane.wang@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have a problem knowing when to show the user debug information and
when not to since the code has already shown the user suitable information
about why a failure is occurring.
This patch adds a bb.BBHandledException exception class which can be used
to identify those exceptions which don't need further explanation to
the user.
This patch uses this class for the bb.providers exceptions and ensures the
command handling code correctly filters the exceptions meaning that
"bitbake invalid"
now shows an simple error message and not a python traceback.
[YOCTO #1141 partial]
(Bitbake rev: eac9249b40ae1e3aa21e016010c862664e59a8d4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The parseConfigurationFiles API calls the related function in
cooker.py to parse config files.
(Bitbake rev: 96c307b9874131ad8c7d9caea6f6dfbd09aab9d4)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no direct use of command in cooker.py, and it is using
bb.command instead. Remove command in the import list.
This fixes a problem of embedded import between command.py and
cooker.py.
(Bitbake rev: c353316b2efcc7a893d6b4aa9a9647d51a6f69e3)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
initCooker is to set the cooker to the initial state with nothing
parsed.
(Bitbake rev: 8ee9c6cc47938604268242d90bd3007218f9a36e)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
for findConfigFiels() and findFilesMatchingInDir() functions, they
don't need to parse all the bb files, thus setting the needcache
value to be False.
(Bitbake rev: 3ef73dee8b08ccfd15a4901cce315a99b22e71d7)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In bitbake/lib/bb/command.py::Command::__init__, we have the following
lines:
for attr in CommandsSync.__dict__:
command = attr[:].lower()
method = getattr(CommandsSync, attr)
sync_cmds[command] = (method)
for attr in CommandsAsync.__dict__:
command = attr[:].lower()
method = getattr(CommandsAsync, attr)
async_cmds[command] = (method)
The sync_cmds and async_cmds are defined as global dictionaries, but it
seems that we've never used them (I did a "grep -r async_cmds bitbake/",
, there is no result except the ones that I have removed), and I can't
find the history of it from "git log -p", I guess that they have been
replaced by the self.cmds_sync and self.cmds_async.
[YOCTO #1791]
(Bitbake rev: 24e99460800856035bb54a84c7aa33b3517436e9)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Turn the reparseFiles logic into a command to reset the cooker's state
machine and a noop which triggers a cache rebuild.
The resetCooker command resets the cookers state machine such that a cache
update will be triggered by any async command which requires the cache.
The reparseFiles command remains as a noop async command that has the
needcache property set to True so that when called it ensures the cache is
built.
Patch from Richard with the addition of removing the force parameter from
the updateCache method.
CC: Richard Purdie <richard.purdie@linuxfoundation.org>
(Bitbake rev: a98f698fe9f38310024013e58475e6d1447ee154)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This command can be used to search each BBPATH for files in the passed
directory which have a filename matching the supplied pattern.
This is implemented for use from the GUI (to determine the available
PACKAGE_CLASSES) but has been written so as to be generically useful and
reusable.
(Bitbake rev: 2a599812a57cb0b964880a6a2b7548423497ea92)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This takes the name of a .conf file and returns the full path to it
(Bitbake rev: 22c8600b885faf841795b872d82f68dfb644a26e)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modify the generateTargetsTree command to allow a list of packages to be
supplied by the caller, in this case we will only generate a target tree
for user requested targets rather than building a tree for the world list.
(Bitbake rev: d4e4f2ecae96e074b2ab3bb9882037af2e385fdd)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Create a standard format server class instance with method calls
for each step in the server setup. There should be enough hooks
for each of the different server types.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some configuration variables (MACHINE, MACHINE-SDK and DISTRO) set which
confguration files bitbake should use.
The added command , findConfigFiles, enables a UI to query which files are
suitable values for a specified parameter.
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Add a new command generateTargetsTree() which returns a dependency tree of
possible targets (tasks and recipes) as well as their dependency information.
Optional parameter 'klass' also ensures any recipes which inherit the
specified class path (i.e. 'classes/image.bbclass') are included in the model
Signed-off-by: Joshua Lock <josh@linux.intel.com>
- Moved the logic for comparing revisions from cooker into command
- Removed 'Cooker' from the event names
- Renamed the 'ExitCode' event into CommandExit, and changed CommandFailed to
be a subclass of CommandExit
(Bitbake rev: c51ed5d7a9971fad6019dac6c35a71b8a54ab16a)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Richard Purdie
Cristiana Voicu
Alexandru DAMIAN
Christopher Larson
Paul Eggleton
Dongxiao Xu
Shane Wang
Robert Yang
Joshua Lock