Currently if errors occur when starting the PR service, there is a race that
occurs since the UI runs various commands including starting builds before
processing the CookerExit(). By adding the error state and refusing to run
async commands in this mode, builds are prevented from starting and the
UI reaches the exit code with the system shutting down cleanly.
(Bitbake master rev: 42fa34142ea685f91115a551e74416ca28ef1c91)
(Bitbake rev: bc2e0796c1846d1567db6343b24b85fd7dba9163)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The patch that implements removeConfigurationVar method was made before
merging the patch that replaces "added by bitbake" with "added by hob".
This patch corrects this issue.
[YOCTO #5284]
(Bitbake rev: 42601a5edef0316767b952b162123534aa8fee18)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In Hob settings, there is a tab to add/remove extra settings. This
patch implements a way to "remove" variables from conf files, through
bitbake. But, to keep the history assigment of the variables synchronized,
instead of removing, it replaces the lines with blank lines.
[YOCTO #5284]
(Bitbake rev: bd720fb63cef6b399619b8fbcaeb8d7710f2d6df)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
MACHINE var is saved using early assignment operator.
Calling MACHINE=x bitbake core-image-... works properly.
Comment "#added by bitbake" is replaced with "#added by hob".
[YOCTO #5070]
(Bitbake rev: 2d0ec8ff083b636a6cf98de3278900eb95c3def6)
Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding the 'getAllKeysWithFlags' read-only command that will
return a dump of the global data state, together with specified
flags for each key. The flag list is passed in as the first
parameter to the command.
This will be used by UI clients to get the build configuration.
(Bitbake rev: 3e094da513e1220319288806cb76ddf804772afd)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The loaded cache modules may add extra attributes to
the recipecache, that will be populated by the cache
classes required by the UI. These attributes
will be used by the UI to display relevant information.
Adds cachefields cache class field to specify
for each cache class which attributes will be set
in the recipecache.
Adds code to automatically expand depends tree with the
fields exported by the extra cache class.
Fixes a cache field name in the HOB UI.
(Bitbake rev: 47c171005fb3803d936e65fcd4436c643883ae16)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding a CookerFeature that allows UIs to enable
receving a dependency tree once the task data has been
computed and the runQueue is ready to start.
This will allow the clients to display dependency
data in an efficient manner, and not recompute the runqueue
specifically to get the dependency data.
(Bitbake rev: 75466a53b6eece5173a9bfe483414148e4c06517)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implementing feature set selection that allows a client
to enable specific features in the server at connection time.
Only enabling of features is supported, as there is
no way to safely remove data loaded into the cooker.
Once enabled, a feature will remain enabled for the
life of the cooker.
Client-server connection now supports specifying the feature
set required by the client. This is implemented in the Process
server using a managed proxy list, so the server cooker
will now load dynamically needed features based on what client
connects to it.
In the XMLRPC server the feature set is requested by
using a parameter for registerUIHandler function.
This allows observer-only clients to also specify features
for the server.
The server code configuration now is completly separated
from the client code. All hardcoding of client knowledge is
removed from the server.
The extra_caches is removed as the client can now specify
the caches it needs using the feature. The UI modules
now need to specify the desired featureSet. HOB is modified
to conform to the featureSet specification.
The only feature available is CookerFeatures.HOB_EXTRA_CACHES
which forces loading the bb.cache_extra:HobRecipeInfo class.
(Bitbake rev: 98e594837aab89ea042cfa9f3740d20a661b14e2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The syntax used for checkPackages implies a copy is returned but it
did not do so. Make it so.
This is fixes universe builds where error messaages were being shown
but there should have only been warnings.
[YOCTO #5222]
(Bitbake rev: 97db2a0792d605f27d434bc6a4acce52857deee1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Clean-up to avoid duplication and promote code reuse to factor
taskdata creation into a common function.
[RP: minor tweaks]
(Bitbake rev: 468c221449290c4f196e87f7d8e23fcd7db86135)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Exiting from the server is antisocial, instead we should raise an exception. This
will correctly fail the current command and reset the server state. We use
the handled exception since for these conditions to occur, something was
already displayed to the user.
(Bitbake rev: dacc94bcace85a2e95aee2dccd8e680c59e4545f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
After running a command on the server, it needs to reset to the initial
state. This ensures that subsequent clients start from a known state
and notice any configuration changes.
Ultimately we may want to do more than this buts a good start and better
than nothing.
(Bitbake rev: dd15648fc2654b8d7c3e00ea7ab3dbf04f24f24b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The shutdown state causes the server to finish what its doing, stop was
them meant to completely stop it. It doesn't mean the server is stopped
though. Renaming the current stop event for forceshutdown gives more
meaning to what it actually does. The stopped namespace then becomes
available to indicate a completely stopped server.
(Bitbake rev: 12e9d33bfae5294e3870dfd1202f63383ad05e92)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the cooker event data isn't rebuilt upon reset and the cache
configuration cannot be changed after init. These are both bad things
and this patch refactors the init/reset code so that it is possible
to reconfigure the server.
(Bitbake rev: 1193b8d76fcb6cb87e9ec135a2514370d7dd90ac)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This call only ever happens in cooker context now so we can drop the
nasty worker check from here.
(Bitbake rev: bc0b30199a8e3624c5b9914430adbcc7c6bd4497)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Cookerdata.findconfigFile method has a new parameter. Changed some calls.
(Bitbake rev: dce0f9d4afe0986e2dd0146944fc4ac9dde275e4)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
updateCache handles the logic for shutting down the parsing so we need
to call it for all cases when we're not running.
This fixes hangs if Ctrl+C is pressed during parsing.
(Bitbake rev: 552b8935dd2f9f11e8d5c08a597a7e966b891480)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hob retrieves the list of recipes and packages using the IMAGE_INSTALL
variable, so a custom image should be saved using this variable.
Changed how the image is saved in a bb file
[YOCTO #4193]
(Bitbake rev: edf3f52c05d86d49b71770cdafde583213e2034d)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Saving an image that requires another image will cause issues when
the second on is removed. So, we have agreed to "require" only the images
from layers.
The functionality is implemented in bitbake, in order to be more abstract,
and it is used by Hob when an image recipe is saved.
[YOCTO #4193]
(Bitbake rev: 28296ca78507ba2e414eb136c81afee65a8e25e5)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It was necessary to append ${TOPDIR}/recipes/images to BBFILES.
Implemented the mechanism to append a value to a variable: a command and
the method in cooker.
[YOCTO #4193]
(Bitbake rev: 4aedbee90bd92395c2460a68702e6ede00e256c9)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When an new image is saved, the dialog for this action has
a field for the description. Changed how an image is saved, by
appending the DESCRIPTION variable at the end of the .bb file.
[YOCTO #4193]
(Bitbake rev: 5629007f2b984005e3a8ac5d9b71422cbc2f1409)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modified generateNewImage function from cooker, in order to be used to
save a template in Hob.
Created a command to ensure that some dirs are created. The templates
(recipes) will be saved in {TOPDIR}/recipes/images folder.
Called these methods from Hob.
[YOCTO #4193]
(Bitbake rev: 96ffa00945c7eb09a0132fa47159aef3ef20fb3e)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In this case, the comment is appended to the end of the file.
Some text editors, do not place a '\n' to the end of the file
after saving it.
[YOCTO #4636]
(Bitbake rev: 2beb9589b1bd9773f587b4dc08afdfe50f4ea913)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As it is implemented now, when parsing the configuration files is
needed, it also needs an initialization.
Also, implemented a function to create an extra configuration files
which can be used as postfile/prefiles when parsing.
(Bitbake rev: 5fe23c1b51da14f58f8c483f43f30b48766a0913)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we reset, the current tracking value may change unexpectedly. This
patch ensures that the default value is preserved over a datastore reset.
(Bitbake rev: 0aa27598d9fdc8c3d18839b3ef93d0ba9698f761)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The command for reparse does nothing, so I have managed to remove it,
because it can be misleading. When a reparse is needed, it can be used
the parse function.
(Bitbake rev: 188eaba121789112ffeb1188f0984d23dfe8df4f)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the change to bitbake-worker we need to ensure the workers know
how to contact the PR service, the magic 0 port and singleton is
no longer enough.
(Bitbake rev: c761751e259bb8e940552a28794b45887b5a72d9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The reasons for combining these objects is ancient history, it makes
sense to clean things up and separate them out now. This follows on
logically from the configuration cleansups and leads well into the
bitbake-worker changes.
(Bitbake rev: 89ffd62661ebcf2a97ce0c8dfd5e4d5bfbe27de7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The collection of the extra caching data should not
be performed by the cooker, but supplied to it.
This patch will also streamline the code for launching servers
without a UI attached.
Based on a patch by Bogdan Marinescu <bogdan.a.marinescu@intel.com>
(Bitbake rev: f0b54280a6bce522508e4741e5f507bc284113a8)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The init function call was clobbering data hob relies on injecting, fix
this so the data injected by hob isn't overwritten.
(Bitbake rev: 64ca1aa1fd5f0473196b6cd805637fa644a92173)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Originally it seemed like a good idea to keep the parameters around. Having
seen this in real life use, its incorrect, we should pull all the data we need
into the cooker's configuguration and then use this to build the datastore.
Being able to just build the datastore from the parameters seemed like a good
idea but having a dummy cooker configuration object is now looking like
the better option.
This also fixes failures in hob since the parseFiles command can call
into cooker directly now and reset the configuration prefiles and postfiles
at will, rather than the indirect calls before which were breaking the datastore
(e.g. BBPATH wasn't set).
The cleanup this allows in tinfoil illustrates how this change makes more sense.
(Bitbake rev: f50df5b891bf318f12fc61c74adfcc626cc6f836)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Similarly to the execution context changes, establish better lifetime
management API of the class event handlers.
(Bitbake rev: 54e35a6cceead9521f8b1dacd48e55064e85c8bd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current execution context management for bitbake is ugly and the
use of a global variable is nasty. Fixing that is hard, however we
can improve things to start to establish an API for accessing
and changing that context.
This patch also adds in an explicit reset of the context when we reparse
the configuration data which starts to improve the lifecycle of the data
in setups like hob.
(Bitbake rev: 6c3281a140125337fc75783973485e16785d05a1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its confusing when an event handler in for example a bbclass receives
events for a recipe which doesn't use the class. This is due to the
class event handlers having confusing scope. Worse, the issue is not
deterministic and the events a handler will see depends on the parse
order.
To avoid these issues, remove the class event handler global scope
and only have class handlers recieve events for recipes using the
class.
(Bitbake rev: 7fb95f3d133e440d463d2faa7151c731f8e1ae96)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Having a global method scope confuses users and with the introduction
of parallel parsing, its not even possible to correctly detect conflicting
functions. Rather than try and fix that, its simpler to retire the global
method scope and restrict functions to those locations they're defined
within. This is more what users actually expect too.
If we remove the global function scope, the need for methodpool is reduced
to the point we may as well retire it. There is some small loss of caching
of parsed functions but timing measurements so the impact to be neglibile
in the overall parsing time.
(Bitbake rev: 4d50690489ee8dc329a9b0c7bc4ceb29b71e95e9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to have a memory resident bitbake and to allow task execution, we need
to be able to rebuild the base configuration without a cooker. This moves the
code into its own class so it can be built independently.
The interface is less than ideal here but I didn't want to add parsing methods
a subclassed DataSmart, at least until we've experimented further with this code
and are certain that makes sense. At the very least, the methods are ugly and need
cleaning up. Spliting the code out seems to be the right thing to do though and
should unblock various activities on BitBake so I believe this code is a step in
the right direction.
Based on a patch from Alexandru Damian <alexandru.damian@intel.com>
(Bitbake rev: 22a0b3cf73d2689db0c118b37aa7492632f8b0a7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Building up a set of actions for the server is tricky since we depend upon the
commandline but fall back to values from the datastore. We should be able to build
a datastore without a commandline and vice versa. Ultimately the UI should send
the commands to the server.
This patch amounts to code rearranging, moving the heavy lifting to the UI, though
a helper in the configuration option. This will need further cleanup/tweaking but
this should be the only update needed to the UIs. The code now queries the server
for any missing data should it need to.
This code allows various knowledge of configuration variables to move to the UI side
only, partcularly pkgs_to_build but also all the command specifiers. It should also
be possible to move cmd eventually, I'm just unsure if any callers call the commands
expecting this to default to something sane right now.
(Bitbake rev: 2dbbb1d51dafd4451fef8fe16f095bcd4b8f1177)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently the UI and server configuration is one big incestuous mess. To
start to untangle this we creater cookerdata, a new module which contains
various confiuration modules and the code for building the base datastore.
To start with we add a ConfigParameters() class which contains information
about both the commandline configuration and the original environment.
The CookerConfiguration class is created to contain the cooker.configuration
options. This means we can transfer new paramters to the server over something
like XMLRPC and then build a new configuration from these on the server.
Based on a patch from Alexandru Damian <alexandru.damian@intel.com>
(Bitbake rev: 35bd5997e8d8e74bc36019030cc10c560a8134f9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
"status" is a relic of very early versions of bitbake, rename it to match what it now
means.
(Bitbake rev: db307fc68773868de2db614a061f51c5c6a90468)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
updateCache() gets called by command.py when needed and needs to be iterated
over. The calls in cooker.py are therefore just plain wrong/confusing now.
(Bitbake rev: 8b7c65bba466bb6773c56849074978ce2c956129)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its clear looking at the code the path argument has never been used and doesn't
work. Tweak the function to be a little more readable.
(Bitbake rev: d7047ceaf29b870ade3257e263e010f65aa8bc6e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The Cooker class is too large and needs to be split up into different
functional units. Splitting out the collections code into its own class
seems like a good place to start to try and disentangle things.
(Bitbake rev: ca1fcbb6e214c155a05328779d3d326e10c5eac0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The <> operator is deprecated, replace with !=.
(Bitbake rev: 4a43e58dd97ec6ea304eaa727c030973a5bc91b7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Current if the PR Service fails to start, bitbake carries on regardless or
hangs with no error message. This adds an exception and then handles it correctly
so the UIs correctly handle the error and exit cleanly.
[YOCTO #4010]
(Bitbake rev: 949c01228a977c3b92bfc0802f6c71b40d8e05b3)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie
Cristiana Voicu
Valentin Popa
Alexandru DAMIAN