bitbake-worker makes use of the signal module
but it doesn't import it. This patch fixes the issue.
[YOCTO #4750]
(Bitbake rev: c2ed639690f135994199eb24d964e37f57259e3a)
Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a file ends with .xz, it currently gets overwritten during unpack:
The decompress command for .xz files is:
'xz -dc %s > %s' % (file, efile)
and as efile == file, we end up overwriting file (the source).
Fix this by adding .xz to the list of suffixes that that need to
be removed from a file name for an extract command, leaving the
bare file name. Now, for a given file foo.xz,
file == foo.xz and efile == foo, similar to how .gz .bz2 and .Z
files are treated.
(Bitbake rev: 2cd2d0a48e12ab4358fb967eaf7a56c17993f48d)
Signed-off-by: André Draszik <andre.draszik@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a warning if 'bitbake -C' is executed with a task that does not
exist.
Fixes [YOCTO #4877]
(Bitbake rev: 6459c1d0eb8f1007246df36149e2496ee531e25f)
Signed-off-by: Petter Mabäcker <petter@technux.se>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Until now the split made searching the string "file://", but
this is not ok when SSTATE_VARIABLE has the following form:
SSTATE_MIRRORS ?= "\
file://.* http://someserver.tld/share/sstate/PATH \n \
file://.* file:///some/local/dir/sstate/PATH"
In the documentation I've found that \n is the list separator.
[YOCTO #4857]
(Bitbake rev: 73bcd96928cb2df390e1fc6d3a8b7ce3e9d546a4)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We can't know the dynamic pacakge's name exactly, there might be a
problem, for example, when we use:
IMAGE_INSTALL_append += "ncurses-lib12344"
The ncurses-lib12344 matches ncurses' dynamic packages pattern:
PACKAGES_DYNAMIC = "^${PN}-lib.*"
so there is no errors before the rootfs creation though there is no
ncurses-lib12344.
We can warn this, but I think that we'd better not since there are many
dynamic packages, or there would be too many warnings, for example, the
perl and kernel modules, maybe we can print a debug message for it.
[YOCTO #4798]
(Bitbake rev: df372ca057f0c8c2152223b3e26ad9a30958bab6)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Modal dialogs doesn't run on the main loop so they cannot
catch any signal from the terminal. This patch makes sure
the dialogs are destroyed when a SIGINT is sent to HOB.
[YOCTO #3329]
(Bitbake rev: 6eee0cc37438cc3f91531b7df524330fba27161b)
Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The method was removed when the process for saving configuration
in Hob was changed. Replace the call with the right function.
[YOCTO #4793]
(Bitbake rev: b6aa2b63d71cbe82850a375381b2dbc750cf1905)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If BBPATH isn't set and bblayers.conf isn't found, improve the message
shown to the user to help their understanding of what the problem might
be.
[YOCTO #3271]
(Bitbake rev: 0e639f5cbc813c8d4719019cfdd4287e9a429610)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A
[YOCTO #4772]
When path:file change to python function, it maybe include '@' character.
So, add the special character to change to '_' for avoid error.
(Bitbake rev: 684bc6dcb11ecb1fd7a4d25c08909ad9879e8342)
Signed-off-by: Li Wang <li.wang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In this case, the comment is appended to the end of the file.
Some text editors, do not place a '\n' to the end of the file
after saving it.
[YOCTO #4636]
(Bitbake rev: 2beb9589b1bd9773f587b4dc08afdfe50f4ea913)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(aka pay the cookie monster for weak defaults)
If you have code like:
MYVAR = "a"
MYVAR_override ??= "b"
then MYVAR will get the value "a" even when override is in OVERRIDES. The
reason is that the value of ??= is set as a flag not a value and the cookie
monster isn't paid.
The fix is to ensure appropriate payment is made for a defaultval varflag
matching the usual setVar case.
(Bitbake rev: 3d8044bc79c482c5ea008ddf12a8128dcd1527ee)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if the flags set against a variable in the base data store
change, it doesn't automatically trigger a reparse when it really
should. For example with the blacklist class setting:
PNBLACKLIST[qemu] = "bar"
PNBLACKLIST[bash] = "foo"
will not trigger a reparse if only one entry is changed and a
blacklisted recipe can still be built.
I did consider using BB_SIGNATURE_EXCLUDE_FLAGS in here however it
doesn't make sense, we want to trigger a reparse when any of the
flags change too (which is different to the sstate signatures which
we wouldn't want to change in those cases).
[YOCTO #4627]
(Bitbake rev: ed74ea50043f6feb698c891e571feda2b9f8513d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Until now the variables were saved in bitbake configuration,
now they are saved in configuration files, in order to be read by
bitbake-worker. This helps to assure the consistency for the rest
of the variables.
(Bitbake rev: ea65ebf43525f173205183aa2fd5d8db303ffd4a)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the cooker initialization is called, the variables history is lost.
It need a reset, in order to load the configuration files.
Also, concatenated the value for INHERIT variable.
(Bitbake rev: 2b8c0c09e09ab69edc53501b84050c797f0686e4)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As it is implemented now, when parsing the configuration files is
needed, it also needs an initialization.
Also, implemented a function to create an extra configuration files
which can be used as postfile/prefiles when parsing.
(Bitbake rev: 5fe23c1b51da14f58f8c483f43f30b48766a0913)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we reset, the current tracking value may change unexpectedly. This
patch ensures that the default value is preserved over a datastore reset.
(Bitbake rev: 0aa27598d9fdc8c3d18839b3ef93d0ba9698f761)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Clean up a number of old style accesses to the datastore.
(Bitbake rev: d872fef2c38749c3c6f5d84344db3ec2f9f134ce)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The command for reparse does nothing, so I have managed to remove it,
because it can be misleading. When a reparse is needed, it can be used
the parse function.
(Bitbake rev: 188eaba121789112ffeb1188f0984d23dfe8df4f)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds a new command to create an extra configuration
file, in order to create it from Hob.
Also, it calls the enable/disable data tracking when needed.
Modified the prefiles and postfiles variables because the cooker
expects some lists.
(Bitbake rev: 92393701adcfa181a1090e3632bc6df7ee7852e6)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Until now, some configuration settings was saved in a hob specific
directory. From now on, it will be saved in conf directory through bitbake.
(Bitbake rev: c53e902b010d1c3f1550f7e60e744f40120f73c2)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
BB_NO_NETWORK can be set by bitbake internally by the use of
BB_FETCH_PREMIRRORONLY so update the error message to give users a
hint about this.
[YOCTO #3222]
(Bitbake rev: cac3060d0bf8c7deeacda18d06d92787911380d0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, tasks like fetchall are slightly broken since if a recipe
has specific [depends] which occur after do_fetch and add items not listed
in DEPENDS and RDEPENDS, they are not caught by recrdeptask. We've gone
around in circles on this issue (e.g
http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake/lib/bb/runqueue.py?id=5fa6036d49ed7befe6ad50ec95c61a50aec48195
) and in many cases the behaviour of recrdepends is correct but tasks like
fetchall need the other behaviour.
To address this we add a recideptask flag which can be used in conjuction
with the recrdeptask flag to specify which task to to the inspection upon.
This means entries like do_rootfs[depends] which have do_fetch tasks are
caught and run.
I'm not 100% happy with needing another flag but I don't see any rational
way to get the correct behaviour in all cases without it.
[YOCTO #4597]
(Bitbake rev: f8c9b292b02ce2c28741b74901205f5e5807ca87)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change allows bitbake to connect to a remote server
by reading the BBSERVER environment variable and effecting
the config. Basically a shortcut to the command line parameters.
(Bitbake rev: 81929f86c57ed0a4ad0cda7aaa820fceabaa61e9)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I add an option to terminate a remote server gracefully
as not to need a kill command.
(Bitbake rev: 7495f835666a9561c2c7d84da7aaa74e4df55b9a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
I add an observer only mode for the knotty UI and
the XMLRPC server that will allow the UI to register
a callback with a server in order to receive events.
The observer-UI is able to send read-only commands to the
server, and also is able to register as an event handler.
Read-only commands are the commands that do not change
the state of the server and have been marked as such in
the command module.
The observer can switch to a full client if it calls addClient
at any time, and the server has no other client running.
(Bitbake rev: 4de9ee21f1fa4d04937cc7430fb1fc8b7a8f61e2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Interestingly the previous version comparision was broken and we were always using
the compatibility code, masking this bug. Oops.
(Bitbake rev: d48e8bcb24e8fa5d4fd60fd2c9927a95976d8d8b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Now we've moved to require python 2.7.3, we can jettison the compatibility
workarounds/hacks for older python versions.
(Bitbake rev: a51c402304f2080a76720f9b31d6dfdbed393bba)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a setscene task has a hard dependency on a task like pseudo-native, its
expected that the setscene task will not run unless the dependency is met.
This adds code to ensure that is the case, otherwise a bug would show up
with a usecase like:
bitbake gnome-common
bitbake pseudo-native -c cleansstate
bitbake gnome-common -c clean
bitbake gnome-common
With the double wrapper script environment, we'd not see issues like
this as it would be masked. The problem theoretically affects code like
useradd too as well as anything using a sstate postinstall.
(Bitbake rev: c54e738e2b5dc0d8e6fd8e93b284ed96e7a83051)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
XMLRPC API ran a XMLRPC server at port+2 in order
to provide endpoints to register an event server
in which to dump the events.
This is no longer used, so we remove it.
(Bitbake rev: e171a363913a86e56266f4c9d107110c7f5221e3)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a flag to event handlers which lists the events a given handler wishes to
process. By default event handlers recieve all events but this means
we can stop running code in many cases if we know it doesn't want the event.
This is part of the fix for YOCTO #3812, but implements filtering only
for class event handlers; the other part (events filter for UIs) will be
the subject of a different patch.
(Bitbake rev: 074003a4e7530a72863b9c685fc5c31b0f08c039)
Signed-off-by: Bogdan Marinescu <bogdan.a.marinescu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the change to bitbake-worker we need to ensure the workers know
how to contact the PR service, the magic 0 port and singleton is
no longer enough.
(Bitbake rev: c761751e259bb8e940552a28794b45887b5a72d9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We're going to need a fakeroot/pseudo version of the worker so
abstract the code to start the worker process.
(Bitbake rev: b5d0f12f9df3ab211700473ed145ee6fbd9ca8e9)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The worker was being executed by each execution queue so would get
constructed twice for each build. This is wasteful so move execution
to the main runqueue so we only have to start the worker once.
(Bitbake rev: 8117f8480125b121b2b5ac0afc31b108d9e670ae)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a pretty fundamental change to the way bitbake operates. It
splits out the task execution part of runqueue into a completely
separately exec'd process called bitbake-worker.
This means that the separate process has to build its own datastore and
that configuration needs to be passed from the cooker over to the
bitbake worker process.
Known issues:
* Hob is broken with this patch since it writes to the configuration
and that configuration isn't preserved in bitbake-worker.
* We create a worker for setscene, then a new worker for the main task
execution. This is wasteful but shouldn't be hard to fix.
* We probably send too much data over to bitbake-worker, need to
see if we can streamline it.
These are issues which will be followed up in subsequent patches.
This patch sets the groundwork for the removal of the double bitbake
execution for psuedo which will be in a follow on patch.
(Bitbake rev: b2e26f1db28d74f2dd9df8ab4ed3b472503b9a5c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
AUTOINC was meant to appear once at the start of the version string.
The list of names may not be sorted meaning it could get inserted in
the middle. This patch simplifies the code and ensures it appears at
the start.
Include cache version bump to ensure the cache picks up these changes.
(Bitbake rev: ad8bf10d873abb94d987860a3f6d06b134fb8a99)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Go through the cached list of file checksums and check if any of the
files no longer exist; if any are missing then invalidate the cached
recipe, which will force it to be reparsed and thus force the list of
files to be collected again. This prevents a warning when moving a file
to a different location that is still picked up by the recipe, e.g.
moving a file from a "files" subdirectory to one named with the recipe
name (${BPN}).
Fixes [YOCTO #4474].
(Bitbake rev: e1aa103593c46612c4d9000524174983b5994dc9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The reasons for combining these objects is ancient history, it makes
sense to clean things up and separate them out now. This follows on
logically from the configuration cleansups and leads well into the
bitbake-worker changes.
(Bitbake rev: 89ffd62661ebcf2a97ce0c8dfd5e4d5bfbe27de7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Two fixes in bitbake related to running remote servers -
* can now specify correctly the bind port
* the information in print conforms to common server infrastructure
(Bitbake rev: b657208ee15ae065e5fcc2dd6e0051e03d246727)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch enables the XMLRPC server to remain
resident in memory after a task is run, and to accept
a new controlling client. To check the server after
task completion, do
lsof bitbake.lock
in the build directory. Kill the server with kill.
(Bitbake rev: e823e1f0675ff3794eb39ef0b4df2d7a220f4013)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In an attempt to minimize code duplication, create
clear interfaces, and maximize code reuse through OOP,
bb.server adds base classes for the BitBakeServer,
BitBakeServerConnection and actual server implementations
instructed in particular server types.
These classes document the minimum interfaces that the
derived classes must implement, and provide boilerplate code.
Changes to None, Process and XMLRPC servers as to use
the common server infrastructure.
(Bitbake rev: 6db4a64cef20f8d0aba804db4c4e1eec7b112b46)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Valentin Popa
André Draszik
Petter Mabäcker
Cristiana Voicu
Robert Yang
Richard Purdie
Li Wang
Peter Kjellerstedt
Alexandru DAMIAN
Bogdan Marinescu
Paul Eggleton