This unlikely looking function was found to be eating a lot of CPU time
since it gets called once per trip through the idle loop if we're not
running a maximum number of processes. This was particularly true in
world builds of 13,000 tasks.
Calling the computation code is pretty pointless because until some
other task finishes nothing is going to become available to build.
We can know when things become available so this patch teaches the
scheduler this knowledge.
It also:
* skips any coputation when nothing can be built
* if there is only one available item to build, ignore the priority map
* precomputes the stamp filenames, rather than doing it every time
* saves the length of the array rather than calculating it each time
(the extra function overhead is significant)
Timing wise, initially, 5000 iterations through here was 20s, with
the patch 200000 calls takes the same time. The end result is that
builds get up and running faster.
(Bitbake rev: 4841c1d37c503a366f99e3a134dca7440e3a08ea)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no good reason to separately track var_references and
references so merge them and remove the unneeded variable.
(Bitbake rev: 64d4cbd6360c96574cece70205ea3aecc3f8bae6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In 6a48474de9505a3700863f31839a7c53c5e18a8d the url parameter to a
number of functions was removed. However, not all calls to
latest_revision() were fixed...
(Bitbake rev: 7c94ca56b2fd85a989089f58b3dcce3172a778f2)
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
VAL = "" (not shown)
VAL = " " (shown as "")
VAL = " x" (shown as "x")
would all show up rather differently to what would be expected in the
bitbake -e output. This fixes things so they appear consistently.
The output for running some shell functions may also change slightly
but shouldn't change in a way that is likely to cause problems.
[YOCTO #5507]
(Bitbake rev: fcba5ef0053dc0ef5360e4912609e5d52f5046b0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With a SRC_URI = " \
p4://depot/folder/...;module=localfolder/localsubfolder;changeslist=${P4CHANGELIST} \
"
the subfolders of //depot/folder/... get renamed when mapped to the
local folder structure. They lose the first 3 letters. This
patch fixes that.
Issue reported by and patch sent from katutxakurra@gmail.com
[YOCTO #5380]
(Bitbake rev: 40e06dc459d9c0b5d42d65b2d2c846196fd36b1f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In recent versions of bitbake, it is not possible to initialise a
BBCooker object without having it load the configuration first. Thus we
should avoid creating the Tinfoil object here in bitbake-layers which
does that internally until we actually need to, so you can run
"bitbake-layers help" and not have to wait several seconds for the
output.
(Bitbake rev: 8f1e280fbbb6432d7bcc1fb4241f402668c6c5ea)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There has been a continuing call for supporting wildcard in bbappend
filenames. The wildcard is actually allow matching of the name and
version up to the point of encountering the %. This approach will
allow for matching of the major or major.minor.
Exampes:
busybox_1.21.1.bb
busybox_1.21.%.bbappend will match
busybox_1.2%.bbappend will also match
if we update to busybox_1.3.0.bb the above won't match, but a busybox_1.%.bb
will.
[YOCTO #5411]
(Bitbake rev: 31bc9af9cd56e7b318924869970e850993fafc5f)
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When using the dry run option (-n), bitbake would still try and fire
a specific fakeroot worker. This is doomed to failure since it might
well not have been built.
Add in some checks to prevent the failures.
[YOCTO #5367]
(Bitbake rev: f34d0606f87ce9dacadeb78bac35879b74f10559)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no good reason to keep passing around the url parameter when
its contained within urldata (ud). This is left around due to
legacy reasons, some functions take it, some don't and its time
to cleanup.
This is fetcher internal API, there are a tiny number of external users
of the internal API (buildhistory and distrodata) which can be fixed up
after this change.
(Bitbake rev: 6a48474de9505a3700863f31839a7c53c5e18a8d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
urldata contains the url so we might as well stop passing around
pointless function parameters. This was done for legacy reasons but
its time to clean this mess up.
This is a first step in cleanup and is a standalone patch but there is
more to be done in a second patch.
(Bitbake rev: 06590cfebbcf6565a17b80cc298e3ecdfaba4656)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A typo in buildinfo helper leads to a bug where
information about tasks is not correctly stored.
This patch fixes the typo.
(Bitbake rev: 67b752993a2c64cba9ccc4fa662f0bddf081e74a)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch changes bitbake to log an error to the
command line when the build is interrupted via Ctrl-C.
This is needed to inform the user that not all tasks
required for the build have been executed, and
the build is not complete.
Internally, the Bitbake server will return a CommandFailed
event that will be logged by Toaster as build failure.
(Bitbake rev: 9a658e8b1511f1b9f91663f546f748fdfbc8965f)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the computed name of a directory contains an undefined variable
reference, bitbake dutifully creates a directory with a name that has
${...} in it. However, the actual task script created then tries to cd
to that directory, and the cd command fails, because no such directory
exists -- because the shell has helpfully removed the ${...} which did
not match any actual variables.
Since we want the name to be used exactly-as-is, add single quotes around
the name so this doesn't cause strange failures running tasks, which
allows us to progress past such failures and get to a point where they
can be diagnosed.
(Bitbake rev: 2809c2e6f2f35f9b08058950be896947ab5a0284)
Signed-off-by: Peter Seebach <peter.seebach@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have do_bundle_initramfs which is a task inserted after compile and
before build. It is not covered by sstate.
If we run a build with a valid sstate cache present, the setsceneverify
function realises it will rerun the do_compile step (due to the
bundle_initramfs task) and hence marks do_populate_sysroot to rerun.
do_install, a dependency of do_populate_sysroot is left as marked as
covered by sstate.
What we need to do is traverse the dependency tree for any setsceneverify
invalided task and ensure any dependencies are also invalidated. We can
stop at any point we reach another setscene task though.
This means the do_populate_sysroot task has the data from do_install
available and doesn't crash.
(Bitbake rev: f21910157d873c030b149c4cdc5b57c5062ab5a6)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If an SCM mirror is in PREMIRRORS, the tarball is downloaded and then found
by the "upstream" check and handled correctly.
If an SCM mirror is in MIRRORS, the tarball is downloaded but not used
since there is no "upstream" run after MIRRORS completes. It therefore
sits there useless and unused. This code change forces the upstream to
run after a mirror tarball is found and fixes the usage of SCM mirrors
in MIRRORS.
(Bitbake rev: a66ee0994645aa5658b2f5ea134ed17d89f8751a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using git merge-base for checking for ancestors is nice but required git 1.8.0
which is not in many distrbutions yet. We therefore revert to a more ugly
check using git branch --contains until such times as we can upgrade.
(Bitbake rev: 31467c0afe0346502fcd18bd376f23ea76a27d61)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The default value of 0.5 seconds before sending the pr-server a
SIGTERM is not enough to guarantee that sqlite has committed all
the pr data to the database. By polling the pid to see if it is
still running, this allows the pr-server process to shutdown
cleanly and finish the final pr data commit.
(Bitbake rev: 22eec978e70794923c85689928c6be0cfe71cdcd)
Signed-off-by: Konrad Scherer <Konrad.Scherer@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The current use of git log to check if a given revision is present can be
a little fragile.
For example if revision X was on branch A, and then later added to branch
B, the update checks would not notice this since they just check for X
being in the repository.
We also had some autobuilder corruption where an older packed-refs file
was copied over a new repository containing newer pack files. There
was no update to the refs file since the revision was present but
not accessible in any branch.
The correct fix is to check that the required revisions are present
on the specific branches. This patch does this using merge-base.
(Bitbake rev: 89abfbc1953e3711d6c90aff793ee622c22609b1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to track the file where a configuration
variable was defined, this patch bring these changes:
* a new feature is defined in CookerFeatures, named
BASEDATASTORE_TRACKING. When a UI requests BASEDATASTORE_TRACKING,
the base variable definition are tracked when configuration
is parsed.
* getAllKeysWithFlags now includes variable history in the
data dump
* toaster_ui.py will record the operation, file path
and line number where the variable was changes
* toaster Simple UI will display the file path
and line number for Configuration page
There is a change in the models to accomodate the recording
of variable change history.
[YOCTO #5227]
(Bitbake rev: 78e58fed82f2a71f052485de0052d7b9cca53ffd)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
0 (not applicable) is not a valid sstate_result for tasks with
outcome 2 (sstate), which should return 3 (restored), 2
(failed) or 1 (missed).
Sstate_result for tasks with outcome 2 is equal to the outcome
of _setscene corespondent task.
[YOCTO #5220]
(Bitbake rev: 8ff8d75318ea88ba80c744b471e486901ef6749a)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the user explicitly asks to build a target that is listed in the
value of ASSUME_PROVIDED, show a warning mentioning that it will be
ignored.
(Bitbake rev: 65be09f5e2a4a41e65c9232e208d8154b822fc4e)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When the build fails and I click "Edit packages", we should be returned
to the tab we launched the build from. Hob should remember the tab,
not the search state.
[YOCTO #5257]
(Bitbake rev: afdea16516084ef6c046d80e12bf244f25a9da50)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The buildstats file path changes based on the
optional PE variable that may be defined for a
recipe.
The toasterui simply ignored the PE value, and
as such it didn't correctly reach buildstats files
for some of the tasks.
This patch fixes the issue.
[YOCTO #5073]
(Bitbake rev: 97b8ab88edc7c8dfb26b4cf305701ec96e52cc4f)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch addresses an issue where a failed sceneQueue task
entry was not updated on the Fail event. As a result, it
always showed the task as not-available.
[YOCTO #5216]
(Bitbake rev: 9b99a417f58381bac4bda412bcfd11de50403318)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The AUTHOR field in most recipes is not defined,
or it's not really consistently set in the metadata,
Also does it seem particularly useful.
This patch removes the AUTHOR variable from the
toaster system
[YOCTO #5449]
(Bitbake rev: da3ac049300be84defab7b32b0b99ab07c7d0a27)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tasks without script type information showed by default
as NoExec; this happens for all Prebuild or Covered
tasks, as script type information comes only on TaskStarted
event. Such a default value may drive confusion, as NoExec value
should be reserved for the NoExec-flagged tasks.
This patch adds a new default value named Unknown that will be
used for all tasks that don't have script type information
available.
[YOCTO #5327]
(Bitbake rev: ec6cac74290f0d4f5b60222019c23416b4b8e1ef)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Also removed some redundant file manipulation code
Based on patch sent by Stefan Stanacar <stefanx.stanacar@intel.com>
(Bitbake rev: e054c1e7c8581f66082fcdfb89769401ca6e78a3)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Since -g is required, tell the user exactly so.
(Bitbake rev: f05f74e98b0dd567a8b0cb85dc8183716619991b)
Signed-off-by: Gary Thomas <gary@mlbassoc.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster needs to record information about packages
installed on a built target image, and dependencies
between these packages.
This patch fixes a bug where the variable from the
server wasn't read correctly leading which caused
the buildhistory to not be processed correctly.
Additionally, two display issues in the package table
were fixed, issues that lead to package information
being displayed incorrectly.
[YOCTO #5197]
(Bitbake rev: ab4bc18409d80de6d069e3dd76c3c54964fe5764)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In order to use toaster, now you have to set INHERIT+="toaster buildhistory"
To keep it simple, I've done some changes in order to automate it. When toaster
is started, this line is added to a new file called toaster.conf.
This file is passed to the bitbake server with the --postread parameter.
Based on a patch by Cristiana Voicu <cristiana.voicu@intel.com>
(Bitbake rev: 029e868044989eda370340f8bf4200cfd2670fca)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In the Configuration table, we need to link prefixed / suffixed
variables to the corresponding variable descriptions in documentation.conf.
[YOCTO #5198]
(Bitbake rev: 641d9c4fda5fe978154fdfab978c3c09e3906eab)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the terminal where the server was started is closed,
the bitbake server should shutdown. Currently the system
is left in hanging state.
This patch uses "trap" command to make sure the servers
are closed on terminal exit.
[YOCTO #5376]
(Bitbake rev: 5f8b97010f7b465753b6ff6275d18426006ee14b)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch fixes an issue where, if not defined,
the timezone defaults to 'America/Chicago'.
The solution is to set the timezone to current computer's
timezone.
[YOCTO #5186]
(Bitbake rev: a4102b549f04a9b52cdcd318bf511a18ab48067d)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When building a single file, the cooker will recreate
the recipecache from scratch.
I suspect this is a remnant of past code, since:
* the current recipecache works fine
* the new recipecache will not have all the fields as
requested by HOB_EXTRA_CACHES setting
This patch disables recreating the recipecache, leading
to shorter times when building single build files
(-b option) and better compatibility with Toaster.
(Bitbake rev: 618d69b00075981b8553513130d7deb1aed61578)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bitbake tasks may be of type 'python' or 'shell',
or they may not be executed at all, which is record
as task type 'noexec'.
In order to record proper task type, this patch:
* creates no exec task type as the default value in
the toaster model definition
* adds full task flags to the bb.build.TaskStarted event
in build.py
* if the task actually starts, the toaster ui will
record the type of the task as either 'python' or 'shell'
based on the task flags.
[YOCTO #5073]
[YOCTO #5075]
[YOCTO #5327]
(Bitbake rev: 6648c57e6d369fc009ea3a9fe939def5d2c67bf5)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Toaster needes to record extra data that needs to
be moved at the time of the dependency tree dump.
This data includes:
* layer priorities for recording in the layer section
* the inherit list for each PN which allows to determine
the type of the PN (regular package, image, etc).
This patch adds this data to the dependency tree dump.
(Bitbake rev: 7636aba37320aaf9b044d3832ddc21af51ccd69c)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some of the data values may come of as None through the event system,
and the UI would encounter a problem saving the Configuration.
It would be trying to save these values as NULL in the
database, which is not allowed.
This patch adds more verification for data coming through
the event system.
Other minor updates:
* update for the event model from toaster.bbclass
* minor code flow fix in the event system
(Bitbake rev: 03fafd086381723c6486522873671515824e49f2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This dialog is not needed. Instead "Edit packages" button should
redirect to Packages list.
[YOCTO #5257]
(Bitbake rev: aa2e86a2ac28414f6d25fcd541a6ed2b50f15a2c)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Trying to use a server with username and password authentication
within the URL of the SRC_URI variable doesn't appear to work.
This patch adds the missing parts to the hg fetcher to make this
work properly.
(Bitbake rev: dc3d6d73e44802c203b3f7247f6f212acc2f69bf)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When Hob receives a bb.command.CommandFailed event, it doesn't show any error,
because the error_msg is empty. If so, it is hard to detect the issue, because
Hob continues to run till it blocks because of an information gap.
[YOCTO #5097]
(Bitbake rev: a5abd1826f34e6a7eefa837620b846e9b62ae758)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have faced a corner case situation where the 'last changed
revision' returned from svn info is wrong. It happens when the last
revision is a directory move. e.g. if we assume that the svn
repository at revA has root/x/y/z/foo/bar and it is moved to
root/a/b/c/foo/bar in revB, then svn info 'last change revision' will
return revA. As such when using AUTOREV, we are going to attempt to
retrieve root/a/b/c/foo/bar (as per SRC_URI) but at revA when it did
not exist.
So this patch changes how we retrieve the latest revision and uses
'svn log --limit 1' which gives correct result in all tested cases.
(Bitbake rev: 17d8ef0b813a05c231e3dbe6e8bc82a4a9b1d2f8)
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Filesystems like btrfs and reiserfs sets the inode count to 0, since
they don't have an inode concept. This is expected, and having a warning
show up every time you run bitbake can cause undue concern.
(Bitbake rev: f3ac2d3678f48c68a250a0a20c08cf8687322d38)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding a new bitbake UI interface named 'toasterui'.
'toasterui' listens for events and data coming from a
bitbake server during a run, and records it
in a data store using the Toaster object model.
Adds a helper class named BuildInfoHelper that
reconstructs the state of the bitbake server and
saves relevant data to the data store.
Code portions contributed by Calin Dragomir <calindragomir@gmail.com>.
(Bitbake rev: 62200ff6694b21fbd5abf009a6f47ad93adf5309)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit adds the 3rd party frameworks used for the web UI.
jQuery is licensed under MIT.
Bootstrap is licensed under APACHE-2.0
(Bitbake rev: 8395e257fed030f0d9b24feba17ed99664f26b2b)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the Toaster component to Bitbake.
Toaster is a module designed to record the progress of a
Bitbake build, and data about the resultant artifacts.
It contains a web-based interface and a REST API allowing
post-facto inspection of the build process and artifacts.
Features present in this build:
* toaster start script
* relational data model
* Django boilerplate code
* the REST API
* the Simple UI web interface
This patch has all the development history squashed together.
Code portions contributed by Calin Dragomir <calindragomir@gmail.com>.
(Bitbake rev: d24334a5e83d09b3ab227af485971bb768bf5412)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A BBHandledException means we already showed an error to the user so
we shouldn't show a stack trace as this just confuses things further.
(Bitbake rev: 8a8bafc8ded98364a31878b23c64503a53affcd1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
These functions in the main bb module have long been deprecated
and moved to other modules. Finally remove the compatibility links.
(Bitbake rev: ccd181c3ed4852e2b9169cf19aaf18aeacddcc18)
(Bitbake rev: d7f817518c5df2524a4bcf008ba63c71a8eb48bb)
(Bitbake rev: 89b31a4fb44f2c2ad0bb4210151652cd3730418d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Avoid storing paths to files in SRC_URI when writing out the the
file checksums to siginfo files. This prevents a move of the source
directory being reported by bitbake-diffsigs as files being removed and
then added (the signature itself is not affected since the file paths
have never been included in the signature).
This has required the format of the file checksums in the siginfo file
to be changed from a dict to a list of tuples (in order to handle
multiple files with the same name under different paths, which is
uncommon but possible); the code remains backwards-compatible with older
siginfo files that use a dict however.
Fixes [YOCTO #5245].
(Bitbake rev: e4d3077c5b0cc57964640512f3646c2d73c1d855)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if errors occur when starting the PR service, there is a race that
occurs since the UI runs various commands including starting builds before
processing the CookerExit(). By adding the error state and refusing to run
async commands in this mode, builds are prevented from starting and the
UI reaches the exit code with the system shutting down cleanly.
(Bitbake rev: 42fa34142ea685f91115a551e74416ca28ef1c91)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This change is needed to inform the user that selecting
live type means that the system will build a hddimg and iso
image.
(Bitbake rev: 5051f59976de4e099bb434aeea414de5a67a069f)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The patch that implements removeConfigurationVar method was made before
merging the patch that replaces "added by bitbake" with "added by hob".
This patch corrects this issue.
[YOCTO #5284]
(Bitbake rev: 42601a5edef0316767b952b162123534aa8fee18)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This utility doesn't take any special arguments, but it's nice if it at
least knows how to deal with no arguments, --help and errors properly.
(Bitbake rev: 0cabdf1d0cde6687bc1372675a0d6242587c87a0)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Set up a logger independent of BitBake so we can log errors ourselves
* Handle common errors without printing a traceback
(Bitbake rev: 77b5f5b8dca4deebb06eeb06a8e7f2ccdbfff46f)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Use OptionParser to parse the two options to -t rather than trying to
pick them out ourselves.
* Add a description shown with --help output
(Bitbake rev: daab42d19463b4108968fc88b207936e5ac84154)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adjust the task name automatically if the -t option is specified with
a task name that doesn't start with do_ (e.g. "configure" instead of
"do_configure").
(Bitbake rev: d182cbc63745303ef2dc9fa2cbbf5d87a68e0b52)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Getting an error message about --remote-server being set when really
BBSERVER was is confusing, clarify the message.
(Bitbake rev: d7b5938a30a9b0ed83f899a06a88786e8392f8bd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When starting a server we don't want to show the debug event queue which
is reserved for when when errors have occurred. This patch copies the UI
code to ensure the user doesn't see confusing output.
(Bitbake rev: a886cda58415085981646fb9a024fa7641f55865)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For BBHandledExceptions, we've already displaced a sensible error to
the user so we don't need to do it again. Just exit with an error
value.
(Bitbake rev: 1ff5ec26eba70ab1c85674a60b7dac77317bf349)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Showing a traceback when unable to connect to a bitbake server is
rather ugly. This change allows us to show a sensible error message.
(Bitbake rev: 26913202f83fbbecdce95da59515af102bcde4a7)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If for example you try "bitbake -m" with an invalid BBSERVER, error
messages are not displayed. This change ensures logging is in place
to catch and display such errors.
(Bitbake rev: 719808f95adc7820fcc09743c592513414d03ce1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In Hob settings, there is a tab to add/remove extra settings. This
patch implements a way to "remove" variables from conf files, through
bitbake. But, to keep the history assigment of the variables synchronized,
instead of removing, it replaces the lines with blank lines.
[YOCTO #5284]
(Bitbake rev: bd720fb63cef6b399619b8fbcaeb8d7710f2d6df)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
MACHINE var is saved using early assignment operator.
Calling MACHINE=x bitbake core-image-... works properly.
Comment "#added by bitbake" is replaced with "#added by hob".
[YOCTO #5070]
(Bitbake rev: 2d0ec8ff083b636a6cf98de3278900eb95c3def6)
Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
print disappears into the ether, so use logger.warn and clean up the messages.
(Bitbake rev: 90f91f7402ff69f3fe9fba5f94a53d371303ce34)
Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Display a busy cursor while the configuration is saved.
[YOCTO #4846]
(Bitbake rev: fdb2e0120f189e97b8fdb6eb72d77a3100ba3018)
Signed-off-by: Valentin Popa <valentin.popa@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The parser never has supported it, the datastore API did happen to work
but whitespace in key names is a really bad idea and not something I think we
should encourage or support.
Fix test case failures after excplitly ignoring it for variable expansion
purposes.
(Bitbake rev: a2074ddaba6f53962d6caf34dbd27bdbc259935b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code is happily trying to expand variable names containing newlines,
spaces and tabs which are illegal characters in variable names. This
patch stops it doing this. This will change dependency checksums
since some rather weird dependencies were being attempted to be expanded.
(Bitbake rev: 37e13b852b33d98fa40f49dc1e815b3bbe912ff0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If my build starts from a YP standard image recipe, the 'Save image
recipe' dialog should be empty.
If my build starts from a custom image recipe, the 'Save image recipe'
dialog should populate the 'Name' and 'Description' fields with the values
set for the custom image recipe.
[YOCTO #5004]
(Bitbake rev: 10757c529fe0b4b9a39740d269831347a3aab4a0)
Signed-off-by: Cristiana Voicu <cristiana.voicu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The --help text was rather inconsistent in style and plain incorrect in places,
using confusing terminology in others. I guess most people know what the options
do and don't read this but its confusing to new users.
This updates it to use the terms recipe and task consistently, remove
the references to stage, bbread and generally try and make the output
more useful.
[YOCTO #4856]
(Bitbake rev: 516311946c7bd14c84947dc44c3bb0563e5a9667)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
On Linux its not possible for processes to regain a previous nice level after
it has changed. Its therefore not possible to have a core low priority and
then raise the priorities of individual tasks.
This variable allows us to do something like:
BB_TASK_NICE_LEVEL = "5"
BB_TASK_NICE_LEVEL_task-testimage = "0"
to give priority to specific tasks which the BB_NICE_LEVEL functionality
doesn't give us the option of.
(Bitbake rev: 94d82997220c6cfc7028f76719df028ba8254a5c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding the sstate-related hash for all runqueue and
scenequeue tasks, as it's needed in the WebHob data.
(Bitbake rev: b6e2ce1cf7a0ede890f08fabf536a556dc4263c5)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding the generic bb.event.MetadataEvent that is
targeted specifically at metadata usage. This is
needed in order to let the metadata code send and receive
events during asynchrous execution without having
to define each event specifically in Bitbake.
Metadata code should subscribe to and fire the MetadataEvent
in order to communicate asynchronously, and identify
the object using event.type field, and parse the
data in the event.data field.
Knotty UI will ignore these event by default.
This deprecates RequestPackageInfo/PackageInfo, and that
event pair will be removed in the future.
(Bitbake rev: ae1ea51aaab73e010d1c3db39df058bebebc11dd)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding the 'getAllKeysWithFlags' read-only command that will
return a dump of the global data state, together with specified
flags for each key. The flag list is passed in as the first
parameter to the command.
This will be used by UI clients to get the build configuration.
(Bitbake rev: 3e094da513e1220319288806cb76ddf804772afd)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The loaded cache modules may add extra attributes to
the recipecache, that will be populated by the cache
classes required by the UI. These attributes
will be used by the UI to display relevant information.
Adds cachefields cache class field to specify
for each cache class which attributes will be set
in the recipecache.
Adds code to automatically expand depends tree with the
fields exported by the extra cache class.
Fixes a cache field name in the HOB UI.
(Bitbake rev: 47c171005fb3803d936e65fcd4436c643883ae16)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds task identifying information for all
runQueue and sceneQueue events, and for bb.build.Task* events.
This will allow matching event to specific tasks in the UI
handlers processing these events.
Adds RunQueueData functions to get the task name and task
file for usage with the runQueue* events.
Adds taskfile and taskname properties to bb.build.TaskBase.
Adds taskfile and taskname properties to the *runQueue* events
(Bitbake rev: b4a5e4be50d871a80dbe0993117d73f5ad82e38f)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adding a CookerFeature that allows UIs to enable
receving a dependency tree once the task data has been
computed and the runQueue is ready to start.
This will allow the clients to display dependency
data in an efficient manner, and not recompute the runqueue
specifically to get the dependency data.
(Bitbake rev: 75466a53b6eece5173a9bfe483414148e4c06517)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Implementing feature set selection that allows a client
to enable specific features in the server at connection time.
Only enabling of features is supported, as there is
no way to safely remove data loaded into the cooker.
Once enabled, a feature will remain enabled for the
life of the cooker.
Client-server connection now supports specifying the feature
set required by the client. This is implemented in the Process
server using a managed proxy list, so the server cooker
will now load dynamically needed features based on what client
connects to it.
In the XMLRPC server the feature set is requested by
using a parameter for registerUIHandler function.
This allows observer-only clients to also specify features
for the server.
The server code configuration now is completly separated
from the client code. All hardcoding of client knowledge is
removed from the server.
The extra_caches is removed as the client can now specify
the caches it needs using the feature. The UI modules
now need to specify the desired featureSet. HOB is modified
to conform to the featureSet specification.
The only feature available is CookerFeatures.HOB_EXTRA_CACHES
which forces loading the bb.cache_extra:HobRecipeInfo class.
(Bitbake rev: 98e594837aab89ea042cfa9f3740d20a661b14e2)
Signed-off-by: Alexandru DAMIAN <alexandru.damian@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The change to use the expansion cache in VariableParse was incorrect as
it was adding in references it shouldn't have been. This patch corrects
the codepaths and ensures the references are correct.
The cache version is bumped since the previous bug could have leave
to invalid checksum calculations and a clean cache is therefore desireable.
The impact of the bug was that sstate was not getting reused when it should
and some tasks were also being rerun when they should not have been.
(Bitbake rev: 8a42d082315bd6ce091d006bf83476db257fa48b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The syntax used for checkPackages implies a copy is returned but it
did not do so. Make it so.
This is fixes universe builds where error messaages were being shown
but there should have only been warnings.
[YOCTO #5222]
(Bitbake rev: 97db2a0792d605f27d434bc6a4acce52857deee1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
1. Added a new section "Appending and Prepending (override style
syntax)". This section shows how the append and prepend operators
work using the override style syntax.
2. Added a new section "Removing (override style syntax)". This
section describes the new "_remove" operator.
(Bitbake rev: 6983afab0ce8d82d102142636d5a570f7d86a844)
Signed-off-by: Scott Rifenbark <scott.m.rifenbark@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Sanity test failures are no longer fatal with this change so whilst
its the right idea, the code paths need more work.
This reverts commit a50017ba71250e1710a6425b60ac7e3f03d88295.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With new bitbake UIs having the cooker exit at 'random' points
in the codebase is problematic. This patch raises an exception
which matches the siutation instead.
(Bitbake rev: a50017ba71250e1710a6425b60ac7e3f03d88295)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Move the variable lookup to the outer loop for performance, replacing
a now unneeded parameter (after the previous changes).
(Bitbake rev: 8a59a8707cecbde257fca169775ce8ff7709928b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When looking up flag variable dependencies, large chunks of the function
aren't needed. Optimise the function flow accordingly for speed.
(Bitbake rev: 1bf3aee698ad35f6815ea2c75471a96511a29d55)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Instead of multiple calls to getVarFlag, make one call to getVarFlags, only expanding
the flags that need to be expanded. This improves performance.
(Bitbake rev: eba1e9545cc933820d40de96f023b2307b3c4d0b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a variable references another but it isn't set at present, the
reference wasn't stored. It really should be marked as a reference
and the higher level dependency code can handle as appropriate.
(Bitbake rev: b05b748b2153c941b95cd36fb22aaafc4dbf3791)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Allow a list of flags to expand to be passed into getVarFlags. This
is useful within bitbake itself to optimise performance of the
dependency generation code.
(Bitbake rev: a3ae7efdf750fc5bb9ff5a75defbcfdab1912dbe)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Simple if xxx checks end up calling len(xxx). We're interested in the specific case
of None which means we can break out the iterator much earlier after the first
item. This adds in the specific tests for None in what is a hot path in the
data store code which gives small performance gains.
(Bitbake rev: a4d81e44a7cd3dafb0bf12f7cac5ff511db18e60)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Compute a cache of the list of potential export variables so
that we don't have to compute the list from scratch.
(Bitbake rev: f41f46f7eaa6889edeb3a4e4ddedc07084686c60)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Profiling shows the creation of keys() has overhead and we're better using
an iterator rather than the memory associated with the huge list of keys
when iterating the whoe datastore. We minimise the number of times
we do this to twice only per recipe.
(Bitbake rev: e63448d9ee331b0f45fb9a0197d0dbee49eb2fa0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie
Peter Kjellerstedt
Florin Sarbu
Paul Eggleton
Valentin Popa
Saul Wold
Alexandru DAMIAN
Peter Seebach
Konrad Scherer
Cristiana Voicu
Trevor Woerner
Gary Thomas
Volker Vogelhuber
Nicolas Dechesne
Olof Johansson
Ross Burton
Erik Botö
Scott Rifenbark