[YOCTO #1126]
A local PR service will be started and stopped automatically along
with the bitbake invocation/ternimation.
This local PR service will be started only and if only when the
PRSERV_HOST is set to 'localhost' and PRSERV_PORT is set to '0'.
When started, the sqlite3 database is stored at
"${PERSISTEN_DIR}/prserv.sqlite3" or "${CACHE}/prserv.sqlite3".
(Bitbake rev: 9d8f45407c67ed0d3c4f820cf646de3c385067c7)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have a problem knowing when to show the user debug information and
when not to since the code has already shown the user suitable information
about why a failure is occurring.
This patch adds a bb.BBHandledException exception class which can be used
to identify those exceptions which don't need further explanation to
the user.
This patch uses this class for the bb.providers exceptions and ensures the
command handling code correctly filters the exceptions meaning that
"bitbake invalid"
now shows an simple error message and not a python traceback.
[YOCTO #1141 partial]
(Bitbake rev: eac9249b40ae1e3aa21e016010c862664e59a8d4)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Show a summary count of warning/errors messages shown to the user during the
build and make it clear when an error exit code is being set.
[YOCTO #1540]
(Bitbake rev: 9943bad611a974e4d37a00c7a4de1752250370c5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The parseConfigurationFiles API calls the related function in
cooker.py to parse config files.
(Bitbake rev: 96c307b9874131ad8c7d9caea6f6dfbd09aab9d4)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no direct use of command in cooker.py, and it is using
bb.command instead. Remove command in the import list.
This fixes a problem of embedded import between command.py and
cooker.py.
(Bitbake rev: c353316b2efcc7a893d6b4aa9a9647d51a6f69e3)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is no direct use of command in cooker.py, and it is using
bb.command instead. Remove command in the import list.
This fixes a problem of embedded import between command.py and
cooker.py.
(Bitbake rev: 92fec2788e33d301cc63848901bc6adc764a2ecf)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pass host and port to BitbakeServerInfo class instead of the "server"
instance. With this change, GUI can connect with server individually
by host address and port.
(Bitbake rev: 5124351d6c287185723c98e6e4400c5402a5c8a8)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
initCooker is to set the cooker to the initial state with nothing
parsed.
(Bitbake rev: 8ee9c6cc47938604268242d90bd3007218f9a36e)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For buildTargets function, we use bb.configuration.data as parameter
to inject events, since in hob environment, some variables are
modified dynamically and bb.configuration.event_data may out of date.
(Bitbake rev: 487a124942fd5cd9de71413b79a4049a6d26f1e8)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
for findConfigFiels() and findFilesMatchingInDir() functions, they
don't need to parse all the bb files, thus setting the needcache
value to be False.
(Bitbake rev: 3ef73dee8b08ccfd15a4901cce315a99b22e71d7)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When debugging setscene problems it was found that some extra debug messages were useful.
This patch adds them.
(Bitbake rev: cff2c258b77fde01d530a5923e553e6111b15eb5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Local cloning of git repositories from DL_DIR into WORKDIR fails when
using ssh URL with port specification e.g.
"ssh://user@host:port/path/to/repo.git". Git fetcher clones such remote
repository into "${DL_DIR}/git2/host:port.path.to.repo.git". However,
when clonging from ${DL_DIR}/git2/host:port.path.to.repo.git into
${WORKDIR}, git fetcher fails with "ssh: Could not resolve hostname
${DLDIR}/git2/host: Name or service not known".
A solution is to replace ":" by "." in host component, similarly as it
is done when replacing "/" with "." in path component, so that local
clone directory in DL_DIR looks like this: "host.port.path.to.repo.git".
(Bitbake rev: 1f2867b79f1cd2bfbdc849ba5677a39db6fa3396)
Signed-off-by: Juraj Hercek <juraj.hercek@jhksoftware.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As reported on the mailing list[1] when changing IMAGE_FSTYPES through the
hob preferences a reparse is required before the changes will be picked up
by the system. This patch sets the reload_required property of the class to
true when the image types have been modified to ensure the reparse is
triggered.
1. https://lists.yoctoproject.org/pipermail/yocto/2011-December/006002.html
(Bitbake rev: 6c0babf08909307ab69a66ed06e77e8818b2a8c5)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The knotty UI just ignores these and so should RunningBuild, if these events
aren't handled the UI appears to hang.
Fixes [YOCTO #1665]
(Bitbake rev: 540ba78075bd525776aa23bf38bee66350c66534)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some BB_HASHBASE_WHITELIST variables are in the lists of variable
dependencies for signatures. Ignore those differences in lists
since this difference does not matter
(Bitbake rev: 71b53a3f0766ca464560a1f6a449f9424fbdf7ae)
Signed-off-by: Matthew McClintock <msm@freescale.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In bitbake/lib/bb/command.py::Command::__init__, we have the following
lines:
for attr in CommandsSync.__dict__:
command = attr[:].lower()
method = getattr(CommandsSync, attr)
sync_cmds[command] = (method)
for attr in CommandsAsync.__dict__:
command = attr[:].lower()
method = getattr(CommandsAsync, attr)
async_cmds[command] = (method)
The sync_cmds and async_cmds are defined as global dictionaries, but it
seems that we've never used them (I did a "grep -r async_cmds bitbake/",
, there is no result except the ones that I have removed), and I can't
find the history of it from "git log -p", I guess that they have been
replaced by the self.cmds_sync and self.cmds_async.
[YOCTO #1791]
(Bitbake rev: 24e99460800856035bb54a84c7aa33b3517436e9)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
BB_STRICT_CHECKSUM caused a backtrace as the FetchError parameters are
incorrectly specified such that FetchError is being passed 8 params
when it's expecting 3. This fixes the parameters so we're passing a
formatted string and the url.
(Bitbake rev: b8b2f4287c9125542b18a294c0a94ed89a7e73a8)
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The assignments of self.configuration.cmd in BBCooker seems duplicated,
have the followings in both BBCooker::__init__ and
BBCooker::loadConfigurationData:
if not self.configuration.cmd:
self.configuration.cmd = self.configuration.data.getVar("BB_DEFAULT_TASK", True) or "build"
The __init__ invokes the loadConfigurationData, and it would make sure
that self.configuration.cmd has been assigned a proper value, so we can
remove the one in __init__.
[YOCTO #1791]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using bitbake -e -b virtual:xxxx:/path/to/the.bb would result in
zero matches since the virtual:xxxx piece wasn't being processed.
This adds in the necessary functionality to handle it correctly.
[YOCTO #1793]
(Bitbake rev: bd5a727c8447bcb747c1d2463b7de2ab6d21a7de)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There is a subtle but nasty problem that a function's cwd can vary
depending on whether ${B} (often ${S}) exists before the funciton is
called or not. Most functions in the system can cope with this but
its bad practise and I've just witnessed build failures resulting
from this during image generation from bootimg.bbclass. I also
suspect this could explain some odd fetcher behaviour witnessed in
the past.
This change ensures we always call funcitons with a specific build
directory making things deterministic.
(Bitbake rev: ef0888f83fa4408eb768257d7e03700202faad18)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This allows the code to safely fall back to dumping the basehash data
if the taskhash data isn't present for some reason. We could effecitvely
obsolete the runtime option and use this approach instead exclusively.
(Bitbake rev: 5ace320ccc01f4e326f90b7ba060dcbff3380dca)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
USER is the correct variable to use, also this can affect sstate
cache as well.
(Bitbake rev: d7f9edda65dae2e046871afa275c5a51dff48fc4)
Signed-off-by: Matthew McClintock <msm@freescale.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We have a problem if we want to inject specific information into the variable
dependency code. There are cases for example where we want a dependency
on the value of X but it doesn't matter how X was constructed or what
dependencies it might have had, we only care about the absolute value.
With the current code, its near enough impossible to do this.
This patch adds such a mechanism so the user can trigger this with code like:
baselib[vardepvalue] = "${baselib}"
It also refactors some of the code so we do variable lookups once
instead of doing this in two different functions.
[YOCTO #1583]
(Bitbake rev: 6c879b44ccf42dc73fe4467076e114700d7ba81b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently if you specify a mirror like:
file://.* http://linux.freescale.net/yocto/sstate-cache
it won't work as you expect whilst:
file://.* http://linux.freescale.net/yocto/sstate-cache/
will since it has the trailing slash.
This patch handles both cases correctly. It also adds some debug to
the uri_replace function since its near impossible to debug it without
some kind of output.
[YOCTO #1578]
(Bitbake rev: a0246bf09c93bb657eaf6ba61d090b247ed33640)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The BB_CONSOLELOG variable is used to specify the console log file
to use. This means people can look up things that happened during a
build by may have scrolled off the screen.
[YOCTO #1771]
(Bitbake rev: a34ff490a46577a64345365fa1ac39c48d0861cd)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Defaulting to any file in DL_DIR as the first match for a file:// url
doesn't make much sense and can lead to unexpected results.
This patch changes the logic so this is the last fallback location
instead. Whether it should be using DL_DIR at all for this is a
good question but something for another patch.
[YOCTO #1710]
(Bitbake rev: 5597a68fac0954c682b67471722c2643e2415f99)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without this patch the tash hash dependencies can be in a order
that is dependent upon directory/filesystem layout. With this
change the data is sorted the same regardless.
Without this the dependent hashes could be in different orders
on different systems and consequently final md5 hash would differ
as well even though nothing else changed.
(Bitbake rev: 9a2029899c946ce9aa8adbc85f2cfe7a85b92182)
Signed-off-by: Matthew McClintock <msm@freescale.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is useful for really longs lists to pinpoint what has
actually changed
(Bitbake rev: f1eb6d3dcc10c42bb09383a87bde3afa69bc6ed9)
Signed-off-by: Matthew McClintock <msm@freescale.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The existing looping code can mask an existing "found = True"
by forcing it to False each time. This can lead to dependency
lists not being fully searched and results in dependency errors.
An exmaple of this was the autobuilder building linux-yocto from
sstate but then rebuilding some of the recipe's tasks for no
apparent reason. Separating the logic into two variables solves this
problem since any "found = True" value is now always preserved.
(Bitbake rev: 61017fc5d30b7a13308d038872ec92efc1a84cef)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If A depends upon B which depends upon C and the setscene for B
succeeds but C is going to get rebuilt, we should wait for C to
try and build A but currently we don't.
This is due to the timing of when we run the task_skip() as this
triggers other tasks to become buildable. This patch moves the timing
of that call to a more appropriate place allowing dependencies to
behave as expected.
(Bitbake rev: b7114d8e5d9b0720339bd5d24d243c0f2a7c1f3b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The OE metadata has cases where it needs to prevent some setscene
functions from running. An example of this is where we know a task
is going to run do_configure (which would clean out do_populate_sysroot)
and hence we don't want do_populate_sysroot_setscene to run.
This change adds in a hook so that the metadata can allow any
such policy decision to filter back up to bitbake.
It removes the existing code which attempted to do this in a generic
way but failed.
(Bitbake rev: 2f6d987d9957a5d713ce119c24c2e87540611f53)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its possible for relative paths to creep into FILE. These confuse the
build system no end as its not clear where they might be releative to.
This patch ensures we always use resolved absolute paths for FILE
so that things behave in a deterministic way.
(Bitbake rev: 658d7daa70e46c2b20973b90ee53f0bbadc8bf5d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that the list of dependencies is included in the hash
as well as their contents
Prior to this, adding or removing dependencies with values
of "None" would not change the hash, despite diffsigs reporting
this difference.
(Bitbake rev: 727ca945177ce9bd44515cf611e3e95a09466d98)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When comparing sig files, if the recipe locations had changed, the
dependent tasks list would show as changed even if the actual hash
had not changed. This updates the code to only compare the base part
of the pathnames.
It also tweaks some of the output to add newlines to aid comparing
two lists of variables as it makes the location of the difference
clearer.
(Bitbake rev: 165a22ddcc647b945707fb5c483146bb336d5f66)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code which populates setscene_covered list was adding a task to the
covered list if all of the tasks that depend upon it were also covered;
however, this means that tasks that would have installed "runtime"
dependencies were being marked as covered also, e.g. gmp-native and
mpfr-native are needed by gcc-cross at runtime since they are shared
libraries that gcc links to, but their do_populate_sysroot tasks were
being marked as covered, resulting in failures later on if gcc-cross was
available from sstate but mpfr-native and gmp-native weren't.
Since we currently have no real way to handle runtime dependencies for
native packages, add a workaround which avoids marking tasks as covered
if one or more of their revdeps are from a different recipe.
Fixes [YOCTO #1536].
(Bitbake rev: e492eb4dc9016cd0bed194377c6f2b85cf0ad113)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds appendVar and prependVar functions to the data store
meaning python code would no longer have to do the getVar, append and
the setVar dance that much of the current python code does.
It also adds corresponding variants for flags.
Currently there is no spacing added by these functions. That could be
added as a parameter if desired.
If these functions turn out to be hotspots in the code, there are tricks
that could potentially be used to increase the speed of these specific
operations within the datastore.
(Bitbake rev: 4a4046268f84b85559eea2c4b6a6004ad8cccb77)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a FAKEROOTNOENV which does the opposite of the FAKEROOTENV variable
and is data loaded into the environment for tasks without the fakeroot
flag.
The intent here is to provide a way to control the environment when we
aren't needing fakeroot context which allows us to unload the preload
from memory entirely and gain a valuable speedup.
I'm not 100% happy with needing this at the bitbake level, particularly
with the cache hit but it does give a valuable speedup.
(Bitbake rev: 05c29ab0b2ad3c521414cabb6a92bca15c6e919c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The bitbake codebase makes assumptions that only one copy is active
against a given build directory at a given time. This patch adds a
lockfile in TOPDIR to ensure that is the case.
Note that no unlock is needed, that is automatically dropped when
execution terminates.
(Bitbake rev: 6004cbf36c980c2574c1c9153df81a7c55317e17)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The lockfile retry parameter is expected to return immediately after
attempting to take the lock. There was a bug in the logic which this
patch fixed to ensure it does that.
(Bitbake rev: f421ef819f00ac659504d9af41bcc8323422ff8c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the vardeps flag is not None, we now silence the warnings about
non-literal usage for that variable.
(Bitbake rev: e724b9f417d1baf898f5afc6376c73c1a2ad8db9)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are two usual cases involving bb.data.expand:
- Calling it with a string literal -- "bb.data.expand('${FOO}/${BAZ}/bleh', d)".
- Calling it on getVar results (legacy) -- "bb.data.expand(bb.data.getVar('FOO', d), d)"
Nothing in any of the usual layers uses it in any other way, and I'm
having trouble coming up with any real use cases beyond this. The first
of the above cases is already tracked, via the expandWithRefs called
on the python code string. The second didn't emit a warning anyway,
since the getVar was already handled.
Given this, I see no reason for us to maintain explicit expansion
tracking. Further, we weren't using its results anyway (the var_expands
member).
(Bitbake rev: 405dfe69e6a608826e599ebf2f83ef8cf5083b96)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- If a name is passed to the parser, prepend the messages with "while
parsing <name>:". This gives a bit more context.
- Tweak the warning messages slightly (they had to be altered anyway to
inject the variable being parsed).
Before:
DEBUG: Warning: in call to 'bb.data.getVar': argument ''%s' % var' is \
not a literal
After:
DEBUG: while parsing emit_pkgdata, in call of bb.data.getVar, argument \
''%s' % var' is not a string literal
(Bitbake rev: 1060193ae4d54e667735dbff5d1d2be49a3f95c9)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
With the previous method, using the compare_name methods, we split the
requested match name by '.', reversed it, then compared them piecemeal
during the node traversal. The new method walks the nodes and hands back
the name of what's being called, and then we check that. This also
consolidates the two different implementations of traversal of the
attribute/name nodes (one in compare_name, one for the execs).
(Bitbake rev: 84e535b5165c7e936c5b1486bdf4626ed3649f5f)
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The split is even less necessary now that we use ast.walk rather than an
actual NodeVisitor subclass.
(Bitbake rev: d6c44fac184abae8395bfa7078f06675218aa534)
Signed-off-by: Christopher Larson <kergoth@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Lianhao Lu
Richard Purdie
Dongxiao Xu
Juraj Hercek
Joshua Lock
Matthew McClintock
Robert Yang
Paul Eggleton
Christopher Larson
Christopher Larson