Currently bitbake only adds files to its dependency list if they exist.
If you add 'include foo.inc' to your recipe and the file doesn't exist,
then later you add the file, the cache will not be invalidated.
This leads to another bug which is that if files don't exist and then
you add them and they should be found first due to BBPATH, again the
cache won't invalidate.
This patch adds in tracking of files we check for the existence of so
that if they are added later, the cache correctly invalidates. This
necessitated a new version of bb.utils.which which returns a list of
files tested for.
The patch also adds in checks for duplicate file includes and for now
prints a warning about this. That will likely become a fatal error at
some point since its never usually desired to include a file twice.
The same issue is also fixed for class inheritance. Now when a class
is added which would be found in the usual search path, it will cause
the cache to be invalidated.
Unfortunately this is old code in bitbake and the patch isn't the
neatest since we have to work within that framework.
[YOCTO #5611]
[YOCTO #4425]
(Bitbake rev: 22e6b1c4c4afb27057689bbc94cbdf1f19f93e3d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* resolve_file was behaving different when relative and absolute
paths were passed to it
* include relative-path/non-existent-file.inc
works correctly resolve_file throws IOError, BBHandler.py:handle()
doesn't catch it, ConfHandler.py:include() catches IOError and shows:
DEBUG: CONF file 'relative-path/non-existent-file.inc' not found
* include /absolute-path/non-existent-file.inc
was failing, because resolve_file just returns fn,
BBHandler.py:handle() calls bb.parse.mark_dependency(d, abs_fn)
which throws:
OSError: [Errno 2] No such file or directory: '/absolute-path/non-existent-file.inc'
and parsing fails.
Ad isfile() test for absolute fn and throw IOError to make
resolve_file behavior consistent for both paths.
* I know we had some issues with -b relative-path-to-recipe.bb and
absolute path, so consider this patch only as RFC and documentation of
this problem
* Catch OSError too in ConfHandler.py:include() e.g. in case the file exists, but user
cannot read it or something like that.
(Bitbake rev: b0bbd89a4f0b98fa1ab28b8e0526cd9ddb76fa57)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Python 3 is stricter about how files are accessed. Specficially:
* Use open(), not file()
* Use binary mode for binary files (when checksumming)
* Use with statements to ensure files get closed
* Add missing file close statements
(Bitbake rev: 9f08b901375ba640f47596f1bcf43f98a931550f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The __config_regexp__ in ConfHandler is quite long, and using re.X to
break the expression onto several lines make it a bit easier to read.
(Bitbake rev: 54dce9e14ab0657d76f0d0ae22eef7fab8e8950d)
Signed-off-by: Olof Johansson <olof.johansson@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have:
FOO = "a"
FOO += "b"
FOO+= "c"
The expected result is "a b c" however we were seeing "a b" with the FOO+
variable being assigned the value "c". This isn't the expected result.
We need to make the name part of the variale non-greedy so that any + character
becomes part of the operator. This patch does that. I compared the configuration
in OE-Core before and after the change and only the test case changed.
[YOCTO #3834]
(Bitbake rev: 2cd8d7fd12a646e6516e2c985e6a54121d19eb59)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- enable the '~' character in bitbake variables
(Bitbake rev: 7c15ff1d50d7b601414f1d55c90e3c59981a0876)
Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This was meant to be squashed into the previous commit for multiline comment
handling. It fixes the case the commented multiline is followed by an empty
line which was resulting in a traceback instead of a sane error message.
(Bitbake rev: 7e7d692e244fe8dca533f842ca143b9c821e317c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Faced with an expression like:
# Some comment \
FOO = "bar"
what should bitbake do? Technically, the \ character means its multiline and
currently the code treats this as a continuation of the comment. This can
surprise some people and is not intuitive.
This patch makes bitbake simply error and asks the user to be clearer
about what they mean.
(Bitbake rev: 589d31ce41e019ee6a7cb6527d67bc76c0b6382a)
(Bitbake rev: 79c00fabe08b4c210a3bd81cfaffbc47ffdc2e2b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Syntax like:
FOO = "bar" # eek"
would result in FOO taking the value 'bar" #eek' which is clearly
not the intention. Whilst our metadata is riddled with mixtures of even
quotes like:
FOO = "d.getVar("X")"
odd numbers of quotes seem rare. This patch adds detection of one odd
quote which we don't have any of in OE-Core so it seems a valid sanity
improvement.
(Bitbake rev: 5f892d9b083550e20e37576070ec7d1a94cc88fe)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
To make the UI settings take effect, we need to hook at the end of each
config file parsing and set UI specific values.
(Bitbake rev: f54e733c7863110896f43900d9e4e791602f9d65)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The export regexp was only meant to catch values like:
export VARIABLENAME
however after the stricter quoting patch was applied, it was also matching
variables like:
export BAR=foo
and setting the export flag on a variable called "BAR=foo". The = character
is an invalid variable name character. This patch tightens up the regexp
match so it only matches the intended character set and only matches variable
names.
(Bitbake rev: 6d1765c2eac8c1958ceb9c81d55d04a9bc961cb1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently, bitbake will accept variables in the forms:
X = 1
X = '1 \
X = "1"
X = '1'
which will all set X=1. This patch removes the first two possibilities
and makes quoting mandatory. There is little metadata out there which
doesn't quote properly and bitbake will exit with an error about the
exact line number and file with any problem so users can easily identify
and fix issues. OE-Core has already been checked/fixed.
The motivation for this is being able to give sane errors if a user
does something like:
IMAGE_INSTALL += # tslib mtd-utils"
which currently gives a really nasty failure.
(Bitbake rev: a8ae80741fea5e0ec0fb9a52a963a4baa38d2564)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ensure that a file and line number are reported for ParseError where
possible. This helps particularly in the case of inherit and require
which previously did not report either of these upon failure.
(Bitbake rev: f588ba69622a2df35417ced184e56c79ac1b40d5)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Its possible for relative paths to creep into FILE. These confuse the
build system no end as its not clear where they might be releative to.
This patch ensures we always use resolved absolute paths for FILE
so that things behave in a deterministic way.
(Bitbake rev: 658d7daa70e46c2b20973b90ee53f0bbadc8bf5d)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When parsing multiline variables in conf files, the last character can
be accidentally removed. s2 contains new data read from the file which
may or may not end with the continuation character. It makes sense to
let the next loop iteration strip this if needed.
We don't often use multiline expressions in .conf files which is why I'd
imagine we haven't noticed this before. Most variables are quoted and
its the closing quotation which often disappears.
(Bitbake rev: 09a9146262d58dfe4a2ea4270026b90ae33f6c91)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We will be needing this information to improve the tracebacks of python code
from the metadata, as well as to give the user information about where
variables were defined, so they know how it ended up the way it is.
(Bitbake rev: 9615c538b894f71a2d1a0ba6b3f260db91e75786)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We use a custom Logger subclass for our loggers
This logger provides:
- 'debug' method which accepts a debug level
- 'plain' method which bypasses log formatting
- 'verbose' method which is more detail than info, but less than debug
(Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
This way we can fully utilize bblayers, you can do everything in bblayers.conf
and avoid setting any environment variables at all.
(Bitbake rev: 5def1c8c31432968349f9b29d6333d7962260a8b)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
??= is a lazy, conditional assignment. Whereas a ?= immediately assigns to
the variable if the variable has not yet been set, ??= does not apply the
default assignment until the end of the parse. As a result, the final ??= for
a given variable is used, as opposed to the first as in ?=.
Note that the initial implementation relies upon finalise() to apply the
defaults, so a "bitbake -e" without specifying a recipe will not show the
defaults as set by ??=. Moving application of the default into getVar adds
too large a performance hit. We may want to revisit this later.
(Bitbake rev: 74f50fbca194c9c72bd2a540f4b9de458cb08e2d)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Better to error as early as possible rather than experience strange behavior
resulting from the use of the largely useless stock bitbake.conf/base.bbclass.
(Bitbake rev: 641e6cf3ec3ab4d26929cf4d2a3704ff07eed4d6)
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Do not attempt to open the file in the resolve_file method
(a lot like bb.which... maybe bb.which can be used). This way
we don't need to open/close a file which we have already parsed.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Evaluate the statements after having parsed one file. This is
referred to as "entwirren" and we can remove the direct evaluation
and postpone a bit, in the future we can use a cached copy instead
of parsing the original.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
When parsing we will collect a number of statements
that can be evaluated...The plan is to be evaluate
things twice (old+new) and then compare the result,
it should be the same.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
With obtain it was possible to use an existing fetcher to
download a bb or config file. In practive no one has used it
and it was likely broken in regard to depends_cache... Remove
it for now, simplfiy the code.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* Make the test functionality work
* Optimise BBPATH handling when changing directory
* Optimise file globing for BBFILES
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
* File licence headers were sanitised causing most of the diff.
* cooker.py was created from bin/bitbake.
* cvs fetcher port option was added
* The -f force option was fixed to work correctly
* Multiple entries in rrecrdeps are now handled correctly
(allows adding do_deploy to image depends)
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1129 311d38ba-8fff-0310-9ca6-ca027cbcb966
Richard Purdie
Martin Jansa
Olof Johansson
Constantin Musca
Paul Eggleton
Chris Larson
Bernhard Reutner-Fischer
Holger Freyther
Richard Purdie
Richard Purdie