Commit Graph

1063 Commits

Author SHA1 Message Date
Paul Eggleton 3bf2d2103e bitbake/cooker: implement layer dependencies, make priority optional
Implement (optionally versioned) dependencies between layers, and if layer
priorities are not specified using BBFILE_PRIORITY_layername (now
optional) then work out the layer priority based on dependencies.

Define LAYERDEPENDS_layername in layer.conf to specify the dependencies
of a layer (list of layer names, split with spaces in the usual way);
LAYERVERSION_layername can be defined for each layer allowing specific
version dependencies to be specified via depname:version in the list of
dependencies. An error will be produced if any dependency is missing or
the version numbers do not match exactly (if specified).

Note: default priority if unspecified for a layer with no dependencies is
lowest defined priority + 1 (or 1 if no priorities are defined).

Addresses [YOCTO #790].

(Bitbake rev: 115b89fa279b64e79da0f72caf7b30965a83fab1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-05 13:36:51 +01:00
Paul Eggleton 44db5d5d76 bitbake-layers: add show_overlayed action
Add a show_overlayed action to list overlayed recipes.

(Bitbake rev: f0c2175dc943160e45ebd72fc932dd16ee361bfb)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-05 13:36:50 +01:00
Paul Eggleton 07dbea44c7 bitbake: track 'overlayed' recipes
Recipes that have been 'overlayed' (where there is a recipe in another
layer where that layer has a higher priority) are now listed within
cooker.overlayedlist for use in bitbake-layers. This is a dict with
keys of the topmost (highest priority) recipe file.

(Bitbake rev: 370fc603d79f9c34cc23b4b520b685256c23df5d)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-05 13:36:49 +01:00
Richard Purdie 81545301d7 bitbake: Switch to use process as the default server
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-05 11:40:23 +01:00
Richard Purdie 56e46f0bc4 bitbake/ast: Fix ??= vs. ?= handling
As the code stands, setting a variable with ??= could result in a ?=
variable not overriding it. This patch fixes the issue by allowing
the ast to make lookups that ignore any ??= set variables.

(Bitbake rev: 32fee2e650dfdd3aa9a7572dad1251e0c24ca34b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-05 11:39:05 +01:00
Richard Purdie ea518846b8 bitbake: Add missing bracket, somehow lost by sync scripts
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:44:12 +01:00
Joshua Lock f3be8e9a7d bitbake: add -R option for loading configuration files after bitbake.conf
Useful if you want to load a configuration file that sets values which may
also be set in bitbake.conf or one of the files it includes.

(Bitbake rev: a8246ae5400c23df0d3ee29c36f4d9f257d1e6d1)

Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:17:36 +01:00
Joshua Lock 22d8fb1fc3 cooker: switch to new universe target rather than world
When the caller doesn't specify a pkgs list we want to generate the tree of
all available packages. To do so use the new universe target list.

(Bitbake rev: 26b0c538ad4f677e0d45a66484c2dca073459282)

Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:17:36 +01:00
Joshua Lock 97e0beb393 cooker: remove code duplication in non trivial functions
The generateTargetsTreeData() and generateDepTreeData() functions perform
are essentially the same function only creating slightly different data
structures. Instead of duplicating non-trivial code drop
generateTargetsTreeData and instead have an optional boolean argument which
defaults to False for generateDepTreeData() which has it include the extra
fields required for the targets tree.

(Bitbake rev: 3e38ea4e5748473740821b6e10c8477c08ab45e2)

Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:17:36 +01:00
Joshua Lock 6c36f4a6aa cooker|command|event: add new command findFilesMatchingInDir
This command can be used to search each BBPATH for files in the passed
directory which have a filename matching the supplied pattern.

This is implemented for use from the GUI (to determine the available
PACKAGE_CLASSES) but has been written so as to be generically useful and
reusable.

(Bitbake rev: 2a599812a57cb0b964880a6a2b7548423497ea92)

Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:17:35 +01:00
Joshua Lock 9fe29fd7db command|cooker|event: add findConfigFilePath command
This takes the name of a .conf file and returns the full path to it

(Bitbake rev: 22c8600b885faf841795b872d82f68dfb644a26e)

Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:17:35 +01:00
Joshua Lock 6769269bea cooker: add generic method to locate configuration files
Convert _findLayerConf(self) to _findConfigFile(self, configfile) so that
the core functionality of the method can be used elsewhere.

(Bitbake rev: c515b76c3a27d57d5ae8dddf15cc836811b24ee1)

Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:17:35 +01:00
Joshua Lock a8afb3b353 command|cooker: allow generating targets tree for specified pkgs
Modify the generateTargetsTree command to allow a list of packages to be
supplied by the caller, in this case we will only generate a target tree
for user requested targets rather than building a tree for the world list.

(Bitbake rev: d4e4f2ecae96e074b2ab3bb9882037af2e385fdd)

Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:17:35 +01:00
Liping Ke cbd00b8927 bitbake: Make bitbake server type configurable.
Add -t options in bitbake for configuring server type.

(Bitbake rev: 5591329948648927154024bcb882c45766defac2)

Signed-off-by: Liping Ke <liping.ke@intel.com>
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:17:35 +01:00
Lianhao Lu 94a57fd87e ui/hob: Fixed the "build again" hang.
Using gobject.threads_init() instead of gtk.gdk.threads_init(). These
two modes are conflict to each other. Using gobject.threads_init()
allows only the main thread to touch GUI(gtk) part.

(Bitbake rev: b9698d0e8d681f1fd6ab8d28530136b85411386f)

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Joshua Lock <josh@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-07-01 17:17:35 +01:00
Richard Purdie f0c5451e59 Update version to 1.13.2
(Bitbake rev: 5d41720d1550c04aac76275614ca15110c1c7f52)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-28 15:28:16 +01:00
Richard Purdie a2b9ef9d78 bitbake/ast: Add optional argument for BBCLASSEXTEND
Add an optional argument to BBCLASSEXTEND entries which gets passed to
the extention class as BBEXTENDVARIANT. Also add BBEXTENDCURR whic
is set to the current extension class name.

This mode functions slightly differently to the previous BBCLASSEXTEND
code in that PN is not changed.

(Bitbake rev: 8d3c899e0a15840c54de26d2f1fc552430517778)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-28 15:27:43 +01:00
Richard Purdie 21c5985812 bitbake/ast: Call expandkeys after the RecipePreFinalise event
This means the event handler can change variables such as PN and those
changes will be reflected in the updated variable key names.

(Bitbake rev: 664b85742d1afc94b291a85fd245abebffacdf3d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-28 15:27:42 +01:00
Richard Purdie 1b5d7b59d3 bitbake/data_smart: Don't track overrides in deleted variable names
When we delete a variable we no longer expect it to override other
variables.

To do this we remove it from the list of active overrides at deletion
time. It turns out we already had to do this at override expansion time
so this cleans up the code to be more consistent as an added bonus.

(Bitbake rev: d924ff9ede57c3dea6e1c738ba3633f18d460b14)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-28 15:27:41 +01:00
Richard Purdie 5c5b0045e8 bitbake/data_smart: Don't export deleted/empty entries in the list of keys
If you d.delVar(), you expect the variable to be gone. Even empty
variables continue to exist in the datastore and are still user visible
unfortunately. The COW siutation means you can't just remove it
since it might unmask a variable from an inner copy.

This patch therefore stops empty variables from appearing in key lists
exposed to the external world making empty variables an internal
implementation detail only.

(Bitbake rev: 2b5548c591d4cfde9238d2cc0959c42cfc08f09c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-28 15:27:41 +01:00
Richard Purdie cc2a8ff522 bitbake/data_smart: Optimise the data store iterator
Since we're going to creat the seen set() anyway, we might as well use
it directly. If we don't do this, we see thousands of function calls
with associated overhead on profiles.

(Bitbake rev: 9d43e3279895639ee4899df635f2546c7ee13737)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-28 15:27:40 +01:00
Robert Yang 671e580111 bitbake: Add task specific stamp file support
This patch, based on proof of concept code from Richard adds code to
bitbake to allow individual tasks to optionally specify their stamp
file using the stamp-base flag. This takes the same form as the STAMP
variable but can be specified on a per task basis.

Code is also added to runqueue to ensure that if two tasks share the
same stamp file, only one will be executed at once.

A significant usecase for this code is to share source code (${S})
between recipes where separate build directories (${B}) are used.

(Bitbake rev: 41bef02bef8379590ba012319aebe05068a8081e)

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-28 13:44:38 +01:00
Mark Hatle a1f79a7896 runqueue.py: Add umask task control
The umask for a task can now be set as:

task[umask] = 022
task[umask] = '022'

If specified as a text string, it must be octal.  (This is due to
recipe parsing where it's always set to a string.)

[RP tweaked to use None instead of -1]
(Bitbake rev: a5caaaaba8f0db1af5d8f2e610021d6d4b56894e)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-24 12:23:12 +01:00
Brandon Stafford 5fbb1b77d0 doc/usermanual.xml: Tweaks for the manual
This patch contains what I hope are non-controversial improvements to
the manual. Most of the changes are single characters, but the
line-by-line diff makes the patch look large.

(Bitbake rev: 5481cc90645e13c4e3cdea41e8e369528a0b1649)

Signed-off-by: Brandon Stafford <brandon@pingswept.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-22 16:57:31 +01:00
Scott Garman 48cd942be1 fetch2/git.py: improve error reporting when an invalid protocol is used
When an invalid 'protocol' parameter is used in a git SRC_URI,
the error reported was not helpful:

ERROR: Function 'Fetcher failure for URL: 'None'.
<environment dump>
fatal: Could not make temporary directory: No such file or directory

So instead check that ud.proto is set to something valid, and if not
raise a meaningful ParameterError which explains that the protocol
type is the source of the problem.

This fixes bug [YOCTO #1142]

(Bitbake rev: a2a29b72275ab03a263f4479a590b92111a0d6a8)

Signed-off-by: Scott Garman <scott.a.garman@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-22 16:27:30 +01:00
Richard Purdie 90dab9783d process.py: Fix issue where early errors weren't making it to the console
(Bitbake rev: d97f7d762e3d2f1b0da038d4d99f2531b2490670)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-16 22:14:01 +01:00
Scott Garman 62d538fbe6 make exception handling syntax consistent
Update exception handling syntax to use the modern style:
except ExcType as localvar

(Bitbake rev: dbf5f42b06bef81749b13aa99945cc1292a6676d)

Signed-off-by: Scott Garman <scott.a.garman@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-15 11:13:13 +01:00
Richard Purdie 039798a4d2 codeparser: When loading the cache, ignore ValueError
(Bitbake rev: 9bff182a4ba9571679985b45b309990a6eddad14)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-15 11:13:13 +01:00
Richard Purdie e5f3cc34df bitbake/fetch2: When replacing URLs in mirror handling mask out empty entries
The symptom of this problem is something like a cvs url which specifies
a username where the username is then passed through to something like
an http mirror.

This patch fixes things by ensuring empty entries are preserved in the
new URL.

(Bitbake rev: c1d978d7bd1ac8eb1e2d50029ab2384be9f72fb4)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-13 15:10:07 +01:00
Richard Purdie ba5db2d81c bitbake/cooker: Fix -b option by ensuring the empty cache structure is present
(Bitbake rev: 1430a36e81737bd92245042710eb9d6ad8b6f1a7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-13 15:10:07 +01:00
Richard Purdie a831dc120f Update version to 1.13.1
(Bitbake rev: c3c2ad6f22e35b893a353d4c21d0e923e46ad07b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-09 22:44:55 +01:00
Richard Purdie b8321c5658 bitbake/event/ast: Add RecipePreFinalise event
One of the implications is we need to register the event handlers before
executing the anonymous python functions. I can't find any issue with making
that change in any existing metadata use cases.

(Bitbake rev: a981df3cc9bf410d24f39919959952bdc6c76d03)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-09 22:43:46 +01:00
Richard Purdie af93b8937e bitbake/parse/ast: We always need to finalize the default data since otherwise overrides to BBCLASSEXTEND may not be applied
(Bitbake rev: f22ac9149cc9ab96510bb377deb82cd9bceb95c1)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-09 22:43:45 +01:00
Richard Purdie eda23733f0 bitbake/data_smart: Change overrides behaviour to remove expanded variables from the datastore
Currently if you do:

OVERRIDES = "z"
DEPENDS_prepend = "a "
DEPENDS = "b"
DEPENDS_z = "c"

d.update_data()
d.getVar("DEPENDS")

gives "a c"

d.update_data()
d.getVar("DEPENDS")

then gives "c"

This patch changes the behaviour such that at the time bitbake expands the DEPENDS_z
override, it removes "DEPENDS_z" from the data store. In the above example this would
mean that it wouldn't matter how often you call d.update_data(), you'd always get
"a c" back.

See the bitbake-devel mailing list for further discussion and analysis of the
potential impact of this change.

(Bitbake rev: 899d45b90061eb3cf3e71029072eee42cd80930c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-09 22:43:44 +01:00
Scott Garman 73871c2712 cache.py: fix bitbake -s command
This uses the correct index of self.pn when setting up cachedata's
pkg_pn, fixing the output of bitbake -s.

This fixes bug [YOCTO #1149].

(Bitbake rev: d000493c09ac5c1dcbab22d3a91296a9cb194ac0)

Signed-off-by: Scott Garman <scott.a.garman@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-09 22:18:50 +01:00
Paul Eggleton dc15ddd161 bitbake-layers: handle skipped recipes
Report bbappends correctly for skipped recipes instead of reporting the
bbappends as not having any matching recipe.

(From Poky rev: 39d72ecf6bc1da85aefb11e6481719185f0cb953)

(Bitbake rev: a71183762e5d7d9f8153832efb9595e3a21d62f1)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-09 18:12:51 +01:00
Paul Eggleton 81e2f52fb8 track skipped packages
Add skiplist to cooker that allows access to the list of packages skipped
via SkipPackage (this includes COMPATIBLE_MACHINE, INCOMPATIBLE_LICENSE,
etc.) This can be used to enhance error reporting.

(From Poky rev: 6c12b7b1099c77b87d4431d55e949cf7c5f52ded)

(Bitbake rev: 7d2363f35350be27a33f568c23eb07fcd3d27e53)

Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-09 18:12:51 +01:00
Richard Purdie e55dc39c37 Drop psyco support
At best it gave 1-2% improvement now, its 32 bit x86 only and isn't
supported after python 2.6.

PyPy is probably a better option now.

(Bitbake rev: 3c3bd0c2fa80d747f25401c17b785c7c2f3787ca)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-09 15:48:23 +01:00
Richard Purdie c7f76a1bc9 bitbake/data/runqueue: Sync up with upstream to clean up environment variable handling
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:28 +01:00
Richard Purdie bdab8e9354 bitbake/build.py: Drop exec_shell environment as its now unneeded
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:28 +01:00
Richard Purdie fe967227dd runqueue.py: Wrap parsing with exception trapping
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:28 +01:00
Chris Larson 95f02a2ab1 Shift traceback pre-formatting into LogHandler
It's cleaner to leave it behaving as usual, passing the exception data in the
exc_info attribute of the LogRecord where it normally lies, and then let
LogHandler make it pickleable so it can be sent to the UI.

(Bitbake rev: 3539c9474a0b53f57e614544c70a7197ecdfb130)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:27 +01:00
Richard Purdie 355338c40c bitbake/cooker.py: Misc sync with upstream
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:27 +01:00
Chris Larson 9f33dde6e7 cooker: use BBHandler.inherit for INHERIT
Ideally we'd avoid direct BBHandler usage, but honestly, to say the whole
bb.parse abstraction is incredibly leaky is an understatement.  If we try to
make handle() not reparse classes, things get ugly fairly quickly, as
inherit() calls handle() itself after adding the class to the inherit cache.

This change fixes it so we no longer risk reparsing a class if:

- it's listed in INHERIT multiple times
- it's listed in INHERIT and is 'inherit'ed from a class in INHERIT

(Bitbake rev: 057c3cddeb72584c6c3908bd702288cece9b66ea)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:26 +01:00
Chris Larson 88f4ba28bc cooker: simplify self.configuration.data vs data usage
(Bitbake rev: 87112adee4e8add0a97ff8be8311d9afe202412d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:26 +01:00
Chris Larson 87dfc1466d cooker: don't choke if we have nothing to parse
If all our recipes were cached, there's no reason to fire off any parsing
progress events at all.

(Bitbake rev: e9e174e5781fc3de4dfd60d01228048a06a62b16)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:26 +01:00
Chris Larson 0f74a38ffd bb.exceptions: don't choke on frames without arguments
(Bitbake rev: c9bb37e588ee7ee95eca798b0eae57bad68e8caf)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:25 +01:00
Chris Larson 3fd34462ed cooker: handle ExpansionError the same way we do ParseError
(Bitbake rev: 12aef37618bc77fb5ef085d24ad85471fb85c111)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:25 +01:00
Chris Larson 96e2ee1d73 Shift exception formatting into the UI
Now we use bb.exceptions to pass pickleable traceback entries to the UI, and
the UI is free to do whatever it wants to do with this information. By
default, the log formatter for the UIs formats it with bb.exceptions.  This
also means that all exceptions should now show 3 lines of context and limit to
5 entries.

(Bitbake rev: ee48d628ee038bd72e1cd94aa75f5ccbacbcee4c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:25 +01:00
Chris Larson a3efbb96f2 cooker: don't show a traceback for ParseError
(Bitbake rev: cae6bf031dc83ba0439d07584fdbbd4a962408a3)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-06-08 20:38:25 +01:00