Commit Graph

63 Commits

Author SHA1 Message Date
Chris Larson ecc68fa4fb Switch bitbake internals to use logging directly rather than bb.msg
We use a custom Logger subclass for our loggers

This logger provides:
- 'debug' method which accepts a debug level
- 'plain' method which bypasses log formatting
- 'verbose' method which is more detail than info, but less than debug

(Bitbake rev: 3b2c1fe5ca56daebb24073a9dd45723d3efd2a8d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson 4855548ffb Use the python logging module under the hood for bb.msg
(Bitbake rev: 47ca82397bc395b598c6b68b24cdee9e0d8a76d8)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:46:33 +00:00
Chris Larson bac0c60b61 Make realfile optional for better_exec, using the existing fn in the code object
(Bitbake rev: 499a2d28d578cdd6df7cd30ccb79cc2b2796fb65)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2011-01-04 14:36:54 +00:00
Yu Ke 6ee0c26e21 bb.utils: check if lock file is writable, to fix bug 606
Bug 606 report that if $DL_DIR is read-only, do_fetch will
simply hang without any error message.

The root cause is that: bb.fetch.go()->bb.utils.lockfile()
will try to lock file ${DL_DIR}/xxxxx.lock. Since ${DL_DIR}
is read-only, it will cause IOError exception. Although
lockfile() can catch the exception, currently code simply
ignore all the exception and continue the loop. it make
sense if the exception is caused by locking contention,
but in the read-only $DL_DIR case, it cause endless waiting
unfortunately.

So this patch add read-only check for lockfile to avoid the
silent hang.

Fix [BUGID #606]

Signed-off-by: Yu Ke <ke.yu@intel.com>
2010-12-30 09:28:45 +00:00
Richard Purdie 0ba9a9fffe bitbake: Overhaul environment handling
Currently, anything whitelisted in the environment makes it into the worker
processes. This is undesireable and the worker environment should be as
clean as possible. This patch adapts bitbake sosme variables are loaded into
bitbake's datastore but not exported by default. Any variable can be exported
by setting its export flag.

Currently, this code only finalises the environment in he worker as doing so
in the server means variables are unavailable in the worker. If we switch
back to fork() calls instead of exec() this code will need revisting.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-09 13:16:14 +00:00
Richard Purdie a0033b622e bitbake/utils.py: Improve traceback to be more helpful/clear to users
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-05 22:52:44 +00:00
Richard Purdie 1a0c39e050 bitbake/utils.py: Ensure the last lines of functions are printed in tracebacks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-12-05 22:51:38 +00:00
Richard Purdie 43595fabbe bitbake: Implement signatures
Includes functionality to find out what changes between two different singature data dumps.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-03 16:08:32 +01:00
Richard Purdie 71a46a44a2 utils.py: Fix bb.copyfile to change the permissions of the file back correctly
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-23 13:01:05 +01:00
Mark Hatle 87dba4254b utils.py: Add a new extend_deps() to easily merge two dependency lists
Add a new extend_deps function to more easily merge two dependency lists.
This avoids adding duplicates, unless the value of the dependency is
different.

Signed-off-by: Mark Hatle <mhatle@windriver.com>
2010-08-20 23:46:08 +01:00
Richard Purdie 75b57d573c bitbake/utils.py: Allow copyfile to copy files which aren't readable
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-20 15:11:44 +01:00
Richard Purdie ac3452c431 bitbake/utils.py: Improve better_exec debug output
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-20 12:52:43 +01:00
Richard Purdie 6e3a3816d5 bitbake/utils.py: Give useful debug information when no lineno information is available
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 23:27:08 +01:00
Richard Purdie 8e8316c8f0 bitbake/utils.py: Allow explode_dep_versions to handle the commas join_deps can introduce into strings
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-17 09:29:08 +01:00
Bernhard Reutner-Fischer 8105d4381f *: Fix typo in documentation
s/dictonary/dictionary/

(Bitbake rev: 0cc632761e75f66a8ce5ca2fe370f7551ccbfdf0)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-04 16:12:39 +01:00
Bernhard Reutner-Fischer ad8c5b61a4 utils: Fix number of arguments to bb.msg functions
the 'fn' argument of them is not used, should be removed anytime:
sed -i -e
'/^def.*fn/s/,[[:space:]]*fn[[:space:]]*=[[:space:]]*None[[:space:]]*)/)/g'
lib/bb/msg.py

(Bitbake rev: 1cb72e371322c271ee7f2d008c6f7899fb38b4fd)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-03 14:07:11 +01:00
Chris Larson ef1de9ecaf Apply some 2to3 refactorings
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:37 +01:00
Chris Larson 22a2179905 Kill unnecessary usages of the types module
types.IntType -> int
types.StringType -> basestring
...

Also moves our ImmutableTypes tuple into our own namespace.

(Bitbake rev: 83674a3a5564ecb1f9d2c9b2d5b1eeb3c31272ab)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson 1180bab54e Apply some 2to3 transforms that don't cause issues in 2.6
(Bitbake rev: d39ab776e7ceaefc8361150151cf0892dcb70d9c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson ad543e2e41 Apply the 2to3 print function transform
(Bitbake rev: ff2e28d0d9723ccd0e9dd635447b6d889cc9f597)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson 51f5346f29 Drop some unused vars
(Bitbake rev: b79ca5f1cc2d262de877047769ce25279ba26672)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 1c74fd768f Import fixups
(Bitbake rev: 4fa052f426e3205ebace713eaa22deddc0420e8a)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 7acc132cac Formatting cleanups
(Bitbake rev: 2caf134b43a44dad30af4fbe33033b3c58deee57)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson bbf83fd988 Drop doctest, as we aren't utilizing it
We should add back in some proper unit testing.  Doctest is rather limited,
and we haven't actually made use of it since the original implementation of
the datastore when the project started, as far as I'm aware.

(Bitbake rev: 3a11c2807972bbbddffde2fa67fc380d159da467)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 943ece8610 Implement bb.msg.domain as a named tuple, drop the Enum class
Also fixes some bb.msg references from within bb.msg.

(Bitbake rev: db95af590f742c8186e84046ad9704fae1733720)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 87f62d7c8d Ensure we always utilize the correct messaging domains
(Bitbake rev: 92a2e2e90981c0615171abe03645a772d84f6986)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:31 +01:00
Chris Larson a7f05ea4af Resurrect the old bb.vercmp as bb.utils.vercmp, and its deps
This is just for compatibility.  We may drop it in the future, or rewrite it,
as it's not particularly pythonic.

(Bitbake rev: c4e31d7fe1d15a1e3ef2a453e7c7812d403d22cb)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:31 +01:00
Chris Larson 3995546222 Add & utilize a simple_exec utility function
(Bitbake rev: 5ead16f14f996bcbb2fb14277b8ce5a9317b8160)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:31 +01:00
Chris Larson 94b60d1247 Consolidate the exec/eval bits, switch anonfunc to better_exec, etc
The methodpool, ${@} expansions, anonymous python functions, event handlers
now all run with the same global context, ensuring a consistent environment
for them.  Added a bb.utils.better_eval function which does an eval() with the
same globals as better_exec.

(Bitbake rev: 424d7e267b009cc19b8503eadab782736d9597d0)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:31 +01:00
Richard Purdie 9a543b75ca bitbake: Add bb.utils.join_deps to handle creation of dependency strings with versions
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-06-30 09:47:36 +01:00
Jeff Dike f9d7e9e3be bitbake: Move the logger initialization from cooker to utils
In order to move the environment cleaning, which wants to log a
message, before cooker creation, the logging facility initialization
needs to happen earlier.  So, it is now in init_logger in utils.py and
called from bitbake before the creation of the data object.

It also seems more correct to initialize a global facility like this
from a more global context than the creation of an object, of which
there could theoretically be many.

Signed-off-by: Jeff Dike <jdike@linux.intel.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-06-29 13:32:04 +01:00
Chris Larson 6028daedcb Fix 7 references to undefined variables, as spotted by pyflakes
(Bitbake rev: e1e4ccf203e38070eeafd31a622671996cff61a1)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-25 17:25:56 +00:00
Chris Larson c6940a9795 bb.utils: remove old portage remnants
The following utility functions were removed.  Note that in this particular
case we're bypassing a proper deprecation process, as these functions are
clearly not utilized, are obvious remnants of old ways of doing things, and
some of which do not even function properly.

- tokenize
- evaluate
- flatten
- relparse

- ververify
- isjustname
- isspecific
- catpkgsplit
- pkgsplit
- pkgcmp
- dep_parenreduce
- dep_opconvert

(Bitbake rev: 62983ad9b151ee8d51e8cf9a31c736c7813edf16)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-25 17:25:54 +00:00
Chris Larson ec3f908948 bb.utils: utilize string.letters and string.digits
(Bitbake rev: 7af05cba87ec9f3ea5e53689b3d9f0a63784d1b5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-25 17:25:51 +00:00
Chris Larson 80a52e7ad6 bb.which: fix broken direction/reverse argument
(Bitbake rev: 4725d83f532cad96168aa9affdedb33b6fc897b7)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-25 17:25:50 +00:00
Bernhard Reutner-Fischer 5d90d90358 utils: improve wording of error message
(Bitbake rev: b2486ec57c6a7adf09d0960fdf6727881b324d2f)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-25 17:15:28 +00:00
Chris Larson 8f83be3338 Fix msg usage in the moved utility functions.
(Bitbake rev: 686288444d22091dee66e20ec49b9c53f8c980b7)

Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-22 14:54:57 +00:00
Chris Larson f8c6db95d7 Move most utility functions from bb into bb.utils.
(Bitbake rev: ff720ec59b30671c951dbf3b96df10ef56b8b505)

Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-22 14:54:42 +00:00
Richard Purdie 62c103ce0e bitbake: utils.py: Don't error if an exported environment variable is unset (i.e. None)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-12 13:50:19 +00:00
Richard Purdie 22c29d8651 bitbake: Switch to bitbake-dev version (bitbake master upstream)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-20 18:46:02 +00:00
Richard Purdie d5a137475a bitbake: Add build_environment function for building an environment based on the data dictonary
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-02 14:33:53 +01:00
Richard Purdie 7695704eab bitbake: Add bb.utils.prune_suffix() 2008-12-31 22:58:57 +00:00
Richard Purdie ffdec425bf bitbake/bitbake-dev: Sync with upstream 2008-12-06 13:16:13 +00:00
Richard Purdie c53608b220 bitbake utils.py: Merge prunedir symlink fix from bitbake upstream
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5551 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-10-20 14:19:36 +00:00
Richard Purdie 7a4b36a7d1 bitbake: Add some sanity checks to the lockfile functions
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5534 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-10-17 11:04:26 +00:00
Robert Bradford 696d4f01d2 bitbake: Refactor the environment variable handling
Refactor the environment variable handling so that it more closes matches the
changes that have been made in bitbake-dev.

git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5527 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-10-17 09:46:35 +00:00
Richard Purdie f4337b9803 bitbake utils.py: Fix ocassional locking glitch with a better retrying mechanism
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5415 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-10-06 08:09:11 +00:00
Joshua Lock e2d321075a Add OEROOT to preserved_envvars_list.
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5351 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-09-30 21:49:11 +00:00
Richard Purdie efe3d75208 bitbake: Sync with upstream
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5350 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-09-30 21:12:36 +00:00
Richard Purdie 2e182847e4 bitbake/bitbake-dev: Allow much better control of which variable influence bitbake from the environment
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5347 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-09-30 20:57:18 +00:00