Commit Graph

230 Commits

Author SHA1 Message Date
Richard Purdie 36f1ae42fe pseudo/fakeroot: Move the pseudo directory creation into bitbake
If sstate was used to accelerate a build, the pseudo directory might not have
been created leading to subsequent task failures.

Also, sstate packages were not being installed under pseudo context meaning
file permissions could have been lost.

Fix these problems by creating a FAKEROOTDIRS variable which bitbake ensures
exists before running tasks and running the appropriate setscene tasks under
fakeroot context.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-22 13:00:40 +01:00
Richard Purdie 229b6630af bitbake-runtask: Ensure logging settings take effect in the worker process (and pass verbose setting)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-14 20:07:15 +01:00
Joshua Lock 812f8835d2 bitbake/runqueue: make the runtask program configurable
Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-10-12 13:32:27 +01:00
Richard Purdie ff539b11ac bitbake-runtask: Ensure logging options are inherited from the parent
[BUGID #291]

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-11 12:52:57 +01:00
Richard Purdie bee7c21f6e bitbake/runqueue.py: Fix invalid variable reference fixing the -f option with setscene tasks
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-07 15:45:02 +01:00
Richard Purdie 35d095c282 bitbake/sstate: Implement a lookup function to speed up setscene processing
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-10-05 22:26:33 +01:00
Richard Purdie ec4d6b989a bitbake: Pass task hash information to subprocesses
Pass task has informaiton to work processes, allowing full manipulation of
the hash data in the task context allowing checksums to be usable.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-28 15:34:27 +01:00
Richard Purdie fe0f86e15c bitbake/runqueue: Remove now unneeded indentation
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 21:00:10 +01:00
Richard Purdie 474c2a2817 bitbake/runqueue.py: Optimise delay values to avoid unneeded delays
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-13 16:57:13 +01:00
Joshua Lock 3e7db72216 bitbake: save the initial PATH for use when starting a subprocess
It was possible for bitbake-runtime to be run against a semi-installed
python-native resulting in tracebacks with ImportError's.

To prevent this we stash the initial PATH in the BBConfiguration when bitbake
is started and then set this in the env when launching bitbake-runtask through
subprocesses Popen() call.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-09-08 19:22:37 +01:00
Richard Purdie 43595fabbe bitbake: Implement signatures
Includes functionality to find out what changes between two different singature data dumps.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-09-03 16:08:32 +01:00
Richard Purdie 53ff069611 bitbake/runqueue.py: Ensure rqexe always exists and that empty task lists cause a graceful exit
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-25 00:59:13 +01:00
Richard Purdie 616e75523b bitbake: Correctly route events from the worker to the server
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-20 12:26:19 +01:00
Richard Purdie 4a8fe7fc99 bitbake/runqueue.py: Ensure the flush happens as late as possible to avoid data mixups
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 23:39:19 +01:00
Richard Purdie 6739a629e9 bitbake/runqueue.py: Fix runqueue UI issues
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 22:36:26 +01:00
Richard Purdie b6bfe14205 bitbake: Switch to use subprocess for forking tasks and FAKEROOTENV to run shell and python under a fakeroot environment
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 22:36:25 +01:00
Richard Purdie 787c1cf811 bitbake: Initial scenequeue implementation (needs major fixes)
bitbake: scenequeue: Skip setscene if the underlying task already ran
bitbake/setscene: Make sure uneeded dependencies are removed recursively

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 20:06:25 +01:00
Richard Purdie 3ce480de27 bitbake/runqueue.py: Factor task skipping code into a function
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 11:41:44 +01:00
Richard Purdie 65b068a5f8 bitbake/runqueue.py: Create RunQueueExecute and RunQueueExecuteTasks classes, further splitting up runqueue
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 11:41:44 +01:00
Richard Purdie 96ec9f8a60 bitbake/cooker.py: Allow idle handlers to pass through a true value
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 11:41:44 +01:00
Richard Purdie 1db9ee5e8d bitbake/runqueue.py: Drop duplicated class
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-19 11:39:59 +01:00
Richard Purdie 5d9f37873d bitbake: Split Runqueue into two classes, a data processor and the execution part
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-18 11:49:58 +01:00
Bernhard Reutner-Fischer 0e3fc6dff2 *: cosmetic whitespace removal
(Bitbake rev: 2ed36a3d57de25e1af31d657a7b0b14857e5056a)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-03 14:06:40 +01:00
Chris Larson e6b6767369 Let the runqueue find the user selected scheduler dynamically
Searches the module (bb.runqueue) for any new style classes which are
instances of RunQueueScheduler, and uses the one whose 'name' attribute
matches the value of BB_SCHEDULER.

(Bitbake rev: 6497cedf9cfc03201250af816995dd2bd85c36ef)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-08-03 14:03:41 +01:00
Richard Purdie 63e6ba8567 bitbake: Add support for .bbappend files (see mailing lists for detais)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 15:10:40 +01:00
Richard Purdie 6598f98ba8 bitbake: runqueue: Allow passing of success/failure handlers to runqueue_process_waitpid()
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 12:31:20 +01:00
Richard Purdie 8c23850453 bitbake: runqueue: Allow taskname to be passed to check_stamp_task()
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 12:31:20 +01:00
Richard Purdie 009343b2a2 bitbake: runqueue: Make child process error scary so user is less likely to ignore
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 12:31:20 +01:00
Richard Purdie 2d263840b0 bitbake: runqueue: Fix number of running tasks accounting (include failed tasks)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 12:31:20 +01:00
Richard Purdie 3cd042f505 bitbake: runqueue: Remove unneeded exception catching from fork_off_task()
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 12:31:19 +01:00
Richard Purdie 5c44f9a78d bitbake: runqueue: Separate out task forking code into a new function
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 12:31:19 +01:00
Richard Purdie 8a5b5080a2 bitbake: runqueue: Combine the waitpid result handling code into a function
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 12:31:19 +01:00
Richard Purdie 90db9f3074 bitbake: Remove two layers of unnecessary functions from task execution
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-16 12:31:19 +01:00
Chris Larson ef1de9ecaf Apply some 2to3 refactorings
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:37 +01:00
Chris Larson 88a257634a Show a traceback when a task fails due to an exception
(Bitbake rev: 5bb24832761bd518cb0249125f67939e0cbd0998)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:36 +01:00
Chris Larson 4fac507dfb Don't tell the user we're sending SIGINT to the remaining 0 tasks
(Bitbake rev: 0b963e52eca0447d5bbfc87d826ad662fe940381)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:35 +01:00
Chris Larson 1180bab54e Apply some 2to3 transforms that don't cause issues in 2.6
(Bitbake rev: d39ab776e7ceaefc8361150151cf0892dcb70d9c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson ad543e2e41 Apply the 2to3 print function transform
(Bitbake rev: ff2e28d0d9723ccd0e9dd635447b6d889cc9f597)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:33 +01:00
Chris Larson 1c74fd768f Import fixups
(Bitbake rev: 4fa052f426e3205ebace713eaa22deddc0420e8a)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 7acc132cac Formatting cleanups
(Bitbake rev: 2caf134b43a44dad30af4fbe33033b3c58deee57)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:32 +01:00
Chris Larson 87f62d7c8d Ensure we always utilize the correct messaging domains
(Bitbake rev: 92a2e2e90981c0615171abe03645a772d84f6986)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-07-02 15:41:31 +01:00
Chris Larson 6028daedcb Fix 7 references to undefined variables, as spotted by pyflakes
(Bitbake rev: e1e4ccf203e38070eeafd31a622671996cff61a1)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-25 17:25:56 +00:00
Richard Purdie 589df80ca1 bitbake: Gross hack to fix packaged-staging with BBCLASSEXTEND packages
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-23 22:33:02 +00:00
Bernhard Reutner-Fischer de7c939a7f bitbake: runqueue.py: improve printing dependent tasks
Print names instead of Task-IDs (and not mentioning they're task ids).

Previously we printed e.g.:
Dependency loop #1 found:
  Task 89 (/there/src/oe/openembedded/recipes/busybox/busybox-native_1.15.2.bb, do_configure) (depends: Set([88, 282, 92, 87]))

Now we say
Dependency loop #1 found:
  Task 89 (/there/src/oe/openembedded/recipes/busybox/busybox-native_1.15.2.bb, do_configure) (dependent Task-IDs ['busybox-native, do_patch', 'update-rc.d, do_populate_staging', 'busybox-native, do_populate_staging', 'shasum-native.bb, do_populate_staging', 'busybox-native, do_unpack'])

(Bitbake rev: 00eaf76fdc32eb515995b47dfa69eb90ca904b37)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-02-15 17:03:47 +00:00
Richard Purdie 5ebad0d500 knotty: Improve task failure message handling using the uihelper code
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-21 23:49:17 +00:00
Richard Purdie 5c62833766 runqueue.py: Use fcntl to make the worker pipes non-blocking
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-21 23:46:20 +00:00
Richard Purdie d7c9b7d8b9 runqueue: When a task fails, make sure the server still processes events from other workers
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-21 15:58:56 +00:00
Richard Purdie 573f6125e6 bitbake/runqueue: Fix typo
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-21 10:25:54 +00:00
Richard Purdie 22c29d8651 bitbake: Switch to bitbake-dev version (bitbake master upstream)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-20 18:46:02 +00:00
Richard Purdie a6d608eb19 bitbake runqueue: Add a minor speedup to the runqueue code
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-29 14:08:05 +01:00
Richard Purdie fa5d458526 bitbake: Factor some runqueue code into common functions making things more readable
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-21 22:38:53 +01:00
Richard Purdie 502bd2ef92 bitbake: Fix a bug in runqueue and optimise slightly further
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-21 22:32:35 +01:00
Richard Purdie 8f5363d16d bitbake: Optimise runqueue recursive dependency calculations removing a bottleneck in world builds
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-21 19:44:23 +01:00
Richard Purdie 81ff3a90f8 bitbake: Update to work without warnings with python 2.6
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-05-12 16:53:22 +01:00
Richard Purdie b296ae263c bitbake: Add tryaltconfigs option to disable the alternative configuration attempts and make the 'continue' more aggresive 2008-10-28 22:15:06 +00:00
Richard Purdie cd872118e7 bitbake: Fix nostamp flag handling bug
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5368 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-10-01 13:55:17 +00:00
Richard Purdie e14d7dcbee bitbake: Sync with 1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@4463 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-05-13 07:53:18 +00:00
Richard Purdie 01bb9af1ec runqueue.py: Fix default whitelist variable value
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@4413 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-05-05 09:21:49 +00:00
Richard Purdie d6addd4969 bitbake: Sync with 1.8 branch upstream for PREFERRED_PROVIDERS message improvements and BB_STAMP_WHITELIST functionality
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@4411 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-05-04 23:22:24 +00:00
Richard Purdie 853280f161 bitbake: Update with changes from bitbake 1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@4006 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-03-14 11:44:34 +00:00
Richard Purdie ab191d21e2 bitbake: Update to bitbake 1.8 branch head
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3892 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-03-03 22:01:45 +00:00
Richard Purdie 6ac3680a2d bitbake: Sync with upstream 1.8 branch for git fetcher and --continue mode fix
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3761 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-02-11 20:33:43 +00:00
Richard Purdie 079e6825aa bitbake: sync with upstream stable branch bugfixes and enhancements
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3554 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-01-20 12:09:31 +00:00
Richard Purdie 7821f2250d bitbake: Sync with bitbake upstream for various fixes
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3411 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-01-06 16:51:51 +00:00
Richard Purdie 14fb92c008 bitbake runqueue.py: Allow multiple options for deptask flag (from poky), fix comment typo.
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3041 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-10-30 12:03:07 +00:00
Richard Purdie e223238b1b bitbake: Update to latest bitbake-1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2651 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-09-02 14:10:08 +00:00
Richard Purdie d8bfa5c6ef bitbake: Sync with 1.8.8 release
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2513 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-20 07:48:43 +00:00
Richard Purdie 7611768e23 bitbake: Sync with 1.8 head. Adds locking to the fetcher to prevent parallel downloads, fixes key expansion issues and occasional missing dependency graph links
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2502 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-16 09:55:21 +00:00
Richard Purdie 6e909eb80e bitbake runqueue.py: Improve the progress indicator by skipping tasks that have already run before starting the build rather than during it
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2423 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-09 10:51:58 +00:00
Richard Purdie b5aa22b972 bitbake lib/bb/runqueue.py: Make sure intertask depends get processed correctly in recursive depends
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2368 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-05 22:43:24 +00:00
Richard Purdie d3076aca9d bitbake: Sync with upstream
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2012 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-06-26 21:23:37 +00:00
Richard Purdie 25cbce31b3 runqueue.py: Correctly use /dev/null as stdin
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1918 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-06-12 09:23:39 +00:00
Richard Purdie d05bcee7f2 bitbake: Merge bugfixes from bitbake-1.8 svn
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1759 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-05-22 11:50:37 +00:00
Richard Purdie 7371e6323c bitbake: Update to 1.8.1 (inc. various bug fixes, epoch support)
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1419 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-04-01 15:04:49 +00:00
Richard Purdie 23ce7821a3 bitbake: merge upstream fixes and fix bug in recrdeptask handling
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1282 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-02-21 20:15:13 +00:00
Richard Purdie f5665d5bfc bitbake: Sync with upstream.
* File licence headers were sanitised causing most of the diff. 
 * cooker.py was created from bin/bitbake. 
 * cvs fetcher port option was added
 * The -f force option was fixed to work correctly
 * Multiple entries in rrecrdeps are now handled correctly
   (allows adding do_deploy to image depends)
 


git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1129 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-01-08 23:53:01 +00:00
Richard Purdie aac2b8d747 runqueue.py: Send SIGINT upon Ctrl+C, not SIGNTERM
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@1014 311d38ba-8fff-0310-9ca6-ca027cbcb966
2006-12-06 16:29:08 +00:00
Richard Purdie 21fb2cbe1c bitbake runqueue.py: Set children as process group leaders and send SIGTERM to the process groups
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@998 311d38ba-8fff-0310-9ca6-ca027cbcb966
2006-12-04 17:30:47 +00:00
Richard Purdie 681d6c18ad bitbake: Sync with bitbake trunk for bugfixes and improved dot file generation code
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@987 311d38ba-8fff-0310-9ca6-ca027cbcb966
2006-11-29 22:52:37 +00:00
Richard Purdie 306b7c7a97 bitbake: Upgrade from 1.4 -> 1.7.4ish
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@863 311d38ba-8fff-0310-9ca6-ca027cbcb966
2006-11-16 15:02:15 +00:00