Commit Graph

61 Commits

Author SHA1 Message Date
Nitin A Kamble c3682bf89b bitbake decodeurl: fix the file:// url handling
Without this patch decoding a url of this kind file://dir/filename gives
path=/filename host=dir.
With the patch it decodes as path=/dir/filename host=""

Probably nobody stumbled on this issue yet because nobody used
file:// urls with directory names in the path.

Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com>
2010-06-07 16:05:00 +01:00
Chris Larson 6028daedcb Fix 7 references to undefined variables, as spotted by pyflakes
(Bitbake rev: e1e4ccf203e38070eeafd31a622671996cff61a1)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-25 17:25:56 +00:00
Enrico Scholz 5060280a7a Expand PREMIRRORS
When not expanding PREMIRRORS, the functions fails/does not work correctly
when PREMIRRORS is not a plain string (e.g. contains ${...} or a ${@...}
statements).

(Bitbake rev: d612d22b073f68b8cf1bb7344e0487820040d80d)

Signed-off-by: Enrico Scholz <enrico.scholz@informatik.tu-chemnitz.de>
Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-25 17:17:12 +00:00
Chris Larson ada2a8494a Avoid unnecessary calls to keys() when iterating over dictionaries.
dict objects provide an __iter__ method for the iteration which gives you the
keys, so calling keys directly is unnecessary, and isn't really a best
practice.  The only time you really need to call the keys is if there's a
danger of the dict changing out from underneith you, either due to external
forces or due to modification of the iterable in the loop.  Iterations over
os.environ are apparently subject to such changes, so they must continue to
use keys().

As an aside, also switches a couple spots to using sorted() rather than
creating a temporary list with keys() and sorting that.

(Bitbake rev: 5b6ccb16c6e71e23dac6920cd2df994d67c2587b)

Signed-off-by: Chris Larson <clarson@mvista.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-22 15:01:59 +00:00
Tom Rini dea828c744 Add initial 'repo' fetcher
(Bitbake rev: f68406e864c9837feb56cbec993b620481445cc2)

Signed-off-by: Tom Rini <tom_rini@mentor.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-22 14:58:15 +00:00
Chris Larson ac392b66a5 Move encodeurl, decodeurl into bb.fetch.
(Bitbake rev: 867d36f9afce2d298874ac7563e5b3852ef04659)

Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-22 14:54:10 +00:00
Chris Larson a3012e1ca9 Move MalformedUrl, VarExpandError into appropriate modules.
(Bitbake rev: e616483b237dafff7f90ba1c09e9ee7c383a2e47)

Signed-off-by: Chris Larson <clarson@kergoth.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-22 14:53:55 +00:00
Richard Purdie 382a630e78 bitbake fetch/__init__.py: Failing urls should raise errors, not fatal exceptions
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-03-15 12:55:47 +00:00
Joshua Lock 52cbb07423 bitbake: fix invalid method and logic
The previous bitbake change included an invalid method call and inverted logic.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-02-05 12:41:08 +00:00
Joshua Lock ec92133d6f bitbake: Test premirrors and mirrors in checkstatus()
checkstatus() is used to ensure we can fetch a copy of each file, so it makes
sense to also test PREMIRRORS and MIRRORS in the method.

This patch adds calls to try_mirrors() to the Fetch.checkstatus() method and
changes the try_mirrors() method to take a check argument, which is False by
default. When check is True try_mirrors() will call a fetchers checkstatus()
with the replaced uri.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-02-05 12:19:05 +00:00
Joshua Lock c9fd126f64 bitbake: Fix try_mirrors() logic, was only trying the first mirror.
The logic in try_mirrors() was buggy such that only the first entry in the
mirrors list was tried.

This patch fixes this. Thanks to Richard for the review.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-02-04 14:08:34 +00:00
Joshua Lock afe12428a9 bitbake: if PREMIRRORS set test for local file in FetchData.setup_localpath
When we are using PREMIRRORS it's possible a mirror in the local namespace
(some filesystem path, i.e. an NFS share) provides read-only files.
This is a perfectly valid scenario so this patch fixes bitbake so that for
such a scenario locapath is set to the files path rather than some child
of DL_DIR.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-02-04 00:18:29 +00:00
Joshua Lock b571168ac7 bitbake: Enhance the fetchers' support for local mirrors
Modify the try_mirrors() function to return the localpath of the fetched file
and update the data dictionary to reflect this.
Secondly the metadata files, lock and md5, should always be stored relative to
the ${DL_DIR} as it is possible that the localpath is a read-only directory,
for example in the scenario where there is a read-only file:// mirror.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-02-04 00:18:29 +00:00
Joshua Lock 0737552c1d bitbake: unify mirror support and make it independant of the fetcher
This patch serves two purposes. Firstly it unifies the concept of mirrors into
PREMIRRORS and MIRRORS. PREMIRRORS are tried before the SRC_URI defined in the
recipe whereas MIRRORS are tried only if that fails.
The tarball stash was conceptually inline with a PREMIRROR only with special
handling within the wget fetcher and therefore only worked with certain
fetch types.
Secondly the patch removes the need for individual fetch implementations to
worry about mirror handling.

With this patch, the base fetch implementation will first try to use a
PREMIRROR to fetch the desired object, if this fails the native fetch method
for the object will be tried and if this fails will try to fetch a copy from
one of the MIRRORS.

Signed-off-by: Joshua Lock <josh@linux.intel.com>
2010-02-04 00:18:29 +00:00
Richard Purdie 22c29d8651 bitbake: Switch to bitbake-dev version (bitbake master upstream)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2010-01-20 18:46:02 +00:00
Richard Purdie 011557c157 bitbake: Fix dict changing size problems
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-11-26 22:45:52 +00:00
Richard Purdie b5a059ff28 fetch/__init__.py: Try and make the sortable_revision code mode readable
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-11-13 12:15:23 +00:00
Richard Purdie 483f85802a bitbake: Sync various functions with those from bitbake-dev and bitbake upstream
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-11-13 12:15:21 +00:00
Richard Purdie c2b5a61728 bitbake: Sync module import cleanups back from bitbake trunk
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-11-13 12:15:21 +00:00
Richard Purdie 8138603680 bitbake/fetch: Ensure SRCREVINACTION always resets the core flags even when parsing failes by using a try/finally
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-09-24 18:57:08 +01:00
Mike Turquette 59040b5d14 __init__.py: export GIT_PROXY_COMMAND for those behind draconian proxies
Signed-off-by: Mike Turquette <mturquette@ti.com>
2009-08-26 10:56:53 -05:00
Richard Purdie 433c2d2318 bitbake: Remove persist_data domain renaming code as it appears to expose sqlite bugs and data consistency issues
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-30 21:24:31 +01:00
Richard Purdie 64b04685b6 bitbake: Add a --revisions-changed commandline option to indicate when floating srcrevs have changed
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-07-29 14:33:14 +01:00
Richard Purdie 522cffdb9e bitbake: Add PN to SRCREV keyhash in the persistent database to avoid conflicts between pacckages (from upstream bitbake)
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
2009-06-10 11:21:43 +01:00
Richard Purdie bb98685a71 bitbake: fetch/init: Add 'HOME' to list of variables exported when runnig fetcher commands 2009-01-19 22:28:49 +00:00
Robert Bragg c1a12b7d20 Adds username support to the git fetcher, and exports SSH_AGENT_PID + SSH_AUTH_SOCK
The SSH_AGENT_PID + SSH_AUTH_SOCK environment variables are needed if you
are using ssh-agent to hold passwords for your ssh keys.
2008-12-03 16:21:45 +00:00
Richard Purdie 610b53fcb9 bitbake: Optionally take username/password options from url parameters 2008-11-05 10:31:40 +00:00
Richard Purdie 4c626238e7 base.bbclass: Set GIT_CONFIG correctly 2008-11-04 13:32:35 +00:00
Richard Purdie b1046ef743 bitbake: Don't export the GIT_PROXY_COMMAND in the fetcher code since it means selective proxying isn't possible 2008-10-31 14:37:24 +00:00
Richard Purdie f0b1d561c7 bitbake: Preserve http_proxy and ftp_proxy in fetcher command execution 2008-10-23 13:34:24 +01:00
Richard Purdie ede20a3245 bitbake: Improve proxy variable handling in fetchers 2008-10-22 22:24:03 +01:00
Richard Purdie 1dc91f00fd fetch/__init__.py: Add in hooks to make sure osc fetcher is seen
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5402 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-10-02 22:15:38 +00:00
Richard Purdie 979f987c44 bitbake cache.py: When SRCREV autorevisioning for a recipe is in use, don't cache the recipe. Based on work from Heikki Paajanen
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@5352 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-09-30 21:54:47 +00:00
Richard Purdie 63e451ebbb __init__.py: Error if SRCREV == INVALID
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@4919 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-07-22 09:13:51 +00:00
Ross Burton 8bfb9258cf Use bb.utils.md5_file isntead of calling md5sum
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@4386 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-05-01 10:59:24 +00:00
Richard Purdie 748039c0e1 bitbake: Sync with bitbake 1.8 branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@4352 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-04-27 11:22:54 +00:00
Richard Purdie 67d5f0483b bitbake: Improve fetcher runcmd function so error messages are visable and various variables are exported for the benefit of the git fetcher
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@4194 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-04-08 11:34:15 +00:00
Richard Purdie ab191d21e2 bitbake: Update to bitbake 1.8 branch head
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3892 311d38ba-8fff-0310-9ca6-ca027cbcb966
2008-03-03 22:01:45 +00:00
Richard Purdie b12795d49f bitbake fetcher: Make invalid url errors clearer
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3262 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-11-27 22:23:04 +00:00
Richard Purdie 62a3ee38ed bitbake: Convert fetcher code to using bb.utils locking functions
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3223 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-11-24 16:49:12 +00:00
Richard Purdie e13102cd66 bitbake: Update SRCREV fetcher code to cope better with multiple SCM packages
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3145 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-11-13 23:03:21 +00:00
Richard Purdie d9a71e8814 bitbake: fetcher init - clear data's internal expand cache since the cached SRCREV value will be wrong
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3104 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-11-08 10:27:16 +00:00
Richard Purdie e9b7bc6914 bitbake: fetcher updates - Add BB_GENERATE_MIRROR_TARBALLS option to allow disabling tarball creation to speedup git fetches, improve srcrev handling to remove some hacks and remove the need for external workarounds
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@3101 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-11-07 23:49:55 +00:00
Richard Purdie 388c2f80ff bitbake: Add hg fetcher (from bitbake upstream)
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2865 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-10-11 21:41:34 +00:00
Richard Purdie 899c451a73 bitbake: Sync with 1.8 upstream branch
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2689 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-09-05 07:48:15 +00:00
Richard Purdie d8bfa5c6ef bitbake: Sync with 1.8.8 release
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2513 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-20 07:48:43 +00:00
Richard Purdie 7611768e23 bitbake: Sync with 1.8 head. Adds locking to the fetcher to prevent parallel downloads, fixes key expansion issues and occasional missing dependency graph links
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2502 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-16 09:55:21 +00:00
Richard Purdie ce800d3aea bitbake: Sync with upstream 1.8 branch for fixes
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2484 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-12 23:06:49 +00:00
Richard Purdie 18026165c3 bitbake: Sync with upstream
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2480 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-11 22:42:15 +00:00
Richard Purdie 62f662256a bitbake fetch/__init__.py: Don't print double newlines
git-svn-id: https://svn.o-hand.com/repos/poky/trunk@2474 311d38ba-8fff-0310-9ca6-ca027cbcb966
2007-08-09 13:54:30 +00:00