- do not use the BB_URI_LOCALCOUNT database for computing revision
incremental numbers anymore
- sortable_revision now generates "AUTOINC+${latest_rev}"
- use one incrementing value rather than several
- PV becomes 0.1+gitAUTOINC+deadbeefdecafbad_decafbaddeadbeef
- remove all localcount code and simplify the fetcher
- this patch addresses the following proposal:
http://lists.linuxtogo.org/pipermail/bitbake-devel/2012-November/003878.html
(Bitbake rev: 61cf01c5c236b4218f40cfae7c059c2b86765dbd)
Signed-off-by: Constantin Musca <constantinx.musca@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Doc cleanup, no functional change.
(Bitbake rev: 5161a84f5dcfe748382a5073349bf10ed21641f9)
Signed-off-by: Robert P. J. Day <rpjday@crashcourse.ca>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wget commands for check and resume were
mixed-up, leading to the following issues:
1. long running "NOTE: Preparing runqueue"
reason: objects were downloaded, not spidered on the mirror
2. Failing network test in Build Appliance, because wget 1.14
(in use in BA) will fail if a file already exists.
During the network connectivity test, index.php file was
actually downloaded, not spidered (checked for existence on
yoctoproject.org website), leading to wget failure.
(Bitbake rev: d7a5185cae975eaca50a9785c6605e895dc7bb51)
Signed-off-by: Cristian Iorga <cristian.iorga@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The last changes introduced an error in some of the logic. Add brackets
to clarify the meaning of the expression and fix certain build failures.
(Bitbake rev: 87aea65bd5d553bd0495b0f1efe6d41d0bb2810f)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently there is code which uses FILESDIR in unpack to ensure
parent directories are created, leading to differing behaviour depending on
which search path is used to locate the directory.
This change standardises the code and takes the data from the fetcher in
question meaning we can standardise the code and deprecate FILESDIR.
(Bitbake rev: 1cccb3bd01ed82e4978acfef0fda1bd797eef72a)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently wildcard matches end up working by FILESDIR being defined
in the metadata to a default of "." in FILESPATH which is hacky at best.
This patch adds the behaviour into the fetcher so its at least slightly
more explicit.
(Bitbake rev: 07b5f84133ac79aac4e939ea5f24390ad7f940a5)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It is a common mistake to use http:// and protocol=git when attempting
to fetch from a git repository using the http protocol; if this is
detected then throw an error explaining that you need to use git:// with
protocol=http instead.
(Bitbake rev: 5bc4930c1638db16bcd5f9c8cfc4081f9ffc192b)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add "-d" option to cpio since it is useful:
-d
--make-directories
Create leading directories where needed.
[YOCTO #3137]
(Bitbake rev: a78f9ded7896432b107f34c0bb608b389fdb676a)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
bitbake-selftest is failing due to directories not being created. This adds in an
appropriate mkdir so the tests can complete. Presumably in general OE use, something
else is ensuring the parent directory is created.
(Bitbake rev: 1270a07713e2a6c6e6fadcc61b785aebc99ae17b)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Don't print the full exception in the initial warning - if we later
succeed in fetching the file from a mirror, we won't usually need the
details (which are in the fetch log if they are needed); otherwise the
full error will be printed when the fetch operation fails. Also adjust
the conditional block so that we don't print another warning just
mentioning we're going to try mirrors.
* Call logger.error() so that with knotty the full log is not printed
* Provide an explanation around the lines we print for easily updating
the checksums in the recipe. We don't want users to be just blindly
updating the recipe in case of a transient failure or deliberately
altered remote file.
(Bitbake rev: 2793413106c925b06783beb7413aa87cbcf246c3)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Most of the time we don't need to see the fetch command; the fetch log
includes the command as a debug message in any case, so omit it. Also
adjust the way command output is printed (we don't need stderr/stdout
labelled, and print "no output" instead of "output:\nNone" when there is
no output.
(Bitbake rev: a75505a52e4da918222100221f79e8a658f90446)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add in misssing space between the parameters. Reported by
Jate Sujjavanich <Jate.Sujjavanich@myfuelmaster.com>.
(Bitbake rev: 55382f0aac84b8f81cad0b82053c0b8295c33e54)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code in the CVS fetcher is elderly and there are simpler ways of
using the data store. This updates to use the modern APIs.
(Bitbake rev: 78eee8c70a80997293df99475153aed0b2ad0a17)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The localdata variable was removed, fix up a lost reference to this.
(Bitbake rev: 02ccc1396005ce0b7a2150a5ce12b723df21d464)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The implementation of NoMethodError and MalformedUrl was broken - if you
just set self.args in an exception class to a string it treats it as a
list and then fails later on with a TypeError due to the number of
arguments not matching up.
This nasty exception during exception handling was breaking the normal
exception flow (fixed separately), which meant that if you had a
malformed URL or invalid protocol in SRC_URI you would get the
following:
ERROR: Command execution failed: Traceback (most recent call last):
File "/home/user/poky/poky/bitbake/lib/bb/command.py", line 84, in runAsyncCommand
self.cooker.updateCache()
File "/home/user/poky/poky/bitbake/lib/bb/cooker.py", line 1207, in updateCache
if not self.parser.parse_next():
File "/home/user/poky/poky/bitbake/lib/bb/cooker.py", line 1694, in parse_next
logger.error('Unable to parse %s', value.recipe,
AttributeError: 'exceptions.TypeError' object has no attribute 'recipe'
A specific fix for [YOCTO #2977].
(Bitbake rev: 9d4150d99051d24ff218e8a43664ceaf524b19c7)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This ensures that if all a MIRRORS entry does is add a slash, this does
not result in a circular loop.
Fixes [YOCTO #3073].
(Bitbake rev: 57055d337a2c9997a6e5d5bdabaec396e3e128e9)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Unpack the ".rpm" binary package (only .src.rpm in the past)
* Unpack the .deb and .ipk binary package, their unpack commands are the same.
* This is useful for binary package recipe.
[YOCTO #1592]
(Bitbake rev: de7ceb9459574f33920ccc06255b533434f0ec25)
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If you have foo and foo.git in GITDIR, the two can end up being confused
by git with some horrible union of the two being cloned. This adds
a workaround to avoid this happening until git 1.7.9.2 onwards is
common enough for this to be removed. We use a symlink to hide
the directories we don't want git to know about.
(Bitbake rev: bbf1f6fe594c721a296ca09ee7c583d4a205c591)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When a fetch failure occurs for a local file, this patch ensures we print the
locations searched making it easier for the user to debug the problem.
(Bitbake rev: a461adbc5f09b41c771a7603370f6f2d1299ae8e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* in form which can be copied into a recipe without modification
* like oe-classic did since:
http://git.openembedded.org/openembedded/commit/?id=68abc465559a68e9201c9525be3c4acc6297eaed
* it shows them in right form when they are missing completely, but in
more verbose form when different
* it needs to print that only when checksums were requested, e.g.
fetching from sstate mirror sets both md5mismatch and sha256mismatch,
but your checksums shouldn't be shown
(Bitbake rev: 9eb34fc866775fd8310759a0111f232a9dc98981)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If a file:// mirror is being used, the fetcher will create a symlink to the
local file. However, if the local file gets removed, that link will be dead,
and os.path.exists() returns False in that case, so it tries and fails to
recreate the link. Now we unlink such a dead link if it exists.
(Bitbake rev: 229ed3857e826e3e215e843cb51f729c1e13ed37)
Signed-off-by: Christopher Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Without the new exception, when the system is configured to use premirrors,
but not allow network access (via BB_NO_NETWORK), when a recipe was lacking a
checksum the wrong error message(s) were being generated.
Instead of complaining about trying to perform network access, if the system
was able to find the item in the premirror, it should inform the user of the
lack of checksums, and the two SRC_URI fields they should use to update their
recipe.
(Bitbake rev: cb10e9c03a3f96d94e27e18330009616dde5e2b3)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This enhances the fetcher to allow preservation of the path component in urls
like: file://xxx/yyy/somefile.patch.
(Bitbake rev: e49a656a499355a5c6e7eb00bf5b8f1795e8dddb)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
this allows wget fetcher to store the downloaded file in a specified custom filename in ${DL_DIR}
Exmaple:
SRC_URI = "https://edc.intel.com/Download.aspx?id=6190;downloadfilename=LIN_IEMGD_1_14_GOLD_2443.tgz"
This fixes bug:
[YOCTO #2570]
(Bitbake rev: ceb5871007f221c4d86a7bee421d4dd8d9100aaf)
Signed-off-by: Nitin A Kamble <nitin.a.kamble@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This problem occurs when fetching a different revision of the same source. Which mean every time you update a bzr package.
Using branch sets the pull location, and are the preferred way of cloning/branching a repository in bzr.
(Bitbake rev: 877a04d0b3cea9d5dbdf3c54fe0feb54cb997dda)
Signed-off-by: Martin Ertsaas <mertsas@cisco.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* it was send in v1 of proto -> protocol changes but then wasn't in V2
http://patchwork.openembedded.org/patch/31617/
where warning about proto= was moved to shared __init__
(Bitbake rev: b2017f493ab730d804ae44ec5a168d464626d046)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
As well, if "proto" is used, get the associated value as "protocol"
(Bitbake rev: 53e6b630f0463d2d07cdaa9c9eb36794dc9b6b69)
Signed-off-by: Andrei Gherzan <andrei@gherzan.ro>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This also changes to use -t 2 -T 30 as the defaults which are more sane for
a modern fetcher and already specified in OpenEmbedded metadata.
(Bitbake rev: bf0e5dddf0f63cdb0648fb6d872af5ceef6fbfb0)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This brings the cvs fetcher more into line with the others and allows consistent usage
of the FETCHCMD variable and option handling.
(Bitbake rev: 390ad59739356422852e976fa246375abc6aba08)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
svn changed working checkout formats between 1.6 and 1.7. Its convoluted to
detect what format a given working copy is in so the simplest solution is simply
to run "svn upgrade" within the working copy.
The base svn command variable is relocated slightly to enable this new code to
work effectively.
(Bitbake rev: ebd3ecdb5f3c6d3fe1fad27cbed4d80f4277e92e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
'git remote prune' at this location does not make much sense because
the following 'git remote rm' will prune stale and non-stale branches.
The 'prune' can cause trouble because it will access the network
bypassing the no-network code in bitbake. When this operation fails and
throws an exception, the next command (--> 'git remote rm') will be
skipped. This in turn, will make all the following operations fail,
because they assume that the remote does not exist yet.
(Bitbake rev: 2ba23df5fad4b94d38a6aed97f7822226d72eb89)
Signed-off-by: Enrico Scholz <enrico.scholz@sigma-chemnitz.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
People are using regexps in the url type field so we need to preserve
this bitbake behaviour. To address the issues with https:// urls mapping
badly to file:// urls we anchor the regexp if its not already anchored.
There should be no expressions in the wild which would break with this
change.
(Bitbake rev: ce0579dc256251e523c6330641f98b9f5a0e5761)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If '*' does end up in mirror urls accidently, some strange things
can break since supports_checksum() looks for this, ud.localpath can
then get ignored and this can lead to empty directories being downloaded
"successfully". '*' is a special case for file urls only at this point
so remove any entries that accidentlly make it in through url mapping.
(Bitbake rev: 1369bec2404d942acc3618a8d005ec6868dcfd41)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When writing mirror specifications, the current regexp syntax can be awkward
and hard to get it to do what you want. For example, extracting the 'basename'
of a repository:
PREMIRRORS = "git://.*/([^/]+/)*([^/]*) git://somewhere.org/somedir/\\2;protocol=file"
can now become:
PREMIRRORS = "git://.*/.* git://somewhere.org/somedir/BASENAME;protocol=file"
which is much clearer. A MIRRORNAME substitution is also added which contains
an encoded form of both host and path. One of the problems with the existing
regexp syntax is you couldn't access HOST information from PATH and vice-versa
which is an issue this patch also addresses.
Tests for the new syntax are also added.
(Bitbake rev: c6b1acbad7b3d2698530eb8b5249adb4ab95da21)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
When we build the mirror urls, its possible an error will occur. If it
does, it should just mean we don't attempt this mirror url. The current
code actually aborts *all* the mirrors, not just the failed url.
This patch catches and logs the exception allowing things to continue.
(Bitbake rev: c35cbd1a1403865cf4f59ec88e1881669868103c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If -l is specified and the source and destination are not on a common
filesystem an error occurs. The -l option is however the default for
git for local paths which the fetcher already now ensures in the
file:// case.
We can therefore safely drop the -l option.
(Bitbake rev: 3aeb268b2aaab4bb8b1cfff1450e0b76aa8ce855)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
A file:// url should use "clone -l" to greatly speed
up the clone in the case of a kernel when it is local.
(Bitbake rev: 2bab2cc3ffe67ee2a308074a6e4c2c7be5636d2f)
Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For mirror mapping, we never use regexps for the type component of
the url. Doing so causes various slightly bizarre behaviour such as
https:// urls being translated to files:// urls which we have no handler
for. This patch forces the type matches to be direct, not regexp based
and gives the expected bahvriour.
(From Poky rev: 695d8aca0999d2d61970b990e83c3132ba6f12cb)
(Bitbake rev: 604df1b25cf114e083f52917df2df64e01279c25)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Using ud.localpath as a basename when it points at a directory causes
problems. The supports_checksum() method gives a good indication of whether
ud.localpath can be used in the way we need.
(From Poky rev: 933ec8a44634e33f92f6f76de3a34094c3d63aa6)
(Bitbake rev: dcd79ae20ab2c72c3312b2251c2b6dc4cabe988e)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We only consider mirror tarballs when the source and target urls are of
differing types. We also should clear all url paramters when handling
mirror tarballs.
(From Poky rev: da140b8b0b3dda5429f9eee68829ef5247cdfe12)
(Bitbake rev: 7619dcad29a6c2405b15a8fbadfa11e81b399ae1)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This means that parameters in the source expression are used as part of the
match. Parameters in the destination are used explicitly in the final
url.
(From Poky rev: c465cb0c5c927dd41d96ad6d6fa1566349574bb7)
(Bitbake rev: 3c468ac3ed6d045561afce19b85ae9dd18d87cea)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Currently we only consider one pass through the mirror list. This doesn't
catch cases where for example you might want to setup a mirror of a mirror
and allow multiple redirection. There is no reason we can't support this
and the patch loops through the list recursively now.
As a safeguard, it will stop if any duplicate urls are found, hence
avoiding circular dependency looping.
(From Poky rev: 0ec0a4412865e54495c07beea1ced8355da58073)
(Bitbake rev: e585730e931e6abdb15ba8a3849c5fd22845b891)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Constantin Musca
Robert P. J. Day
Cristian Iorga
Richard Purdie
Paul Eggleton
Robert Yang
Martin Jansa
Christopher Larson
Mark Hatle
Nitin A Kamble
Martin Ertsaas
Andrei Gherzan
Enrico Scholz
Jason Wessel