Commit Graph

126 Commits

Author SHA1 Message Date
Richard Purdie 37624b9745 bitbake/fetch2: Update forcefetch and mirror handling to clean up, simplfy and bug fix the code
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 16:52:42 +00:00
Richard Purdie 681bcf4e6b bitbake/fetch2/git: use clonedir as ud.localfile too since the mirror tarball may not exist
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 10:13:29 +00:00
Richard Purdie 3308868267 bitbake/fetch2/git.py: Ensure that forcefetch operates in the correct directory for calling _contains_ref()
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:34:24 +00:00
Richard Purdie 917d3e9697 bitbake/fetch2/git: Fix localpath to point at the clone repo since we no longer always generate a mirror tarball and it isn't a good guide to fetcher success
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:13:44 +00:00
Richard Purdie 7202a77134 bitbake/fetch2: Use True instead of integer values
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:37 +00:00
Richard Purdie ab0dd13974 bitbake/fetch2: Rename Fetch class to FetchMethod
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie d08397ba4d bitbake/fetch2: Rewrite and improve exception handling, reusing core functions for common operations where possible
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:36 +00:00
Richard Purdie 5a52cb80d6 bitbake/fetch2: Define a sane localpath function and remove code duplication
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00
Richard Purdie 972eb5faba bitbake/fetch2: Move ud.localfile setup into urldata_init
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-07 09:06:35 +00:00
Richard Purdie e4c1e99f23 bitbake/fetch2/git: Switch to completely use bare clones
We no longer need index/workdir support in the mirror tree and it causes all
kind of reference naming problems.Simplifying the code to remove this and use
just bare clones addresses this problem.

We increase the "version" number on the mirror tarballs to reflect the change
and ensure older mirror tarballs are not used as they would break.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-06 11:44:12 +00:00
Saul Wold 192f631129 fetch2/git.py: Fix problem when fetching with no origin
When updating a git repo, its possible to not have an origin,
so fail gracefully with a debug message.

Signed-off-by: Saul Wold <sgw@linux.intel.com>
2011-02-05 14:14:21 -08:00
Richard Purdie 96810cade0 bitbake/fetch2/git: Ensure we prune origin to remove stale branches
Thanks to Bruce Ashfield for the tip.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-05 15:43:36 +00:00
Richard Purdie dfc5faf717 bitbake/fetch2/git: Ensure deleted branches upstream are removed, all branch tags are forced to the upstream revisions and all revisions and tags are always fetched
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-03 20:18:52 +00:00
Richard Purdie adfa6c40da bitbake/fetch2/git: Ensure target directory exists when copying files
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-02 17:46:37 +00:00
Yu Ke 0e837e6844 fetch2: Allow multiple src rev to be used in one src_uri
* SRC_URI format:
the SRC_URI are extended to allow multiple src rev:
name=<name1>,<name2>,...<name-n>
branch=<branch1>,<branch2>,...,<branch-n>
also SRCREV can be defined with
SRCREV_<name1> = xxxxx
SRCREV_<name2> = xxxxx

* FetchData extention
to support multiple src rev, several FetchData data are added:
- FetchData.names: list of name in SRC_URI,  one name per srcrev. name is the index of revision and branch
- FetchData.revisions: dictionary of name->revision.
- FetchData.branches: dictionary of name->branch.

For example, linux-yocto recipes becomes:
SRC_URI = "git://git.pokylinux.org/linux-yocto-2.6.37;protocol=git;branch=${KBRANCH},meta;name=machine,meta"

FetchData.names = ['machine', 'meta']
FetchData.revisions = { 'machine':xxxxx, 'meta':xxxxxx }
FetchData.branches = { 'machine':${KBRANCH}, 'meta':'meta'}

* generic revision handling extension
the related revision handling code in fetch2.__init__.py are changed accordingly. the major change is add name parameter to indicate which src rev to handling. originally there is one src rev per FetchData, so FetchData parameter is enough. now since one FetchData has multiple src rev, it is necessary to use FetchData + name to specifiy src rev.

* git extension
git fetcher are also revised to take advantage of the multiple src rev in FetchData. especially the download() method are enhanced to fetch multiple src rev.

* other fetcher (svn, hg, ...) does not support multiple src rev. they just sync the API to add name, and then simply ignore the name. no actually functional change

Signed-off-by: Yu Ke <ke.yu@intel.com>
2011-02-02 17:19:18 +00:00
Yu Ke affcfe468a git.py: Remove the source tree tarball archives
Since we're now always providing the git source control files it becomes
pointless to handle the tarballs of specific git revisions so drop this
part of the fetcher.

Signed-off-by: Yu Ke <ke.yu@intel.com>
2011-02-02 17:13:30 +00:00
Richard Purdie 05048e0f91 bitbake/fetch2/git: Add nocheckout option to disable checkouts and always add scm data to checkouts
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-02-02 17:11:41 +00:00
Yu Ke 3a3fc6b071 bb.fetch2: Add git unpack
The git download method clones the git repository to the local machine. The unpack process
can be optimised to be a local to local machine clone or a direct readtree operation to the
destination using git.will clone git repo to local, so git unpack can be simplified
to only checkouting the code to the work dir. For fullclone case, we also
need to manually copy all the ref info, which is needed by the later do_kernel_checkout().
Rather than use hardlinks, we reference the repository using alternatives since the
download directory may be on a different filesystem.

[Change to use -s by Richard Purdie]

Signed-off-by: Yu Ke <ke.yu@intel.com>
2011-02-02 17:05:48 +00:00
Yu Ke 029f8584d5 bitbake/fetch2: Instrument fetchers when making network access
Signed-off-by: Yu Ke <ke.yu@intel.com>
2011-01-25 12:32:50 +00:00
Yu Ke 733de7596c bb.fetch2: remove the obsolate Fetch.try_mirrors referrence
Fetch.try_mirrors is no longer exists, so the code is obsolate

Signed-off-by: Yu Ke <ke.yu@intel.com>
2011-01-25 12:25:47 +00:00
Yu Ke ef918a72de git.py: split download to download() and build_mirror_data()
the download is to fetch the source from URL, the build_mirror_data is
to create the mirror tar ball. the original go() method mix them together,
it is more clean to split them.

Signed-off-by: Yu Ke <ke.yu@intel.com>
2011-01-25 12:25:47 +00:00
Yu Ke e05918937c bb.fetch2: rename "go" with "download" to better reflect its functionality
no functional change

Signed-off-by: Yu Ke <ke.yu@intel.com>
2011-01-25 12:25:47 +00:00
Yu Ke 1589a1172f Fetcher: break the "SRCREVINACTION" deadlock
Current fetcher has annoying "SRCREVINACTION" deadlock,
which occurs when SRCREV=${AUTOREV}=@bb.fetch.get_srcrev():
get_srcrev()->setup_localpath()->srcrev_internal_helper()
->evaluate SRCREV->get_srcrev()

current fetcher resolve the deadlock by introducing a
"SRCREVINACTION" condition check. Althoguh it works, it is
indeed not clean.

This patch use antoehr idea to break the deadlock: break
the dependency among SRCREV and get_srcrev(), i.e. assign
a specific keyword "AUTOINC" to AUTOREV. when Fetcher meet
this keyword, it will check and set the latest revision to
urldata.revision. get_srcrev later can use the urldata.revision
for value evaluation(SRCPV etc). In this case, SRCREV no longer
depends on get_srcrev, and there is not deadlock anymore.

Signed-off-by: Yu Ke <ke.yu@intel.com>
2011-01-10 20:24:33 +00:00
Yu Ke 54e34f6e25 bb.fetch2.git.py: add git urldata_init
move the git specific urldata init from localpath to urldata_init
so that it can be called early

Signed-off-by: Yu Ke <ke.yu@intel.com>
2011-01-10 20:21:35 +00:00
Yu Ke c64b445f38 bb.fetch2: replace bb.fetch with bb.fetch2 in the bb.fetch
bb.fetch2 is copied from bb.fetch, and has many bb.fetch referrence.
Fix these referrence with bb.fetch2 referrence

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-10 18:05:07 +00:00
Yu Ke 4dccd92439 bitbake: copy bb.fetch to bb.fetch2 as initial code base for fetcher overhaul
Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2011-01-10 14:23:36 +00:00