Before this change:
a) Ownership and permissions of files copied from packages to
package-split could get lost during the copy process. This change
ensures they are preserved.
b) Ownership and permissions of directories could also get lost.
Most of the complexity in this patch is addressing this problem
ensuring newly created directories match the source ones being
copied.
c) There was no warning about directories being created but not
shipped by any package.
This patch fixes all of the above issues.
(From OE-Core rev: 6021e309e69d823e1467648aee12a32182945569)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Previously python dependencies were of the format "python(abi) = ..."
This format is not yet supportable within OE, so revert to a form
we know we can handle.
Introduce a change to package.bbclass that ensures it will causes the
sstate-cache's "package" to invalidate. Since pythondeps changed, the
output of rpmdeps changes, which causes the per-file dependency
information to change.... thus we need to invalidate the cache!
(From OE-Core rev: 30064a98dc9049db4a37f119d15fbb59aa3c8377)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #1486]
1. image.bbclass: In multilib_sanity_check, duplicated files except
those matching to MULTILIBRE_ALLOW_REP are allowed if they're equal.
2. pacakge.bbclass: Allow DEBIAN_NOAUTONAMES variable to be
automatically mapped in multilib case.
(From OE-Core rev: 66f9045e17c39f5c64a0699b72ea460a0701717a)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For packages that have files installed that aren't in a subdirectory,
the following build WARNING is emitted (this for initramfs-live-boot
as an example):
WARNING: For recipe initramfs-live-boot, the following files were
installed but not shipped in any package:
WARNING: init
The problem is that the filenames added to the 'seen' array are always
added with a path separator at the beginning of the filename, but when
the package dir is walked for comparison, any files at the top-level
will be missing the beginning path separator and the comparison will
fail despite the fact that the file was actually packaged. This
because the remainder between the dirname and the dvar base name is
used in the path join and in the case of files at the top-level, the
remainder is the empty string, where it should be '/' for comparison
purposes.
(From OE-Core rev: cb19503139b9102f02ba3e5be63d5e85f280f2ef)
Signed-off-by: Tom Zanussi <tom.zanussi@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #1388]
This change is needed to correctly add the dependencies for the do_package
task which bitbake is unable to automatically detect itself.
(From OE-Core rev: 0614b9aa62a46f81d334ca4230080cc707347f3c)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
For image type of recipes, we need to define do_package_write as its
do_build recrdeptask, which ensures that all packaging types listed in
PACKAGE_CLASSES will be built out.
[YOCTO 1370]
(From OE-Core rev: b68cd33673fdc17711d7b9fde9cc23b0d8498c17)
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
[YOCTO #1345]
The new variable ALL_MULTILIB_PACKAGE_ARCHS contains all the values of
PACKAGE_ARCHS for each multilib variants. The opkg backend now uses this
new value insteald of the PACKAGE_ARCHS to update the opkg indexes and
to generate the opkg configuration files. This allows the normal
packages and multilib packages may be installed into the same rootfs.
(From OE-Core rev: b774bf44ef004276da12a83ebd69715c00b596ac)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
in multilib case, the PACKAGE_DYNAMIC is overrided with multilib
prefix. Take multilib:lib64-perl as example. the "perl-module-*"
will become "lib64-perl-module-*"
the output_pattern in do_split_packages is designed to work with
PACKAGE_DYNAMIC, so it should be applied with the same logic, i.e.
overriding with multilib prefix. otherwise the do_split_package will
split incorrect files
this patch implements the mulitlib override logic for do_split_packages
We also need to rename the extra_depends to support multilib case
(from Dongxaio Xu).
(From OE-Core rev: a4bc86713892502aeefbbdb3053e8cf1e1fc0bdb)
Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Xu Dongxiao <dongxiao.xu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This patch adds the core multilib class which can be used along with a
parameter specifying the mutlilib to use in BBCLASSEXTEND.
The MLPREFIX variable is added and can be used in cases where its too
difficult to dynmaically work out where a mutltilib prefix is needed
to be added to a variable.
This includes:
* SHLIBSDIR and PACKAGE_ARCH fixes from Lianhao Lu.
* PACKAGE_DYNAMIC mapping from Yu Ke
* PACKAGE_INSTALL mapping from Yu Ke
* RPROVIDES mapping from Yu Ke
* TARGET_VENDOR fix from Mark Hatle
* Ignorning *-native-runtime dependnecies as well as *-native from Yu Ke
* Map PKG and ALLOW_EMPTY from Dongxiao Xu
* Ensure RCONFLICTS and PKG field dependencies are remapped (from Dongxiao Xu)
* Ensure PN and MLPREFIX are set at the same time to ensure consistent BPN values (Yu Ke)
(From OE-Core rev: 5d9453049915db48ec4b5972e12287417ebb61a2)
Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Xu Dongxiao <dongxiao.xu@intel.com>
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We switch to using os.lchown in order to avoid following a symlink.
We also now check if an item is a symlink, if so we avoid the
os.chmod as a symlink inherits the mode of it's target.
(From OE-Core rev: c64d075b3d367e6c76aafa17782585d026b1f31e)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Add a new function that is responsible for fixing directory and file
permissions, owners and groups during the packaging process. This will fix
various issues where two packages may create the same directory and end up
with different permissions, owner and/or group.
The issue being resolved is that if two packages conflict in their ownership
of a directory, the first installed into the rootfs sets the permissions.
This leads to a least potentially non-deterministic filesystems, at worst
security defects.
The user can specify their own settings via the configuration files
specified in FILESYSTEM_PERMS_TABLES. If this is not defined, it will
fall back to loading files/fs-perms.txt from BBPATH. The format of this
file is documented within the file.
By default all of the system directories, specified in bitbake.conf, will
be fixed to be 0755, root, root.
The fs-perms.txt contains a few default entries to correct documentation,
locale, headers and debug sources. It was discovered these are often
incorrect due to being directly copied from the build user environment.
The entries needed to match the base-files package have also been added.
Also tweak a couple of warnings to provide more diagnostic information.
(From OE-Core rev: 8c720efa053f81dc8d2bb604cdbdb25de9a6efab)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Value of USE_PR_SERV is either "1" or "0" looking at
settings in bitbake.conf
USE_PR_SERV = "${@[1,0][(bb.data.getVar('PRSERV_HOST',d,1) is None) or (bb.data.getVar('PRSERV_PORT',d,1) is None)]}"
So we compare the strings
(From OE-Core rev: 5f6179324ac3956ad87123005bfcab4e8f30e67b)
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
1. Added package_get_auto_pr to PACKAGEFUNCS to get the auto
incremented value(PRAUTO) from remote PR service.
2. use PKGV/PKGR for pkgdata which will be used by package_write_xxx.
3. Added supporting functions in prserv.bbclass.
(From OE-Core rev: b1f6120c04731d3d66f322ce550b0d223c118dab)
Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is a backport of the corresponding package.bbclass functionality
(which is needed by micro) from the openembedded tree.
(From OE-Core rev: 600dbb7cb384c2290af38b993a9bea3a4dfc4494)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The previous change used egrep instead of fgrep. We need to use fgrep because
there are expression like syntaxes in some file names, we need exact matches.
(From OE-Core rev: 0de88dc9aa30f29ec1ab5cc0c541c8be859392ab)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The debug directory before was below ${WORKDIR}. Unfortunately if
something was based on a git tree, it meant that "git" was the
directory name being preserved for usr/src/debug usage. The patch
moves to using "${WORKDIR}/.." as the base, to ensure that the
WORKDIR naming is used in usr/src/debug.
(From OE-Core rev: dbc752c75786b0985fbeb4986467ae01290f424a)
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Refactor the function to eliminate additional interations/walks of the
filesystem..
Elimiate multiple runs of the external 'file' command as well.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
We now support two styles of debug information generation, the '.debug' style,
which is the same as previously implemented. This style simply splits the
debug information and makes it available in the same general directory.
/bin/foo -> /bin/.debug/foo
The new 'debug-file-directory' style splits the debug information and places
it into the single debug-file-directory, /usr/lib/debug:
/bin/foo -> /usr/lib/debug/bin/foo.debug
Both also find and copy all referenced source code to a new /usr/src/debug
directory. This allows the -dbg files to be used for stand-a-lone debugging
on or off the target device.
File stripping is now handled as a seperate operation from file splitting.
This allows us to split the debug information, but also leave it in the
original file -- or prevent the debug information from being split.
Also enhance the comments within local.conf.sample to provide a better
understanding of the control the user has over debug file generation.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
When doing package split, we use hard link instead of copy, which can
save about 10% disk space when building poky-image-minimal.
If fail, it will fall back to the copyfile function.
[Updated by Richard to use os.link and avoid an exec() call per file]
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
Hard links were not being preserved in the move from the install image
-> package copy. Again they were being discarded in the package ->
packages-split copy as well.
By preserving the hard links we have the potential to save a ton of rootfs
space.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
A new default SUMMARY and DESCRIPTION was defined for any package that
previously did not contain one. This value is based on the original
SUMMARY_${PN} value.
The new default SUMMARY and DESCRIPTION is used as a basis for all of
the automatic summary and descriptions for the various package splits,
include ${PN}, ${PN}-dbg, ${PN}-dev, ${PN}-doc, and locales.
A recipe may also override any of the automatic summaries by simply
specifying the value.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Setting this flag disables the normal additions to RRECOMMENDS for a package
(useful when splitting dev / dbg packages).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com>
As discovered by Lianhao Lu and Kevin Tian:
"""
do_package checksum changes with switching the order In PACKAGE_CLASSES, e.g:
From
PACKAGE_CLASSES = "package_rpm package_ipk"
To
PACKAGE_CLASSES = "package_ipk package_rpm"
This is undesired since the order only matters for final rootfs generation. The actual
cause is related to how do_package is generated, which depends on when
package.bbclass is first brought in.
"""
Since we never need the EXPORT_FUNCTIONS funcationality for do_package we just drop
it which removes this problem and stablisised checksums.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit changes the sysroots path to be machine specific.
Changes includes:
1) STAGING_DIR_TARGET and STRAGING_DIR_HOST points to machine specific
paths.
2) task stamp files. Adding ${MACHINE} info into stamp files for
do_populate_sysroots and do_package tasks. Add a BB_STAMPTASK_BLACKLIST
to keep native, nativesdk, crosssdk, and cross-canadian stamp unchanged.
3) siteconfig path. Separate the site config path for different machines
to avoid one machine adopting the cache file of another machine.
4) sstate. Add machine name to sstate manifest file.
Change relocation code for sstate paths since sysroot is machine.
Keep native, nativesdk, crosssdk, and cross-canadian unchanged.
5) toolchain scripts. Change the environment path to point to machine
specific sysroots in toolchain scripts bbclass.
6) Relocate la files when populating to a different machine of the same
architecture.
7) Exclude STAGING_DIR_TARGET and STAGING_DIR_HOST parameter from sstate
siginfo since they contain ${MACHINE} information.
Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
package sstate requires a lock under sysroots/, which however may not be
created when sstate_setscene functions are executed and then causes failures.
here we make sure 'sysroots' created before do_package_setscene is executed.
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
If sstate was used to accelerate a build, the pseudo directory might not have
been created leading to subsequent task failures.
Also, sstate packages were not being installed under pseudo context meaning
file permissions could have been lost.
Fix these problems by creating a FAKEROOTDIRS variable which bitbake ensures
exists before running tasks and running the appropriate setscene tasks under
fakeroot context.
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
In order to resolve a host-contamination problem, we re-work the way that
autoconf and friends are invoked during the compilation of RPM.
This has a side effect of fixing another bug where RPM was being renamed
HOST_ARCH-HOST_OS-rpm. So we remove the "fixes" for that behavior as well.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Switch the per file dependency handing to passing the information to
rpm via the standard 'external' dependency scripting. This ensures that
the dependencies found by RPM exactly match the ones presented by
package.bbclass.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
The overall file list was being constructed with the wrong variable, it
should have had FLIST in the name.
Also it was possible to construct the system with some illegal variable
names. Names that would have include "[]". So translate these away...
Finally fix an issues where a filename with an _ could cause a package
variable conflict. Again translate this away..
Signed-off-by: Mark Hatle <mhatle@windriver.com>
Richard Purdie
Mark Hatle
Lianhao Lu
Tom Zanussi
Dongxiao Xu
Chris Larson
Yu Ke
Khem Raj
Phil Blundell
Koen Kooi
Paul Eggleton
Kevin Tian
Richard Purdie
Mark Hatle