sysmo-bts
/
generic-poky
9
0
Fork 0
Code Pull Requests Releases Activity

sstate.bbclass: Improve sstate_installpkg performance 

In a pathological case, lots of files to process, the sstate_installpkg
performance was very poor.  It interated over each file and ran 3
individual sed commands per file.  Changing this to keep iterating
but running only a single command took about 1/3 time time.

However, when looking at the corresponding sstate_hardcode_path
function, it was clear we could optimize this further.

Using the same encoding logic to specify only the minimumal sed
operation necessary, and using xargs to avoid the os.system call the
install step was able to be performed in 13% of the original time.

Example timing numbers for perl:

3m7s original code
1m20s single sed, but interating
0m26s using xargs and limited sed

(From OE-Core rev: d9f655753fbdc8cbd8e705577430fed4f23732b3)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This commit is contained in:
Mark Hatle 2012-05-10 18:13:38 -05:00 committed by Richard Purdie
parent 31122b03bf
commit b85bbc52d3
1 changed files with 21 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

28
meta/classes/sstate.bbclass
View File

@ -174,18 +174,29 @@ def sstate_installpkg(ss, d):
bb.build.exec_func('sstate_unpack_package', d)
# Fixup hardcoded paths
#
# Note: The logic below must match the reverse logic in
# sstate_hardcode_path(d)
fixmefn = sstateinst + "fixmepath"
if os.path.isfile(fixmefn):
staging = d.getVar('STAGING_DIR', True)
staging_target = d.getVar('STAGING_DIR_TARGET', True)
staging_host = d.getVar('STAGING_DIR_HOST', True)
fixmefd = open(fixmefn, "r")
fixmefiles = fixmefd.readlines()
fixmefd.close()
for file in fixmefiles:
os.system("sed -i -e s:FIXMESTAGINGDIRTARGET:%s:g %s" % (staging_target, sstateinst + file))
os.system("sed -i -e s:FIXMESTAGINGDIRHOST:%s:g %s" % (staging_host, sstateinst + file))
os.system("sed -i -e s:FIXMESTAGINGDIR:%s:g %s" % (staging, sstateinst + file))
if bb.data.inherits_class('native', d) or bb.data.inherits_class('nativesdk', d) or bb.data.inherits_class('crosssdk', d) or bb.data.inherits_class('cross-canadian', d):
sstate_sed_cmd = "sed -i -e 's:FIXMESTAGINGDIR:%s:g'" % (staging)
elif bb.data.inherits_class('cross', d):
sstate_sed_cmd = "sed -i -e 's:FIXMESTAGINGDIRTARGET:%s:g; s:FIXMESTAGINGDIR:%s:g'" % (staging_target, staging)
else:
sstate_sed_cmd = "sed -i -e 's:FIXMESTAGINGDIRHOST:%s:g'" % (staging_host)
# Add sstateinst to each filename in fixmepath, use xargs to efficiently call sed
sstate_hardcode_cmd = "sed -e 's:^:%s:g' %s | xargs %s" % (sstateinst, fixmefn, sstate_sed_cmd)
print "Replacing fixme paths in sstate package: %s" % (sstate_hardcode_cmd)
os.system(sstate_hardcode_cmd)
# Need to remove this or we'd copy it into the target directory and may
# conflict with another writer
os.remove(fixmefn)
@ -300,6 +311,9 @@ python sstate_cleanall() {
def sstate_hardcode_path(d):
# Need to remove hardcoded paths and fix these when we install the
# staging packages.
#
# Note: the logic in this function needs to match the reverse logic
# in sstate_installpkg(ss, d)
staging = d.getVar('STAGING_DIR', True)
staging_target = d.getVar('STAGING_DIR_TARGET', True)