generic-poky/bitbake/lib/bb/remotedata.py

117 lines
3.8 KiB
Python
Raw Permalink Normal View History

bitbake: tinfoil: rewrite as a wrapper around the UI Rewrite tinfoil as a wrapper around the UI, instead of the earlier approach of starting up just enough of cooker to do what we want. This has several advantages: * It now works when bitbake is memory-resident instead of failing with "ERROR: Only one copy of bitbake should be run against a build directory". * We can now connect an actual UI, thus you get things like the recipe parsing / cache loading progress bar and parse error handling for free * We can now handle events generated by the server if we wish to do so * We can potentially extend this to do more stuff, e.g. actually running build operations - this needs to be made more practical before we can use it though (since you effectively have to become the UI yourself for this at the moment.) The downside is that tinfoil no longer has direct access to cooker, the global datastore, or the cache. To mitigate this I have extended data_smart to provide remote access capability for the datastore, and created "fake" cooker and cooker.recipecache / cooker.collection adapter objects in order to avoid breaking too many tinfoil-using scripts that might be out there (we've never officially documented tinfoil or BitBake's internal code, but we can still make accommodations where practical). I've at least gone far enough to support all of the utilities that use tinfoil in OE-Core with some changes, but I know there are scripts such as Chris Larson's "bb" out there that do make other calls into BitBake code that I'm not currently providing access to through the adapters. Part of the fix for [YOCTO #5470]. (Bitbake rev: 3bbf8d611c859f74d563778115677a04f5c4ab43) Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2016-12-13 07:07:06 +00:00
"""
BitBake 'remotedata' module
Provides support for using a datastore from the bitbake client
"""
# Copyright (C) 2016 Intel Corporation
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 2 as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
import bb.data
class RemoteDatastores:
"""Used on the server side to manage references to server-side datastores"""
def __init__(self, cooker):
self.cooker = cooker
self.datastores = {}
self.locked = []
self.nextindex = 1
def __len__(self):
return len(self.datastores)
def __getitem__(self, key):
if key is None:
return self.cooker.data
else:
return self.datastores[key]
def items(self):
return self.datastores.items()
def store(self, d, locked=False):
"""
Put a datastore into the collection. If locked=True then the datastore
is understood to be managed externally and cannot be released by calling
release().
"""
idx = self.nextindex
self.datastores[idx] = d
if locked:
self.locked.append(idx)
self.nextindex += 1
return idx
def check_store(self, d, locked=False):
"""
Put a datastore into the collection if it's not already in there;
in either case return the index
"""
for key, val in self.datastores.items():
if val is d:
idx = key
break
else:
idx = self.store(d, locked)
return idx
def release(self, idx):
"""Discard a datastore in the collection"""
if idx in self.locked:
raise Exception('Tried to release locked datastore %d' % idx)
del self.datastores[idx]
def receive_datastore(self, remote_data):
"""Receive a datastore object sent from the client (as prepared by transmit_datastore())"""
dct = dict(remote_data)
d = bb.data_smart.DataSmart()
d.dict = dct
while True:
if '_remote_data' in dct:
dsindex = dct['_remote_data']['_content']
del dct['_remote_data']
if dsindex is None:
dct['_data'] = self.cooker.data.dict
else:
dct['_data'] = self.datastores[dsindex].dict
break
elif '_data' in dct:
idct = dict(dct['_data'])
dct['_data'] = idct
dct = idct
else:
break
return d
@staticmethod
def transmit_datastore(d):
"""Prepare a datastore object for sending over IPC from the client end"""
# FIXME content might be a dict, need to turn that into a list as well
def copy_dicts(dct):
if '_remote_data' in dct:
dsindex = dct['_remote_data']['_content'].dsindex
newdct = dct.copy()
newdct['_remote_data'] = {'_content': dsindex}
return list(newdct.items())
elif '_data' in dct:
newdct = dct.copy()
newdata = copy_dicts(dct['_data'])
if newdata:
newdct['_data'] = newdata
return list(newdct.items())
return None
main_dict = copy_dicts(d.dict)
return main_dict