342 lines
18 KiB
XML
342 lines
18 KiB
XML
<!DOCTYPE appendix PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
|
|
"http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
|
|
|
|
<appendix id='ref-bitbake'>
|
|
|
|
<title>Reference: Bitbake</title>
|
|
|
|
<para>
|
|
Bitbake a program written in Python which interprets the metadata
|
|
that makes up Poky. At some point, people wonder what actually happens
|
|
when you type <command>bitbake poky-image-sato</command>. This section
|
|
aims to give an overview of what happens behind the scenes from a
|
|
BitBake perspective.
|
|
</para>
|
|
|
|
<para>
|
|
It is worth noting that bitbake aims to be a generic "task" executor
|
|
capable of handling complex dependency relationships. As such it has no
|
|
real knowledge of what the tasks its executing actually do. It just
|
|
considers a list of tasks with dependencies and handles metadata
|
|
consisting of variables in a certain format which get passed to the
|
|
tasks.
|
|
</para>
|
|
|
|
<section id='ref-bitbake-parsing'>
|
|
<title>Parsing</title>
|
|
|
|
<para>
|
|
The first thing BitBake does is work out its configuration by
|
|
looking for a file called <filename>bitbake.conf</filename>.
|
|
Bitbake searches through the <varname>BBPATH</varname> environment
|
|
variable looking for a <filename class="directory">conf/</filename>
|
|
directory containing a <filename>bitbake.conf</filename> file and
|
|
adds the first <filename>bitbake.conf</filename> file found in
|
|
<varname>BBPATH</varname> (similar to the PATH environment variable).
|
|
For Poky, <filename>bitbake.conf</filename> is found in <filename
|
|
class="directory">meta/conf/</filename>.
|
|
</para>
|
|
|
|
<para>
|
|
In Poky, <filename>bitbake.conf</filename> lists other configuration
|
|
files to include from a <filename class="directory">conf/</filename>
|
|
directory below the directories listed in <varname>BBPATH</varname>.
|
|
In general the most important configuration file from a user's perspective
|
|
is <filename>local.conf</filename>, which contains a users customized
|
|
settings for Poky. Other notable configuration files are the distribution
|
|
configuration file (set by the <glossterm><link linkend='var-DISTRO'>
|
|
DISTRO</link></glossterm> variable) and the machine configuration file
|
|
(set by the <glossterm><link linkend='var-MACHINE'>MACHINE</link>
|
|
</glossterm> variable). The <glossterm><link linkend='var-DISTRO'>
|
|
DISTRO</link></glossterm> and <glossterm><link linkend='var-MACHINE'>
|
|
MACHINE</link></glossterm> environment variables are both usually set in
|
|
the <filename>local.conf</filename> file. Valid distribution
|
|
configuration files are available in the <filename class="directory">
|
|
meta/conf/distro/</filename> directory and valid machine configuration
|
|
files in the <filename class="directory">meta/conf/machine/</filename>
|
|
directory. Within the <filename class="directory">
|
|
meta/conf/machine/include/</filename> directory are various <filename>
|
|
tune-*.inc</filename> configuration files which provide common
|
|
"tuning" settings specific to and shared between particular
|
|
architectures and machines.
|
|
</para>
|
|
|
|
<para>
|
|
After the parsing of the configuration files some standard classes
|
|
are included. In particular, <filename>base.bbclass</filename> is
|
|
always included, as will any other classes
|
|
specified in the configuration using the <glossterm><link
|
|
linkend='var-INHERIT'>INHERIT</link></glossterm>
|
|
variable. Class files are searched for in a classes subdirectory
|
|
under the paths in <varname>BBPATH</varname> in the same way as
|
|
configuration files.
|
|
</para>
|
|
|
|
<para>
|
|
After the parsing of the configuration files is complete, the
|
|
variable <glossterm><link linkend='var-BBFILES'>BBFILES</link></glossterm>
|
|
is set, usually in
|
|
<filename>local.conf</filename>, and defines the list of places to search for
|
|
<filename class="extension">.bb</filename> files. By
|
|
default this specifies the <filename class="directory">meta/packages/
|
|
</filename> directory within Poky, but other directories such as
|
|
<filename class="directory">meta-extras/</filename> can be included
|
|
too. Adding extra content to
|
|
<glossterm><link linkend='var-BBFILES'>BBFILES</link></glossterm> is best
|
|
acheived through the use of Bitbake
|
|
<link linkend='usingpoky-changes-layers'>"layers"</link>.
|
|
</para>
|
|
|
|
<para>
|
|
Bitbake parses each <filename class="extension">.bb</filename> file in
|
|
<glossterm><link linkend='var-BBFILES'>BBFILES</link></glossterm> and
|
|
stores the values of various variables. In summary, for each
|
|
<filename class="extension">.bb</filename>
|
|
file the configuration + base class of variables are set, followed
|
|
by the data in the <filename class="extension">.bb</filename> file
|
|
itself, followed by any inherit commands that
|
|
<filename class="extension">.bb</filename> file might contain.
|
|
</para>
|
|
|
|
<para>
|
|
Parsing <filename class="extension">.bb</filename> files is a time
|
|
consuming process, so a cache is kept to speed up subsequent parsing.
|
|
This cache is invalid if the timestamp of the <filename class="extension">.bb</filename>
|
|
file itself has changed, or if the timestamps of any of the include,
|
|
configuration or class files the <filename class="extension">.bb</filename>
|
|
file depends on have changed.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='ref-bitbake-providers'>
|
|
<title>Preferences and Providers</title>
|
|
|
|
<para>
|
|
Once all the <filename class="extension">.bb</filename> files have been
|
|
parsed, BitBake will proceed to build "poky-image-sato" (or whatever was
|
|
specified on the commandline) and looks for providers of that target.
|
|
Once a provider is selected, BitBake resolves all the dependencies for
|
|
the target. In the case of "poky-image-sato", it would lead to
|
|
<filename>task-oh.bb</filename> and <filename>task-base.bb</filename>
|
|
which in turn would lead to packages like <application>Contacts</application>,
|
|
<application>Dates</application>, <application>BusyBox</application>
|
|
and these in turn depend on glibc and the toolchain.
|
|
</para>
|
|
|
|
<para>
|
|
Sometimes a target might have multiple providers and a common example
|
|
is "virtual/kernel" that is provided by each kernel package. Each machine
|
|
will often elect the best provider of its kernel with a line like the
|
|
following in the machine configuration file:
|
|
</para>
|
|
<programlisting><glossterm><link linkend='var-PREFERRED_PROVIDER'>PREFERRED_PROVIDER</link></glossterm>_virtual/kernel = "linux-rp"</programlisting>
|
|
<para>
|
|
The default <glossterm><link linkend='var-PREFERRED_PROVIDER'>
|
|
PREFERRED_PROVIDER</link></glossterm> is the provider with the same name as
|
|
the target.
|
|
</para>
|
|
|
|
<para>
|
|
Understanding how providers are chosen is complicated by the fact
|
|
multiple versions might be present. Bitbake defaults to the highest
|
|
version of a provider by default. Version comparisons are made using
|
|
the same method as Debian. The <glossterm><link
|
|
linkend='var-PREFERRED_VERSION'>PREFERRED_VERSION</link></glossterm>
|
|
variable can be used to specify a particular version
|
|
(usually in the distro configuration) but the order can
|
|
also be influenced by the <glossterm><link
|
|
linkend='var-DEFAULT_PREFERENCE'>DEFAULT_PREFERENCE</link></glossterm>
|
|
variable. By default files
|
|
have a preference of "0". Setting the
|
|
<glossterm><link
|
|
linkend='var-DEFAULT_PREFERENCE'>DEFAULT_PREFERENCE</link></glossterm> to "-1" will
|
|
make a package unlikely to be used unless it was explicitly referenced and
|
|
"1" makes it likely the package will be used.
|
|
<glossterm><link
|
|
linkend='var-PREFERRED_VERSION'>PREFERRED_VERSION</link></glossterm> overrides
|
|
any default preference. <glossterm><link
|
|
linkend='var-DEFAULT_PREFERENCE'>DEFAULT_PREFERENCE</link></glossterm>
|
|
is often used to mark more
|
|
experimental new versions of packages until they've undergone sufficient
|
|
testing to be considered stable.
|
|
</para>
|
|
|
|
<para>
|
|
The end result is that internally, BitBake has now built a list of
|
|
providers for each target it needs in order of priority.
|
|
</para>
|
|
</section>
|
|
|
|
<section id='ref-bitbake-dependencies'>
|
|
<title>Dependencies</title>
|
|
|
|
<para>
|
|
Each target BitBake builds consists of multiple tasks (e.g. fetch,
|
|
unpack, patch, configure, compile etc.). For best performance on
|
|
multi-core systems, BitBake considers each task as an independent
|
|
entity with a set of dependencies. There are many variables that
|
|
are used to signify these dependencies and more information can be found
|
|
found about these in the <ulink url='http://bitbake.berlios.de/manual/'>
|
|
BitBake manual</ulink>. At a basic level it is sufficient to know
|
|
that BitBake uses the <glossterm><link
|
|
linkend='var-DEPENDS'>DEPENDS</link></glossterm> and
|
|
<glossterm><link linkend='var-RDEPENDS'>RDEPENDS</link></glossterm> variables when
|
|
calculating dependencies and descriptions of these variables are
|
|
available through the links.
|
|
</para>
|
|
|
|
</section>
|
|
|
|
<section id='ref-bitbake-tasklist'>
|
|
<title>The Task List</title>
|
|
|
|
<para>
|
|
Based on the generated list of providers and the dependency information,
|
|
BitBake can now calculate exactly which tasks it needs to run and in what
|
|
order. The build now starts with BitBake forking off threads up to
|
|
the limit set in the <glossterm><link
|
|
linkend='var-BB_NUMBER_THREADS'>BB_NUMBER_THREADS</link></glossterm> variable
|
|
as long there are tasks ready to run, i.e. tasks with all their
|
|
dependencies met.
|
|
</para>
|
|
|
|
<para>
|
|
As each task completes, a timestamp is written to the directory
|
|
specified by the <glossterm><link
|
|
linkend='var-STAMPS'>STAMPS</link></glossterm> variable (usually
|
|
<filename class="directory">build/tmp/stamps/*/</filename>). On
|
|
subsequent runs, BitBake looks at the <glossterm><link
|
|
linkend='var-STAMPS'>STAMPS</link></glossterm>
|
|
directory and will not rerun
|
|
tasks its already completed unless a timestamp is found to be invalid.
|
|
Currently, invalid timestamps are only considered on a per <filename
|
|
class="extension">.bb</filename> file basis so if for example the configure stamp has a timestamp greater than the
|
|
compile timestamp for a given target the compile task would rerun but this
|
|
has no effect on other providers depending on that target. This could
|
|
change or become configurable in future versions of BitBake. Some tasks
|
|
are marked as "nostamp" tasks which means no timestamp file will be written
|
|
and the task will always rerun.
|
|
</para>
|
|
|
|
<para>Once all the tasks have been completed BitBake exits.</para>
|
|
|
|
</section>
|
|
|
|
<section id='ref-bitbake-runtask'>
|
|
<title>Running a Task</title>
|
|
|
|
<para>
|
|
It's worth noting what BitBake does to run a task. A task can either
|
|
be a shell task or a python task. For shell tasks, BitBake writes a
|
|
shell script to <filename>${WORKDIR}/temp/run.do_taskname.pid</filename>
|
|
and then executes the script. The generated
|
|
shell script contains all the exported variables, and the shell functions
|
|
with all variables expanded. Output from the shell script is
|
|
sent to the file <filename>${WORKDIR}/temp/log.do_taskname.pid</filename>.
|
|
Looking at the
|
|
expanded shell functions in the run file and the output in the log files
|
|
is a useful debugging technique.
|
|
</para>
|
|
|
|
<para>
|
|
Python functions are executed internally to BitBake itself and
|
|
logging goes to the controlling terminal. Future versions of BitBake will
|
|
write the functions to files in a similar way to shell functions and
|
|
logging will also go to the log files in a similar way.
|
|
</para>
|
|
</section>
|
|
|
|
|
|
<section id='ref-bitbake-commandline'>
|
|
<title>Commandline</title>
|
|
|
|
<para>
|
|
To quote from "bitbake --help":
|
|
</para>
|
|
|
|
<screen>Usage: bitbake [options] [package ...]
|
|
|
|
Executes the specified task (default is 'build') for a given set of BitBake files.
|
|
It expects that BBFILES is defined, which is a space separated list of files to
|
|
be executed. BBFILES does support wildcards.
|
|
Default BBFILES are the .bb files in the current directory.
|
|
|
|
Options:
|
|
--version show program's version number and exit
|
|
-h, --help show this help message and exit
|
|
-b BUILDFILE, --buildfile=BUILDFILE
|
|
execute the task against this .bb file, rather than a
|
|
package from BBFILES.
|
|
-k, --continue continue as much as possible after an error. While the
|
|
target that failed, and those that depend on it,
|
|
cannot be remade, the other dependencies of these
|
|
targets can be processed all the same.
|
|
-f, --force force run of specified cmd, regardless of stamp status
|
|
-i, --interactive drop into the interactive mode also called the BitBake
|
|
shell.
|
|
-c CMD, --cmd=CMD Specify task to execute. Note that this only executes
|
|
the specified task for the providee and the packages
|
|
it depends on, i.e. 'compile' does not implicitly call
|
|
stage for the dependencies (IOW: use only if you know
|
|
what you are doing). Depending on the base.bbclass a
|
|
listtasks tasks is defined and will show available
|
|
tasks
|
|
-r FILE, --read=FILE read the specified file before bitbake.conf
|
|
-v, --verbose output more chit-chat to the terminal
|
|
-D, --debug Increase the debug level. You can specify this more
|
|
than once.
|
|
-n, --dry-run don't execute, just go through the motions
|
|
-p, --parse-only quit after parsing the BB files (developers only)
|
|
-d, --disable-psyco disable using the psyco just-in-time compiler (not
|
|
recommended)
|
|
-s, --show-versions show current and preferred versions of all packages
|
|
-e, --environment show the global or per-package environment (this is
|
|
what used to be bbread)
|
|
-g, --graphviz emit the dependency trees of the specified packages in
|
|
the dot syntax
|
|
-I IGNORED_DOT_DEPS, --ignore-deps=IGNORED_DOT_DEPS
|
|
Stop processing at the given list of dependencies when
|
|
generating dependency graphs. This can help to make
|
|
the graph more appealing
|
|
-l DEBUG_DOMAINS, --log-domains=DEBUG_DOMAINS
|
|
Show debug logging for the specified logging domains
|
|
-P, --profile profile the command and print a report</screen>
|
|
|
|
</section>
|
|
|
|
<section id='ref-bitbake-fetchers'>
|
|
<title>Fetchers</title>
|
|
|
|
<para>
|
|
As well as the containing the parsing and task/dependency handling
|
|
code, bitbake also contains a set of "fetcher" modules which allow
|
|
fetching of source code from various types of sources. Example
|
|
sources might be from disk with the metadata, from websites, from
|
|
remote shell accounts or from SCM systems like cvs/subversion/git.
|
|
</para>
|
|
|
|
<para>
|
|
The fetchers are usually triggered by entries in
|
|
<glossterm><link linkend='var-SRC_URI'>SRC_URI</link></glossterm>. Information about the
|
|
options and formats of entries for specific fetchers can be found in the
|
|
<ulink url='http://bitbake.berlios.de/manual/'>BitBake manual</ulink>.
|
|
</para>
|
|
|
|
<para>
|
|
One useful feature for certain SCM fetchers is the ability to
|
|
"auto-update" when the upstream SCM changes version. Since this
|
|
requires certain functionality from the SCM only certain systems
|
|
support it, currently Subversion, Bazaar and to a limited extent, Git. It
|
|
works using the <glossterm><link linkend='var-SRCREV'>SRCREV</link>
|
|
</glossterm> variable. See the <link linkend='platdev-appdev-srcrev'>
|
|
developing with an external SCM based project</link> section for more
|
|
information.
|
|
</para>
|
|
|
|
</section>
|
|
|
|
</appendix>
|
|
<!--
|
|
vim: expandtab tw=80 ts=4 spell spelllang=en_gb
|
|
-->
|