diff --git a/documentation/dev-manual/dev-manual-common-tasks.xml b/documentation/dev-manual/dev-manual-common-tasks.xml index f792c7694f..067b5dfcc2 100644 --- a/documentation/dev-manual/dev-manual-common-tasks.xml +++ b/documentation/dev-manual/dev-manual-common-tasks.xml @@ -1726,6 +1726,484 @@ so that there are some definite steps on how to do this. I need more detail her process. + +
+ Debugging With the GNU Project Debugger (GDB) Remotely + + + GDB allows you to examine running programs, which in turn help you to understand and fix problems. + It also allows you to perform post-mortem style analysis of program crashes. + GDB is available as a package within the Yocto Project and by default is + installed in sdk images. + See the "Reference: Images" appendix + in the Yocto Project Reference Manual for a description of these images. + You can find information on GDB at . + + + + For best results, install -dbg packages for the applications + you are going to debug. + Doing so makes available extra debug symbols that give you more meaningful output. + + + + Sometimes, due to memory or disk space constraints, it is not possible + to use GDB directly on the remote target to debug applications. + These constraints arise because GDB needs to load the debugging information and the + binaries of the process being debugged. + Additionally, GDB needs to perform many computations to locate information such as function + names, variable names and values, stack traces and so forth - even before starting the + debugging process. + These extra computations place more load on the target system and can alter the + characteristics of the program being debugged. + + + + To help get past the previously mentioned constraints, you can use Gdbserver. + Gdbserver runs on the remote target and does not load any debugging information + from the debugged process. + Instead, a GDB instance processes the debugging information that is run on a + remote computer - the host GDB. + The host GDB then sends control commands to Gdbserver to make it stop or start the debugged + program, as well as read or write memory regions of that debugged program. + All the debugging information loaded and processed as well + as all the heavy debugging is done by the host GDB. + Offloading these processes gives the Gdbserver running on the target a chance to remain + small and fast. + + + + Because the host GDB is responsible for loading the debugging information and + for doing the necessary processing to make actual debugging happen, the + user has to make sure the host can access the unstripped binaries complete + with their debugging information and also be sure the target is compiled with no optimizations. + The host GDB must also have local access to all the libraries used by the + debugged program. + Because Gdbserver does not need any local debugging information, the binaries on + the remote target can remain stripped. + However, the binaries must also be compiled without optimization + so they match the host's binaries. + + + + To remain consistent with GDB documentation and terminology, the binary being debugged + on the remote target machine is referred to as the "inferior" binary. + For documentation on GDB see the + GDB site. + + +
+ Launching Gdbserver on the Target + + + First, make sure Gdbserver is installed on the target. + If it is not, install the package gdbserver, which needs the + libthread-db1 package. + + + + As an example, to launch Gdbserver on the target and make it ready to "debug" a + program located at /path/to/inferior, connect + to the target and launch: + + $ gdbserver localhost:2345 /path/to/inferior + + Gdbserver should now be listening on port 2345 for debugging + commands coming from a remote GDB process that is running on the host computer. + Communication between Gdbserver and the host GDB are done using TCP. + To use other communication protocols, please refer to the + Gdbserver documentation. + +
+ +
+ Launching GDB on the Host Computer + + + Running GDB on the host computer takes a number of stages. + This section describes those stages. + + +
+ Building the Cross-GDB Package + + A suitable GDB cross-binary is required that runs on your host computer but + also knows about the the ABI of the remote target. + You can get this binary from the the Yocto Project meta-toolchain. + Here is an example: + + /usr/local/poky/eabi-glibc/arm/bin/arm-poky-linux-gnueabi-gdb + + where arm is the target architecture and + linux-gnueabi the target ABI. + + + + Alternatively, the Yocto Project can build the gdb-cross binary. + Here is an example: + + $ bitbake gdb-cross + + Once the binary is built, you can find it here: + + tmp/sysroots/<host-arch>/usr/bin/<target-abi>-gdb + + +
+ +
+ Making the Inferior Binaries Available + + + The inferior binary (complete with all debugging symbols) as well as any + libraries (and their debugging symbols) on which the inferior binary depends + need to be available. + There are a number of ways you can make these available. + + + + Perhaps the easiest way is to have an 'sdk' image that corresponds to the plain + image installed on the device. + In the case of core-image-sato, + core-image-sato-sdk would contain suitable symbols. + Because the sdk images already have the debugging symbols installed, it is just a + question of expanding the archive to some location and then informing GDB. + + + + Alternatively, Yocto Project can build a custom directory of files for a specific + debugging purpose by reusing its tmp/rootfs directory. + This directory contains the contents of the last built image. + This process assumes two things: + + The image running on the target was the last image to + be built by the Yocto Project. + The package (foo in the following + example) that contains the inferior binary to be debugged has been built + without optimization and has debugging information available. + + + + + The following steps show how to build the custom directory of files: + + Install the package (foo in this case) to + tmp/rootfs: + + $ tmp/sysroots/i686-linux/usr/bin/opkg-cl -f \ + tmp/work/<target-abi>/core-image-sato-1.0-r0/temp/opkg.conf -o \ + tmp/rootfs/ update + + Install the debugging information: + + $ tmp/sysroots/i686-linux/usr/bin/opkg-cl -f \ + tmp/work/<target-abi>/core-image-sato-1.0-r0/temp/opkg.conf \ + -o tmp/rootfs install foo + + $ tmp/sysroots/i686-linux/usr/bin/opkg-cl -f \ + tmp/work/<target-abi>/core-image-sato-1.0-r0/temp/opkg.conf \ + -o tmp/rootfs install foo-dbg + + + +
+ +
+ Launch the Host GDB + + + To launch the host GDB, you run the cross-gdb binary and provide + the inferior binary as part of the command line. + For example, the following command form continues with the example used in + the previous section. + This command form loads the foo binary + as well as the debugging information: + + $ <target-abi>-gdb rootfs/usr/bin/foo + + Once the GDB prompt appears, you must instruct GDB to load all the libraries + of the inferior binary from tmp/rootfs as follows: + + $ set solib-absolute-prefix /path/to/tmp/rootfs + + The pathname /path/to/tmp/rootfs must either be + the absolute path to tmp/rootfs or the location at which + binaries with debugging information reside. + + + + At this point you can have GDB connect to the Gdbserver that is running + on the remote target by using the following command form: + + $ target remote remote-target-ip-address:2345 + + The remote-target-ip-address is the IP address of the + remote target where the Gdbserver is running. + Port 2345 is the port on which the GDBSERVER is running. + +
+ +
+ Using the Debugger + + + You can now proceed with debugging as normal - as if you were debugging + on the local machine. + For example, to instruct GDB to break in the "main" function and then + continue with execution of the inferior binary use the following commands + from within GDB: + + (gdb) break main + (gdb) continue + + + + + For more information about using GDB, see the project's online documentation at + . + +
+
+
+ +
+ Profiling with OProfile + + + OProfile is a + statistical profiler well suited for finding performance + bottlenecks in both userspace software and in the kernel. + This profiler provides answers to questions like "Which functions does my application spend + the most time in when doing X?" + Because the Yocto Project is well integrated with OProfile, it makes profiling applications on target + hardware straightforward. + + + + To use OProfile, you need an image that has OProfile installed. + The easiest way to do this is with tools-profile in the + IMAGE_FEATURES variable. + You also need debugging symbols to be available on the system where the analysis + takes place. + You can gain access to the symbols by using dbg-pkgs in the + IMAGE_FEATURES variable or by + installing the appropriate -dbg packages. + + + + For successful call graph analysis, the binaries must preserve the frame + pointer register and should also be compiled with the + -fno-omit-framepointer flag. + In the Yocto Project you can achieve this by setting the + SELECTED_OPTIMIZATION + variable to + -fexpensive-optimizations -fno-omit-framepointer -frename-registers -O2. + You can also achieve it by setting the + DEBUG_BUILD + variable to "1" in the local.conf configuration file. + If you use the DEBUG_BUILD variable you will also add extra debug information + that can make the debug packages large. + + +
+ Profiling on the Target + + + Using OProfile you can perform all the profiling work on the target device. + A simple OProfile session might look like the following: + + + + + # opcontrol --reset + # opcontrol --start --separate=lib --no-vmlinux -c 5 + . + . + [do whatever is being profiled] + . + . + # opcontrol --stop + $ opreport -cl + + + + + In this example, the reset command clears any previously profiled data. + The next command starts OProfile. + The options used when starting the profiler separate dynamic library data + within applications, disable kernel profiling, and enable callgraphing up to + five levels deep. + + To profile the kernel, you would specify the + --vmlinux=/path/to/vmlinux option. + The vmlinux file is usually in the Yocto Project file's + /boot/ directory and must match the running kernel. + + + + + After you perform your profiling tasks, the next command stops the profiler. + After that, you can view results with the opreport command with options + to see the separate library symbols and callgraph information. + + + + Callgraphing logs information about time spent in functions and about a function's + calling function (parent) and called functions (children). + The higher the callgraphing depth, the more accurate the results. + However, higher depths also increase the logging overhead. + Consequently, you should take care when setting the callgraphing depth. + + On ARM, binaries need to have the frame pointer enabled for callgraphing to work. + To accomplish this use the -fno-omit-framepointer option + with gcc. + + + + + For more information on using OProfile, see the OProfile + online documentation at + . + +
+ +
+ Using OProfileUI + + + A graphical user interface for OProfile is also available. + You can download and build this interface from the Yocto Project at + . + If the "tools-profile" image feature is selected, all necessary binaries + are installed onto the target device for OProfileUI interaction. + + + + Even though the Yocto Project usually includes all needed patches on the target device, you + might find you need other OProfile patches for recent OProfileUI features. + If so, see the + OProfileUI README for the most recent information. + + +
+ Online Mode + + + Using OProfile in online mode assumes a working network connection with the target + hardware. + With this connection, you just need to run "oprofile-server" on the device. + By default, OProfile listens on port 4224. + + You can change the port using the --port command-line + option. + + + + + The client program is called oprofile-viewer and its UI is relatively + straightforward. + You access key functionality through the buttons on the toolbar, which + are duplicated in the menus. + Here are the buttons: + + Connect: Connects to the remote host. + You can also supply the IP address or hostname. + Disconnect: Disconnects from the target. + + Start: Starts profiling on the device. + + Stop: Stops profiling on the device and + downloads the data to the local host. + Stopping the profiler generates the profile and displays it in the viewer. + + Download: Downloads the data from the + target and generates the profile, which appears in the viewer. + Reset: Resets the sample data on the device. + Resetting the data removes sample information collected from previous + sampling runs. + Be sure you reset the data if you do not want to include old sample information. + + Save: Saves the data downloaded from the + target to another directory for later examination. + Open: Loads previously saved data. + + + + + + The client downloads the complete 'profile archive' from + the target to the host for processing. + This archive is a directory that contains the sample data, the object files, + and the debug information for the object files. + The archive is then converted using the oparchconv script, which is + included in this distribution. + The script uses opimport to convert the archive from + the target to something that can be processed on the host. + + + + Downloaded archives reside in the Yocto Project's build directory in + /tmp and are cleared up when they are no longer in use. + + + + If you wish to perform kernel profiling, you need to be sure + a vmlinux file that matches the running kernel is available. + In the Yocto Project, that file is usually located in + /boot/vmlinux-KERNELVERSION, where + KERNEL-version is the version of the kernel. + The Yocto Project generates separate vmlinux packages for each kernel + it builds. + Thus, it should just be a question of making sure a matching package is + installed (e.g. opkg install kernel-vmlinux. + The files are automatically installed into development and profiling images + alongside OProfile. + A configuration option exists within the OProfileUI settings page that you can use to + enter the location of the vmlinux file. + + + + Waiting for debug symbols to transfer from the device can be slow, and it + is not always necessary to actually have them on the device for OProfile use. + All that is needed is a copy of the filesystem with the debug symbols present + on the viewer system. + The "Launching GDB on the Host Computer" + section covers how to create such a directory with + the Yocto Project and how to use the OProfileUI Settings dialog to specify the location. + If you specify the directory, it will be used when the file checksums + match those on the system you are profiling. + +
+ +
+ Offline Mode + + + If network access to the target is unavailable, you can generate + an archive for processing in oprofile-viewer as follows: + + # opcontrol --reset + # opcontrol --start --separate=lib --no-vmlinux -c 5 + . + . + [do whatever is being profiled] + . + . + # opcontrol --stop + # oparchive -o my_archive + + + + + In the above example, my_archive is the name of the + archive directory where you would like the profile archive to be kept. + After the directory is created, you can copy it to another host and load it + using oprofile-viewer open functionality. + If necessary, the archive is converted. + +
+
+