Every tutorial on how to use Python to control GDB (the GNU Debugger) starts with a line like "say you have hundreds of threads."
Well, I have hundreds of threads. But I also have hundreds of socket objects in the C++ program I'm looking at! I can't really afford to examine them one at a time and get any insight. I'm not even sure what I'm looking for in the deadlock situation I'm investigating. Python's a good language for this sort of exploratory data analysis. (I have not yet checked whether I can get pandas and matplotlib to work within GDB--- an adventure for another time.)
The documentation on using Python inside GDB can be found here: https://sourceware.org/gdb/onlinedocs/gdb/Python.html
You type python, enter some Python code, finish with end, and it gets executed by a Python interpreter embedded in the GDB. Just in case you have a burning need to do some bignum arithmetic:
(gdb) python
>print pow( 2, 104 )
>end
20282409603651670423947251286016
The gdb package provided by GDB contains an enormous breadth of functionality. You can define new commands, access types and values, decorate stack frames, even replace an existing C++ method.
Inside Python, you can call gdb.parse_and_eval to use GDB's ability to parse C++ syntax and evaluate it. This is usually the starting point--- "print the value of some variable." But what you get back is actually an object, and most of what I need is in that Values object. This does magic so that fields in an object show up as dictionary entries in the resulting Python object.
(gdb) python
>foo = gdb.parse_and_eval( "this->mutex_" )
>print foo['cleanupPolicy_']
>end
MutexBasic::Destroy
It turns out this doesn't always work the way you'd expect. While the documentation claims that types that can be converted to Python basic types will be, I experienced some very strange behavior trying to call max() and do subtraction on 64-bit values. I worked around this by explicitly converting to long.
Also. C++ strings failed to convert to Python strings:
(gdb) python
>foo = gdb.parse_and_eval( "*('Framework::PthreadThread' *)(this->thread_.ptr_)" )
>bar = foo['name_'].string()
>print bar[7:]
>end
Traceback (most recent call last):
File "", line 2, in <module>
gdb.error: Trying to read string with inappropriate type `String'.
Error while executing Python code.
This might just be a problem with the rather old compiler and debugger combination I happen to be using. A workaround is to cast to a char * (which happens to work with the way the GCC standard library implements std::basic_string.)
(gdb) python
>char_ptr = gdb.lookup_type( 'char' ).pointer()
>foo = gdb.parse_and_eval( "*('Framework::PthreadThread' *)(this->thread_.ptr_)" )
>bar = foo['name_'].cast( char_ptr ).string()
>print bar[7:]
>end
/GSReceiveThread
But, one thing that does work is incrementing a pointer by 1 to move to the next object; the Python wrapper overloads addition so that the "right thing" happens.
Here's the script I ended up using to walk an array of threads and an array of sockets in each thread, and print a few key stats neatly.
def sendSockets():
sockets = []
maxTime = 0
char_ptr = gdb.lookup_type( 'char' ).pointer()
ggss = gdb.parse_and_eval( 'GlobalGenericSocketState' )
for i in xrange( 0, ggss['dataSendEpollThreadCount_'] ):
thread = (ggss['dataSendEpollThreadArray_'] + i).dereference()
for j in xrange( 0, thread['socketCount_'] ):
socket = ( thread['socketArray_'] + j ).dereference().dereference()
client = socket['client_'].cast( char_ptr ).string()
sockets.append( (client, socket) )
maxTime = max( maxTime,
long( socket['lastEpollWaitRecvNs_'] ),
long( socket['lastEpollWaitSendNs_'] ) )
sockets.sort( key=lambda x:x[0] )
print len( sockets), "sockets"
for ( c, socket ) in sockets:
print "{0:20} {1:4} {2:15} lastWaitRecv {3:7} lastWaitSend {4:7}".format(
c, socket['fd_'], socket['socketStats_']['sendBytes_'],
( long( socket['lastEpollWaitRecvNs_'] ) - maxTime ) / 1000000000,
( long( socket['lastEpollWaitSendNs_'] ) - maxTime ) / 1000000000 )