I've been working on GStreamer a lot recently, and I'm slowly becoming comfortable with debugging problems with gdb. I assume this could be useful for other people, so this is part of one of a series on debugging GStreamer.

Using gdb to debug crashes

I recently had a problem with GStreamer crashing if I requested the "active" property of an input-selector pad after it had been removed from the input-selector (this is possible because I still held a reference to the pad). This is a completely trivial bug, but it would have been almost impossible to fix without gdb.

To figure out where and why it was crashing, I re-ran my program with gdb:

# without -args, gdb just runs the program and ignores arguments
gdb -args ~/workspace/webkit/WebKitBuild/Debug/bin/QtTestBrowser ~/workspace/tracktest/demo.html
# type "run" at the prompt

Then I got my program to crash again:

Program received signal SIGSEGV, Segmentation fault.# g_mutex_get_impl (mutex=0x130) at /build/buildd/glib2.0-2.32.3/./glib/gthread-posix.c:119

gdb will immediately give you a prompt at this point (which is why this is easy to debug). You can just type backtrace to get a backtrace (run set pagination 0 first if your backtrace is long and you want to be able to copy it).

#0  g_mutex_get_impl (mutex=0x130) at /build/buildd/glib2.0-2.32.3/./glib/gthread-posix.c:119#
#1  0x00007fffedc8f599 in #g_mutex_lock (mutex=<optimized out>) at /build/buildd/glib2.0-2.32.3/./glib/gthread-posix.c:208#
#2  0x00007fff88c5492f in gst_input_selector_is_active_sinkpad (pad=0x7fff3c026140, sel=0x0) at gstinputselector.c:1579#
#3  gst_selector_pad_get_property (object=0x7fff3c026140, prop_id=<optimized out>, value=0x7fffffffd880, pspec=<optimized out>)
    at gstinputselector.c:310#
#4  0x00007fffedf1893e in object_get_property (value=0x7fffffffd880, pspec=<optimized out>, object=0x7fff3c026140)
    at /build/buildd/glib2.0-2.32.3/./gobject/gobject.c:1289#
#5  g_object_get_valist (object=0x7fff3c026140, first_property_name=<optimized out>, var_args=0x7fffffffd8e8)
    at /build/buildd/glib2.0-2.32.3/./gobject/gobject.c:2014#
#6  0x00007fffedf18d97 in g_object_get (_object=0x7fff3c026140, first_property_name=0x7ffff6bca522 "active")
    at /build/buildd/glib2.0-2.32.3/./gobject/gobject.c:2104#
#7  0x00007ffff5ec7f7b in WebCore::VideoTrackPrivateGStreamer::notifyTrackOfActiveChanged (this=0x9df390)
    at /home/blong/workspace/webkit/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp:122#
#8  0x00007ffff5ec7b51 in WebCore::videoTrackPrivateActiveChangeTimeoutCallback (track=0x9df390)
    at /home/blong/workspace/webkit/Source/WebCore/platform/graphics/gstreamer/VideoTrackPrivateGStreamer.cpp:49#
#9  0x00007fffedc5491b in g_timeout_dispatch (source=0x7fff4407c380, callback=<optimized out>, user_data=<optimized out>)
    at /build/buildd/glib2.0-2.32.3/./glib/gmain.c:3882#
[...]

The backtrace starts at the point where it crashed and moves backwards up the stack. The way to read these is to start from the top, and go backwards until you find something suspicious. In my case, I started by assuming that g_mutex_lock was fine, since a bug in that would cause problems for every program that uses GLib. Line #2 is interesting though:

#2  0x00007fff88c5492f in gst_input_selector_is_active_sinkpad (pad=0x7fff3c026140, sel=0x0) at gstinputselector.c:1579

Looking at the parameters, sel is NULL (0x0). That's probably bad. Why is gst_selector_pad_get_property calling it with a NULL parameter?

I happen to be running GStreamer from git, but if you're using a package, you can get the source from your package manager. Looking at the documentation for input-selector, it's part of coreelements, which is in the main GStreamer package:

sudo apt-get source gstreamer1.0

Or if you want to grab it with git:

git clone git://anongit.freedesktop.org/git/gstreamer/gstreamer

We can find gstinputselector.c with find:

$ find gstreamer* -name gstinputselector.c
gstreamer/plugins/elements/gstinputselector.c
gstreamer/plugins/elements/gstinputselector.h

Or locate (you may need to run sudo updatedb beforehand to get an up-to-date index):

$ locate gstinputselector.c
/home/blong/gst/git/gstreamer/plugins/elements/gstinputselector.c

And looking at the source for gst_selector_pad_get_property, we see:

static void
gst_selector_pad_get_property (GObject * object, guint prop_id,
    GValue * value, GParamSpec * pspec)
{
  GstSelectorPad *spad = GST_SELECTOR_PAD_CAST (object);

  switch (prop_id) {
    case PROP_PAD_RUNNING_TIME:
      g_value_set_int64 (value, gst_selector_pad_get_running_time (spad));
      break;
    case PROP_PAD_TAGS:
      GST_OBJECT_LOCK (object);
      g_value_set_boxed (value, spad->tags);
      GST_OBJECT_UNLOCK (object);
      break;
    case PROP_PAD_ACTIVE:
    {
      GstInputSelector *sel;

      sel = GST_INPUT_SELECTOR (gst_pad_get_parent (spad));
      g_value_set_boolean (value, gst_input_selector_is_active_sinkpad (sel,
              GST_PAD_CAST (spad))); // crash on this line
      gst_object_unref (sel);
      break;
    }
    case PROP_PAD_ALWAYS_OK:
      GST_OBJECT_LOCK (object);
      g_value_set_boolean (value, spad->always_ok);
      GST_OBJECT_UNLOCK (object);
      break;
    default:
      G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
      break;
  }
}

It looks like the input-selector pad is calling a function on its parent, without checking if it's NULL. That's easy to fix (patch generated with git format-patch:

@@ -, +, @@
 selector is NULL.
 plugins/elements/gstinputselector.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
--- a/plugins/elements/gstinputselector.c
+++ a/plugins/elements/gstinputselector.c
@@ -307,9 +307,10 @@ gst_selector_pad_get_property (GObject * object, guint prop_id,
       GstInputSelector *sel;

       sel = GST_INPUT_SELECTOR (gst_pad_get_parent (spad));
-      g_value_set_boolean (value, gst_input_selector_is_active_sinkpad (sel,
-              GST_PAD_CAST (spad)));
-      gst_object_unref (sel);
+      g_value_set_boolean (value, sel &&
+          gst_input_selector_is_active_sinkpad (sel, GST_PAD_CAST (spad)));
+      if (sel)
+        gst_object_unref (sel);
       break;
     }
     case PROP_PAD_ALWAYS_OK:

And now we're done, the last step is to submit a bug report and the patch, and the problem is fixed.