Further to my previous post, Video goodness with Python and GStreamer, I’ve now mastered playback of video along with audio using Python and GStreamer. This is my code to play back an MPEG2 transport stream:
# declare our pipeline and GST elements
self.pipeline = gst.Pipeline(“mypipeline”)
self.src = gst.element_factory_make(“filesrc”, “src”);
self.src.set_property(“location”, “file.mpeg”)
self.demux = gst.element_factory_make(“ffdemux_mpegts”, “demux”)
self.queue1 = gst.element_factory_make(“queue”, “queue1″)
self.queue2 = gst.element_factory_make(“queue”, “queue2″)
self.deinterlace = gst.element_factory_make(“ffdeinterlace”, “deinterlace”)
self.vdecode = gst.element_factory_make(“mpeg2dec”, “vdecode”)
self.adecode = gst.element_factory_make(“mad”, “adecode”)
self.vsink = gst.element_factory_make(“xvimagesink”, “vsink”)
self.asink = gst.element_factory_make(“alsasink”, “asink”)
# add elements to the pipeline
self.pipeline.add(self.src)
self.pipeline.add(self.demux)
self.pipeline.add(self.queue1)
self.pipeline.add(self.queue2)
self.pipeline.add(self.vdecode)
self.pipeline.add(self.deinterlace)
self.pipeline.add(self.adecode)
self.pipeline.add(self.vsink)
self.pipeline.add(self.asink)
# we can’t link demux until the audio and video pads are added
# we need to listen for ‘pad-added’ signals
self.demux.connect(‘pad-added’, self.on_pad_added)
# link all elements apart from demux
gst.element_link_many(self.src, self.demux)
gst.element_link_many(self.queue1, self.vdecode, self.deinterlace, self.vsink)
gst.element_link_many(self.queue2, self.adecode, self.asink)
def on_pad_added(self, element, src_pad):
caps = src_pad.get_caps()
name = caps[0].get_name()
# link demux to vdecode when video/mpeg pad added to demux
if name == ‘video/mpeg’:
sink_pad = self.queue1.get_pad(’sink’)
elif name == ‘audio/mpeg’:
sink_pad = self.queue2.get_pad(’sink’)
else:
return
if not sink_pad.is_linked():
src_pad.link(sink_pad)
def startPlayback(self):
# start playback
self.pipeline.set_state(gst.STATE_PLAYING)
Naturally you begin by declaring your pipeline and elements (for an introduction to GStreamer, see my previous post linked above and the link to Jono Bacon’s post included in that post).
Then you link your ‘filesrc’ to your demux element (variable depending on the type of content you’re playing).
Now, you cannot link anything to your demux element until you’ve started playback – this is because your demux element hasn’t yet looked at the data its being given, so it doesn’t know whether it should have audio and/or video pads. So you need a local function to handle the ‘pad-added’ signal which is sent when, you guessed it, a pad is added to an element – we only care about this signal when it relates to the demux element, so we connect that signal from that element to a local function. In this function we first need to determine what type of pad has been added – audio, video, or something else. In my case, because I’m playing MPEG, I know that I’m waiting for audio/mpeg and video/mpeg pads. The precise pads you need to wait for will depend on the content you’re playing.
When each pad is added, you link both the video and audio pads to separate ‘queue’ elements. Queue elements are essentially buffers and you need these for video and audio playback to work at the same time. You can link the audio queue element to your audio decoder and the decoder to your audio sink during initialisation – you don’t need to wait until you can link in the demux element. Likewise for the video queue element.
The equivalent if you want to launch playback from the command-line would be:
Things become simpler if you’re using a ‘decodebin’ or ‘playbin’ element rather than specifying your elements individually (which is what you’ll do if you want to play more than one type of content). In this case with a playbin you can simply add that one element to your pipeline, after giving it a URI to play, without worrying about what the content is.
# declare our pipeline and GST elements
self.pipeline = gst.Pipeline(“mypipeline”)
self.pbin = gst.element_factory_make(“playbin”, “pbin”);
self.pbin.set_property(“uri”, “file:///home/user/file.mpeg”)
# add elements to the pipeline
self.pipeline.add(self.pbin)
def startPlayback(self):
# start playback
self.pipeline.set_state(gst.STATE_PLAYING)
And on the command line:
You’re probably wondering why I went through all the complicated code at the beginning of this post, when you can just use a playbin as above? Well, if you are only going to be playing one type of content you can make initialisation of GStreamer quicker and more efficient if you specify your elements manually rather than asking GStreamer to detect which elements it needs and then set them up. If you are in the position of being able to specify your elements manually, you should do so. If not, then you should aim to use a decodebin element to automatically detect the decoders needed for the content and specify the src and sink element(s) manually.
I haven’t given an example of using a decodebin, because I couldn’t make it work (GStreamer continues to be pretty buggy – you may well find that you need to use development versions of plugins or even core libraries in order to make things work as they should).
Edit: I’ve put together a quick, fairly simplified PlayBin example for those wanting something downloadable they can play with. playbin-example.py.
