Tags: linux, gstreamer
by Sanchayan Maity
Some time has passed since the GStreamer spring hackfest took place in Thessaloniki, Greece in the month of May. Second time attending the GStreamer hackfest and thought about summarizing some of my thoughts this time around.
Thanks
Before getting into the details, want to send out a thank you to:
- The GStreamer foundation for sponsoring the event as a whole
- Sebastian, Vivia and Jordan for making all the arrangements
- Asymptotic, for sponsoring my presence at the event
The event
It was good to see some familiar faces at the event, folks whom I had met at the previous hackfest and conference. Also nice when you finally meet people you have only conversed with online and get to put a face on the online persona you have been conversing with.
Work
Originally the plan was to work on adding stream multiplexing support to QUIC elements. However, missed pushing some of the work to GitLab which was on desktop and decided to work on that later.
HTTP Live Streaming (HLS)
A merge request for adding multi-variant playlist support with HLS has been pending review for a while. One of the features missing from that merge request was support for codec string generation when using MPEG-TS with H-264 and H-265. Decided to work on that.
H-264 or H-265 has what are known as stream-formats. H-264 or H-265 can be stream oriented or packet oriented. In the case of former, stream-format is said to be byte-stream, while in the case of latter, stream-format is said to be avc. For byte-stream, the required parameter sets are sent in-band with the video, but for avc in GStreamer, the video metadata is conveyed via an additional caps field named codec_data which can be considered as out-of-band. codec_data is only present when the video is packet oriented, that’s when stream-format is avc, this value represents an AVCDecoderConfigurationRecord structure.
GStreamer already has helper functions in codec utilities which can provide information like profile-level which are required for constructing codec strings. However, these helper functions require the existence of codec_data.
When using MPEG-TS as the container, the only possible stream-format is byte-stream with H-264 or H-265. In this case, one needs to parse the in-band information for getting information like profile-level or other video metadata. In Rust, there is the cros-codecs crate which has a parser module. Using this, it was easy to parse the in-band data and then generate the codec string required for HLS playlist.
Threadshare
Before the hackfest, had spend some time on understanding the threadshare elements. Met François Laignel at the hackfest who helped with clearing doubts I had with how some of the code was laid out in threadshare.
If you are interested in understanding about what makes the threadshare elements different, highly recommend going through the blog post here.
There was some end-of-stream handling missing with the threadshare, tcpclientsrc and udpsrc elements. Spend some time working on adding support for that, which has now been merged upstream.
Play
After the three days of hackfest, a day trip was planned to the Palace of Aigai.
GStreamer hackers & co. exploring the Palace of Aigai.
Conclusion
All in all, this hackfest turned out to be a productive and fun filled hackfest. Also, have to add that Greek cuisine is excellent and look forward to the next hackfest and visiting Thessaloniki/Greece again.