BigBlueButton enable parallel recordings processing workers

Issue

Processing of BigBlueButton (later BBB) recordings can be very slow. Even if you selected MP4 as playback format in
/usr/local/bigbluebutton/core/scripts/presentation.yml

video_formats:
  # - webm
  - mp4

The problem is that, until BBB version 2.3 will be released, there’s no plan to configure how many workers of records processing can be run simultaneously.

There’s a little trick however you can use to add a workaround for this limit.

Solution

This solution is based on this howto on an issue on github. I’ve rearranged based on my experience also to support more than two recordings. I will keep here my solution to have it at my fingertips when I’ll need it again.

A little consideration not to overcommit your machine.

ffmpeg in bigbluebutton runs with multiple trheads, so you’ll consider every 4 core 1 running encoding process.

You can divide the processes of encoding passing a parameter by matching ID with a regexp. In this example is on the last digit.

rap-process-worker.rb -p "[0-4]$" /* encode recordings with ID finishing by 0 1 2 3 4 */
rap-process-worker.rb -p "[5-9]$" /* encode recordings with ID finishing by 5 6 7 8 9 */

In this way we can double the processing queue:

edit /usr/lib/systemd/system/bbb-rap-process-worker.service to have ExecStart as follows

ExecStart=/usr/local/bigbluebutton/core/scripts/rap-process-worker.rb -p "[0-4]$"

then duplicate the systemd service

cp /usr/lib/systemd/system/bbb-rap-process-worker.service /usr/lib/systemd/system/bbb-rap-process-worker2.service

and edit it to have ExecStart as follows

ExecStart=/usr/local/bigbluebutton/core/scripts/rap-process-worker.rb -p "[5-9]$"

edit /usr/lib/systemd/system/bbb-record-core.target and add bbb-rap-process-worker2.service in the “Wants=” row as follows

Wants=bbb-rap-archive-worker.service bbb-rap-sanity-worker.service bbb-rap-process-worker.service bbb-rap-process-worker2.service bbb-rap-publish-worker.service bbb-rap-events-worker.service

add bbb-rap-process-worker2.service to the monitored processes by /usr/bin/bbb-record, circa line 639

systemctl --no-pager status bbb-rap-archive-worker.service bbb-rap-sanity-worker.service bbb-rap-process-worker.service bbb-rap-process-worker2.service bbb-rap-publish-worker.service

let systemd know that there’s something new in its files

systemctl daemon-reload

and restart bbb-record-core.target

systemctl restart bbb-record-core.target

now you’ve doubled the processing capacity of your bigbluebutton service.

If you’re plenty of cores, you can always split the processing queue in more than 2 workers modifying this notes woth different workers with different ranges of regexp ie:

rap-process-worker.rb -p "[0-3]$" /* encode recordings with ID finishing by 0 1 2 3 */
rap-process-worker.rb -p "[4-6]$" /* encode recordings with ID finishing by 4 5 6 */
rap-process-worker.rb -p "[7-9]$" /* encode recordings with ID finishing by 7 8 9 */

and so on.

Feel free to drop a comment if you need some advice.

4 thoughts on “BigBlueButton enable parallel recordings processing workers

  1. Hello Mattia,
    I can not find /usr/lib/systemd/system/bbb-rap-process-worker.service file to edit in BBB 2.4 version.

    Do you have any idea or advice?

    1. You should create it.
      But this is old documentation, in 2.4 version I suggest to select webm format and the parallel processing will be true.

  2. Hi, Mattia.

    Thank God I found your post.

    I am responsible for the LMS of a large university. With the spread of the pandemic, the use of video conferencing has become more than mandatory for online classes.

    In my case, teachers are required to have their lessons recorded. As a result, we had a substantial increase in the demand for processing recordings at BBB.

    I haven’t had time to implement your solution in my installation yet, but it’s something promising.

    Thanks for sharing your knowledge.

    Ricardo

Leave a Reply to Ricardo Caiado Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.