HLS support

[anderdev]

Hi there,

It is not really an issue, but I could find another way to talk with you.

Does videojs-wavesurfer works with HLS live Stream?

Sorry to attack you like that, here where I work we have couple HLS videos on a monitoring page that I need to display its audio without unmute the videos.

thanks in advance,
Anderson Santos

[thijstriemstra]

hi @anderdev, have you tried https://github.com/videojs/videojs-contrib-hls in combination with this plugin?

[anderdev]

Not yet, but will right now....thanks.

[anderdev]

Hi @thijstriemstra,

Today together with a bunch of other things I had a chance to try it as per your suggestion.

Unfortunately, I might be doing something wrong...I create my videos through an iteration....working very well, but not with wavesurfer
like:

$.ajax({
                type: 'GET',
                url: '/spi/live-events/monitor',
                dataType:'json',
                success: function (data) {
                    $("#body").html(template(data));
                    for (i = 0; i < data.devices.length; ++i) {
                        devices.push(data.devices[i].id);
                        var player = videojs("video_"+data.devices[i].id,
                                {
                                    plugins: {
                                        wavesurfer: {
                                            src: "https://tvnzioslive01-i.akamaihd.net/hls/live/245926/tvnzhlsingest/Duke/playlist.m3u8",
                                            msDisplayMax: 10,
                                            waveColor: "grey",
                                            progressColor: "black",
                                            cursorColor: "black",
                                            hideScrollbar: false
                                        }
                                    }
                                });
                        // error handling
                        player.on('error', function(error) {
                            console.warn(error);
                        });
                        getNagiosInfo(data.devices[i].id);
                    }
                }
            });

my problem here is that on console I get this error:

videojs.wavesurfer.min.js:4 Uncaught TypeError: Object prototype may only be an Object or null: undefined

I dont know what am I doing wrong.

Looks like that my src is not been accepted, is that the correct way to use wavesurfer?

thanks very much for you help.

[thijstriemstra]

@anderdev could you try this with a development version (e.g. non-minified) of videojs.wavesurfer so we can see the exact stacktrace in the console?

[anderdev]

Hey @thijstriemstra , thanks very much for been helping me.

I just change to use the dev version and console still showing same error.

note that I changed src to my HLS stream.

[thijstriemstra]

@anderdev it looks like it cannot load WaveSurfer, are you sure it's included? Have you tried with the latest wavesurfer.js? Also make sure wavesurfer is loaded before this plugin.

[anderdev]

So, do we have an order here?

I will try:

1. wavesurfer
   2.videoJS
   3.handlebars

let you know the results.

[thijstriemstra]

@anderdev yea, see https://github.com/collab-project/videojs-wavesurfer#using-the-plugin. So videojs/wavesurfer/plugin(s)

[anderdev]

other way around...hehehe videojs, wavesurfer and handlebars.

thanks a lot @thijstriemstra .

[thijstriemstra]

and probably the HLS plugin before or after videojs.wavesurfer..

[thijstriemstra]

@anderdev any news?

[anderdev]

Hi buddy, not really...I have changed imports order and haven't had much success.

In the end, I get very busy with my Scrum Master role that I have put it aside for now...I might be able to be back on this during next sprint, let you know,.

thanks for asking.

[zdenham]

I have been trying to do this without any luck. Tried combining the two plugins but wavesurfer throws DOMException: Unable to decode audio data when you set the source to an HSL format audio even with the contrib-hsl plugin installed. I was wondering if it would be possible to do a live streaming like the microphone but with hsl metadata instead? I really don't know ¯\(ツ)/¯

[thijstriemstra]

With videojs-wavesurfer v2.0.0 I'm seeing the same error using:

var player = videojs('myAudio', {
    controls: true,
    autoplay: true,
    fluid: false,
    width: 600,
    height: 300,
    plugins: {
        wavesurfer: {
            src: 'https://d2zihajmogu5jn.cloudfront.net/bipbop-advanced/bipbop_16x9_variant.m3u8',
            msDisplayMax: 10,
            debug: true,
            waveColor: 'grey',
            progressColor: 'black',
            cursorColor: 'black',
            hideScrollbar: true
        }
    }
});

VIDEOJS: ERROR: Error decoding audiobuffer
The buffer passed to decodeAudioData contains an unknown content type.

When I comment out the wavesurfer plugin, e.g.:

var player = videojs('myAudio', {
    controls: true,
    autoplay: true,
    fluid: false,
    loop: false,
    width: 600,
    height: 300,
    plugins: {
        hls: {
            src: 'https://d2zihajmogu5jn.cloudfront.net/bipbop-advanced/bipbop_16x9_variant.m3u8',
            type: 'application/x-mpegURL',
            withCredentials: true
        },
        /*
        wavesurfer: {
            msDisplayMax: 10,
            debug: true,
            waveColor: 'grey',
            progressColor: 'black',
            cursorColor: 'black',
            hideScrollbar: true
        }*/
    }
});

it still gives an error:

Error: plugin "hls" does not exist

which is unexpected. Checking videojs.getPlugins in the browser console shows:

> videojs.getPlugins();
Object { plugin: t(), reloadSourceOnError: o(), wavesurfer: Wavesurfer() }

Not sure what that reloadSourceOnError is all about.. Maybe related to domain I'm running examples on (http://127.0.0.1:9999/examples/hls.html)? See https://github.com/videojs/videojs-contrib-hls#hosting-considerations

It's possible to save the stream to the local server using using ffmpeg:

ffmpeg -i https://d2zihajmogu5jn.cloudfront.net/bipbop-advanced/bipbop_16x9_variant.m3u8 -c copy -bsf:a aac_adtstoasc output.mp4

But this takes forever and generates a single mp4 file. Trying a different stream, e.g. https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8 still results in the same error. How do you even get the hls plugin to work on it's own?

[thijstriemstra]

Eventually got it to load using:

var player = videojs('myAudio', {
    controls: true,
    autoplay: true,
    fluid: false,
    loop: false,
    width: 600,
    height: 300,
    plugins: {
        /*hls: {
            src: 'https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8',
            type: 'application/x-mpegURL',
            withCredentials: true
        },*/
        /*
        wavesurfer: {
            msDisplayMax: 10,
            debug: true,
            waveColor: 'grey',
            progressColor: 'black',
            cursorColor: 'black',
            hideScrollbar: true
        }*/
    }
});

player.on('ready', function(error) {
    var url = 'https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8';
    console.log('Loading', url);
    player.src({
        src: url,
        type: 'application/x-mpegURL',
        withCredentials: true
    });
});

But then died with CORS error:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at ‘https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8’.
(Reason: Credential is not supported if the CORS header ‘Access-Control-Allow-Origin’ is ‘*’).

[jessefalzone]

I set up a jsfiddle to demonstrate the issue. I used unminified versions for easier debugging, and confirmed that swapping the order of this plugin and videojs-contrib-hls still produces these errors:

VIDEOJS: ERROR: Error decoding audiobuffer
Uncaught (in promise) DOMException: Unable to decode audio data

Maybe I'm off base, but it looks like videojs-wavesurfer calls WaveSurfer.load(url) directly.
The load method expects an audio URL, not a playlist, so maybe there's a way to grab the data coming from videojs-contrib-hls and use loadBlob instead. If not, this might be a problem for wavesurfer.js to solve.

[thijstriemstra]

@jessefalzone thanks for the feedback but you're not actually using their video.js plugin in that example (the plugins section of the config only lists wavesurfer but not hls).

See https://github.com/videojs/videojs-contrib-hls#how-to-use, e.g.:

// html5 for html hls
videojs(video, {html5: {
  hls: {
    withCredentials: true
  }
}});

// or

// flash for flash hls
videojs(video, {flash: {
  hls: {
    withCredentials: true
  }
}});

// or

var options = {hls: {
  withCredentials: true;
}};

videojs(video, {flash: options, html5: options});

but this doesn't work for me, always throws the plugin "hls" does not exist error. Does it work for you @jessefalzone ?

[jessefalzone]

@thijstriemstra you're welcome.

I believe it is working, since videojs-contrib-hls registers itself automatically. You can test this by commenting out the wavesurfer options

var player = videojs('my-video', {
	controls: true,
	plugins: {
		/*wavesurfer: {
			src: src,
			normalize: true,
			debug: true,
			barWidth: 3,
		},*/
	},
});

and the m3u8 video will play correctly (and then it will break if you remove the videojs-contrib-hls script). The hls plugin option appears to be optional.

[thijstriemstra]

That's weird, how would I specify any initialization options? video.js provides an API that allows me to detect if other plugins are in use and videojs-contrib-hls is currently not able to be detected using that API, making it hard to interact and work with that plugin.

So unfortunately it looks like that plugin cannot be used but this looks more promising, a pure JS library for HLS: https://github.com/video-dev/hls.js/

[jessefalzone]

I've changed your earlier example to the following (notice hls is nested in html5, not plugins):

var player = videojs('myAudio', {
    controls: true,
    autoplay: true,
    fluid: false,
    loop: false,
    width: 600,
    height: 300,
    html5: {
        hls: {
            src: 'https://bitdash-a.akamaihd.net/content/MI201109210084_1/m3u8s/f08e80da-bf1d-4e3d-8899-f0f6155f6efa.m3u8',
            type: 'application/x-mpegURL',
            withCredentials: true
        },
    },
    plugins: {
        wavesurfer: {
            msDisplayMax: 10,
            debug: true,
            waveColor: 'grey',
            progressColor: 'black',
            cursorColor: 'black',
            hideScrollbar: true
        }
    }
});

CORS headers will need to be correct though, but I'm not sure it's necessary to specify these options since it's able to use the source in the <source> tags.

At any rate, if you can't detect contrib-hls then I'm stuck :-) I'm not too familiar with the internals of videojs.

[thijstriemstra]

What about this API? https://github.com/video-dev/hls.js/blob/master/doc/API.md

If we can get that to play nice with wavesurfer.js it should be fairly easy to get it work in this plugin.

References:

• How to save video and audio? video-dev/hls.js#1322

[thijstriemstra]

See katspaugh/wavesurfer.js#1078 which basically has to be fixed before we can fix this ticket.

[thijstriemstra]

Figured out how to create a local playlist.m3u8 for local streaming.

Grab latest ffmpeg build from https://www.johnvansickle.com/ffmpeg/ and unpack it somewhere.

Create a test dir:

mkdir ffmpeg-hls
cd ffmpeg-fls

Create a convert.sh script containing:

export app=./ffmpeg-git-20171018-64bit-static/ffmpeg

# convert the input file to TS segments using the libx264 and aac encoder
# and create an M3U8 live playlist to use for live HLS source
${app} -i $1 -map 0 \
-c:v libx264 -c:a aac \
-f ssegment -segment_list playlist.m3u8 \
-segment_list_flags +live -segment_time 10 \
out%03d.ts

cat playlist.m3u8

Make it executable:

chmod +x convert.sh

And run the script on some mp4 file, e.g. source.mp4:

./convert.sh source.mp4

This will create a playlist.m3u8 file and related media files.

Now enable a local Apache server to serve the file. Add new mimetypes to /etc/apache2/mods-available/mime.conf:

AddType application/x-mpegURL .m3u8
AddType video/MP2T .ts

And add this line to your apache vhost:

Header set Access-Control-Allow-Origin "*"

Restart the Apache server.

Finally, make the file accessible for Apache:

mv ffmpeg-hls /var/www/html/

The stream is now available on http://localhost/ffmpeg-hls/playlist.m3u8

[thijstriemstra]

HLS is now supported in video.js (since v7.0):

Play HLS, DASH, and future HTTP streaming protocols with video.js, even where they're not natively supported. Included in video.js 7 by default! See the video.js 7 blog post

More at https://github.com/videojs/http-streaming

So this should already be possible.. testing..

[thijstriemstra]

I tested the multi example with this:

const audioPlayerOptions = {
    controls: true,
    autoplay: false,
    loop: false,
    muted: false,
    fluid: false,
    width: 600,
    height: 300,
    bigPlayButton: true,
    plugins: {
        wavesurfer: {
            backend: 'MediaElement',
            displayMilliseconds: false,
            debug: true,
            waveColor: '#3e5226',
            progressColor: 'black',
            cursorColor: 'black',
            interact: true,
            hideScrollbar: true
        }
    }
};

// create audio player
let audioPlayer = videojs('myAudio', audioPlayerOptions, function() {
    // load m3u8 file
    audioPlayer.src({
      src: 'https://d2zihajmogu5jn.cloudfront.net/bipbop-advanced/bipbop_16x9_variant.m3u8',
      type: 'application/x-mpegURL'
    });
});

And see this error:

 Security Error: Content at http://localhost:8080/examples/multi.html may not load data from blob:http://localhost:8080/81fa0ed5-0627-43e0-9122-8c0baa5044a5.

It seems video.js vhs implementation creates a blob url here:

this.mediaSourceUrl_ = window.URL.createObjectURL(this.masterPlaylistController_.mediaSource);

this.tech_.src(this.mediaSourceUrl_);

And passes it to the videojs-wavesurfer middleware. Unfortunately there is some Security Error when trying to load this blob:http://localhost:8080/81fa0ed5-0627-43e0-9122-8c0baa5044a5 Blob.

I'm probably not approaching this the right way, because it seems like a temporary blob url and the actual segment data is stored elsewhere in video.js.

I opened videojs/http-streaming#1103 asking for more info.

[Noahs98]

Sorry to bother, any news about this problem? I just bumped with this problem, wavesurfer.js cannot get the file while using m3u8 to load audio. The problem is similar to "Unfortunately there is some Security Error when trying to load this blob:http://localhost:8080/81fa0ed5-0627-43e0-9122-8c0baa5044a5 Blob.".