Could HLS Benefit Audio Too?

Could HLS benefit audio too?

This is the future of Podcasting, where we ponder what awaits the podcasters of today.

From the school of podcasting, here's Dave Jackson.

And from the Audacity to Podcast, here's Daniel J.

Lewis.

Daniel, future of podcasting.

Episode 61, we're going to talk about HLS was so much fun.

On the last episode.

We said, hey, let's do a round two on this.

But this time you kind of wanted to talk a little bit about HLS and audio because last time we were talking about video.

Yeah, because HLS can also benefit audio.

Now its primary benefit is for video.

And just a quick rehash, what HLS does is it basically splits a media file into lots of little pieces and then makes all of those little pieces accessible through a playlist.

And then the benefit to that is that it's very easy to switch between those pieces for different quality versions, different media versions, like even switching between audio and video very seamlessly.

It's also much easier to stitch things into there.

Think about trying to put in dynamic content into a video and the rendering power that would be necessary because you have to re render the video to do that, and that takes processing.

You can't do that kind of thing on the fly.

But you can put in just very quickly something additional in that playlist.

That's all of those little files stitched together.

So there are huge benefits to video.

And while the storage requirements are a little bit higher, the bandwidth requirements are actually lower.

But there's also potential benefit to audio.

And after we finished our episode last time, I thought, oh man, we didn't even talk about audio.

But then thinking more about it since then, I realized this could be a whole other subject of how audio podcasting could benefit.

Because if you want to follow the misreported news, Adam Curry says podcasting is only audio, which isn't the way he said it, by the way, just clarity.

He said he wouldn't mind if we decided podcasting was only audio.

And I wouldn't mind that either.

Yeah, well, isn't one of the things it's doing does is it still doing like transcoding, where you have like a, a super high version and a mid version and a I have no bandwidth version?

You know, you could.

And that's the nice thing about HLS is that it is a structure that can do a lot of beneficial things.

If you split it up, let's just assume you're splitting it up into one second segments.

So just like on YouTube, if you are watching a video on YouTube and your Internet connection goes a little bit wonky, or you're on cellular or something like that, you'll notice the quality decreases.

But it doesn't always decrease immediately.

When the connection goes a little weird because it's buffered ahead a little bit.

It switches when it realizes, oh, your connection isn't strong enough to support these files.

So we're going to just seamlessly switch you over to the lower quality version or the other way around, where even if you're watching a video and maybe you're watching it in 360p and you want to jack it up to 4k, you press a little button to change the quality and force it up to 4k.

That is using HLS to be able to easily jump to that same segment in that playlist of files, but now from the 4K folder essentially.

So think about that.

For audio 2, we have alternate enclosures in audio.

And alternate enclosures are where you have access to the full file, but in different versions or the full episode, but in different formats.

Like an MP3 file, a low quality MP3, or maybe even an Ogg Vorbis or something like that.

Lower quality, maybe even a video.

But that's still built around the idea of delivering the entire file in that quality.

Whereas HLS is let's deliver little bits at a time of whatever quality level you want so that you can easily switch to a different quality level.

Or you could switch to a different format too, because HLS can work for audio.

So imagine you're watching a video and then you decide, well, I want to switch to just listening to it.

Well, HLS could handle that switchover, whether that's you switch browser tabs and so you're not watching the video actively, you don't have picture in picture enabled, anything like that.

Or maybe you're watching a video on the phone and you lock the phone so the video view of the video stops.

Now you're just listening to it.

HLS could handle switching that over to the audio right where you were without having to do byte range requests.

It's requesting a specific point in that playlist of one second files.

Yeah, we guess the different apps might have a new option, much like YouTube where you can pick what level of quality you want.

That might be a thing.

Then if we're using this for audio where people can say, oh, you know what, it's audio.

Especially when you get outside the US we're so used to just having bandwidth like water here, you just turn on the faucet and Hear some bandwidth, but you get outside the US and that's not the case and you got to pay more.

So you might say, look, it's just audio.

I just need, you know, the lower version.

It's fine.

And going back to give me that OG Vorbis, which I believe, wasn't that a villain in like Star Wars Episode 4?

He was a James Bond villain.

That's it.

Well, think about the interface improvements this could offer.

So you gave that example of international bandwidth.

Think of just someone traveling somewhere and maybe while they're at home on their WI fi, their phone is playing the high resolution video version.

And then, then when they leave their home WI fi, their phone could pop up a little notification to say, you're now on mobile data.

Would you like to switch to audio only?

And some people might then prefer that so they're not missing any part of the episode, they're just dynamically switching to what they need.

Now that is with the assumption that the entire video didn't already download or buffer while they were on wifi, which that can sometimes happen.

Sometimes it doesn't happen because some of these video files can be really big.

And so some of the apps actually only buffer a certain amount.

Like for a while, Apple podcasts would only buffer.

I think it was 50 megabytes at a time.

I can't remember the exact amount, but it was something like that.

Even with an audio only episode, you could see that the line that showed how much had buffered stopped after a certain point, like far into the future.

And then when you got to that point, then it would move forward beyond that.

And it was somewhere around 50 megabytes, I think.

Yeah, that's the thing because I know, like I download all my stuff, but if everything does shift more to a video, that would not be the default setting because that'll fill up your phone real quick.

That would be interesting to see the different things that would be added.

There are some other potential audience benefits with this too.

Like think about if this whole ecosystem was built around hls.

That is one of the biggest frustrations I've had when I switch a podcast app.

You've switched podcast apps before, right?

I have, yeah.

I've multiple times, yeah.

Have you exported and imported the OPML file?

I have.

And what's interesting is you will see in the features where it'll say like OPML import, export and what I have found in some cases most of them can export, not all of them can import.

And then you're like, oh, you go to.

And you're like, well, here, where's.

Here's the file.

So not all of them import.

And then either a somebody's doing a bad job of exporting or somebody's doing a bad job of importing.

Because there are times also when you try to bring them over and it just sad trombones everywhere, you know, and you're like, well, I'll guess I'll just manually put these back in.

So, you know, and then sometimes it works brilliantly.

It all depends on which app you're moving to and which one you're doing.

But, yeah, I've tried it.

What about you?

I've done it only a couple of times.

And one of the frustrations that I had was that it didn't bring over my position in the episode.

So if I had a partially played episode, it would not move that over, I think.

I can't remember if it even moved over the uncompleted episodes that I had.

It might have.

I don't remember that it's been so long, but I do remember, of course, that it didn't sync the playback position.

Now, right now, certainly we could have something that's added to the open OPML that would include playback position for any unplayed episodes that could be added to an OPML spec that podcast apps would support.

But there's the chance, with the rise of dynamic content insertion and especially dynamic ads, there is the chance that even if you simply saved a time code that wouldn't necessarily return you to the exact spot in the episode.

But I believe, and this is where my knowledge of HLS becomes foggy, because I don't know this for sure, so I could be misrepresenting this, But I believe that you can pinpoint a position in HLS that is not tied directly to the time code in the case of dynamically inserted content.

So, like, look at it this way.

If we say since HLS is playlist driven, if it knows that in the playlist you're at point number 100, we'll just say that's 100 seconds in.

And it knows in the playlist that is this portion of the audio, even if there's some dynamic audio inserted before it, or dynamic video that might be labeled as dynamic instead of 1, 2, 3, 4, 5, 6, and so on.

But it's just, it's in that position in the playlist.

Maybe the way that it resumes your position in the playlist is basically looking at the ID of where you were instead of the time code of where you were.

I could be misrepresenting that.

So please don't quote me on that.

Please don't get angry.

Anyone out there if I got that wrong.

But if we had that ability, then think about how that could tie in now.

How many people actually affected by that?

Not many.

Yeah, it really only comes out of those times when you move from one app to the other.

One of the things I love right now about pocketcast is I can listen on my computer and then basically pick up my phone, walk to the kitchen, and it will pick up right where I left off.

And in fact, in some cases, it's a little bit like overcast, where it'll back up about three seconds.

So when you hit play, you kind of get a quick little review of what they were talking about and then you're into new content.

So that would be cool if it would work across different apps.

If you move in my travels, that whole process has been clunky, but, you know, and then that's.

You have, you know, the guy in a basement that's making this app and nothing wrong with that person.

But there's only so many people to squash the bugs.

Because if you think about that, that's not a feature that you do a lot like volume or speed up or speed down or whatever.

So it's one of those, like, oh, yeah, we'll get to that.

You know, So I totally understand that.

There are some potential privacy concerns, though, along with this, because there is the benefit that it would be easier to track how much was downloaded of a file and played if it's only getting one second at a time.

Not like one second ahead of you, but it would still buffer.

But still it is a kind of streaming data to you.

So there is that benefit of the publishers being able to see how much is streamed.

And.

But that does also come with a privacy concern, because as those chunks are downloaded, if they're being downloaded directly by the device, let's say a smartphone, then the IP address and some kind of identifiable information is being included with that.

So that means potentially, okay, worst case here, someone starts downloading something at home.

So their home IP address is registered.

They keep listening to it, streaming it while they're in the car.

So now their cellular phone provider IP address is registered.

So now we know, oh, that's a Verizon customer.

And then as they travel in different places, that IP address will shift a little bit as their signal changes from tower to tower.

I'm not quite sure how the mobile networks handle that, but there is the potential that the IP address can change, potentially tying them around to a certain area so we could see, oh, every time they're mobile, they always stay within this metropolitan area, or they're always going to.

From this metropolitan area to this one and then back to this one.

And then when they're at work, they're on a different IP address and different network, so that can be registered.

Then if they're playing the same episode as, they then go to the coffee shop.

So now, basically, just by tracking their IP address alone in their HLS segment requests, you'd be able to see their home IP address, their cell phone network provider, their possible work, and where they go for recreation.

That starts to get.

I know advertisers are like, yes, I want that.

That's one of the other biggest benefits, I think, is advertisers now know if they heard my advertisement or not, because they can tell.

And I believe that's probably one of the biggest pushes for this is, hey.

Because it always makes me cringe when I hear, well, the podcasting space can't exist without advertising.

And I'm like, pretty sure it did for about, you know, I get it that you can't make a living on that.

But there are other things besides advertising to bring in money.

But that's one of the biggest, I would think.

And I kind of want to see that happen just because it's always like, well, what advertisers need is this.

And then we kind of give it to them and then we prove, hey, look, this is really working.

And we're still kind of waiting for some of that big dollars to come in, and it'd be interesting to see if, okay, like, we can prove somebody actually heard this, if the money would then actually flow in or not, because it's like, come on, the water's warm.

We've been telling you that for years.

Unfortunately, though, or maybe fortunately, depending on which side of the issue you're on.

I'm not so sure that advertisers could even trust that, though, because, yes, we could.

See, we'll say blip number five is an ad in the HLS playlist, and we can see how many people downloaded that little blip.

But does that mean they listen to it?

No, because if you're on wifi, there's the potential, depending on how the app is configured, there's the potential that the app will buffer everything in advance.

Basically, what we have right now, progressive download, it's progressively downloading, except instead of a single file, it's progressively downloading the entire HLS playlist.

And therefore you still can't know.

Just because that little section was downloaded doesn't mean it was played.

Unless the app prevents a lot of pre buffering.

That's what I think a lot of the advertisers really want is they don't want the apps pre buffering stuff too far in advance.

They basically want true streaming, just streaming exactly the amount that you're playing right now.

So if you lose your connection, playback stops right then.

And that's not a good user experience.

No, especially when every kid in the neighborhood starts playing video games at 3:15.

You're like, wait, where'd all the bandwidth go?

Yeah, that could be interesting.

I mean, they're doing that now because I hear very hyper local ads on different shows that are not made in my area.

So I'm like, okay, well somebody had to somehow go put an ad at this point.

And at that point they then check my phone and go, oh, he's in Ohio.

Let's use, you know, so and so's fun car lot, come down and buy a car kind of guy.

That's always kind of creepy.

And that's a whole other.

You ever notice how that discussion.

I don't want to derail us, but you ever notice how the whole attribution discussion kind of just evaporated?

Like we're all going like, hey, is that invading privacy?

And then it just kind of was like, yeah, well, whatever, here we go.

Well, and there could be some other user facing benefits to using HLS as well.

Some things that are edge cases and the software has to be designed to handle this kind of stuff well enough.

But just to give you an example of.

Have you ever published an episode that had a glaring mistake in it?

Dave, you've never ever done that, right?

Never.

No, I've never done that before.

I've never published an episode where my co host track was accidentally muted the whole time.

Muted?

I was going to say that's it.

The one where there's just 20 seconds of nothing.

And you're like, oh, should have unmuted that channel.

Yeah, that's.

I've never done that.

We have John Buchenis editing the Future of Podcasting now, and he's my editor for the Audacity to Podcast too, which by the way, is back.

So if you like hearing me, then go subscribe also or follow the Audacity to Podcast.

But I think it was before.

Well, obviously it was before I hired him that that actually happened to me.

That published an episode where the co host's track was completely muted.

But here's the point though is think of it this way.

Now the ecosystem would have to support this, but this is a potential.

Just to get some gears turning in your head, if you have a mistake like that, you could replace only those segments in that HLS playlist that were affected by the error.

So not having to replace the entire file and for the people who get the episode, as long as it's not pre buffered, that amount, then they will get the latest bit of audio at that time that they're requesting it, whatever that latest bit of audio is.

Now, that kind of gets messed up if the timing changes.

So that might not work so well in that case.

But if you're changing the content, it's possible to do that with HLS changing just part of that content if you're just replacing a second at a time with a different file.

Yeah, you had me and you lost me because I get it how if they haven't streamed that part of the show yet.

But that's when you said you can only replace like a section.

How do you do that without uploading the MP3 file?

That's why I'm saying the ecosystem would have to support this kind of thing where like maybe as the podcaster, there's a programming way that's a little bit easier to explain this, but think of it like this way.

Have you ever looked at two documents to try and find the differences between the two?

In programming, we have git.

It's a version management system where the way it basically works is it applies the changes to the documents so you can see those changes side by side, you can see the differences.

If You've worked in WordPress and looked at the revisions on a post that you have, you'll see something similar where this is what it was, this is what it is.

Now imagine that instead of documents you have timelines.

And as long as those timelines match up but just what's in those times is different, then the HLS system could then instead of delivering the messed up portions, it could deliver the replaced portions.

But the only way the system could work to do that would be if those files that make up the HLS playlist were replaced.

Actually you wouldn't even have to have a diff.

I realize you wouldn't have to have something tracking what's changed, what sections have changed.

It's really just you put anything, you change those files in the HLS playlist and as long as the podcast app knows to redownload that HLS playlist every time someone presses play, then whatever is in that playlist, it will download.

So if the 1/2 segments have been replaced, then it will download whatever is the new one.

So it is closer to streaming in that the source of the stream could be switched out mid episode.

Interesting.

Which also means dynamic content insertion.

If everything was hls, you could even get dynamic content inserted in episodes people haven't listened to yet but do have on their device.

Not completely downloaded.

But if it's all HLS playback, then they would get your latest dynamic content when they press play.

That's not too different from what we have now with progressive downloads, where if it just doesn't download it at all until they press play.

But this might be technologically maybe a little bit easier to manage because instead of having to stitch that into the audio or especially the video, which is much harder, you're just adding something into that playlist.

Yeah, that could be very interesting.

And like you said, it's one thing to stitch something into audio.

It's a completely different ballgame.

Now if I go into we use Captivate for the future of podcasting.

If I want to do a pre roll, you know, announcing that Daniel's new episode is out, you know, 15 lessons from 15 years or whatever the title was.

I can see in Captivate where it's like, okay, we're stitching that new beginning into every episode.

Where if I get this right, you wouldn't have to do that because it would do it when someone hits play.

Basically the new part would then just go, and do I have that right?

Like, you wouldn't have to pre stitch everything.

You as the podcaster would still have to define your points.

But take Captivate as the example.

I believe they would not need to redo any of the audio files they've already transcoded and adapted for hls.

They would simply change that tiny file which is the HLS playlist.

And that's what you basically press play on is the playlist.

Think of it like a music playlist.

You're just inserting another song at the top or at some point in the playlist.

You're not re downloading all of the other songs, you're just adding another song into it.

The processing requirements then are much lower for hosting providers that they don't have to reprocess or re encode or anything like that.

They just update that HLS playlist so that it includes whatever content you've dynamically inserted, whether that's an ad, an announcement, news segment, anything like that.

Yeah, that'll be the fun thing, especially you Know, right now nobody's using this.

And to have a system, a live system, how do you set that up?

I would think there's a lot of programming and a lot of backups and, you know, to be able to flip a switch and go, okay, starting now, like, then here comes hls.

I'm like, that doesn't sound like an.

Easy solution, but it might be easier than alternate enclosure.

And I think the HLS could have the potential to make alternate enclosure unnecessary.

Yeah.

Because it'll just play whichever one based on your bandwidth or whatever.

And.

And then like you said, if they had it in the app, where it's like, hey, we notice your bandwidth is getting low.

Would you like to switch to audio now instead of video or vice versa?

Yeah, that could be interesting.

So I could see, though, if you wanted to use alternate enclosure for different language versions of an episode, which is not the ideal use for it, but some people might want that.

For whatever reason, the timing would be completely different between the versions.

So their HLS would not be good.

Because a minute into English could be very different from a minute into Spanish or French.

That would be weird.

Yeah, but then again, that's not exactly something that should even be done, because having audio files in a language that's different from the RSS feed is not really a good user experience.

Because although it could, alternate enclosure could support that, and that is certainly one of the ways that you could use it, that doesn't mean that's the way you should.

Yeah, it's not, maybe best practice to do that.

That could get kind of messy.

So HLS has a lot of potential not only for video, but also for audio.

And that's what I wanted to highlight more in this episode.

Now, a lot of this stuff is still far out into the future, but it's being discussed primarily inside psp, the Podcast Standards Project.

Because HLS is not a podcasting 2.0 thing, but it could be a Podcast standards thing.

So follow Podcast Standards Project if you're interested in seeing how that's developed, or if you want to provide feedback on that, go there.

We'll have a link to their website.

It's podstandards.org and we'll have that in the notes too.

But go there.

If you want to provide feedback on the ideas of HLS and potentials and such for that.

Because they're the people who are really discussing this.

Because this is more of a standards kind of thing than a cutting edge podcasting 2.0 development.

Because it's like, hey, what if we all decided to stop supporting MP3 and start supporting whatever other audio format.

That's a bigger standards discussion than podcasting 2.0 innovations.

Yeah.

Cause it's a big move for the media hosts, but really, in the end, it's the apps that have to then be able to, you know, handle a different type of technology, which it sounds like many of them are already.

But nonetheless, we gotta make sure everybody's on the same page before we go flip into a, you know, a different.

Standard and let's see if hell freezes over again.

And maybe Spotify will be one of the first people to support this, because Spotify supporting transcripts now.

That's awesome.

It is awesome.

I haven't heard what was the thing James was going to test.

If you could upload your own transcript, would they put it in the feed or something like that?

Is that what he was testing?

I forget what it was exactly, but it was kind of like they'd come part way, but not all the way just yet.

Well, they don't ingest the transcript.

If you are using Spotify for Podcasters just to get your podcast into Spotify, they're not using the transcript you provide.

But if you host your podcast with Spotify for Creators, I said podcasters earlier.

It's now Spotify.

We should just call it Anchor for Creators, Maybe just mix it all up together.

If you're hosting with Anchor for Creators, then you can create a transcript or let Spotify create a transcript for your show if you have that ability inside your account.

And they will publish that out to the RSS feed.

So it's kind of weird how they're doing it, but it's a step.

It's a big step.

Like from Spotify, the company that pretty much ignores everything in podcast standards.

Right.

There's a reason we call them the walled garden.

It's a very tall wall.

Very much so.

All right, well, I think that will do it for this episode.

Again, our website, the future of podcasting dot net.

Thanks to everyone who's been streaming SATs.

But then we'll put a bow on this one for this episode of the Future of Podcasting.

Keep boosting and keep podcasting.