Support "default captions enabled/showing"

[cjpillsbury]

For the Open Elements Mux Player, we want to support automatically having closed captions/subtitles on by default. Currently, this behavior is tied to implementation details in <media-captions-button/>, for 2 reasons:

• "ownership" of using subtitles fallback (via no-subtitles-fallback) belongs to <media-captions-button/>, which determines whether or not subtitles should be considered when applying default captions enabled/showing
• the logic for which textTrack to show is also "owned" by <media-captions-button/>.

Additionally, the slotted media's available textTracks will likely be asynchronous for HLS/DASH content, both due to details of "tech"/"playback engines" such as hls.js and due to the fact that they can/often are delivered within the manifest/playlist itself. This is automatically accounted for based on the defined behavior of <media-controller/> (and may be further improved by #134 if "UNKNOWN" availability is supported).

Because of these details and in order to future-proof any potential changes in Media Chrome, Mux Player will need to duplicate some logic (or other workarounds) to wait for textTracks availability and then dispatch a click event from the <media-captions-button/> instance (to ensure identical behavior).

Having some kind of support of "default captions enabled/showing" feels like a reasonable Media Chrome feature, not just because of the specific needs from Mux Player, as these considerations would be a high burden/barrier for non-experts and it doesn't seem like an outlandish feature.

My initial proposal was to add a simple boolean attribute to <media-captions-button/>, in part because of the aforementioned considerations. That said, I'm open to any proposal that captures the relevant details and provides a reliable solution.

[heff]

I understand the aforementioned shenanigans a little better now, thanks. :)

From a purely design perspective, an attribute on the controller itself I think is the right answer. Ideally we're not creating a pattern where devs have to look at different buttons to understand why the player is behaving in a certain way with no user interaction.

It sounds like, though, adding default-on to the button may be the more pragmatic solution considering where we went with the captions button already. I can get on board with that.

Taking a quick stab at the first option, if we were to add logic to the addtrack listener to handle this, is the main complication knowing which track to turn on as addtrack is fired?

I have some open questions on the feature, that apply to either case.

• Are we letting them choose which language to default to?
• If no language, do we just default to the first captions or subtitles track that triggers addtrack? I think the answer is yes.

In retrospect, I could make an argument that no-subtitles-fallback should be a media-controller feature/attr, so that anybody building their own CC button or using the media controller APIs can benefit from that. It doesn't get around the CC button still needing to be aware of both captions and subtitles, and even that it should fallback or not. Unless media controller were to merge/populate the captions list with subtitles when fallback was on. Consider that an idea for the future.

[cjpillsbury]

Taking a quick stab at the first option, if we were to add logic to the addtrack listener to handle this, is the main complication knowing which track to turn on as addtrack is fired?

That's the main reason it's easier/more reliable (with the current implementations) to add the attr to the <media-captions-button/> vs. the <media-controller/>, yeah.

Are we letting them choose which language to default to?
Since we currently only have a simple captions button, I'd treat that as a separate issue, though it's definitely a good callout.

If no language, do we just default to the first captions or subtitles track that triggers addtrack? I think the answer is yes.
I agree. That's effectively how the captions button works right now. At least with US vernacular, "closed captions" are typically only one language (the same primary language as the original, non-dubbed audio) for folks with hearing impairments, whereas "subtitles" are alternate languages for folks who aren't fluent in the audible language. The primary motivation for the "subtitles fallback" is because many folks simply forget to "properly" identify a track as closed captions, and so they'll show up as subtitles in their multivariant playlists/manifests.

In retrospect, I could make an argument that no-subtitles-fallback should be a media-controller feature/attr, so that anybody building their own CC button or using the media controller APIs can benefit from that. It doesn't get around the CC button still needing to be aware of both captions and subtitles, and even that it should fallback or not. Unless media controller were to merge/populate the captions list with subtitles when fallback was on. Consider that an idea for the future.

Our original discussions for the current implementation were forward looking to a "subtitles/captions menu" component. If we want to "promote" these things to <media-controller/> we could without too much effort, but we'd probably need:

• 1 or 2 new events to handle a simple toggle or ON/OFF captions, which would use subtitles fallbacks in the same sort of way we currently do
• an attribute that represents the corresponding ON/OFF state (word-smithing the name may be tricky here, since it could be confusing when subtitles fallbacks are in play)

I think that's a reasonable approach, but it does mean more discussion on working through these details.

[heff]

Cool. I think what I had in mind was that media-controller would abstract away the fallback by just merging captions into the subtitles list (or maybe vice versa, or both). So then a subtitles menu would only have to care about one list and one set of events by default. And then of course you'd be able to turn that off for more advanced use cases where you know you want to treat them both intentionally.

[cjpillsbury]

Since this is not explicitly called out in the informal discussion, regardless of any of these details, we will want to be clear on what "default captions enabled/showing" means. I think the two most reasonable interpretations are:

• per src/loaded media (my S1)
  • Whenever a new src is loaded on the slotted media, if there are captions available on the content or via <track/>s (or subtitles if subtitles fallback), automatically show it. This would be roughly analogous to the behavior of autoplay
• per "instantiation" of a <media-controller/>
  • On the initial instantiation of a <media-controller/>, the first time a slotted media loads a src with captions available (or subtitles if subtitles fallback), automatically show it. If the cc/sub tracks are subsequently disabled (e.g. via user toggling), captions/subtitles will remain disabled, even if subsequent loaded srcs contain cc/sub tracks. This would be roughly analogous to the behavior of e.g. muted (more like an "initial value", which should imo inform naming details and setting/unsetting the attr value programmatically).

It may be worthwhile to loop @dylanjha in on this one just to clarify if he had any specific thoughts on what this requirement should entail from the Mux Player side.

[heff]

Whenever a new src is loaded on the slotted media

There's the rub. Right now I don't think there's a way for the captions button to know that the source has changed. Just that the captions list has been updated, but the list could be updated for reasons other than the source has changed. Or the list might go from "en" to "en", which we could decided to not trigger a change. It's possible that for this button we just assume a list change is a source change, with the expected use case being only ever a single captions or subtitles file, but that won't extend to a menu well. Buttons being aware of source changes so they can reset their own internal state, beyond simple attribute updates, has a design smell.

For this iteration we could still go with the button attribute, and assume a single captions file and no source changes. I do think that covers 80+% of the use cases perfectly fine. When we get to building a captions menu, I'm feeling that we need to centralize the subtitles fallback in the controller.

[cjpillsbury]

Buttons being aware of source changes so they can reset their own internal state, beyond simple attribute updates, has a design smell.

I agree 100%.

For this iteration we could still go with the button attribute, and assume a single captions file and no source changes. I do think that covers 80+% of the use cases perfectly fine. When we get to building a captions menu, I'm feeling that we need to centralize the subtitles fallback in the controller.

I think we'll need to assume subtitles fallback. I also think for the 80+% case, we can assume that if the subtitles list and the captions list go from "something" to "nothing", that is a src change. Otherwise, I agree with everything here.

I'll proceed with everything other than my last callout re:80+%, and hopefully we can pin that down before I hit that point (we can also always treat that as a followup if it's not resolved by the time I get there.

[heff]

we can assume that if the subtitles list and the captions list go from "something" to "nothing", that is a src change.

Yeah I feel good about that assumption. That should be what happens if you pull the current video element out of the controller and then append a new one. Not sure if that happens if you append the new video element first and then remove the old one. I don't feel like we need to solve for that in this iteration though.