R&D Progress on TV 3.0 Application Coding Layer

Abstract

Television plays a social role of immense relevance in Brazil. TV is more than a technological object in the room: it is also an important cultural artifact and an element of national integration. Therefore, any technological evolution that represents a change in the way of watching television may imply some cultural change for the society itself. Thus, the development process of SBTVD Forum’s TV 3.0 Project, at least for its most highlighted application coding use cases and features, are being carried out not only as a technology research, but also as a social study.

Brazil has been watching TV since 1950, with the inauguration of TV Tupi in São Paulo. In seven decades, Brazilian television has experienced technological, social and content development that makes the national experience one of the richest in the world. Remarkably, the current terrestrial DTV system specifies the Ginga middleware, a national technology, as the standard for multimedia interactivity since 2007. Ginga has been proven to support a consistent evolution that made it the first Brazilian technology adopted as an international standard by ITU-T in 2009 and recognized by ITU-R as an integrated broadcast-broadband system in 2017.

TV 3.0 Project is currently under Phase 3, carrying out further tests and evaluations over the physical and video layers, and developing a reference mux/demux. Regarding the application coding layer, most of the innovative requirements established by the Call for Proposals (CfP) are under study by selected Academia research groups, since those requirements were not appropriately addressed in previous phases.  The CfP specified 17 requirement groups for application coding, including basic aspects on backward compatibility with Ginga specifications and its implementation reuse, besides support for TV 3.0 underlying technologies. The advanced requirements include support for application-based TV experience, immersive audiovisual content, multimodal interaction, sensory effects, multi-user profiling, audience measurement, IP convergence, extensibility, just to name a few.

This paper focuses on the methodology, progress and early achievements of the Academia R&D team on addressing the high-priority application-coding requirements for TV 3.0. The team is composed of 40 researchers from 6 academic institutions, namely PUC-Rio, UFPB, UFF, UFJF, UFMA and CEFET-RJ. The work started in April 2023.

As a means of actively collaborating with the research methodology, SBTVD Forum’s Technical and Market Modules jointly decided on a prioritization of requirements to determine the sequence of studies for the R&D team. In addition, the Forum’s Application Coding Working Group (WG) specified an initial guideline on how to tackle each requirement, based on the evaluation results from Phase 2 and the WG's experience in standardizing/implementing DTV middleware. Finally, the WG specified a total of 7 use cases to be prototyped, aiming at validating the R&D solutions and publicly demonstrating the new TV 3.0 application features. 

The R&D team diligently incorporated all SBTVD Forum contributions into its methodology, allowing for consistent progress on certain requirements and already presenting the initial findings of the studies. 

First, relevant architectural changes in the application coding layer were already proposed and agreed, based on the fundamentals of Ginga specifications for DTVPlay receivers. The requirements on application-based TV experience impose changes that include a new user interface for listing each broadcaster's initial application. In addition, a new media player is needed, capable of keeping running during app switches, regardless of whether the current audiovisual content is delivered over-the-air (OTA, broadcast) or over-the-top (OTT, broadband).

There is also significant effort in requirements engineering and social studies regarding this application-based TV experience, as mentioned in the beginning of this abstract. The team is running focus groups and opinion polls with a probabilistic sample, as well as prototyping the entire viewer's journey based on the quanti/qualitative data obtained. This prototype will be further refined following the principles of Design Thinking, under discussion with a team of experts proactively assigned by RNP. 

Another area of focus involves evaluating the features introduced by the adopted audio and video codecs, with the objective of identifying properties that can be utilized by applications and determining the necessary implementation support. 

The extensibility requirement is also under study, with a focus on identifying Ginga-NCL and Ginga CCWS APIs as metadata so that new applications can obtain granular information about the functionality support of their interest in the receiver, thus allowing them to adapt according to the available resources.

A further R&D task has focused on the accessibility requirements, more specifically on the captioning part, where IMSC1 standard is adopted for encoding and transmitting subtitles. It uses a subset of TTML, which consists of an XML file with several possible settings for captioning, such as position, color, font, display time, synchronism, emojis and images. In order to test and validate the forwarding of captions to mobile devices over the local network, a prototyping environment was developed, composed of a partial Ginga CCWS implementation and NCLua and HTML5 applications. New required APIs are added to CCWS prototype, in this case for the real-time forwarding of captions in TTML format over a sockets or websockets. The synchronism between the applications is performed through the socket, and the results are being evaluated.

The team has also been working on the implementation of use cases related to sensory effects, immersive content and multimodal interaction. To accomplish this, the team is working on harmonizing the adopted proposals NCL 4.0 and Guaraná. Combined, they allow for the inclusion of sensory effects (wind, scent…) in interactive multimedia applications and the execution of parts of the application on head-mounted displays connected to the TV, in a 360° scene including 3D objects, immersive MPEG-H 3D audio and traditional multimedia objects. In addition, users will be able to interact with applications using different modalities (voice, gestures…) using input recognition devices.

Concluding, partial implementations of prototyped use case apps will be demonstrated during SET Expo 2023 in the SBTVD Forum booth. This paper will include diagrams and technical details for deepening the discussion on each study of this challenging project of unparalleled opportunity.