Deep sea test: To test a new video technology, researchers dived to the wreckage of the Titanic. There, at a depth of 4,000 meters, they tested a transmission method that makes video conferencing possible with extremely low bandwidth – even a submarine’s sonar is sufficient as a carrier signal. This is possible because the audio and image are converted to text and only converted to an industrially generated video at the recipient.
Video conferencing has been a trend since the start of the coronavirus pandemic at the latest – more and more people are communicating via Zoom, WebEx, Skype and Co. Since there is a shortage of bandwidth, there are hindrances with poor transmission quality, drops and disconnections in the connection.
The wreck of the Titanic as a test site
A team led by Alex Waibel of the Karlsruhe Institute of Technology (KIT) and Carnegie Mellon University may have found a solution to such problems. They have developed a method by which video data can be transmitted even with low bandwidth. This is possible because the data is not sent as image and sound, but is converted to text before transmission. This includes significantly less data and therefore passes even with poor connectivity and limited line capacity.
The researchers tested whether and how the system worked in an unusual place: during a dive to the wreckage of the Titanic in the North Atlantic. “It is very difficult to transmit data from a depth of four kilometers through salt water without losing it,” Waibel explains. Because sea water absorbs almost all short-wave radio waves, so transmission from the submarine to the mother ship on the surface of the water only works with sonar – bandwidth is correspondingly limited.
From text to synthetic video
Concretely, the new system works as follows: the sound and image information recorded by the computer camera and microphone are processed by a special program. This converts the speaker’s dialogue into text and also reduces the image information to what is necessary for the portrait. This greatly attenuated information is now sent to the addressee – in the practical test in the submarine, this was done via sonar.
In the receiver, the abbreviated data is reconverted using additional software: the system uses the image information to create a synthetic video clip for the speaker. At the same time, the text is converted back into audio speech via the audio output system. “Then, the video contains a synthetic sound that is mapped to the speaker’s voice so that it sounds like the speaker’s voice,” Waibel explains. “In addition, the composition of the video is controlled in such a way that the lips of the speaker move in synchrony with the sound.”
Also suitable for synchronization
The result is a fully synthetic video that is created only at the recipient’s end, but thanks to the latest technology, it still feels like a video conference. In contrast to these, such video can also be transmitted over low bandwidth – even from the seabed. “This will make communication in spoken language easier in the future,” Waibel says.
As the researcher explains, the technology is also suitable for translating films into another language or for lip-sync videos.
Source: Karlsruhe Institute of Technology
“Certified tv guru. Reader. Professional writer. Avid introvert. Extreme pop culture buff.”
More Stories
Classic espionage, cyber attacks, creating economic dependencies, and technology withdrawal –
How the technology behind ChatGPT powers this bot
Mercedes G580 with EQ technology: The G-Class has become electric