The Advantages of Text-Based Information Versus Videos, Audio or Images

• Updates
  • 2022-01-17: section "Optimizing for Text Consumption"
  • 2022-06-05: comment by Heinz Wittenbrink about the environmental impact

I want to start with a brief episode from my past. When I was about to finish my PhD thesis in 2012, I was looking for options of starting a career in academia. Therefore, I was speaking to great PIM researchers at international conferences and asking them if I may join their research groups as a post-doc. One of my favorite researchers was Gloria J. Mark. Back then, she was analyzing the effects of multi-tasking settings on efficiency and so forth.

After I have introduced myself and my research on tagstore where I was using user-generated tags to support filing and retrieving arbitrary files on the local file system, she explained that she doesn't see any future in that research. To my surprise, she argued that the future is not text-based. She wants to shift her focus on video-based information where she thinks that this will be much more relevant for the people. According to her, text is getting less important while video is getting more and more important.

This is why I do think that we should think about the importance of information locked into video, audio and image content.

I feel very sad when relevant information is locked away in walled gardens we're about to lose in the future. Even worse is relevant information in walled gardens in form of video, audio or images only.

The current popularity of Tiktok, Instagram and YouTube is impressive. Other services like Twitter do implement more and more features to mimic them. On the other hand side, I'm frightened when I think of all the good content that we are going to lose with them.

Why are we going to lose that information?

Information in video, audio and image files are much more difficult to curate, consume, distribute:

• difficult to share except as a link to something that may get offline
• hard or impossible to index for independent search engines
• therefore hard to be searched and retrieved
• no decentralized storage solution
  • except open solutions like PeerTube which is that unpopular it is almost non-existent
  • almost no chance of distributed backups
• walled gardens aren't built for the eternity

There is the term digital dark age which stands for "a lack of historical information in the digital age as a direct result of outdated file formats, software, or hardware that becomes corrupt, scarce, or inaccessible as technologies evolve and data decay."

With information stored in non-directly accessible form in video, audio and images files, this digital dark age es even getting more darker. We can not even access it properly right now. Now you can imagine the impact on retrieving currently locked-in video information in a few decades from now.

I don't think that this is rooted in some evil plan or similar. This happened by coincidence. It enables even the illiterate people to contribute more or less relevant content to our modern world. You can be a famous YouTube influencer or Insta-star without being able to write down you thoughts in an ordered manner.

Furthermore, people who can not even read or write are able to consume this content in almost any situation with their mobile phones. It's ubiquitous and ready to be consumed any time on any decent Internet-connected device.

Even worse, it's addictive and makes us sick. It all starts with FOMO. Therefore, it's good to practice digital detox every now and then.

Back then, when Gloria Mark told me that text is dead and video is the future (I exaggerate a bit), I was irritated and my initial thoughts were that she has to be wrong.

But she was right. Obviously.

I did not want to accept the fact that people develop different workflows and usage behavior compared to what I tend to identify as a good and efficient computer usage pattern and environment. Therefore, I did not want to support this with research as well. Don't get me wrong, I still got rejected by Gloria and not the other way round. ;-)

My reaction was really stupid from a scientific perspective. Maybe my reaction is understandable from an efficiency and PIM perspective. Maybe I'm more of a teacher than a researcher? I would like to identify, teach and spread good ideas on how to use computers as a general tool instead of "just" observing, how people actually use computers. I love to help people to get more efficient instead of following trends I don't think are a step forward in long-term perspective.

I don't seem to be alone with this opinion.

So what do I think would be a better approach?

In contrast to video/audio/images, information in text-based form can be:

• easily shared as a copy if necessary
• easily indexed to its full extend
• easily searched by multiple independent search services
• easily copied and stored in a decentralized fashion

YouTube gets 500 hours of video content uploaded every minute(!) and this figure is from 2020. Almost everything of that is stored exclusively on YouTube only. Just imagine the day when YouTube will be shut down. Human kind will face a heavy loss of information and knowledge comparable to the loss of the Library of Alexandria. This single event is most likely the reason that we've lost hundreds or even thousands of year's passed-on knowledge from the druids, medicine men, wise witches and so forth. That's not progress. This is the exact opposite of progress.

Decentralization needs to be a goal for relevant content for the future. If something is originated in video/audio/images, everything relevant needs to be extracts in text form.

Current technology is able to do this in a mediocre way. OCR and speech to text does come with its downsides. Their output is not good enough that it can be trusted blindly. If you want to make sure that all relevant information got extracted as text, you currently have to invest manual effort in fixing the remaining errors to reach a satisfying result.

I got the impression that we stopped improving on those technologies. When I did some research and test on paper digitalization many years ago, I was disappointed on the tools I was able to use and their end results. I'm afraid that improvements are made by commercial companies like Evernote and agencies like the NSA which does not get shared with the general public.

By teaching the downsides and long-term effects of information lock-in to children, we may be able to make text more sexy. When I'm demoing extremely advanced tools that are text-based (command line, vim, Emacs, LaTeX and many others), the majority of non-tech-savvy people are blocking right away. They stop listening solely because of its look and appeal.

Even something simple like Orgdown is too sophisticated or nerdy for the average person. This way, I can't explain all the advantages of those powerful tools even for them as normal users. That's not progress. This is the exact opposite of progress.

This comment on reddit made a valid point:

[...] For me to see something in a huge wall of text, my brain shuts down from wanting to learn it. So while I think text based documentation is great, you'll eventually have users who fundamentally don't learn that way with no real means of support.

If you do see in a huge wall of text and you do get a problem because of that might have multiple reasons.

One of the reasons might be that you can't read properly. In most cases, this is not the true reason.

One of the remaining reasons is that information is brought to a text representation which is not optimized for consumption by others. For some things like personal notes or ephemeral text, this might be totally fine. However, for most texts that address other people as readers the author is required to prepare it well. This way, the information can flow from the written word to the brain of the readers in an optimal way. Research showed that even in an optimal case, only about 85 percent of the information of a text gets to the brain of the reader. So please do mention important things multiple times.

Not the only one but also an important aspect of optimizing the reading experience is typography. I've written about its role in that article using the example of LaTeX typesetting.

So if we are talking about information being provided in text-based form versus video/audio/images, there are plenty aspects that still need to be taken care of for the optimized path from the author's idea to the reader's brain. The whole story is not finished when the author has decided on a text-based form at all.

If the one thing you're remembering from that article is that you need to be aware of the lock-in effect of information because of its format, that's a good start.

Help me promoting this notion which might be too subtle for others otherwise.

2022-06-05: Heinz Wittenbrink wrote a nice comment which adds an important aspect: the much larger environmental impact of video compared to text:

Thank you for this article! I agree with everything... A further important aspect is how energy hungry videos are. Many authors - one of them is Gerry McGovern - have written on that. Online Video is one of the things that eat up the planet.