I love books and wish I could read more than I am able to. During the Christmas break, we got hooked on to the science fiction series “The Flash” on Netflix. In one episode Barry read a book in seconds, and although he couldn’t retain the knowledge forever, he retained it just long enough to utilize it. That got me thinking …
Could technology empower each of us to become Barry? Could technology enable us to access the wisdom imparted in the millions of books, published in various languages across the world, over centuries. And so, started my Quest.
I am encouraged to find that the beginning of the solution is already in place!
In the world of computational linguistics and probability:
- An n-gram is a contiguous sequence of n items from a given sequence of text or speech. For example: consider a sequence “… to be or not to be …”. This can be seen as a 1-gram sequence (…, to, be, or, not, to, be, …), a 2-gram sequence (…, to be, be or, or not, not to, to be, …), or a 3-gram sequence (…, to be or, be or not, or not to, not to be, …)
- An n-gram model is a type of probabilistic language model that models sequences using the statistical properties of such n-grams. They can be used to predict the next item in such a sequence and are widely used in statistical natural language processing.
This concept can be applied to the text written in the millions of books to a) understand patterns from past wisdom and b) possibly predict some wisdom for future. Thankfully Google took upon the initiative to digitize books and libraries (https://books.google.com) to jump start the project. Furthermore, Google created the Ngram Viewer (https://books.google.com/ngrams) that lets you search for words and ideas in a database of 500 billion words from 5 million books across the global since 1980, based on the research by Jean-Baptiste Michel, Erez Lieberman Aiden.
There is a lot that can be learnt with this tool. Here’s something I learnt within a few seconds.
Trend 1 – God and Human: Faith and God used to be a key conversation point in past years. Around 1860s the religious conversations started on a steadily declining path. The mention of God was lowest around 1940s, at the same time when conversation about “Human” started picking pace.
Trend 2 – Human, Technology, Intelligence and Artificial Intelligence: Around 1860s as the conversation on “Humans” started picking pace, Technology conversations also started. The human race was moving on from religion to science. In 1940s when Religion was at the lowest trough, Science and technology conversations started accelerating.
Between 1960 – 1980 as focus on “Technology” over passed the focus on “Intelligence”; the seed for Artificial Intelligence was planted. Interestingly now we see the usage trends for Technology and Artificial Intelligence declining as compared to increasing trend for “Intelligence”. My rationale for this pattern is that in the era of AI, AR and VR the boundaries between Artificial Intelligence and Intelligence are merging, with the Digital world powering the physical world to new heights!
So, what is the next Quest?
Perhaps to evaluate if artificial Intelligence could impart the true intelligence Barry poses. His ability to preconize. His ability to think through the multiple outcomes for a future event by concentrating hard on the past events. And to predict future outcomes for each outcome until we reach the final outcome. And thus, by recursively computing the future possible actions and the respective consequences, the ability deduce the most appropriate path to take now. To define our own most favorable destiny based on the cards we are dealt with at every step