The Ocean of 0s and 1s: Understanding the "Complexity is Success" of Big Data
Department of Automation
The theme of this lecture is "Introduction to Big Data". Big data has always been one
of the focuses of modern society, but as we all know, I know very little about this topic.
I am fortunate to listen to Professor Wang Jianmin's explanation this time and begin to
truly understand the focus of this era.
Big Data - Precise "Inaccuracy"
The term big data was first proposed by the famous American futurist Alvin Toffler
in his book "The Third Wave". After 2012, the term big data was frequently mentioned
and used[1]. It was not until Victor Mayer-Schönberger's monograph "The Age of Big
Data" that he elaborated on the concept and basic characteristics of big data in detail,
and then the long road of big data discussion began. Small data is limited and difficult
to be universal, so it requires a strict amount of information to "know the whole story
from a single leaf"; while big data can obtain as much information as possible. Whether
effective or ineffective, precise, or rough, when the sample and the population are close
to being equal, after careful screening and verification, the result must be widely
popularized and relatively small in error. Big data achieves a more meaningful goal
than meticulousness by "allowing imprecision"[2].
Take photo classification as an example. When the number is small, our classification
method is to name them one by one. Once the number increases to 100, 1,000, or even
tens of thousands, our best choice becomes to divide them by time, place, style, and
pixel. Currently, rough becomes the most "precise" solution, because although naming
one by one achieves infinite precision, it makes screening and selection extremely
troublesome, which is inconsistent with the modern requirement of efficiency; at the
same time, one-to-one correspondence prolongs the time, which is equivalent to
increasing the intensity of work. Not only is the result unsatisfactory, but the cost is
high and the loss is large. No matter how it is measured, the expenditure and output are
always unbalanced.
Therefore, although complexity is imprecise, if it is applied to the appropriate
development direction, it is not a kind of alternative "precision".
Global and partial - focus on a broader vision
A famous man once said: "If you understand the global things, you will be more able
to use the local things, because the local things are subordinate to the global things."
Big data is the inclusion of small data. It is more of an outline and context, and a mixture
of information from all directions. But this does not mean that big data is a fog. On the
contrary, small data is a specific exploration of limited space. It will not expand or lose.
It is easy to confine people's thoughts to the original place, which is the "fog of the
soul" at certain times.
Taking the shopping APP in life as an example, we found that the list of "you may
be interested" will be pushed in the message. This is the effect that big data can produce
by analyzing "correlation". If only the list related to the purchased goods appears in the
push, and no other extensions appear, then the user will soon lose interest, the desire to
shop will decrease, and the merchant's profit will be difficult to obtain; once the channel
of random push based on the browsed goods is opened, the possibility of users' interest
will be greatly increased. In the acquisition of new information, the desire to shop
increases, thus creating more opportunities for profit.
Therefore, it is not difficult to see that big data is a panoramic view, providing the
right to look at problems from all angles, and giving the convenience of understanding
and adjustment from a God's perspective. Complexity no longer means chaos, but more
like the night sky reflecting countless stars, vast and broad, with hope and future[2].
In summary, big data is indeed a mixture of various information, and information
does have good and bad advantages and disadvantages. The previous analysis method
is to eliminate the complexity and retain the essence. Although it is feasible, it is not
practical. Big data is a new way for us to understand the world. It is a tool invented by
people to better understand the world, allowing people to have a deeper understanding
of the world. It truly realizes unified discussion[3]. There is no need to focus on
distinguishing information categories. If you can get real results from this "ocean" and
meet your needs, you will win. In other words, complexity is also success.
References
[1] Pang Jingan. The era of big data: changes in thinking, industrial transformation and
the rise of data science [J]. Advances in Information Science, 2016, 11(00): 186218.
[2] Viktor Mayer-Schönberger, Kenneth Cukier: The Age of Big Data, translated by
Sheng Yangyan and Zhou Tao, 2013, Hangzhou: Zhejiang People's Publishing
House, 46-47, 65.
[3] Liang Hui. Changes in philosophical thinking in the era of big data [J]. Jilin
Education, 2020(17): 93-94.