Integral Musings | Towards a Holistic Vision

An Integral Approach to management and human development based on the spiritual vision of Sri Aurobindo and the Mother with an emphasis on its application to various domains of knowledge and life.

Big Data: What it Can and Cannot Do – M.S. Srinivasan

yellow_flower_195139At present, the corporate world is enamoured of “Big Data” or Analytics, which is regarded as the new “Management Revolution” Harvard Business Review has come out with a special issue on it. Is it just another hype which will fade out? How much of the “revolution” is hype and how much of it is real?

Deluge of Data
Harvard Business Review brought out a special issue on this theme on October 2012. The editorial in the issue states “Yes there’s a lot of hype. But companies that make a sophisticated analysis of the huge data streams now available can unlock deep insights and value – without having to make major new investments in technology.” This reminds me of a similar phenomenon in the 1970’s when the Quantitative Techniques entered into management. More or less similar claims are made, like for example intuition is no longer needed for decision-making and computers and mathematics will soon replace human judgement or intuition. But after a few years of flirting with “quants” management thinkers again started talking about the importance of intuition in decision-making.

Let us first listen carefully to experts on big data. First question is what it is all about. Andrew MacAfee and Eric Brynjolfson; data scientists from MIT and Sloan School of Management, in their article in Harvard Business Review “Big Data: The Management Revolution”, state that the main objective of big data movement is to “gleam intelligence from data and translate that into business advantage”. There is undoubtedly a need for such a movement because in recent years there is a tremendous increase in the volume, velocity and variety of data. This has its advantages as well as drawbacks. Too much of data becomes difficult to handle but if we can find a way to manage this big data which represents facts of the world in movement, it holds the key to a better understanding of the changing world. If we can find a way to manage this data creatively, which means to discern the hidden patterns, answer questions or indicate solutions, then it is a big help in decision-making. This is what Big data movement can do.

Data and Insight
But can it entirely replace intuition? MacAfee and Brynjoffson state: “Data-driven decisions are better decisions… it’s as simple as that. Big data enables managers to decide on the basis of evidence rather than intuition.” However, in the last part of their article in HBR they say: [“Companies succeed in big data era not simply because they have more or better data but because they have leadership teams that set clear goals, define what success looks like and ask the right question. Big data’s power does not erase the need for vision or human insight. On the contrary we still must have leaders who can spot a great opportunity, understand how a market is developing, think creatively and propose truly novel offering articulate a compelling vision persuade people to embrace it and deal effectively with customers, employees, stockholders and other stakeholders. The successful companies of the next decade will be the ones whose leaders can do all that while changing the way their organisations make many decisions”.]

So Big data can’t be a substitute for intuition and insight. Almost all the factors listed above involve decisions which require human insight. Big data can’t be a substitute for it though it can help in arriving at a better decision by providing well-structured and appropriate information based on facts. However when the decisions involve a deeper or wider understanding of human factors like motivation or well-being or a holistic comprehension of the totality of life, then data becomes less important and human insight acquires a much greater significance.

Right decisions depend on right understanding which can be at many levels. At the first level, there is the understanding of life as it is, made of the known, visible, manifest, and predictable and the mobile actualities of life. At a deeper level is the unknown, invisible unmanifest, dormant, unpredictable and the flux of future possibilities. At a wider level is the understanding of the ecological dimension, which means connectedness, interdependence, unity and wholeness of life, which is necessary for knowing the long-term consequences of our decisions. At another level is the realm of values, the ethical, aesthetic and spiritual dimension, and the factors which lead to human well-being and fulfilment, individual and collective, inner and outer, which may involve reconciling many dualities, dilemmas and conflicts thrown by life.

Big data, analytics, and whatever future development in IT which enhances the capacity to process information can only help in better understanding at the first level of visible facts and when combined with Artificial Intelligence provide a certain level of insight into the second level of the invisible like for example hidden patterns behind facts. But at all other deeper, wider or higher levels, analytics may not be of much help and we need a deeper and higher intuition. In the future, as we grow as a race in our inner consciousness, more and more of these higher non-material dimension will begin to manifest and get incorporated into human life. This is already happening. For example, factors like ethics, values, long term vision, social responsibility, sustainability, holistic perspective and human well-being are now increasingly recognised as key to success in the future world. All these factors and many more which may come in the future will throw up problems, situations and decision-contexts which cannot be solved entirely by analytics but requires a higher intuitive understanding.

Even in the first level, right decisions require a certain subjective freedom from excessive self-interest. Without this freedom, big data will be used to support personal decisions driven consciously or subconsciously by self-seeking motives. Secondly, quite often right questions are the spark which ignites creativity and innovation and analytics can’t ask questions. As McAfee and Brynjoffson quote Pablo Picasso as telling “Computers are useless they can only give you answers.”

23 comments on “Big Data: What it Can and Cannot Do – M.S. Srinivasan

  1. Deshawn Howdyshell
    November 22, 2015

    I have read several good stuff here. Certainly worth bookmarking for revisiting. I surprise how much effort you put to make such a magnificent informative website.

  2. Ramageethan
    December 28, 2015

    Awesome Sir! It it true that intuition plays a major role and it cannot be replaced by big data analytic …

  3. Samantha
    January 27, 2016

    Hi my name is Samantha and I just wanted to drop you a quick note here instead of calling you. I discovered your Big Data: What it Can and Cannot Do – M.S. Srinivasan | Integral Musings | Towards a Holistic Vision page and noticed you could have a lot more traffic. I have found that the key to running a popular website is making sure the visitors you are getting are interested in your niche. There is a company that you can get targeted visitors from and they let you try the service for free for 7 days. I managed to get over 300 targeted visitors to day to my site. Check it out here:

  4. cs go skins database
    April 12, 2016

    Thank you so much for sharing your amazing web page.|

  5. Lakeesha Poetzsch
    April 23, 2016

    Hi there! I am so grateful that I found your blog. I really admire the dedication you put into presenting the detailed information for free. I will definitely come back to your site for more. Keep up the good work. By the way, this system helps me making around $650 per day, if you allow me, I suggest you to take a look at this secret offer page to access your very own turnkey software business to increase your online revenue. Wish you more success, all the best!

  6. Nolan Drowne
    May 17, 2016

    I hope to give something again and help others such as you helped me.|

  7. nba
    June 2, 2016

    I enjoy checking your websites. Thanks for your time!

  8. Armandina Winrow
    August 19, 2016

    Way cool, some valid points! I appreciate you making this article available, the rest of the site is also high quality. Have a fun.

  9. hairy teen
    October 15, 2016

    Excellent site үou һave hеrе.. It’s hard too fijd higҺ-quality writing like yourѕ nowadays.
    I ѕeriously ɑppreciate individuals ⅼike yоu!
    Take care!!

    Ⅼook at my site :: hairy teen

  10. xserseva
    November 21, 2016

    Hello! If you have been involved in the binary options markets I have something to interest you. Even if you have not been involved in the financial markets and are interested in earning some money with your free time I have something which may interest you. it�s called AutoBinarySignals!. It is a software system which will help you trade on the Binary Options Market. This is a �Plug-And-Play� system which you can easily set up and have operating in just a few moments.
    With the AutoBinarySignals software system there are no complex charts, no difficult to understand analysis, or no complex methods. This is an easy to learn system and you can have it running to earn more money.

  11. corburt erilio
    February 10, 2017

    Some truly interesting details you have written.Helped me a lot, just what I was looking for : D.

  12. corburt erilio
    February 13, 2017

    It?¦s really a great and useful piece of info. I?¦m satisfied that you simply shared this helpful information with us. Please keep us informed like this. Thank you for sharing.

  13. Emeline Oborne
    March 1, 2017

    This is the best place I found to get targeted traffic.

  14. điềm máy mắt
    August 16, 2017

    Its like you read my mind! You seem to know so much about this, like you wrote the book in it or something. I think that you could do with a few pics to drive the message home a little bit, but instead of that, this is wonderful blog. A great read. I’ll certainly be back.|

  15. cheap logo
    August 28, 2017

    Respect to website author , some wonderful entropy.

  16. Website scripts
    August 29, 2017

    Thanks for another great post. Where else could anybody get that kind of information in such a perfect way of writing? I have a presentation next week, and I am on the look for such information.

  17. Scripts
    September 9, 2017

    An impressive share, I just given this onto a colleague who was doing a little evaluation on this. And he in reality bought me breakfast as a result of I found it for him.. smile. So let me reword that: Thnx for the treat! But yeah Thnkx for spending the time to discuss this, I really feel strongly about it and love studying extra on this topic. If potential, as you develop into expertise, would you mind updating your weblog with extra particulars? It’s extremely useful for me. Huge thumb up for this weblog submit!

  18. management script
    September 16, 2017

    its great as your other articles : D, thanks for posting .

  19. NicePhpScripts
    September 19, 2017

    Hello I will be thus excited I discovered the webpage, I seriously found you unintentionally, while I had been exploring upon Google regarding another thing, Anyhow I will be here today and would likely want to point out cheers to get a amazing post plus a all-round thrilling weblog (I additionally really like the actual theme/design), I don’t have enough time in order to browse all of it in the second but I possess saved that and also added your own Bottles, when I have time I am returning to examine far more, Please do maintain the truly amazing job.

  20. PHP auctions script
    September 22, 2017

    Hello! I would choose to supply a huge thumbs up with the great info you have here with this post. I will be returning to your website to get more soon.

  21. classified script
    September 26, 2017

    you have got a great blog here! do you want to make some invite posts in my blog?

  22. Jobs Script
    September 29, 2017

    Man that was very entertaining and at the same time informative.’:;;.

  23. This article was great. My friend sent me a link to it from his email.

Leave a Reply

Your email address will not be published. Required fields are marked *


This entry was posted on November 5, 2015 by in Integral Management.