Data profession gone viral

Before Covid-19, the only people excited by data were those in related fields. Covid-19 changed that. How did this change come about? We all can remember, looking back now, the news items about an infectious virus spreading in Wuhan, China. It was tucked away as news item number 4 or 5. People didn’t think too much about it at the time. However, things changed very soon and public health officials started issuing cautious warnings that this could potentially be cataclysmic for us here in the UK as well. We remember the attempts to isolate people coming in from China, back in Feb 2020. Things deteriorated very quickly as the situation took a turn for the worse in Spain & Italy. It became evident overnight that the UK was only 3-4 weeks behind with a similar outcome. The public policy makers had to very quickly make decisions that would not only impact public health but every aspect of our lives in the coming months. 

Public policy makers rely on macroscopic data to make sense of the world they are trying to shape. This situation was no different, in that respect. Their decisions with respect to Covid-19 would be guided by data. Putting aside political decisions for a moment, we can all agree that the health officials relied on data to inform the government about what tools it could be deploy at its command to contain the spread of the virus. The government recognizing the widespread impact, started doing the daily Covid-19 briefings. In many ways, thanks to open data initiatives, we were able to see for ourselves the public health situation as it unfolded in the country. We all were suddenly glued to the television to watch these briefings filled with numbers, hypothesis, graphs, tables, percentages and we overnite became familiar with terms like “flatten the curve”,”reduce the R rate”, “NHS capacity”, “hospital admission rate” etc. If not for the extraordinary situation we all found ourselves in, these terms would be anything but arcane, confined to the small groups statisticians. However, it seemed we all had been put through a bootcamp and crash course combined in a matter of days. We all were expected to understand this new language of the nation. As lockdown swooped on us, our discussions moved to online forums and were rife with analysis of the previous day’s Covid-19 figures. All of us had an opinion on how we should tackle this contagion. Even today there is no agreed single approach to tackling Covid-19 anywhere in the world. What we have is a set of measures that can be deployed alone or in combination usually in a balacing mode to reduce the impact. Literally overnight, people have been schooled in how data alone is not enough to inform and provide the best course of action. We are all aware of the profound insight that a world without data analysis is bad but a world with data can equally be bad if we draw the wrong inferences from it. The challenges of data analysis are self evident now to the wider public.

We all live in a world where everything happens at a scale unthinkable even a decade ago. It is not only that scale that is massive now, but also the interconnectedness of our lives and the avenues this provides for effects to ripple through at a pace never seen before – whether it be a social media meme, a physical virus or awareness about data analytics.

News organisations, leading news websites, universities and internet giants like Google played an equally important role in familiarising us with data lingo. They presented regular, timely, clear, simple & visual Covid-19 related data that all of us could make sense of. Other than sports statistics, never before have so many of us been presented with so much data so regularly. Never before have we all analysed such data so regularly and discussed its impact on our lives. It seems we all have  turned in armchair data scientists overnight. This is not a slight but a reality and it is for the better. 

It is a good thing for the data professionals, that the general population is aware of the role data plays in our daily lives. It is good that they are more tuned to graphs, statistical insights and the challenges in data inferences. The past few months have softened everyone towards data science and made them more receptive to data professionals. Data profession has moved from rarified field to our drawing rooms, to a certain extent. This will only help the already burgeoning demand for more data professionals. Let us not fear this wider awareness / education as a threat to the craft but as a validation and an opportunity. 

Data in our daily lives

We all used data throughout our daily lives without even realising it. Of course, we do not perform the rigorous statistical exercise that is usually associated with the craft. However, if you were tasked with opening someone’s eyes to the world of data science and how they do it too, albeit at a different scale, examples are abound around us. Let us look at three examples where we do this in our lives regularly.

Buying a car or a house

For most of us the most expensive thing we buy would be a house. Obviously there are many factors to consider when embarking on this project. We want to get this right as not only does it cost a lot of money but most of the time we might have an emotional connect to the place, because we intend to live in it. Instantly we recognise that we’re dealing with a lot of different factors ranging from location, design, finance to regulations.  We try to make sense of this complex set of data through spreadsheets. We deal with statistical concepts like average, what if analysis, trends etc. We also familiarise ourselves with calculated terms like ROI, rates (interest rates, inflation rates, rates of change of house prices etc). Not only do we educate ourselves with these things but also realise very quickly that the data itself isn’t enough. In spite of our best efforts we are left with the feeling that we are still best guessing a lot of the decisions in the process. This process of buying a house, especially the analysis that goes into it, is no different from what happens across data terms around the world. They collect some data, clean it, ask questions that need answering, run data through statistical analysis to try and answer those questions, with the hope that a decision can be arrived at confidently backed by data.

Biometric exercise watches

This is the most subtle way we are all exposed to the craft of data science daily. Our humble watches have evolved from being just mere time keepers, to fashion statements to biometric data hoovers & SciFi level dashboards! Countless times a day we look at our watches to see how we are doing with our metrics. Have we walked enough steps? Have we burnt enough calories? Are we being stationary for long periods during the day? How is our heart rate doing? Maybe once in a while we ask more detailed questions like – Is my gait changing? Am I walking slower or faster compared to the last year? Is my stride reducing? what is my BMI? etc. If you ever get a time you should google and find out how to look at the data that is on your phone. You would be surprised the amount of lines of data that’s constantly written throughout the day. It will run to tens of thousands of rows of data just for one days!! The quality that I most admire about this whole affair is that it is subtle when it comes to end user interaction but complex in the background and is at a massive scale. All of the data collection and analysis is hidden away and wrapped up. All that we the user is provided are insights! Constant and repeated exposure to performance dashboards is making all of us into analysts. All of us have already made the first leap into data science without knowing. Of course, the real craft is more rigorous and requires a mathematical and computational grounding. However, the craft it is not a distant for anyone thinking about exploring a career in it.

Buying a cup of coffee

The last example I would like to visit to show that we all use data analysis in our daily lives is the mundane task of buying a cup of coffee from a local cafe. I have always felt overwhelmed when I walk into a coffee shop with the plethora of choice available. What size drink, what strength coffee, how much milk, how hot, sugar, milk alternative, flavourings, price, eating in/ take out. Phew!!! Our brains take in all of this information and work our way through sequentially arriving at our final choice as we approach the person across the counter. I have always found this a bit overwhelming and fascinating. Most of the data science when condensed to a super simplistic approach is almost akin to this. We collect data, process it and run through an algorithm to arrive at a decision. Imagine doing a similar task at a much larger scale – with electricity grids, forex prices, rail networks, drug discovery with molecular shapes, composition and pathways to select etc. All of them go through the same steps albeit with a more mathematical approach.


Anyone thinking about taking up a profession in the field of data should not think no that it is a distant unattainable profession. They are most likely already using tools, performing tasks and employing those aspects of their grey cells that will be called upon if they get into the profession. It is just a question of putting oneself through a structured programme to learn rigorous techniques that are employed by professionals. It is as simple as – Decide, plan, sign up for a course, train, qualify and become a data professional.