Common Mistakes in Data Science – Part I
Part 1
Here are 10 Data Science bloopers that industry-observers advice all data science professionals to steer clear of:
The Data Science domain is constantly on an upwards swing across all industries. Data scientist and data analysts enjoy a tremendous demand in the job market. And going by historical trends, it is a steady growth. In terms of demand for qualified Data Science professionals, India is second only to the US. A study conducted by Analytics India Magazine early this year estimated that nearly 50,000 Data Science vacancies are currently available in India alone.
However, with growth and acceptance comes responsibility. While it is a fact that Data Science is still not a primary career choice, this means that the current talent shortage will continue despite great demand. And Data Scientists are coming under relentless scrutiny owing to the high amount of visibility their work garners. Experts advise that at this crucial juncture, all data science professionals – current or aspiring – must steer clear of the common mistakes they usually tend to commit. Let us cast a quick glance over a few such blunders that industry-observers point out.
In this first part, we consider 10 Data Science bloopers:
- Lack of a proper plan: Data analysts are supposed to proceed from piecemeal data towards the big picture. Any given data science problem seeks the answer to why the concerned data behaves the way it does – and looks at the narrative that is emerging from the dataset. Addressing this situation requires unambiguous methodology. Thus, setting off without an appropriate plan or strategy that can serve as a roadmap can be suicidal.
- Aimless data collection: Even with the best of plans, things will go awry if data is collected the wrong way. While including as many parameters as possible for ensuring data quality and quantity is crucial, it is equally important to record the data correctly. Too many independent variables can be detrimental in designing predictive ML models.
- Lack of visual thinking: Visualisations are critical in data exploration and help you either spot patterns or trends. Choosing an inappropriate visualisation means not being able to make the most of the collected data – or, even worse, missing the data trends completely. A data professional must know the suitable visualisation tools and what graphs and charts can best describe the gathered data.
- Relying on intuition: Data scientists often work on theories based on intuitive predictions regarding what a particular data set might reveal. This is common – but can lead to vital misinterpretation of data. Doing an exploratory analysis of the collected data is more important, rather than relying on individual hunches.
- Not considering inherent bias in data: Data professionals often don’t have a say in how or where the data is collected. Hence, while planning a methodology to use a given data set, they should consider whether there is inherent bias present in the data. They also need to judge whether the collected dataset is an adequate representation of the entire population. Looking for inherent biases in the data early on helps avoid skewed models.
- Overreliance on algorithms: While algorithms are crucial to data science for detecting patterns, they are not always infallible. Depending too much on past data-driven algorithms might actually be counterproductive. Looking into the past as a sole predictor could prevent data professionals to seek out new possibilities from an existing dataset.
- Not optimising the model: Your model must be optimised for the data you have and follow the change in data over time. And such optimising should not be a one-time activity. Every time there are changes in the data itself, the parameters must be modified to match the changes. In machine learning, this falls under optimising the values of hyperparameters to reach peak performance.
- Optimising for the wrong goal: This is another extreme of the optimisation game. It happens primarily because data scientists prefer automation to monitor the outcome. With control groups, it is equally important to measure the quality of the output and monitor it throughout the process.
- Ignoring control groups: Not using control groups to test a new data model can be suicidal. An overenthusiastic data professional might start randomly deploying a model to make the most of it, you might want to use it everywhere. However, that is impractical. It is acutely important to identify groups who do not use the model but would want to trust it.
- Borrowed implementations: It is tempting to adopt implementations with proven success – but in data science, what worked for one might not work for another. Given the large number of open-sourced algorithms that exist in the market, it is easy to prototype a model. However, not all implementations come to the rescue when it comes to finding a solution that fits your problem.
Look out for more Data Science blunders in Part 2
Know more about the syllabus and placement record of our Top Ranked Data Science Course in Kolkata, Data Science course in Bangalore, Data Science course in Hyderabad, and Data Science course in Chennai.
https://praxis.ac.in/old-backup/data-science-course-in-chennai/
