The world of Data & Analytics is rarely a one-person show. In fact almost every large organization will have entire teams and departments dedicated to capturing, storing, and using data. With that comes a variety of different specialized roles that each play a vital part in the process. While each company will be setup a little bit differently and may use roles in their own way, what is outlined below represents the general industry definitions of the roles.
Core Disciplines
Data Analytics
This discipline is all about answering business questions with data but the work comes in all sorts of shapes and sizes. It can be an simple as plotting a few points on a graph and visually looking for a trend all the way to complex statistical or machine learning techniques on large datasets. In most cases data analytics is focused on answering new business questions on more of an ad hoc basis rather than answering the same question over time. When an important insight or KPI (key performance indicator) is found through analytics and is turned into a dashboard or report that is updated on a regular cadence we generally consider that to be business intelligence work even if one person is doing all of it.
Generally speaking data analytics work will follow the same pattern regardless of complexity and time to complete. It starts with a business stakeholder who has a question or hypothesis about the business but wants to verify using data. An analyst or team of analysts will work with that stakeholder to refine the question and talk about what possible solutions or output may look like. Once that is moderately well defined the analyst will explore various data sources to see what data is available and what data is missing. This could involve anything from exploring a well defined defined data warehouse, to looking through raw tables straight from applications, to looking through spreadsheets. The analyst then pulls together the various data elements that are required and starts to put together their analysis or solution. Their output could be anything from looking for simple trends in Excel, to some analytical models created in R or Python, or in many cases is just a report with useful KPIs. The analyst then brings this back to the business stakeholder and presents their findings. This could be the end of the analysis or more commonly turns into a feedback session where the business question is better refined and the next iteration of the analysis follows.
Typical Data Analytics Roles
Data Analyst: Of all of the job titles on this list this one may be the mostly broadly used. In most cases a data analyst does all of the steps outlined above meaning they need a strong combination of technical skills to organize and analyze data, analytical thinking skills so that they can pull the right inferences from the data, communication skills for when the meet with and present back to stakeholders, and business knowledge so that they can properly understand the business problem.
To be an effective data analyst you need both technical and soft skills. From our experience it is much more common for someone to not get the role they are interviewing for because they are unable to effectively demonstrate their handle of the soft skills rather than their inability to code in SQL. That’s why we offer our services, because 99% of the content out there is about the technical side and very little exists on the other pieces you need to be effective in the role. We strongly consider you setup time with us to help asses where you are currently at, but if nothing else please read this deeper dive on what it takes to be a data analyst. Data Analyst Skills
Business Analyst: Generally speaking the term business analyst can be used two different ways. First it refers to someone in a role similar to a data analyst but where they spend more time analyzing business problems and less time on wrangling data. They generally sit on a specific business team and get deep into a particular subject area. While they are capable of wrangling some data, they usually use fit for purpose data marts that can be used over and over again to answer their various questions.
The second way that the term business analyst is used is for someone who spends an extensive amount of time with business stakeholders so that they can capture their requirements and translate them for developers and engineers. While this term has historically been used to talk about software and application development the role is also starting to expand into the data world. People who have a good balance of technical understanding (though not necessarily coding skills themselves) and business understanding will gather requirements for the technical team to execute on. Unlike the data analyst role that does more end to end development this role is part of a team where roles and responsibilities are more specialized. We generally would not consider the work here part of the data analytics discipline. Instead it falls more under the data engineering work we will talk about later.
Business Intelligence
Business Intelligence is the practice of using data to evaluate and monitor the health of a business so that it can be run more efficiently. This often comes in the form of metrics and KPIs (key performance indicators) that appear in dashboards and reports. While business intelligence can be used by anyone in an organization the majority of BI is built for high level leadership who has to make strategic decisions. Depending on the intended audience the dashboards and reports usually focus on a particular division or aspect of the business (ex. HR, sales, operations, etc.) and the content of the dashboard tells an easy to understand story for what is happening.
The line between business intelligence and data analytics is often blurry, however there are a few key differences that generally separate them. First business intelligence usually refers to reports and dashboards that have long shelf lives where the KPIs are static or slowing changing. For example you may have a sales dashboard that shows total sales revenue month by month. While the value of that metric obviously is constantly changing, the business rules are not. If at some point there is a dip in total sales revenue leadership can identify that through business intelligence and investigate. The investigation is likely a one-time deeper dive into the data to try and discover the root cause. This would be considered a data analytics exercise, generated from a business stakeholder who used a business intelligence dashboard.
If you want a deeper dive into the nuts and bolts of business intelligence check out this article from TechTarget https://searchbusinessanalytics.techtarget.com/definition/business-intelligence-BI
Typical Business Intelligence Roles
Report Developer: This role has been around for many years and likely has the lowest barrier to entry of but also the lowest pay grade. Generally this role is focused on creating simple to moderately complex reports that go to business stakeholders or leadership on some recurring basis. The reports are usually some mixture of grid reports (think of an Excel file), summary reports (values rolled up into KPIs), and basic charts and graphs. The reports usually have little interactivity and are very fit for purpose with little value outside of their intended use. While this may not sound like the most glamorous of work in reality these are great roles to break into the industry, learn about a company’s data and business processes, and build relationships with important stakeholders. Also while many organizations are striving to have cutting edge reporting and analytics, these types of reports are still the backbone of many organizations and likely will be for another 5-10 years.
Business Intelligence Analyst/Developer: These roles are the next evolution of the report developer. Much like the the report developer they are focused on providing data, KPIs, and charts/graphs to their end users. The key difference is the sophistication of the delivery methods and the flexibility/potential these provide for their users. Rather that static reports, BI developers are building interactive dashboards in tools such as Tableau and Power BI. When combined with flexible data models this allows the dashboards to do things such as drill downs, variable filtering, and in some cases even embedded analytical models where users can input their own parameters. This means that these dashboards can often be used by multiple areas of the business to answer questions on their own even if they weren’t the original intent of the dashboard.
When building these more advanced dashboards organizations tend to use a full spectrum of team structures. We have seen cases where a BI developer may do full end to end development including gathering the requirements, building the datasets, and building the dashboards while in other orgs a full project team will be assembled where each person plays a specific role for the build and support. If you are interested in a role like this it is important to be very meticulous in your job search and interviewing process to ensure the organization works in a way that you are comfortable with. If not you could end up in a role where you won’t be happy because you are doing too many or too few things relative to your preferences.
Data Science
Data science is likely the most well known while perhaps being the least understood of the disciplines that fall under data & analytics. At its core it is about using advanced analytical techniques and large amounts of computing power to pull insights from data to answer business questions. What makes data science different from traditional statistical methods is that most of the data science techniques do not rely on the strong assumptions about distributions of data that traditional statistics does. The modern techniques also allow the processing of raw, semi-structured, and unstructured datasets that were impossible to analyze in the past. While this allows data science to be applied in nearly any situation with any dataset it also makes the results much more difficult to interpret. Where many practitioners and companies fall short is not in applying the data science techniques, it is in applying the techniques in a way that produce a result that can be acted upon. This in my opinion is the number one differentiator between the haves and the have nots today.
For a more in depth overview we suggest you check out this article https://searchenterpriseai.techtarget.com/definition/data-science or some some more research on your own.
Typical Data Science Roles
Really only one role worth talking about here and that’s the role of data scientist. There are likely some other roles that do similar type work such as actuary, statistician, or big data analyst but for the most part the term data scientist dominates the market today. If you see other roles out there it is likely the work differs in some significant way from what you’d expect with a data science role
Data Scientist: Data scientists are widely considered the rock stars in the world of data analytics. They are highly educated, highly technical, and highly in demand and they command salaries that reflect that. Their job is to answer complex business problems using large amounts of data and/or advanced techniques that a typical data analyst won’t be able to do. While data scientists usually have the SQL skills needed to wrangle their data, they differentiate themselves through programming skills (such as Python and R) and are comfortable using raw, semi-structured, and unstructured data sources.
Adjacent Roles
Data Engineering
Data engineering refers to the processes of creating database objects and moving data from one place to another. Usually this is done by an IT team so that the recurring processes can be built in a robust manner that will live for years. In the world of data & analytics data engineering usually centers around the process of pulling data out of source systems, where most people wouldn’t have access and the data would be hard to use, and putting it into a data warehouse or data lake layer in formats that are easier to consume, thus allowing a larger portion of the company to make use of it.
Data Engineering Roles
Data Engineer/ETL Developer: Many people may argue that these roles are not the same but for the purpose of how we think of the world we will lump them together as any differences are nuanced and immaterial for those focused on analytics. These roles are almost always squarely in IT and focus on creating recurring processes to Extract, Transform, and Load (ETL) data from one place to another. While the data that is being moved is critical to analytics, the people who fill these roles are generally not good candidate to move into data analyst type roles and vice versa. The processes that are built are very focused on repeatability, resource effectiveness, and quality. The people in these type of roles generally have little interaction with the business and get requirements from business analysts or other tech savvy users of their data.
Data Quality Analyst: This is a newer role and may not exist in the majority of organizations yet. As the name suggests this role is focused on data quality, basically making sure an organization’s data is accurate and complete. Because data quality is expensive organizations that have roles dedicated to it they tend to focus on the widely used areas such as a centralized data warehouse or data marts. Much like their software development counterparts, these roles focus on creating recurring tests that can be automated and reused over time as data grows and changes. Also like many other aspects of data & analytics, tools and AI promise to greatly improve data quality with minimal effort. While this may be true one day the majority of organizations are reliant on humans to ensure their data is correct.
Data Governance
Data Governance refers to the people and processes that focus on maintaining the quality of a company’s data. This can manifest itself in a few different ways. First there will be people who have roles dedicated to keeping data accurate and complete. That means not only preventing mistakes but ensuring business rules actually make sense in how they describe the business. Data governance, as the name implies, also involves setting policies and standards for those using and creating data. What many people don’t realize is that a lot of data is actually created by end users who might not have traditional data roles. Often times they create reports with logic that becomes widely consumed but hard or impossible to replicate outside of that one report. Data governance helps to break down those silos so that people don’t use data in a way that they shouldn’t while also create ways for valuable business rules and logic to make it back upstream so that it can be used by many.
Data Governance Roles
Data Steward/Owner/Custodian: Similar to the data quality analyst the data steward role is there to help with data quality. In many organizations there is not consensus when it comes to who “owns” the data. Not in terms of legal ownership (that is obviously the company) but rather who gets to make decisions, most commonly around business rules and access to the data. The steward role makes it perfectly clear by assigning a business SME to oversee a portion of the company’s data and make those decisions in an effective way. They can also serve as someone to help answer questions about the data, especially when an organization is lacking good metadata (and most are). In some cases this may be a full time dedicated role (meaning that person has no other responsibilities) and in others the stewardship duties may be in addition to their primary role. Either way you can expect to see rapid growth in this area as companies continue to increase their investment in data.
Platform Administration
Platform Administrator: Platform administrators, or platform admins as they are more commonly referred to, handle the day to day administration of tools and applications within a company. These roles are not specific to data & analytics but instead apply to pretty much any application that is used by a company. However from my experience admins usually focus on one particular area at a time. For example the administrators you have for analytics tools probably don’t also work on client facing applications. They stay in their particular space where integrations across platforms are most common and the user base is likely similar across applications. They do things such as provide new users access, monitor server usage and capacity, and turn on new features as they are available. They generally know a lot about what a tool can do even if they aren’t experts in the tools themselves. For example your Tableau admin likely knows a lot about how Tableau works but may not be very adept in making dashboards themselves.
Platform Engineer: These are the people behind the scenes getting all of the technology up and running so that data analysts and others can use it. For example they may set up some servers and then install Microsoft SQL Server so that you have a usable database. They will have to make important decisions about how much CPU and memory to provide, how to back up the system in case of failure, and how to plan out upgrades to minimize down time. Even as more organizations move to the cloud platform engineers continue to have roles to ensure everything is setup and configured correctly and that the overall user experience of the platform is a positive one. If not you may struggle to get the business to adopt your dashboards and reports if they hit technical issues while trying to do so.
Database Administrator: This role has been around a long time and as the name implies they are administrators of databases, similar to the platform administrator above. Depending on where you work and your day to day responsibilities you may talk to database administrators (DBAs) daily or almost never. If for example you work in a place where you aren’t permitted to have your own schemas and tables but still own recurring processes that are widely used and often need some sort of updating then chances are you’ll be on a first name basis with your DBAs. They’ll likely help you to create your tables/objects, security roles, and implement changes for you as needed. If you work in a place where you have a lot of freedom to self-serve and storage and compute power aren’t overly stressed then chances are you’ll only talk to DBAs on occasion when you need new schemas or have run-away processes. Overall having good DBAs who really take care of the system while not neglecting their users makes a huge difference for your day to day work in data & analytics.
Data Modeling & Architecture: Any time that you create a table or a view you are in essence doing data modeling. Having said that almost all large organizations are going to have high level technical staff who specializes in creating efficient data models that can be scaled while maintaining performance. Almost any Enterprise Data Warehouse (EDW) table will be built by some sort of architect and in some cases they may also build the fit for purpose datasets used by smaller audiences, though this is less common. Architects are generally some of the companies most technically advanced employees and really need to understand both the systems as well as the business processes really well to be effective. When architects do their job well you end up with datasets that are flexible enough to be used for a wide range of purposes while also being robust enough to withstand the inevitable changes that will happen over time.
As you can see there are a wide spectrum of work and roles that go into making use of a company’s data. What I have listed above are the generic high level roles. There are likely many more specialized roles and titles that are variations of the generics.