We’ve all heard that “numbers don’t lie.” Technically that’s accurate. What numbers don’t do, however, is tell the entire truth. In our current society’s demand for more information delivered at a faster pace, I’d argue that most Americans are accepting exactly what they are shown without questioning it, which has led to the state of constant disagreement over subjects such as the severity of COVID, the impact and value of social equality protests, and who’s leading in the political polls. Read one article that validates your stance, and you’re a genius. You’re likely to read a second and discover you’re flat-out wrong. Read a third, and you’ll get confused to the point where you give up trying to figure it out and resort to whatever your base instinct or desire commands. Is using that same approach how we want to operate a business?
Every business uses data. HOW they use their data is what differentiates poor from marginal, and good from great. As an example, most businesses that transitioned all employees to working remotely during the past 6 months have asked themselves how they could accurately track employee performance without being in the same physical space. Popular metric groups include productivity (how many activities, projects, or tasks get completed in a given time frame), attendance (how long are employees logged in each day? Are they actually working when logged in?), and performance (feedback from customers, co-workers, etc.). Your agency management system (AMS), customer relationship management system (CRM), e-mail client, and phone system can easily provide you with numbers to assess those. Many even have built-in analytics programs to provide you with an easy-to-read dashboard to help drive your business decisions. You have all the information you could ever need, right at your fingertips. Wrong.
Numbers aren’t the treasure; they are simply clues on the treasure hunt. Numbers are designed to signal when things are out of the norm. Once you notice the signal, it’s time to start asking questions and designing experiments to provide us with other metrics that will help us understand why they are out of the norm. Let’s use the following example: your business needs to furlough one employee due to COVID-related impacts. You’re considering furloughing either Employee A or Employee B. If Employee A handles 150 customer requests a day, and Employee B handles 75, clearly Employee A is the more productive employee. It certainly appears that way… until you start asking questions.
Would your opinion change if I told you that Employee A handles requests that take an average of 2 minutes to complete, and Employee B handles requests that take an average of 5 minutes to complete? Using simple math, Employee A is responsible for 5 hours of productive work per day (150 requests x 2 minutes / 60 minutes per hour), while Employee B is responsible for 6.25 hours (75 x 5 / 60). Knowing that Employee B is now the more productive employee, right? Let’s furlough Employee A!
Now imagine that I tell you both Employee A and B are processing the same types of requests. The first question I’d ask is why it takes Employee B more than twice as long to process a request as it does Employee A. What might lead to that? Our phone system indicates that Employee A’s average customer request phone call lasts 60 seconds, while Employee B’s last an average of 180 seconds. Why does Employee B spend three times as long on a phone call as Employee A? Is a higher quality of customer service being provided? Possibly. We could cross-reference our individual call times with the corresponding customer reviews to determine that. Does Employee B include more complete information in processing the request than Employee A? We could look at quality control auditing scores to determine that.
“Listen, you’ve got my head spinning. I don’t know who the more productive employee is. Just tell me what to do. If you don’t, I’m going to furlough Employee B because he/she/they complete half the number of requests as Employee A, and I’m afraid keeping Employee B would lead to a backlog.”
Based on the data we’ve looked at above, there is no correct answer to the question, “which employee should we furlough?” We need a better question. Is our goal to avoid backlogs, the resulting overtime pay to remaining employees, and the potential frustrations from overwork? Or is our goal to provide the best experience for our clients? Perhaps client retention is our top priority? Until we know the question we want to answer, the numbers we can point to aren’t applicable. Our solution needs to fit the problem we’re trying to solve.
“Ugh. Explain to me why I can’t use the information on my analytics dashboard to determine who is the more effective employee?” Here’s one simple answer: almost every “average” statistic you see is a calculated mean. Is the mean the statistic you want to use? Or do you want to use a more accurate measure of central tendency? (If you’re asking yourself what a “measure of central tendency” is, let me take you back to junior high math class, where you learned that measures of central tendency consist of – at minimum – the mean, median, and mode.) Do you really want to know the mean? Or would it be more effective to know the most prevalent outcome? If it’s the latter, is your dashboard showing you the mode? Is it even capable of that? Perhaps you want a statistic that’s more advanced than what your average 13-year-old could calculate?
As stated earlier, numbers are not the treasure; they are clues on the treasure hunt. No one stumbles upon the “X” more than once or twice. When you are making decisions about your business’s success, your employees’ livelihoods, and the safety of your clients, would you want to rely on stumbling upon the “X” for success? That means it’s time to put in the work to effectively use data, which means questioning EVERYTHING.
Questions you need to ask when presented with data (quantitative or qualitative):
- What problem are we trying to solve?
- What data will help us understand the problem?
- What sources are we getting our data from?
- How reliable are they?
- What agenda might they be trying to push?
- How did they gather that data?
- What inconsistencies may exist in their collection methods?
- How representative is the sample?
- What information are they NOT sharing with me?
- What reasons exist for why they might NOT be sharing that?
- How reliable are they?
- Do I understand all the potential factors that may have contributed to the outcome represented by the data?
These questions are the treasure hunt. With each possible explanation, an additional series of questions arise that require additional data. If your organization can learn to love the process of the hunt, you’ll be much richer in the long run.