Click here to download a pdf version that is tagged for screen readers.
 |
| www.lumaxart.com (Flikr Commons) |
People and
organizations sometimes calculate percentage statistics in
an effort to assess the effectiveness of traditional or innovative approaches
to increasing success or efficiency, or reducing problems, and so forth.
NOTE:
Percentage statistics are simple numbers such as 2% or 87%. They range from 0% to 100%. Percentages are another way of stating a
fraction. For example, the fraction 1/4
can be stated as 25%. The way to convert
a fraction to a percentage is to convert it to a decimal format then multiply
it by 100 (e.g., 1/4 = .25, .25 x 100 = 25%).
 |
| Frits Ahlefeldt-Laurvig (Flikr Commons) |
Although
percentage statistics can provide a useful view on a particular phenomenon, the
use of percentage statistics brings with it substantial risk of interpretive
error, particularly when they are compared to the last measurement of the same
phenomenon, e.g., a year ago, AND, they are interpreted with a confident and
authoritative guess.
Stated by
themselves (i.e., without statistically valid comparison), percentage
statistics require a qualitative interpretation--essentially someone's best
guess. As with all guessing, they might
be right, and they might be wrong.
For example,
suppose someone said "In 2008 the Department
of Audits had a 10.9% rate of employment of workers with disabilities, and one
year later they raised the percentage to 28.3%.
As can be seen, the Department of Audits has gotten serious about hiring
people with disabilities."
Before we drill down, let's first ask: What are the statistics, and what is the qualitative guess?
Before we drill down, let's first ask: What are the statistics, and what is the qualitative guess?
Statistic: 13 employees with disabilities reported in 2008 (10.9%), and 41 in 2009 (28.3%).
Qualitative
guess:
- Audits is serious about employing PWD's.
- Audits hired a lot of PWD's in 2009.
Now, let'sfact check that guess. Here is the state's data on the Department of Audits for 2008 and 2009:
Indeed the number of PWD's reported by Audits approximately
tripled in a single year, from 2008 to 2009. Thus, the statistic is confirmed. What about the qualitative guess? What other
factors might explain the number of PWD's tripling in a single year?
 |
Alternative hypothesis: Audits didn't hire more PWD's, it merely reclassified some of its employees as disabled. If someone wears glasses, they can be counted as disabled. If they start to wear a hearing aid they can be counted as disabled--even if these state workers
don't consider themselves to be disabled.
We have two competing explanations!
Let us look to trend statistics to see if they favor one explanation more than the other. The trend figure below was created from 10 years of state data for the Department of Audits. The state's data is presented in two tables a bit further below.
Let us look to trend statistics to see if they favor one explanation more than the other. The trend figure below was created from 10 years of state data for the Department of Audits. The state's data is presented in two tables a bit further below.
- In two years, (04/05 & 08/09) the percentage of PWD's approximately triples. In one year (12/13) it increases very slightly.
- The rest of the time--7 years--the percent of PWD employees decreases.
- None of the four main race groups show similar doubling or tripling of representation rates. (Because you can't reclassify someone's race).
- Importantly, notice there are no hiring spikes in 2005 or 2009, which you would expect to see if Audits was hiring people with disabilities. If they were, there would be increases in most or all races, corresponding to the number of PWD's hired. Remember, a PWD is counted twice, by disability and race.
- Audits didn't hire people with disabilities.
- Audits reclassified a bunch of employees as "disabled."
- Audits should not get credit for increasing rates of employment for PWD's, because they didn't do the work.
- Audits should use the LEAP program and hire some PWD's who are looking for work.
In another blog article, coming soon, we will examine many state organizations with the same, strange spikes in the number of employees with disabilities. Taken together, these suggest the state may be claiming higher rates of employment of PWD's, when actually all they are doing is re-classifying already hired, already working employees from not disabled to disabled.
We already learned in another blog that the state's rate of hiring people with disabilities is much lower than for the four race groups (click here to see that blog article).
In another forthcoming blog article we will see that the state's rate of hiring of PWD's under the age of 30 has dropped 68% in the last 10 years. So, it may be the state has some explaining to do. There is an appearance of a shell game.
We already learned in another blog that the state's rate of hiring people with disabilities is much lower than for the four race groups (click here to see that blog article).
In another forthcoming blog article we will see that the state's rate of hiring of PWD's under the age of 30 has dropped 68% in the last 10 years. So, it may be the state has some explaining to do. There is an appearance of a shell game.
TAKE HOME
LESSONS THAT WILL SERVE YOU WELL AS YOU STRIVE TO DEVELOP SCIENTIFIC THINKING
AND SEARCH FOR CREDIBILITY IN DATA AND EXPLANATIONS
- When you hear interpretations of percentage statistics, be cautious.
- Develop the habit of questioning percentage statistic data.
- Develop the habit of looking at trend data to evaluate claims based upon percentage statistics.
- Develop the habit of asking to see source data, then checking to see if it is correctly represented.
- When possible, avoid seeing any significance in isolated percentage statistics.
Please cite this
as: Nelson, Eric L. (2015). The Problem
With Percentage Statistics. Trends in
State Work, http://trendsinstatework.blogspot.com/2015/05/the-problem-with-percentage-statistics.html
