Vicissitudes in Appraising Reading Difficulty

Chuck Schultz

Writing difficult sentences was once seen as a mark of advanced learning. Now, it is more likely seen as insensitivity to the reader. To the test developer, it is seen as bias. Tests should ve written to measure content unencumbered with reading difficulty.

Many indexes or reading difficulty have arisen to help the writer simplify understanding. Lucid writing is less likely to contain long words and complex sentences. Several factors account for this. Abstractions are more likely to be expressed in long words than are concrete concepts and abstractions are harder to understand. The most familiar words tend to be shorter. Readers are likely to loose their train of thought in complex sentences.

It is not surprising that simple indexes based on word length and sentence length are related to reading difficulty. But the indexes tell only part of the story. Interesting material, vivid language, familiar references, personal references, quotations, white space, effective organization, and credibility all contribute to reading ease. In addition, a reading index can be tricked by following the formula without attending to the other factors just mentioned.

The meaning of a difficulty measure's index value or grade level is questionable. Although the developers attempt to establish meaningful values, I consider them rough guides. Word® Six provided grade levels from three different indexes and they did not agree. The later versions of Word® give only one grade level so that users are not as aware of being ignorant of what it means.

I have used the fog index for the past 25 years because I was familiar with it and it was easy to use. To compute grade level, you add the average sentence length to the number of words per hundred that have more than three syllables then multiply by 0.4. Now I get the Fleisch-Kincaid grade level from my word processor. It is based on sentence length and the number of characters per word. Below, I present sentences and paragraphs to illustrate some readability concepts. I show the fog index and the Flesch-Kincaid grade level for each of them; these indexes often show quite different results.

When you want to use terms like criterion-related validity and correlation coefficient, it is hard to keep down your number of long words per hundred. I learned years ago to lower the fog index by inserting short words. Paragraph A below presents a straightforward statement about validity, but yields a fog index of the 27^th grade level, whatever that means. Paragraph B use exactly the same words plus some shorter words, and the fog index goes down from 27 to 15.

A. Criterion-related validity, or empirical validity, is a statistical relationship between job performance measures and selection procedure scores. Criterion-related validity is expressed as a statistical index, the correlation coefficient.

[fog = 27, Fleisch-Kincaid = 12.0]

B. The most common way of showing how good a test is has long been criterion-related validity, or empirical validity, which uses a kind of statistical relationship between job performance measures of people in a job and their test scores, which are also called selection procedure scores. This gauge, criterion-related validity, is most often expressed as a statistical index, which is called the correlation coefficient.

[fog = 15, Fleisch-Kincaid = 12.0]

Is the paragraph with the lower fog easier to read? Some readers get lost in the short words of the second paragraph that mechanically reduce the fog index. Both paragraphs have the Flesch-Kincaid of 12.0.

In Paragraph C and D, I attempt to express the same content in simpler and simpler terms. The readability indexes decrease accordingly.

C. How good a test is for selecting workers depends on how well it reflects what the workers do on the job. The worker's output is one criterion for judging a test. Comparing test scores to output shows whether the test identifies good workers. A correlation coefficient is used to show how well test scores match criteria. Researchers call the way of telling whether a test is good, criterion-related validity.

[fog = 11, Fleisch-Kincaid = 8.6]

D. We want to see if a test selects good workers. One way to see this is to look at what the workers do. Do people with high test scores do better work than people with low scores? If those with high scores are better workers, we say the test is valid. A Criterion shows how well people do on their job. A correlation coefficient gauges criterion-related validity. It measure how well work and test scores match.

[fog = 7, Fleisch-Kincaid = 4.9]

Which paragraph, A through D, would convey the material most effectively to readers of this newsletter? Or would you pitch them all and focus on flow rather than fog?

What does it seem that I am doing with Paragraph E?

E. An index of accord of a value that depicts a trait and a gauge of toil yields the symbol of test merit. This index is, in fact, the cosine of the vectors for the trait and the toil. Test scores often depict traits. The toil trace denotes a criterion. The term required cosine means the cosine mentioned above. Therefore, finding the required cosine denotes criterion-related validity of a test.

[fog = 8, Fleisch-Kincaid = 6.6]

This is what I call tricking the indexes. The sentences are short, most of the words are short, but how easy is it to read? If you already understand the concept, you may be able to discern what the author means by the various words and phrases, but I'll bet you went back and read part of it more than once. Even then, you may feel the author has made some awfully poor word choices.

In this discussion of the paragraph, why do you suppose "the author" has been used instead of "I"?

You can write interesting or dull passages using long or short sentences and long or short words. Let's look at some topics that are arguably interesting or dull: such as, sailing and geology. Within either topic, writing can be interesting or dull depending on style and word selection. Can geology ever be more interesting than sailing? The number paragraphs below are my attempt to contrast interest and difficulty levels for both topics.

1. The hold on a rope could save a sailor from waves washing over the boat deck.

[fog = 6, Fleisch-Kincade = 4.6]

2. Can Mike maintain his tenuous grasp on the main sheet as wave after wave rips at everything on deck to see how securely it is fastened down?

[fog = 15, Fleisch-Kincade = 11.1]

3. Mountains are made by lava forced out of the ground or by tilting of the earth's crust..

[fog = 6, Fleisch-Kincade = 4.9]

4. Mountains appear when a violent crack in the earth's surface issues forth liquid rock or when a gradual shifting of the earth's crust stands several square miles of rock on end.

[fog = 16, Fleisch-Kincade = 12.0]

I hope I have made the point that you can have highly readable material although readability indexes say differently. The more practical message is that a low readability index does not guarantee readable writing. We must still look at our writing to see how it flows. The fog index is a nice tool, but writing is still an art.

Chuck Schultz may be reached at (360) 923-5340, 2941 B Firwood Loop SE, Olympia, WA 98501-4844.