Juicy Studio Readability Test: Contradictory Results

For the current text of the “Clear Helper” Web site home page, I conducted the Juicy Studio Readability Test.  Results from the readability test’s automatic calculation were significantly different from those derived from my manual calculation.  I don’t know how to account for this.  It makes me distrust the test.

Automatic Summary Results

I submitted the URL of the Clear Helper home page.  A summary of reading level results was produced.

Total sentences 59
Total words 370
Average words per Sentence 6.27
Words with 1 Syllable 211
Words with 2 Syllables 66
Words with 3 Syllables 47
Words with 4 or more Syllables 46
Percentage of word with three or more syllables 25.14%
Average Syllables per Word 1.81
Gunning Fog Index 12.56
Flesch Reading Ease 47.73
Flesch-Kincaid Grade 8.16

Note the last three results.  They indicate that someone with almost 13 years of education or someone in the eighth grade could understand the current text of the Clear Helper home page.  The Flesch Reading Ease scorer of 47.73 falls below the ideal of 60 – 70.

These scores and their indications are worse than those I determined by following Juicy Studio’s instructions on how to calculate the scores manually.

Gunning-Fog Index: Manual Calculations

  • This test “… is a rough measure of how many years of schooling it would take someone to understand the content.”
    • To calculate this score, as instructed, I added the average number of words per sentence (6.27) to the number of words with three or more syllables (93) and multiplied the total by .04. Score = 4.
      • This indicates a fourth-grader would be able to understand the home page text.
        • Note: I think there is an error in Juicy Studio’s instructions.  It first says to use the percentage of 3-syllable words but, in the formula, indicates the number of 3-syllable words should be used.  Calculating the formula with the percentage produced a score of 1.  Because it seemed quite unreasonable that a first-grader could read the home page text, I instead calculated the formula using the number of 3-syllable words.

Flesch Reading Ease: Manual Calculations

  • For this test, “… the higher the score, the easier it is to understand the document.” The ideal score is 60 to 70.
    • To calculate this score, as instructed, I subtracted the sum (153.126) of 84.6 multiplied by the average number of syllables per word (1.81) from the sum (6.36405) of 1.015 multiplied by the average number of words per sentence (6.27). Result = -146. 76195. I then subtracted this result from 206.835 to achieve a score of 60.
      • The score of 60 falls within the ideal range.
        • Note: In calculating the score, I had to change the negative result number (-146. 76195) to a positive number.  It was the only way to produce a reasonable result.

Flesch-Kincaid Grade Level: Manual Calculations

  • This test “… is a rough measure of how many years of schooling it would take someone to understand the content.”
    • To calculate this score, as instructed, I multiplied the average number of words per sentence (6.27) by 0.39 and added its sum (0.24453) to the sum (21.358) of the average number of syllables per word (1.81) multiplied by 11.8. Result = 21.60253. I then subtracted 15.50 from the result. Score = 6.
      • This indicates the home page text requires a sixth-grade education to understand.

Conclusions & Speculation

The results from the manual calculations indicate someone with a fourth- to sixth grade education should understand the home page text, and that its readability falls within the ideal scale.  This is significantly better than the results and the indications of the readability test’s automatic calculations.

I wonder if the different results are due to the consequence of following contradictory- and confusing instructions about how to perform the manual calculations.  Perhaps the errors are mine.

I will have to revisit this at a later date.

Tags: ,


%d bloggers like this: