Flesch-Kincaid: Threat or Menace?

[mabfan]

For the past week I've been leaving the novel behind. Instead, I've been working on an outline for a young adult novel I've wanted to write. It's a time travel story about a teenage boy who has to stop his future self from destroying history. I'm using it for my application to the Boston Public Library's Children's Writer-in-Residence program. I imagine I have a whelk's chance in a supernova of getting this fellowship, but it can't hurt to apply.

As I was working on the proposal, I started thinking about readability. Although I aim for a transparent style in my fiction, this was the first time I was developing a story for a younger audience. If this book is going to sell to teenagers or preteens, I'm going to need to make sure I write it at their level.

One of the many books on writing I own is FICTION WRITER'S BRAINSTORMER by James V. Smith, Jr. Smith has a chapter called "A Brainstormer's Guide to Revision and Editing" in which he describes the way he analyzed bestselling fiction. He typed into his computer selections from ten bestselling novels, one each by Fannie Flagg, Kaye Gibbons, John Grisham, Jan Karon, Stephen King, Elmore Leonard, Terry McMillan, Anna Quindlen, Danielle Steel, and Wallace Stegner. And then he ran them through his grammar checker to find out what the Flesch-Kincaid scale said about them.

I'm not going to go into all the details about the Flesch-Kincaid scale; you can find information on it by searching the web or picking up a good reference book. All we need to know for our purposes is that there are two ways to rank prose. Either we can give it a "readability" score that is cast as a percentage, or we can give it a grade level, which is based on the twelve standard grades in the American educational system.

So, for example, a story with a 75% readability will be understood by 75% of readers. A story with a grade level of 8 will be understood by anyone with an 8th grade education or higher.

Smith did his analysis with an open mind. I imagine that most of us would expect that the more commercial writers would score higher on readability and lower on grade level. And perhaps we'd see the reverse for the more literary writers.

Smith's results surprised him. Despite deliberately choosing a mix of commercial and literary writers, he found that many of the results fell into the same range. The average in four separate categories was as follows:

The amount of passive voice the writers used ranged from 2.3% to 13.43%.

The number of characters per word ranged from 3.72 to 4.58.

The readability ranged from 72.34% to 91.84%, with an average of 83.1%.

Finally, on the Flesch-Kincaid Grade Level scale, the range was 2.68 to 6.3, with an average grade level of 4.4.

In other words, he found that the bestselling writers were aiming their prose, prose that is read by a majority of adult readers in the country, at a fourth grade level.

He also analyzed one his own work, and to his chagrin discovered that his readability was only in the 60% range, and that his prose aimed at an 8th grade level. He decided to use this information as a tool to revise his future work.

He created his Ideal Writing Standard, which I'm calling the Smith Writing Ideal Standard or SWID, because I can pronounce it "swid." He said that from now on, he revises all his work to the following four standards PER ANY SCENE:

No more than 4.25 characters per word.
No more than 5% passive voice
No less than an 80% readability on the Flesch Reading Ease scale.
A Flesch-Kincaid Grade Level of 5 (although on the chart, he lists that as 4-6).

Now, as it so happens, today HarperCollins released the first section of the upcoming Neil Gaiman novel ANANSI BOYS. On a whim, I decided to copy the text into my computer and run the grammar checker to see what it would say about its readability.

The results were astonishing:

4.2 characters per word
Passive voice: 3%
Readability: 79.9%
Flesh-Kincaid Grade Level: 5.4

In other words, according to the numbers, Gaiman easily fulfills the Smith Ideal Writing Standard in three of four categories. On the fourth, readability, he misses by a tenth of a percent. I'm inclined just to give it to him.

Now, what are we to make of these results? Does this mean that all bestselling writers are sitting at their word processors, analyzing their scenes by formulae, and thus ensuring their literary stardom? Is literary success nothing but a cold, emotionless application of an equation? (Having two degrees in Physics myself, I must admit that the thought appeals to me.)

But, as useful as this tool might be, I don't think the rest of us need to run every single thing we write through the grammar checker. Furthermore, I doubt this is how the bestselling writers do it either.

On the contrary, I suspect that what happens is that as writers write, they learn how to write better. I doubt that Gaiman (or King, or any of the bestselling writers Smith analyzed) are using their grammar checker in such a mechanical fashion. Instead, I think that they've developed an instinct, a knack if you will, for language. Somewhere inside their minds they've learned what works and what doesn't, and it just so happens that as they write, their instincts kick in, and they make their prose as accessible as possible.

I also think they revise a lot.

But for the rest of us...well, I'm not going to be as evangelical as Smith and endorse his method 100%. But I will suggest that it can't hurt us to try a few scenes from our work through the grammar checker and see what pops out at the end. Because the higher the grade level at which we pitch our prose, the fewer our readers.

By the way, for those of you wondering, just before posting I ran this blog entry through the grammar checker. Here are the results:

4.4 characters per word
Passive voice: 3%
Readability: 63.7%
Flesh-Kincaid Grade Level: 8.7

Hm. I've got a lot of learning in front of me.

Comments

[cogitationitis.livejournal.com]

Probably all those proper names, along with words like 'chagrin' and 'evangelical,' raised the grade level.

My latest entry, btw: 4.1 char/word, 6% passive, 78.6 reading ease, grade 5.8. I'm doing better than you! How come I don't get any Hugos?

[mabfan.livejournal.com]

Well, those entries aren't science fiction. :-)

Seriously, though, I just did an analysis of one of my Hugo-nominated stories. It came out as follows:

4.3 char/word
0% passive voice
81.7% reading ease
grade level 4.0

So perhaps when I write my blog entries, I tend to write at a different level.

[chaos-wrangler.livejournal.com]

MSWord at work is set to automatically does the Flesch-Kincaid analysis at the end of spell-check on a section/document, so I've been seeing it's results on my writing at work. My memos to my boss generally fall above the 10th grade level (I hadn't been noticing readibilty so much, maybe I will more now) and often hit 12. The daily memos to staff are usually much lower, possibly because I'm deliberately aiming for a more cheery "good morning & have a happy day" style. I wonder if shorter words and sentences read as more friendly?

Characters/Word: 4.5, Passive: 0%, Reading Ease: 49.1, Grade Level: 12.0 *sigh*

oops

[chaos-wrangler.livejournal.com]

The computer I'm using doesn't properly keep me logged in for LJ, so I can't edit the verb in the first sentence above. Oddly enough, MSWord didn't notice the grammar oops either.

And this is why....

[lordavon.livejournal.com]

I read through books at the speed of light.

Because everything always seems so EASY to read.

Now I know why.

[magid.livejournal.com]

At [former employer], we were told (look! passive voice! :-) to have the text be at least two grade levels below the target audience, so the kids could, hopefully, learn the math, rather than struggle with the language. Plus, there's English-language learners (ELL, previously known as ESL students). It can be challenging, when there's lots of technical terms in a lesson (There's just no way you can get "hypotenuse" shorter, and if you mention it once, chances are you're going to mention it a number of times.).

[sethg]

If you type the first few paragraphs of Portrait of the Artist as a Young Man or Dahlgren into one of these tools, what readability and grade level are they reported to have? What about the song "Supercalifragilisticexpialidocious" from Mary Poppins?

If you give one of these "fourth-grade level" best-selling novels to a group of average fourth-graders, how well will they understand the text?

[mabfan.livejournal.com]

I'm afraid that I don't have answers to either of your questions. But I can guess on the first one.

I think that the two novels you cite would probably score at a high grade level and at a lower than average readability.

As for your second question, I don't know enough about real fourth-grade ability or the Flesch-Kincaid scale to make an educated guess. My suggestion would be to do the actual experiment. Pick one of the novels that really comes out to a fourth grade level and see how well a real fourth-grader grasps it.

[lisafeld.livejournal.com]

The complexity of language is only part of the deeper issue of complexity and depth of the work. Gaiman, for instance, tends to bring in obscure legends from here and there, detailed plots that require remembering dozens of relationships and interactions which aren't mentioned for a hundred pages at a time, and mature themes like serial killers and pregnancy which in my opinion, he doesn't use gratuitously. But if you compare him with Chabon, who I think turns each sentence into a madlib and tries to figure out the most obscure words to plug in even when they aren't the words he means, just to sound more literary, then yeah, his language is kinda basic, but that's neither a bad thing nor a strictly profit-driven one. It's about enthusiastically communicating ideas, and sometimes that means being Mary Doria Russell (who hasn't done too badly for herself, and whose language can scarcely be called easy) and sometimes that means being Mark Twain.

[mabfan.livejournal.com]

What you said about Gaiman is part of the reason I decided to run the posted selection of his upcoming novel through the analysis. I tend to think of him as a very stylistic writer, but as he's also a New York Times bestselling author, I was curious to see where his work would fall on the scale.

Remember that nowhere in his discussion does Smith claim that this analysis leads to literary merit. The one thing he was looking for was if bestselling novels had any sort of commonality on the Flesch-Kincaid. We could debate all day as to the difference between a bestseller and a quality novel.

[cellio]

The study results surprise me. I find myself wondering how well-calibrated the scales are -- do typical fourth-graders understand the fourth-grade-level material? But it sounds like it's well worth keeping these factors in mind.

[zhaneel69.livejournal.com]

FYI: (having done some of this a while ago) Nonfiction/blogging is much higher on passive voice than fiction will and should be. Also, talking about statistics is a gaurenteed way to move your Grade Level up, so I wouldn't take this entry to heart.

Run a passage from your fiction (either your novel or one of your shorts) and see what the result is. Better yet: Run something from your first short story and your latest and see what the difference is.

Note: The longer the scene the better the stats will be for averages. Dialogue heavy scenes will probably score better, on average, than description heavy scenes, as people speak with the intent of greater "readability".

Zhaneel

[mabfan.livejournal.com]

I'm kind of afraid to run too much of the already published fiction through the analysis, as I'm worried as to what the results might be. :-)

However, I did run one Hugo nominee through, and posted the results above. I'll restate them here:

4.3 char/word
0% passive voice
81.7% reading ease
grade level 4.0

[exsanguinatrix.livejournal.com]

you don't know me, but I got here from Neil Gaiman's links. I'm curious as to what the results would be if you feed excerpts from 'bestselling books' like Harry Potter or The Da Vinci code, aren't you?

just a thought.

[mabfan.livejournal.com]

Well, what Smith did for his book was type in 2000 words from each novel. He selected the opening paragraphs, a chunk of text from the middle, and an emotionally charged scene near the end. I suppose if you wanted to, you could type in 2000 words from Harry Potter and Da Vinci code and see what happens. My guess is that the results would be within Smith's Ideal Writing Standard.

[mel-musing.livejournal.com]

Another, here, who came via Neil's blog. I'm an artist who tends to gravitate toward the writers and mathematicians of this world. The latter group tends to speak, write, and think off-scale, while the former keep telling me that I simply must write down to capture an audience. I suppose I should also say that I have done some writing when Calliope saw fit to inspire me. What I have learned is that I must "dumb-down" a piece so that the vast majority of people can understand it.

I call this tragic.

So I must be stifled that a moron can access it? No. I refuse. In this fair land of public education, where everyone who has passed through school in the past fifty years or so has been expected to pass all of the 12 mandatory grades, how is it that the reading level for the average purchaser of a best-seller is most comfortable with the same book a fourth grader is supposed to be able to enjoy? My view is that whatever story I display for public consumption, it should assume a readership of at least those whom have successfully passed out of the school system. If it challenges the reader, I have done a good job, if it fails that, then the average American is an insufferable moron and I don't care that they cannot understand what they should be able to.

Great subject, I'm happy to see it brought up.

[disturbed-kiwi.livejournal.com]

While I completely understand what you're saying about 'dumbing down' your writing being a pain and stupid seeming; isn't writing supposed to be about communicating ideas? If you can tell a complex strong emotional story in a way that appeals to everyone from age 5 up, aren't you a better writer?

At the same time, if you write more advanced stuff, thats not wrong per se. If you set out to say "haha I'm better than you because my stuff is harder" it might be.

here via Neil Gaiman's blog...

[happy-spinster.livejournal.com]

I just quit a graduate program in history because I decided I want to write fiction, instead. I told one of my fellow graduate students before I left that writing for academia was wrecking my prose. I wasn't kidding, either.

Here's the Flesch-Kincaid for the last history paper I handed in:
Words per sentence: 23.9 (my personal best is 24.8, by the way)
Characters per word: 4.9
Passive sentences: 3%
Reading ease: 38.6%
Grade level: 12.0

Here's the Flesch-Kincaid for the excessively wordy first draft of a 6,000-word short story I wrote last week:
Words per sentence: 13.4 (there is very little dialogue)
Characters per word: 4.2
Passive sentences: 1%
Reading ease: 79.9%
Grade level: 5.3

Clearly, I'm on the right track. Part of the reason I quit academia is because I no longer wanted to write solely for my fellow geeks. If I'm going to go to all the trouble to research and write a book, I might as well write something my parents would enjoy reading, too.

Now I want to run this test on Valley of the Dolls. I got in big trouble for writing a book report on it in fifth grade, even though I stated quite bluntly that it was the dumbest book I'd ever read. If that one's at a fifth-grade reading level, I'll eat my desk chair.

[mel-musing.livejournal.com]

it should assume a readership of at least those whom have successfully passed out of the school system

Okay, this post didn't even qualify for a high school reading level, but my point is that whatever my piece's grade level comes out as, I have not targeted an ignorant audience. Do people even look up words they don't understand anymore? Harrumph.

[kaolinfire.livejournal.com]

I read Freedom & Necessity with a dictionary by my side, and I recommend everyone do the same. ;) [[Steven Brust & Emma Bull]]

[riddled.livejournal.com]

*wanders in via Neil!blog*

Ooh, this is nifty. :O I suspect I'll be obsessively checking my stats from now on~



Interesting. :D I seem to be going in the right direction.

Neil sent me (thanks, Neil!)...

[shewhomust.livejournal.com]

Very interesting. And a bit scary: all those writers checking their stats, and adjusting their prose accordingly...

mabfan, you have a particular aim in applying Flesch-Kincaid to your writing: if you're applying for a library residency, I can see this sort of consideration carrying some weight with the selectors. Fair enough. When [author's name withheld], whose natural style is quite mannered, is told by their agent to write against the grain and make the current book simpler because it's a YA fantasy, not so fair enough.

As your statistics show, this isn't a contest between art and commerce; it's perfectly possible for stylists to rate well because some of what is scored for is just good style (minimising the passive voice, using the simplest language possible...).

We are using "readability" ambiguously here, since it actually means "scoring well on a particular test" rather than "something I will find easy to read". What's more, readability, whatever it may be, isn't everything (as lordavon implies!). If you can make your target audience want to read you, they will take a lot in their stride: if we are talking about YA fantasy, then consider Tolkien, who - whatever the analysis may say (and I suspect you could get different results from different sample passages!) - is not an easy read!

Re: Neil sent me (thanks, Neil!)...

[mabfan.livejournal.com]

You've hit on the exact reason why I decided to apply Flesch-Kincaid; as I said above, I wanted to make sure I could write to a younger audience.

And, of course, we are defining "readability" here in only one way -- the way Flesch and Kincaid did. There are surely many othr ways to define the concept.

[charliesmum.livejournal.com]

Hi. I popped over here through Neil Gaiman's web log - he posted your link - and I decided you're neat. I'm a wannabe writer, so I'm thinking reading the blog of an actual writer might help inspire me - so I'd like to friend you, if that's okay.

Plus - Neil Gaiman posted your blog. How cool is that?

[mabfan.livejournal.com]

and I decided you're neat

Um, thanks!

so I'd like to friend you, if that's okay

My policy is anyone can friend me, and I'll friend them back. Can't promise to read everyone else's posts, though. And I hope you actually find the rest of this blog interesting.

Plus - Neil Gaiman posted your blog. How cool is that?

Oh, it's way cool. Neil's a nice guy, and hard to dislike, even if one comes in second to him on the Hugo balloting. :-)

[braider.livejournal.com]

One of the odd things about story writing: using large words can actually get in the way of the story. One wants the reader to fall wholeheartedly into the story. If it's in the mouth of a character who likes to use large words, this will not pull the reader out of the story; if it appears as part of the prose, it can get tedious.

[mabfan.livejournal.com]

Funny what you said about character's dialogue. Smith suggests running individual characters' dialogue through the analysis so you can make sure your more sophisticated characters speak at a higher grade level.

[kathrynt.livejournal.com]

(here via Neil, like so many others. . .)

One of the things that really makes Gaiman a dead brilliant author is that he uses language in a very simple way to tell a very complex story. I've seen American Gods and Neverwhere used in adult literacy programs, because they're easy to read without being boring. The 'level' that something's written to refers, iirc, only to the language and grammar used, not to the thoughts that are being expressed. Which writer is better, the one who tells an epic myth using fourth-grade vocab or the one who writes "Dick and Jane" using language only a graduate student understands?

[shsilver.livejournal.com]

Now to run it with a work by Gene Wolfe. And then, just for kicks and giggles, the Gene Wolfe-Neil Gaiman collaboration A Walking Tour of the Shambles. The same experiment can be run with a Terry Pratchett novel and Good Omens, but I expect the results would be most interesting with Wolfe/Gaiman.

[eminence-gris.livejournal.com]

No fooling. I love Gene Wolfe dearly, but the man has a passion for making things hard for the reader. It's just that he's nearly always worth the effort.

However, I think that his style has changed since his Book of the New Sun days. I found The Wizard Knight to be a very easy read compared to Wolfe's earlier works.

I think perhaps Gaiman's rubbed off on him...

[blzblack.livejournal.com]

Fascinating post.

Good luck getting the residency!

[mabfan.livejournal.com]

Thank you! I expect the competition to be tough, but the nice thing is that I've had this particular novel in mind for a while. Even if I don't get the residency, I'm still planning to write this book one of these days.

kinsleycastle.blogspot.com

[(Anonymous)]

It's interesting to read all the comments here. I suspect they show which posters fall within Smith's standard and which dont. ;-) Personally, I don't. I have reading ease scores in the 70s and grades of 6 or 7. I'm not going to lose sleep over that, and I'm certainly not going to try and "improve" my writing by hacking the individuality out of my prose style.

Do you know what sort of writer I want to be? Imagine some guy finds a book on the street. The cover and the first few pages with all the copyright information have been torn off. But in spite of that, and in spite of the fact he's never read this book before, the guy can still tell it's me after a couple of pages.

[chasingtides.livejournal.com]

I checked Shakespeare's Sonnet 104 on the grammar check, and he works out to a .5 grade level. So according to Flesh-Kincaid, Shakespeare should be aimed at Kindergarten and Pre-K. They're a bit wonky.

[mabfan.livejournal.com]

My understanding is that Flesch-Kincaid doesn't work well if the selection is too short. You mighyt try running it on a file filled with about twn to twenty sonnets in a row, to see what happens. (Also, I doubt it's supposed to be used on poetry either.)

When I was a teacher

[theal8r.livejournal.com]

And in my MA program (for literacy aka reading) we were told to use F-K with other readability tests, as that one was more inclined to give higher scores when you ran into words of many syllable. It is a guideline, sure, but one of many.

I do think it is interesting, but, the NY Times, which is supposed to be the hardest paper in the world to read tests out only around 8th grade. Most newspapers are around 4th grade. That best-selling fiction would follow suit is not so surprising. You want people to be able to read it with ease, not have to struggle through it!

[lorna-w.livejournal.com]

interesting! Of course I had to go try my own writing. I F-K'ed a chapter of a mainstream commercial novel of mine and got 1% passive, level 3.9 (are the readability and grade level inversely proportional on this scale?). I tried a literary short story of mine, thinking it would be a higher grade level, and it was still only 4.7.

I pulled up an article I had written a couple years ago and it was grade 11.8, 5% passive.

I wonder to what extent my dialog lowers my writing grade level. I have a good many one-word "sentences" in dialog.

[mabfan.livejournal.com]

I think that readability and grade level do have some sort of inverse relationship on this scale, which would be one its flaws, in my opinion.

[e360.livejournal.com]

hello!

you've written this entry ages ago, but i found it recently through a literary journal..

where did you get this "grammar checker"? Is it an actual program? I would like to obtain it and use it to check just how juvenile my writing is...

[mabfan.livejournal.com]

Most word processing programs have the grammar checker as part of the program. If you're using Microsoft Word, just look under the options for Spelling and Grammar. And you might need to click a box labeled "Calculate reading statistics" or something like that.

Flesch-Kincaid and Bible version reading levels

[wleman.livejournal.com]

Hi, I have just linked to your post from my Better Bibles Blog at http://englishbibles.blogspot.com. I blogged on tests such as Flesch-Kincaid and reading levels indicated for various English Bible versions.

Interesting, I have run some of my own posts through the Microsoft Word grammar checker, and, to my dismay, discovered that the reading level of what I had written was much higher than I had expected. That ran counter to what I try to do when writing and how I believe "standard" prose for most readers should be written. I think the length of many of my sentences, some quite run-on, raised my F-K score.