I'm migrating to Writing Is My Jam, which is where I'll henceforth be freely expressing my many opinions about everything including but not limited to writing, writing for kids, writing for educational publishing, and writing assessments. In fact, if you are so inclined, you can find all of my previous posts from Leslie Hall@Inkspot at the new digs of Writing Is My Jam.
I feel l should close with a song.
The music of my misspent middle school years comes to mind, probably because this song is on my current writing playlist. Theme of magic, because the genre I'm working in is--surprise!--magic realism.
Here's a sampling:
Witch Doctor
Magic
Superstition
That Old Black Magic
Magic Road
Black Magic Woman
And also Entry of the Gladiators,Sideshow, and She Said, none of which particularly have to do with magic (or magic realism), but they do in some way relate to an aspect of plot, character, or theme.
Make a playlist for your writing. It's inspiring. Then come on over to Writing Is My Jam and post a sample in the comments. Because it's fun. A work playlist is fun, too.
Monday, July 7, 2014
Thursday, May 8, 2014
Louis C.K. Hurt My Feelings
Louis C.K. hates the Common Core standards.
I first saw it here, on the HuffPo, from David Letterman. Later, a friend/colleague (I don't name her only because she is in the business, too, and I don't want to get her in trouble) sent me a message to make sure I saw it--thanks for keeping me in the loop.
Toward the interest of full disclosure, I have to say I’ve always had a little crush on Louis C.K.
Why, you may ask?
You may ask this because you’re thinking of the fictional-but-based-on-real-Louis and the elevator fantasy scene or fictional-but-based-on-real-Louis passed out and surrounded by empty pizza boxes and ice cream containers or fictional-but-based-on-real-Louis being rejected by a woman because she witnessed him shrinking from confrontation with a high school bully.
Why do I have a crush on Louis C.K.?
Oh, let me count the ways: for the pure and sheer humanity and vulnerability that he just goes ahead and expresses, seemingly without any filter, and most of all, for the breathtaking courage it must take for him to expose his humanity and vulnerability to the world. He’s willing to be naked, figuratively and literally, when most of us are frantically swaddling ourselves with ego padding, trying to keep our humanity and vulnerability zipped up, buckled tight, under wraps, armored up. We post only flattering glamorous pictures online, nothing that makes us look dumpy or frumpy or dorky, even though surely all of us spend more time being dumpy, frumpy, or dorky than we do being smooth and suave and glamorous and elegant and unruffled.
Unless we’re, you know, Kimye.
(That picture on my blog? Taken three years ago. I've aged. I hate having pictures taken of myself and probably won't update it until I'm seventy.)
Besides, I hail from the working class, as does Louis C.K.,and so I applaud and cheer preach it, brother! whenever he criticizes entitlement or laziness or ingratitude.
All right, so we have this comedian who is a father--a good father, if by “good,” we mean someone who engages in thought about parenting and participates in his kids’ lives, which is all fantastic, and those of us who didn’t have fathers like that think he is really amazing for being that kind of father, and probably those of us who did have fathers like that feel a bit of fondness for him because this is familiar territory--who pays attention to his daughters and their inner lives and who worries when his daughters suffer, and so when his daughters, upon encountering mandatory statewide standardized testing, feel anxious, Louis C.K. has something to say about it. Something really not flattering to the people who write the tests. Something really not flattering to me.
In the spirit of respectful discourse and intellectual debate, I’d like to address these points:
1. Who writes these tests?
I do. Not the bad ones--unless instructed by a client to write badly, and sometimes that do happen, much to my chagrin--and only English language arts. Someone else is to blame for math, science, and social studies. Not my areas.
You could talk to my friend Scott about math or my colleague Jim about science, I guess, but they don't write bad tests, either.
There are lots of bad tests, yes. It's a systemic problem. More on that.
2. Why do I do this horrible, horrible thing?
To earn a living and support my two children.
3. What are my qualifications?
I have a bachelor’s degree in literature from the University of California at Santa Barbara, a master’s degree in English with an emphasis on writing from Sonoma State University, almost two years community college teaching experience, twenty years experience in educational publishing, and five years elementary classroom volunteer experience, as well as other miscellaneous tutoring experience from college and grad school. Also I was a TA in grad school, for the creative writing class, oh God, was that awful, all those stories about cats and sexual abuse and suicide mixed with the occasional fantasy of being a wealthy celebrity writer driving a red Corvette, clearly no student in that class had ever read a word written by any writer other than their favorite writers: themselves. In addition, I’ve put in many, many hours of study--in education, in reading and language acquisition and of literature and literary criticism, and especially in assessment, and even more especially in the writing of test questions.
4. What’s with the Common Core?
It’s a good idea to have national standards. Other countries do, and that’s how they make sure that all the kids in the country are learning the same things at the same pacing. It’s a good idea to consider career- and college readiness, and how to make that happen, particularly when kids in the United States are undereducated to a degree that must make us the laughingstock of industrialized nations. Finland and South Korea especially must snigger at our national ignorance and celebration thereof--is there any country in the world that makes a point of being so dang proud of being stupid? I ask you.
5. Why do people hate the Common Core so very much?
People fear change. People hate what they fear. No one understands what the Common Core standards are, or what the shift means, or that it's really a good thing that kids in Alabama learn the same things as kids in Connecticut. There’s too much hype and not enough real education about the standards and their purpose. Teachers are scared because tests are being used for wrongful purposes (never a good idea to link teacher pay to test scores), and scared teachers are scaring the kids.
6. What is Louis C.K. really upset about?
Like any caring parent, he’s upset that his daughters are upset.
He doesn’t know enough about the Common Core to be upset about them.
That’s not his fault; it’s the fault of the top-secret test publishing industry that keeps all information under lock and key, supposedly to preserve confidentiality, but, really? Wouldn’t it be smarter to explain what’s happening and why? No. Because then they would have to explain everything else, like the billions of dollars spent on testing and how little of it changes anything really, and also how little of it trickles down to the people who are doing the actual work which means that the majority of content developers (not me, I'm the exception, this is my career) are inexperienced hobbyists or inexperienced part-time teachers or hustlers who think they're getting away with something by getting paid to do something they don't know anything about and how much of the big money in testing gets bottlenecked up at the executive and shareholder level.
7. What should Louis C.K. really be upset about?
Capitalism. The one percent. The war on poor people instead of a war on poverty. The state of education in the United States. Dogs that need rescue at animal shelters.
What’s so unfortunate here is that Louis C.K. is someone who’s got a public forum--people (including me) listen to him, laugh at his jokes, care about his opinions. He has an opportunity to make people think (at least a little) and that would be a really great thing if--and I don’t at all intend this as a snarky sarcastic dig--he knew what he was talking about. I'm sure there are a gazillion things he knows plenty about, but the Common Core standards are not on that list. Really, do you think he has even read them? I mean no offense, but I would be surprised by an affirmative.
There are so many things that are terribly wrong in education in general and in educational assessment in particular, but from my perspective--as someone who does know what she’s talking about here--the Common Core is a paper dragon. Let’s talk instead about the corporatization of education.
How about that fewer than half a dozen test publishing companies rule assessment, and the king is Pearson? (Which company is now involved in a controversy over the award of the PARCC assessments as the result of a lawsuit filed by AIR.)
How about that the people who are actually doing the work are paid woefully inadequately (Hello? My yearly income today is the same as it was twelve years ago when I started my business, but guess what, inflation--can you see why this is a problem?) while the companies continue to earn profits that are obscene in comparison?
Let’s take me, partly because I am monumentally self-absorbed, but also because my experience is what I know. I do know many other people in this line of work, but very few have the depth and breadth of experience that I have in educational assessment: I’ve worked in hand-scoring, program management, content development from the ground up (item writer to editor to supervisor to manager to director and back to item writer and editor). I’ve worked directly with state department of education officials. For seven years, I had an annual contract with Miami-Dade County Public Schools, one of the largest districts in the country, a district that has more students than some states.
For the last twelve years, I have concentrated mainly (with some side jobs involving higher level consulting of test design and product research) on the hands-on work of content development: writing and editing material (reading passages and questions) for tests. That is unheard of. In this industry, as soon as anyone shows a spark of initiative, and especially if that initiative is accompanied by a pebble of intelligence, that person gets promoted. Anyone else with twenty years' experience has been in management for at least ten of those years, and management is not the same as actually doing the work, as any line cook at KFC could tell you.
I mean no arrogance when I say that I’m the perfect person to write tests, considering the combination of education, experience, and dedication--because I care about what I do, quality matters to me, the kids matter to me--when I write reading passages and test questions, I’m thinking about the experience of the kids who are going to take the test just as much as I’m thinking about my paycheck. Maybe more.
Not that I don’t think about my paycheck. I do. I have to. I’m a single mother with two kids.
Thinking about those kids who take these tests breaks my heart. Not so much kids like my daughters and Louis C.K.’s daughters--these girls are all going to be fine. They have parents who love them, ready access to books, music, art, libraries, documentaries on penguins and whales and volcanoes and subscriptions to the National Geographic and visits to the Smithsonian. Their parents talk to them all the time (maybe too much, in my case; Louis C.K. is probably a lot more interesting and a lot less pedantic when he talks to his daughters) and are willing to listen and answer questions and explain all about why everything in the world is the way it is. We the parents will support our daughters, consider their happiness, find ways to challenge them, look for opportunities to help them navigate the complexities of relationships, communication, education, and, eventually, careers.
And Louis C.K.’s kids? They’re especially going to be fine. They’re rich. They’ll have their pick of colleges, go wherever they want, do whatever they want from now until they die and leave their truckloads of dollar bills (remember investment income is taxed at about half the rate of labor income, so their money is constantly making money, they'll have more money than they could ever spend) to their kids and their kids' kids.
That is awesome for them, and while I envy their good fortune (which I acknowledge comes from the hard work and talent of their father), I don’t begrudge it them. If they get a little upset about a test, I understand and I sympathize and it's nice that their dad sympathizes, too, but really, there are a lot worse things in the world to happen when you’re a kid, and a lot worse things do happen to many of the kids in the world. Maybe some of that righteous indignation could go to someone else’s kids, kids who really don’t ever get a chance. Not that I mean to be all sassy to Louis C.K.
Note: A little crush. Not a stalker crush. Have I ever written to or tried to contact him in any way? No. Geez. Of course not. Would I ever? No. Oh, God, no. What do you take me for? Have I watched his show and stand-up routines? Yes. Do I laugh at his jokes? Most. Some of the humor is a bit past my endurance, but I celebrate his right to express himself.
I first saw it here, on the HuffPo, from David Letterman. Later, a friend/colleague (I don't name her only because she is in the business, too, and I don't want to get her in trouble) sent me a message to make sure I saw it--thanks for keeping me in the loop.
Toward the interest of full disclosure, I have to say I’ve always had a little crush on Louis C.K.
Why, you may ask?
You may ask this because you’re thinking of the fictional-but-based-on-real-Louis and the elevator fantasy scene or fictional-but-based-on-real-Louis passed out and surrounded by empty pizza boxes and ice cream containers or fictional-but-based-on-real-Louis being rejected by a woman because she witnessed him shrinking from confrontation with a high school bully.
Why do I have a crush on Louis C.K.?
Oh, let me count the ways: for the pure and sheer humanity and vulnerability that he just goes ahead and expresses, seemingly without any filter, and most of all, for the breathtaking courage it must take for him to expose his humanity and vulnerability to the world. He’s willing to be naked, figuratively and literally, when most of us are frantically swaddling ourselves with ego padding, trying to keep our humanity and vulnerability zipped up, buckled tight, under wraps, armored up. We post only flattering glamorous pictures online, nothing that makes us look dumpy or frumpy or dorky, even though surely all of us spend more time being dumpy, frumpy, or dorky than we do being smooth and suave and glamorous and elegant and unruffled.
Unless we’re, you know, Kimye.
(That picture on my blog? Taken three years ago. I've aged. I hate having pictures taken of myself and probably won't update it until I'm seventy.)
Besides, I hail from the working class, as does Louis C.K.,and so I applaud and cheer preach it, brother! whenever he criticizes entitlement or laziness or ingratitude.
All right, so we have this comedian who is a father--a good father, if by “good,” we mean someone who engages in thought about parenting and participates in his kids’ lives, which is all fantastic, and those of us who didn’t have fathers like that think he is really amazing for being that kind of father, and probably those of us who did have fathers like that feel a bit of fondness for him because this is familiar territory--who pays attention to his daughters and their inner lives and who worries when his daughters suffer, and so when his daughters, upon encountering mandatory statewide standardized testing, feel anxious, Louis C.K. has something to say about it. Something really not flattering to the people who write the tests. Something really not flattering to me.
In the spirit of respectful discourse and intellectual debate, I’d like to address these points:
1. Who writes these tests?
I do. Not the bad ones--unless instructed by a client to write badly, and sometimes that do happen, much to my chagrin--and only English language arts. Someone else is to blame for math, science, and social studies. Not my areas.
You could talk to my friend Scott about math or my colleague Jim about science, I guess, but they don't write bad tests, either.
There are lots of bad tests, yes. It's a systemic problem. More on that.
2. Why do I do this horrible, horrible thing?
To earn a living and support my two children.
3. What are my qualifications?
I have a bachelor’s degree in literature from the University of California at Santa Barbara, a master’s degree in English with an emphasis on writing from Sonoma State University, almost two years community college teaching experience, twenty years experience in educational publishing, and five years elementary classroom volunteer experience, as well as other miscellaneous tutoring experience from college and grad school. Also I was a TA in grad school, for the creative writing class, oh God, was that awful, all those stories about cats and sexual abuse and suicide mixed with the occasional fantasy of being a wealthy celebrity writer driving a red Corvette, clearly no student in that class had ever read a word written by any writer other than their favorite writers: themselves. In addition, I’ve put in many, many hours of study--in education, in reading and language acquisition and of literature and literary criticism, and especially in assessment, and even more especially in the writing of test questions.
4. What’s with the Common Core?
It’s a good idea to have national standards. Other countries do, and that’s how they make sure that all the kids in the country are learning the same things at the same pacing. It’s a good idea to consider career- and college readiness, and how to make that happen, particularly when kids in the United States are undereducated to a degree that must make us the laughingstock of industrialized nations. Finland and South Korea especially must snigger at our national ignorance and celebration thereof--is there any country in the world that makes a point of being so dang proud of being stupid? I ask you.
5. Why do people hate the Common Core so very much?
People fear change. People hate what they fear. No one understands what the Common Core standards are, or what the shift means, or that it's really a good thing that kids in Alabama learn the same things as kids in Connecticut. There’s too much hype and not enough real education about the standards and their purpose. Teachers are scared because tests are being used for wrongful purposes (never a good idea to link teacher pay to test scores), and scared teachers are scaring the kids.
6. What is Louis C.K. really upset about?
Like any caring parent, he’s upset that his daughters are upset.
He doesn’t know enough about the Common Core to be upset about them.
That’s not his fault; it’s the fault of the top-secret test publishing industry that keeps all information under lock and key, supposedly to preserve confidentiality, but, really? Wouldn’t it be smarter to explain what’s happening and why? No. Because then they would have to explain everything else, like the billions of dollars spent on testing and how little of it changes anything really, and also how little of it trickles down to the people who are doing the actual work which means that the majority of content developers (not me, I'm the exception, this is my career) are inexperienced hobbyists or inexperienced part-time teachers or hustlers who think they're getting away with something by getting paid to do something they don't know anything about and how much of the big money in testing gets bottlenecked up at the executive and shareholder level.
7. What should Louis C.K. really be upset about?
Capitalism. The one percent. The war on poor people instead of a war on poverty. The state of education in the United States. Dogs that need rescue at animal shelters.
What’s so unfortunate here is that Louis C.K. is someone who’s got a public forum--people (including me) listen to him, laugh at his jokes, care about his opinions. He has an opportunity to make people think (at least a little) and that would be a really great thing if--and I don’t at all intend this as a snarky sarcastic dig--he knew what he was talking about. I'm sure there are a gazillion things he knows plenty about, but the Common Core standards are not on that list. Really, do you think he has even read them? I mean no offense, but I would be surprised by an affirmative.
There are so many things that are terribly wrong in education in general and in educational assessment in particular, but from my perspective--as someone who does know what she’s talking about here--the Common Core is a paper dragon. Let’s talk instead about the corporatization of education.
How about that fewer than half a dozen test publishing companies rule assessment, and the king is Pearson? (Which company is now involved in a controversy over the award of the PARCC assessments as the result of a lawsuit filed by AIR.)
How about that the people who are actually doing the work are paid woefully inadequately (Hello? My yearly income today is the same as it was twelve years ago when I started my business, but guess what, inflation--can you see why this is a problem?) while the companies continue to earn profits that are obscene in comparison?
Let’s take me, partly because I am monumentally self-absorbed, but also because my experience is what I know. I do know many other people in this line of work, but very few have the depth and breadth of experience that I have in educational assessment: I’ve worked in hand-scoring, program management, content development from the ground up (item writer to editor to supervisor to manager to director and back to item writer and editor). I’ve worked directly with state department of education officials. For seven years, I had an annual contract with Miami-Dade County Public Schools, one of the largest districts in the country, a district that has more students than some states.
For the last twelve years, I have concentrated mainly (with some side jobs involving higher level consulting of test design and product research) on the hands-on work of content development: writing and editing material (reading passages and questions) for tests. That is unheard of. In this industry, as soon as anyone shows a spark of initiative, and especially if that initiative is accompanied by a pebble of intelligence, that person gets promoted. Anyone else with twenty years' experience has been in management for at least ten of those years, and management is not the same as actually doing the work, as any line cook at KFC could tell you.
I mean no arrogance when I say that I’m the perfect person to write tests, considering the combination of education, experience, and dedication--because I care about what I do, quality matters to me, the kids matter to me--when I write reading passages and test questions, I’m thinking about the experience of the kids who are going to take the test just as much as I’m thinking about my paycheck. Maybe more.
Not that I don’t think about my paycheck. I do. I have to. I’m a single mother with two kids.
Thinking about those kids who take these tests breaks my heart. Not so much kids like my daughters and Louis C.K.’s daughters--these girls are all going to be fine. They have parents who love them, ready access to books, music, art, libraries, documentaries on penguins and whales and volcanoes and subscriptions to the National Geographic and visits to the Smithsonian. Their parents talk to them all the time (maybe too much, in my case; Louis C.K. is probably a lot more interesting and a lot less pedantic when he talks to his daughters) and are willing to listen and answer questions and explain all about why everything in the world is the way it is. We the parents will support our daughters, consider their happiness, find ways to challenge them, look for opportunities to help them navigate the complexities of relationships, communication, education, and, eventually, careers.
And Louis C.K.’s kids? They’re especially going to be fine. They’re rich. They’ll have their pick of colleges, go wherever they want, do whatever they want from now until they die and leave their truckloads of dollar bills (remember investment income is taxed at about half the rate of labor income, so their money is constantly making money, they'll have more money than they could ever spend) to their kids and their kids' kids.
That is awesome for them, and while I envy their good fortune (which I acknowledge comes from the hard work and talent of their father), I don’t begrudge it them. If they get a little upset about a test, I understand and I sympathize and it's nice that their dad sympathizes, too, but really, there are a lot worse things in the world to happen when you’re a kid, and a lot worse things do happen to many of the kids in the world. Maybe some of that righteous indignation could go to someone else’s kids, kids who really don’t ever get a chance. Not that I mean to be all sassy to Louis C.K.
Note: A little crush. Not a stalker crush. Have I ever written to or tried to contact him in any way? No. Geez. Of course not. Would I ever? No. Oh, God, no. What do you take me for? Have I watched his show and stand-up routines? Yes. Do I laugh at his jokes? Most. Some of the humor is a bit past my endurance, but I celebrate his right to express himself.
Sunday, November 17, 2013
What Are We Waiting for
November is National Novel Writing Month (NaNoWriMo, that crazy venture in which you write 50,000 words of a consecutive narrative thread in 30 days). The NaNoWriMo motto is The world needs your novel.
I don't know whether the world needs my novel, but I do know that every November for the last 5 years, I considered participating in NaNoWriMo but didn't follow through.
This year I am doing NaNoWriMo. Or I should say I did NaNoWriMo, because I'm done.
In the first 7 days I wrote 20,661 words. As of midnight yesterday, I had written 50,425 words, which means I finished the novel and accomplished what I had set out to do. (Almost. I'd set myself the ridiculous goal of 50,000 words in 15 days. So I was a day late.)
NaNoWriMo critics will agree with Laura Miller of Salon who says "I am not the first person to point out that 'writing a lot of crap' doesn’t sound like a particularly fruitful way to spend an entire month, even if it is November." (To which one might be tempted to respond thusly.) I would agree with Laura Miller that if a writer is using some of the silly tricks I've seen for completing NaNoWriMo, such as adding a nonsense word ("potato") after every word in order to pad the word count, it's potato pretty potato much potato a why-bother potato exercise potato.
Certainly writing fast doesn't always mean writing badly. Mental Floss provides a list of NaNoWriMo novels that got published.
As for me, I found the NaNoWriMo project a delightful and fruitful way to spend two weeks and a day.
If you are a writer who has considered participating in NaNoWriMo, I encourage you to do it.
Because if your novel offers the world your unique voice, your quirky perspective, your way of looking at and thinking about the world, and your story--the story only you can tell--then yes, the world really does need your novel.
We need all the stories we can get.
I don't know whether the world needs my novel, but I do know that every November for the last 5 years, I considered participating in NaNoWriMo but didn't follow through.
This year I am doing NaNoWriMo. Or I should say I did NaNoWriMo, because I'm done.
In the first 7 days I wrote 20,661 words. As of midnight yesterday, I had written 50,425 words, which means I finished the novel and accomplished what I had set out to do. (Almost. I'd set myself the ridiculous goal of 50,000 words in 15 days. So I was a day late.)
NaNoWriMo critics will agree with Laura Miller of Salon who says "I am not the first person to point out that 'writing a lot of crap' doesn’t sound like a particularly fruitful way to spend an entire month, even if it is November." (To which one might be tempted to respond thusly.) I would agree with Laura Miller that if a writer is using some of the silly tricks I've seen for completing NaNoWriMo, such as adding a nonsense word ("potato") after every word in order to pad the word count, it's potato pretty potato much potato a why-bother potato exercise potato.
Certainly writing fast doesn't always mean writing badly. Mental Floss provides a list of NaNoWriMo novels that got published.
As for me, I found the NaNoWriMo project a delightful and fruitful way to spend two weeks and a day.
If you are a writer who has considered participating in NaNoWriMo, I encourage you to do it.
Because if your novel offers the world your unique voice, your quirky perspective, your way of looking at and thinking about the world, and your story--the story only you can tell--then yes, the world really does need your novel.
We need all the stories we can get.
Tuesday, August 6, 2013
PSA: Conducting a Performance Review of the Inner Critic
For a few days I've been considering what Freud identified as the super-ego, what has become popularly known as the inner critic, or if you're abundantly blessed, your personal busload or committee or parliament of inner critics.
Generally speaking, the role of the inner critic seems primarily protective, as wrongheaded as its tactics are for that purpose. If you're a writer, your inner critic might advise you to clean the house instead of writing, or may suggest there's no point in writing, because everything you write is horrible, bad, and no-good, and that one 7th grade English teacher was just being nice or had bad taste or that the one story you wrote that got published or that your writing group liked was an anomaly. No attempt means no risk means safety from rejection and criticism. Probably there are inner critics for every occupation.
There are as many strategies of dealing with inner critics as there are types of inner critics: we can embrace them, treat them with compassion, or murder them. I'd like to offer a new one: conduct a performance review to evaluate their effectiveness on the job.
Frankly, if my inner critics were my employees, I'd have fired them by now. I always sort of knew they were making a mess of it--how did they even make it past the interview?-- but I'd never taken the time to consider the nature of their incompetence. I made one of the biggest mistakes employers make with performance reviews, according to Forbes: I haven't been conducting them.
To prepare for the review, I must specify the responsibilities of the inner critic: what is the job description?
A critic approaches a work with curiosity, is open to experiencing a work and responding to the work, observes those responses, and investigates the work in order to consider the artist's purpose and message, and how the artist achieves that purpose and conveys that message. A critic pays close attention to the work. A critic is dedicated to a particular art (and should be an expert of that art or field), and in the raising up of that art, and so the critic is responsible for celebrating achievements as much as for observing attempts that may fall short.
These are questions a critic might ask in examining a work of art:
Generally speaking, the role of the inner critic seems primarily protective, as wrongheaded as its tactics are for that purpose. If you're a writer, your inner critic might advise you to clean the house instead of writing, or may suggest there's no point in writing, because everything you write is horrible, bad, and no-good, and that one 7th grade English teacher was just being nice or had bad taste or that the one story you wrote that got published or that your writing group liked was an anomaly. No attempt means no risk means safety from rejection and criticism. Probably there are inner critics for every occupation.
There are as many strategies of dealing with inner critics as there are types of inner critics: we can embrace them, treat them with compassion, or murder them. I'd like to offer a new one: conduct a performance review to evaluate their effectiveness on the job.
Frankly, if my inner critics were my employees, I'd have fired them by now. I always sort of knew they were making a mess of it--how did they even make it past the interview?-- but I'd never taken the time to consider the nature of their incompetence. I made one of the biggest mistakes employers make with performance reviews, according to Forbes: I haven't been conducting them.
To prepare for the review, I must specify the responsibilities of the inner critic: what is the job description?
A critic approaches a work with curiosity, is open to experiencing a work and responding to the work, observes those responses, and investigates the work in order to consider the artist's purpose and message, and how the artist achieves that purpose and conveys that message. A critic pays close attention to the work. A critic is dedicated to a particular art (and should be an expert of that art or field), and in the raising up of that art, and so the critic is responsible for celebrating achievements as much as for observing attempts that may fall short.
These are questions a critic might ask in examining a work of art:
- What place does this work have in the tradition or genre?
- How has this work been influenced by earlier works?
- How does this work reflect a social, historical, or cultural context?
- How may different levels of interpretation be applied to this work?
- What are the messages of this work?
- Is there a universal message or is the message specific to a group, time, place, instance?
- How does this work convey those messages?
- Does this work have a unique voice and style that set it apart from others of its type?
- What contributes to the uniqueness of this work?
- What emotional responses does a person have to this work?
- How does this work elicit those emotional responses?
- insulting, bullying, ridiculing, mocking, belittling, name-calling, disparaging the artist or the work, listing past mistakes, predicting future failure
Critics address the work with specific questions and use the information to discuss the work and explore the meaning of the work. It's a lot easier to approach a work from opinion (I like it, I hate it) or judgment (rotten, all right, awesome), but so much less interesting and less effective.
What good does opinion do, really? Even nice opinions aren't much use, as much as it feels better when a reader says that he likes one's writing than when he says that he hates it.
What good does opinion do, really? Even nice opinions aren't much use, as much as it feels better when a reader says that he likes one's writing than when he says that he hates it.
If you heard the way my inner critics talk, you'd agree that they've fallen down on the job. Or maybe they got handed the wrong job description in the first place. I can't recall their ever having performed satisfactorily.
Writers need to write, and we need to figure out ways around and through and over and under the obstacles to writing, even while we're constructing the obstacles.
And there are other reasons:
And there are other reasons:
- We parents, teachers, coaches, aunts, uncles, grandparents--we all of us adults--create the voices for future generations of inner critics. I'd like my kids to go out into the world with inner critics who contribute to, rather than sabotage, their happiness and well-being.
- Many of us supervise others. In so doing, we can't help but impose the messages of our inner critics on others. We have the choice to be constructive or destructive. We have the choice to be reasonable (or not) with ourselves and others.
- No good writing ever came from performance anxiety.
I'm concerned that how we think about, talk about, teach, and measure performance in writing trains students to believe that not only are they terrible writers, but they are not capable of writing well. The reality is that everyone can tell a good story, and if you can tell a good story, you can write a good story, and if you can write a good story, you should, because we all need stories, as many as possible, from as many perspectives as possible.
Thursday, August 1, 2013
The Big Idea, or Focus, Cross-Referenced to Basic Rules of Item Writing
What comes before preparation is intention, which we previously discussed here. Still, the concept of the Big Idea bears further exploration.
Let's consider how we might approach this grade 4 standard from the CCSS, RL.4.2:
This standard requires two distinct subskills: determining a theme and summarizing text.
Either may be assessed with multiple-choice, constructed-response, or technology-enhanced items, although I note that in an ideal world, we wouldn't use multiple-choice for summarizing, but would instead ask students to create the summary. Again in that ideal world, it's best if we provide the student with opportunities to demonstrate mastery of a particular skill by allowing the student to perform the skill; however, we often operate under constraints that exclude the ideal. That's okay.
After we've read all of our ancillary support materials and have thoroughly acquainted ourselves with the story, poem, or play (for less experienced item writers and for all item writers without a strong background in literary analysis, I suggest making an outline of and annotating the passage in order to avoid the trap of writing superficial and repetitive items), we determine the theme(s). There may be more than one. Out of fairness, choose the strongest theme that is most clearly supported and most thoroughly developed in the passage. The theme may be stated explicitly or may be implied by the characters' words and actions.
Here is our passage, "A Boy's Song" by James Hogg.
We would probably use call-out boxes to define some of the vocabulary--"lea" and "nestling" stand out as words likely to interfere with student understanding.
If we're writing a multiple-choice item, the stem will look like this:
Often at the lower grades, we use "theme" and "main idea" as synonyms; depending on curriculum, grade 4 students may not yet be familiar with the specific terms for narrative elements, and we don't want to erect unnecessary obstacles for those students, so we might write a stem that looks like this:
What is a theme of the poem?
A the joy of spending time in nature
B [TK]
C [TK]
D [TK]
Next we'd write three distractors (wrong answers). Each distractor should have a rationale--that is, each should embody a specific mistake or breakdown in comprehension or literary analysis that might hinder a student en route to determining the theme. The rule in item writing is that, given the evidence in the text, distractors must be "plausible but not possible." The distractors should be clearly wrong to the student who is able to "determine a theme...from details in the text."
Many clients require item writers to provide rationales or justifications for the wrong answer; I support this wholeheartedly as valuable practice for inexperienced item writers. Experienced item writers have rationales in their minds already, so it's just a matter of typing them.
When we write the distractors, we must stay focused on our Big Idea. In order to do that, we'd consider the breakdowns that occur when students attempt to identify a theme. In order to do that, we'd think about the process of making meaning from text. We read the poem and step back and come up with the overarching meaning: the joy of spending time in nature. Then we think about how a student might falter in putting the pieces of the poem together to see that big picture. A student might get stuck on a detail of the poem, and mistake that for a theme. A student might confuse theme and subject. A student might focus too narrowly.
Next up: constructing plausible but not possible distractors.
What I'm reading: The Reivers by Faulkner and Imaginings of Sand by Andre Brinks.
Let's consider how we might approach this grade 4 standard from the CCSS, RL.4.2:
Determine a theme of a story, drama, or poem from details in the text; summarize the text.This standard is passage-dependent; students read a story, poem, or play (or excerpts of the same) and then answer questions about what they read.
This standard requires two distinct subskills: determining a theme and summarizing text.
Either may be assessed with multiple-choice, constructed-response, or technology-enhanced items, although I note that in an ideal world, we wouldn't use multiple-choice for summarizing, but would instead ask students to create the summary. Again in that ideal world, it's best if we provide the student with opportunities to demonstrate mastery of a particular skill by allowing the student to perform the skill; however, we often operate under constraints that exclude the ideal. That's okay.
After we've read all of our ancillary support materials and have thoroughly acquainted ourselves with the story, poem, or play (for less experienced item writers and for all item writers without a strong background in literary analysis, I suggest making an outline of and annotating the passage in order to avoid the trap of writing superficial and repetitive items), we determine the theme(s). There may be more than one. Out of fairness, choose the strongest theme that is most clearly supported and most thoroughly developed in the passage. The theme may be stated explicitly or may be implied by the characters' words and actions.
Here is our passage, "A Boy's Song" by James Hogg.
Where the pools are bright and deep,
Where the gray trout lies asleep,
Up the river and o'er the lea,
That's the way for Billy and me.
Where the blackbird sings the latest,
Where the hawthorn blooms the sweetest,
Where the nestlings chirp and flee,
That's the way for Billy and me.
Where the mowers mow the cleanest,
Where the hay lies thick and greenest,
There to trace the homeward bee,
That's the way for Billy and me.
Where the hazel bank is steepest,
Where the shadow falls the deepest,
Where the clustering nuts fall free.
That's the way for Billy and me.
Why the boys should drive away,
Little sweet maidens from the play,
Or love to banter and fight so well,
That's the thing I never could tell.
But this I know, I love to play,
Through the meadow, among the hay;
Up the water and o'er the lea,
That's the way for Billy and me.
If we're writing a multiple-choice item, the stem will look like this:
What is a theme of the poem?Or we might identify the poem only by its title ("What is a main theme of 'A Boy's Song'?") if we plan to write another item about genre characteristics ("How does the reader know 'A Boy's Song' is a poem?").
Often at the lower grades, we use "theme" and "main idea" as synonyms; depending on curriculum, grade 4 students may not yet be familiar with the specific terms for narrative elements, and we don't want to erect unnecessary obstacles for those students, so we might write a stem that looks like this:
What is a main idea of the story?I prefer "a" rather than "the" in order to allow for variety in literary interpretation; we'd follow the client's preference on this. In this case, a clear theme is the joy of spending time in nature. Now we have a stem and the correct response:
What is a theme of the poem?
A the joy of spending time in nature
B [TK]
C [TK]
D [TK]
Next we'd write three distractors (wrong answers). Each distractor should have a rationale--that is, each should embody a specific mistake or breakdown in comprehension or literary analysis that might hinder a student en route to determining the theme. The rule in item writing is that, given the evidence in the text, distractors must be "plausible but not possible." The distractors should be clearly wrong to the student who is able to "determine a theme...from details in the text."
Many clients require item writers to provide rationales or justifications for the wrong answer; I support this wholeheartedly as valuable practice for inexperienced item writers. Experienced item writers have rationales in their minds already, so it's just a matter of typing them.
When we write the distractors, we must stay focused on our Big Idea. In order to do that, we'd consider the breakdowns that occur when students attempt to identify a theme. In order to do that, we'd think about the process of making meaning from text. We read the poem and step back and come up with the overarching meaning: the joy of spending time in nature. Then we think about how a student might falter in putting the pieces of the poem together to see that big picture. A student might get stuck on a detail of the poem, and mistake that for a theme. A student might confuse theme and subject. A student might focus too narrowly.
Next up: constructing plausible but not possible distractors.
What I'm reading: The Reivers by Faulkner and Imaginings of Sand by Andre Brinks.
Wednesday, July 31, 2013
The Big Idea, Cross-Referenced to Basic Rules of Item Writing
What endeavor doesn't benefit from planning and preparation? What endeavor succeeds without preparation?
A test blueprint identifies the skills and/or knowledge to be assessed, provides the item-to-skill distribution, and specifies item formats.
Let's think about creating a blueprint to assess writing at grade 6. We'll base the blueprint on the Common Core State Standards.
In the CCSS, English conventions are addressed in the language standards, and what we might call writing strategies and application are addressed in the writing standards. The language standards could be assessed with a variety of formats: standalone or passage-dependent multiple choice items, standalone or passage-dependent technology-enhanced items, or as one component of an extended-constructed-response item.
Here is a writing standard:
Generally the above standard would be assessed with an extended-constructed-response item, because multiple-choice items and short constructed-response items don't allow students sufficient opportunity to demonstrate the ability to "write arguments to support claims...." However, the subskills may be (and frequently are) assessed with multiple-choice items; this is more common at the district or classroom level than at the state level. You might see a question that addresses W.1.a by asking the student to choose the best opposing claim for a given argument. Such multiple-choice items may help teachers isolate specific areas in which a student needs instruction and support.
Here are language standards:
In a typical statewide high-stakes assessment program, the decisions that inform the development of a test blueprint and item specifications are made by committees, which is as it should be, and committees should include classroom teachers. Committees often include other stakeholders, e.g., business leaders who may be asked to identify skills and knowledge necessary in the workplace.
Once all of that preparation is complete, item development begins.
Now let's say we've received an assignment to write those multiple-choice language items and that ECR writing prompt. We've read all of the project documentation and support materials; we have the item specifications in front of us.
It is a truth universally acknowledged that a test item should target one and only one skill or bit of content knowledge. Each idea should have one big idea; every part of the item should support that focus.
If we were going to write a multiple-choice item for W.2.b, our big idea would be how to spell grade-level appropriate words. We might write an item that looks like this:
Which word is spelled correctly?
A absense
B boundery
C civilizashion
D dissolve*
This item clearly targets one skill: correctly spell grade-level-appropriate words. The stem tells the student exactly what to do. The item is phrased simply and concisely. The content is neutral; there are no highly-charged words. All of the answer choices are grade 6 words (according to EDL Core Vocabularies); all are words likely to be known to grade 6 students and are words that are significant to academic content areas. There are no tricky or esoteric rare words. The answer choices appear in a logical order (here we use alpha order). All of the distractors address common spelling mistakes: using s instead of c, using e instead of a, and writing phonetically. None of the words are homonyms and so none are context-dependent; each of these words have one correct spelling.
Here is a poor item addressing the same skill:
Which word is written correctly?
A musheenz
B rabby
C anker
D pistol
This item has multiple flaws. First, the big idea is not specified in the stem; the student doesn't know what s/he is expected to do until s/he reads the answer choices. The answer choices are not grade-level-appropriate; "machine" is a grade 2 word, while "anchor" is grade 3. The word "rabbi" may not be familiar to grade 6 students. Answer choice A ("musheenz") is plural, while the other ACs are singular. Answer choice A also offers mistakes that are unlikely to be made by students at the targeted grade level. The answer choices do not appear in any logical order. Finally, the correct response is a type of weapon.
As bad as this item is, though, we could make it even worse by
Ah, fatal words! Too late in moving here, too late in arriving there, too late in coming to this decision, too late in starting with enterprises, too late in preparing.These first guidelines of the CCSSO/TILSA Quality Control Checklist for Item Development and Test Form Construction should be considered in early stages of planning, long before item writing assignments are made:
1A. Each item should assess content standard(s) as specified in the test blueprint or assessment frameworks.
2A. Items must measure appropriate thinking skills as specified in the test blueprint or assessment frameworks.
3A. Items should be written at appropriate cognitive levels and reading levels according to the item specifications guidelines.
A test blueprint identifies the skills and/or knowledge to be assessed, provides the item-to-skill distribution, and specifies item formats.
Let's think about creating a blueprint to assess writing at grade 6. We'll base the blueprint on the Common Core State Standards.
In the CCSS, English conventions are addressed in the language standards, and what we might call writing strategies and application are addressed in the writing standards. The language standards could be assessed with a variety of formats: standalone or passage-dependent multiple choice items, standalone or passage-dependent technology-enhanced items, or as one component of an extended-constructed-response item.
Here is a writing standard:
1. Write arguments to support claims in an analysis of substantive topics or texts,using valid reasoning and relevant and sufficient evidence.a. Introduce precise claim(s), distinguish the claim(s) from alternate oropposing claims, and create an organization that establishes clearrelationships among claim(s), counterclaims, reasons, and evidence.
b. Develop claim(s) and counterclaims fairly, supplying evidence for eachwhile pointing out the strengths and limitations of both in a manner thatanticipates the audience’s knowledge level and concerns.
c. Use words, phrases, and clauses to link the major sections of the text,create cohesion, and clarify the relationships between claim(s) and reasons,between reasons and evidence, and between claim(s) and counterclaims.
d. Establish and maintain a formal style and objective tone while attending tothe norms and conventions of the discipline in which they are writing.
e. Provide a concluding statement or section that follows from and supportsthe argument presented.
Generally the above standard would be assessed with an extended-constructed-response item, because multiple-choice items and short constructed-response items don't allow students sufficient opportunity to demonstrate the ability to "write arguments to support claims...." However, the subskills may be (and frequently are) assessed with multiple-choice items; this is more common at the district or classroom level than at the state level. You might see a question that addresses W.1.a by asking the student to choose the best opposing claim for a given argument. Such multiple-choice items may help teachers isolate specific areas in which a student needs instruction and support.
Here are language standards:
1. Demonstrate command of the conventions ofstandard English grammar and usage whenwriting or speaking.
a. Ensure that pronouns are in the proper case(subjective, objective, possessive).
b. Use intensive pronouns (e.g., myself,ourselves).
c. Recognize and correct inappropriate shifts inpronoun number and person.*
d. Recognize and correct vague pronouns(i.e., ones with unclear or ambiguousantecedents).*
e. Recognize variations from standard Englishin their own and others’ writing andspeaking, and identify and use strategies toimprove expression in conventional language.*
2. Demonstrate command of the conventions ofstandard English capitalization, punctuation, andspelling when writing.
a. Use punctuation (commas, parentheses,dashes) to set off nonrestrictive/parentheticalelements.*
b. Spell correctly.
All of the above language skills may be assessed with multiple-choice questions. These could be standalone, or could offer a stimulus: an editing passage with embedded errors. More on language items as previously discussed here.
For our imaginary grade 6 writing test, we might decide that we'd like to use multiple measures in order to obtain as much information as possible in as many different ways as we can, so we're going to create a blueprint that specifies a combination of item formats and includes x number of multiple-choice and technology-enhanced items, along with one extended-constructed-response to a writing prompt; this response will be scored with a holistic rubric that addresses organization, style and voice, and conventions. We would develop a test blueprint that specified the standards and subskills to be assessed, along with the number of items and item formats for each standard or subskill.
In our blueprint, we may also use Bloom's Taxonomy or Norman Webb's Depth of Knowledge Guide to determine the cognitive level for each item. Although the cognitive levels of some skills are relatively simple to determine, based on what is required from students, some skills may be addressed at multiple levels of cognitive complexity.
We may instead indicate the cognitive levels, item difficulty, and content or domain limits, and reading levels in the item specifications, as suggested in the CSSO/TILSA checklist.
In a typical statewide high-stakes assessment program, the decisions that inform the development of a test blueprint and item specifications are made by committees, which is as it should be, and committees should include classroom teachers. Committees often include other stakeholders, e.g., business leaders who may be asked to identify skills and knowledge necessary in the workplace.
Once all of that preparation is complete, item development begins.
Now let's say we've received an assignment to write those multiple-choice language items and that ECR writing prompt. We've read all of the project documentation and support materials; we have the item specifications in front of us.
It is a truth universally acknowledged that a test item should target one and only one skill or bit of content knowledge. Each idea should have one big idea; every part of the item should support that focus.
If we were going to write a multiple-choice item for W.2.b, our big idea would be how to spell grade-level appropriate words. We might write an item that looks like this:
Which word is spelled correctly?
A absense
B boundery
C civilizashion
D dissolve*
This item clearly targets one skill: correctly spell grade-level-appropriate words. The stem tells the student exactly what to do. The item is phrased simply and concisely. The content is neutral; there are no highly-charged words. All of the answer choices are grade 6 words (according to EDL Core Vocabularies); all are words likely to be known to grade 6 students and are words that are significant to academic content areas. There are no tricky or esoteric rare words. The answer choices appear in a logical order (here we use alpha order). All of the distractors address common spelling mistakes: using s instead of c, using e instead of a, and writing phonetically. None of the words are homonyms and so none are context-dependent; each of these words have one correct spelling.
Here is a poor item addressing the same skill:
Which word is written correctly?
A musheenz
B rabby
C anker
D pistol
This item has multiple flaws. First, the big idea is not specified in the stem; the student doesn't know what s/he is expected to do until s/he reads the answer choices. The answer choices are not grade-level-appropriate; "machine" is a grade 2 word, while "anchor" is grade 3. The word "rabbi" may not be familiar to grade 6 students. Answer choice A ("musheenz") is plural, while the other ACs are singular. Answer choice A also offers mistakes that are unlikely to be made by students at the targeted grade level. The answer choices do not appear in any logical order. Finally, the correct response is a type of weapon.
As bad as this item is, though, we could make it even worse by
- increasing the reading load by burying the spelling words in sentences and offering four sentences as the answer choices;
- obscuring the targeted skill by adding in other types of conventions errors, such as mistakes in capitalization and punctuation;
- using homonyms, or words that are spelled differently depending on the context;
- using above-grade-level vocabulary.
Item writing is both an art and a science. There's so much to consider, even in writing the simplest spelling item.
Saturday, July 27, 2013
How to Get the Best from Item Writers
Many years ago, I was a development manager at a Great Big Huge Test Publishing Company. I've already told the story of how I began as a temp employee in hand-scoring, as so many recruits to the test publishing industry do. Armed with my book-learnin' and a new but hardly marketable M.A. in English, emphasis in creative writing, I was thrilled to get a job that paid slightly more than $10 an hour, a job that had to do with words and writing. Yay me, illustrating the joy of low expectations.
When the Great Big Huge Test Publishing Company was awarded what was then considered a big statewide assessment contract (back in the days when we tested at grades 3, 6, 8, and 10, or grades thereabouts), I was plucked from hand-scoring, handed the title of associate editor and deposited in a cubicle in a cavernous upstairs honeycomb which cubicle I shared with another associate editor who'd also come from hand-scoring. Within 5 years, I'd gone from the windowless cubicle of associate editor to content editor to supervisor to program manager to the window office of development manager. You can probably guess at my success as a manager, given I had no training and little experience in management. Oh, if only I had read Bringing Out the Best in People: How to Apply the Astonishing Power of Positive Reinforcement.
Which I have since read, and which principles I endeavor to apply when I'm called to supervise others, and to the effectiveness of which I can attest. We do what we know; when we know better, we do better.
It may be tempting, when we consider the plummeting quality of what we see on test materials, to blame the writers. But as this video from ETS reminds us, the writer is only one of many contributors.
Getting the best from item writers has to start long before an editor sends out that Are you available? email. The foundation of a project must be sound; there must be a blueprint and prototypes; there must be a clear vision of what the product is intended to look like, how it is intended to perform, what skills/knowledge it is intended to measure, and how it should measure those skills/knowledge.
These decisions should not be left to the item writer; few item writers are equipped to make such decisions. In the past, item writers worked in-house, or were mostly former employees of test publishing companies, and so were at least minimally conversant with principles governing the design and construction of assessments. Sometimes item writers were corralled to help assemble the tests and were given Xeroxed sheets containing lists of item numbers and associated data. That is no longer the case. I don't know of any test publishing company who maintains a staff of in-house item writers. Today test publishing companies commonly hire item writers who have never worked for test publishing companies and who have little experience writing items for high-stakes assessment (they may have written for curriculum and textbooks, if they have any experience at all). They may not have any classroom experience; they may not even have kids, and so the world of education--the real world of education and of what kids really are able to know and do at a given grade level--is a mystery to them. Or they develop their own ideas about what K-12 students know and can do, ideas that are as inaccurate as they are ambitious and inflated. (This is through no fault of their own, but the remedy is simple: volunteer in the classroom. Go to a school and offer to spend an hour a week in a classroom.)
Even if item writers were equipped, they shouldn't make decisions which should rightfully be made at a much higher level, by folks with greater knowledge, experience, and authority. Such decisions take time. There must be time to consider, reflect, think about it in the shower and in the car, time to return to one's colleagues and say Well, what if and how will it work if. The what-ifs must be given time to rise to the surface.
Rushing inevitably creates chaos. Whatever writers produce under slippery circumstances--when the expectations are not specified-- will fail to meet those unspecified expectations.
Assuming, however, that the big decisions have been made, and that the writers have been provided with everything they need to do a good (or excellent) job, what else can companies do to get the best from writers?
1. Take care of all housekeeping details upfront. Provide the writer with written information about the scope of work, schedule, deadlines, pay rates, and points of contact. Preferably all in one email message. Send the contract and the W-9. Tell the writer whom to invoice and how. Remove possible sources of worry. Worry is destructive to creativity and productivity.
2. Provide training. The training should be as brief as possible, and should be conducted at the commencement of the project. A training that is offered a month before writing begins is useless, because writers will have forgotten the information they learned. Materials for the training should be emailed in advance. The writers should be told whom to call if they have questions.
3. There should be a dedicated content lead available to respond to writers' questions and to provide timely guidance throughout the course of the project.
4. Give writers the chance to do it right. The content lead's ducks must be lined up and ready to waddle. There must be a clear style to follow, preferences to comply with, and so on. The directions and feedback should be clear. To be effective, feedback must be immediate. Feedback must have the purpose of informing work in progress. Consider how disheartening it is to submit 50 items and then be told that there is now a new requirement, please revise those items accordingly and resubmit.
5. Allow the writers to work as they work best. More and more companies are requiring writers to input items directly into an online authoring system. While some of these are better than others, all add time and effort on the part of the writer, thus siphoning off energy better spent on item development. For each project, writers must learn how to use a new system; they might finish the project before they become proficient. Then it's off to a new system. I often decline opportunities to work in authoring systems, because I find the levels of clickage annoying--seconds add up to minutes add up to hours over the course of a year, hours I would much rather have spent reading or looking out the window or talking to my daughters or whatever else.
6. Let the writers do the work they do best. Writers write. Now that companies are operating on principles of leanness akin to corporate anorexia, companies are expecting writers to take on the work that used to be the province of content editors and desk-top publishers. With no increase in pay and no increase in time allotted to do the work.
7. Give writers enough space to write. Some assignments are so rigid and exacting, with so many criteria of so many types, that they become impossible.
8. Allow the writers to contribute their unique knowledge, experience, and skills. Writers work for all the educational assessment, test preparation, and curriculum publishers. They have access to a depth and breadth of knowledge about what's happening in educational publishing that is denied to the folks whose only job in educational publishing has been to work at the one company at which they are currently employed. Being open to the possibility that the writers know something and giving the writers freedom beyond the stricture This is how we do it will only serve the company and ultimately, the kids.
9. Be a human and let the writer be a human. We are none of us robots. We all have strengths and weaknesses. We have the skills we shine at and the skills we don't. This is normal and the nature of being human; it's not a flaw unique to a particular writer if she has trouble juggling multiple spreadsheets (not to name any names, me). No one in this world is capable of doing everything perfectly; no one is guilty of never making a mistake. The industry used to understand that; the protocol for test publishing included many rounds of editorial review prior to submitting materials to proofreading, and then to QA.
If these principles were applied, quality would improve.
That's all I got for today. I'm off to Valencia, to the Cal Arts campus, to visit my daughter, Twin A ("A" being the initial written on the knitted cap the nurses placed on her head after her birth) who is a creative writer in the California State Summer School for the Arts program.
What I'm reading: I finished As I Lay Dying. I love Faulkner. It always takes me at least half the book to marshall my resources to focus on his writing, I find it so challenging, but once I'm in, I'm there. I have a novel by Andre Brinks next, I think.
When the Great Big Huge Test Publishing Company was awarded what was then considered a big statewide assessment contract (back in the days when we tested at grades 3, 6, 8, and 10, or grades thereabouts), I was plucked from hand-scoring, handed the title of associate editor and deposited in a cubicle in a cavernous upstairs honeycomb which cubicle I shared with another associate editor who'd also come from hand-scoring. Within 5 years, I'd gone from the windowless cubicle of associate editor to content editor to supervisor to program manager to the window office of development manager. You can probably guess at my success as a manager, given I had no training and little experience in management. Oh, if only I had read Bringing Out the Best in People: How to Apply the Astonishing Power of Positive Reinforcement.
Which I have since read, and which principles I endeavor to apply when I'm called to supervise others, and to the effectiveness of which I can attest. We do what we know; when we know better, we do better.
It may be tempting, when we consider the plummeting quality of what we see on test materials, to blame the writers. But as this video from ETS reminds us, the writer is only one of many contributors.
Getting the best from item writers has to start long before an editor sends out that Are you available? email. The foundation of a project must be sound; there must be a blueprint and prototypes; there must be a clear vision of what the product is intended to look like, how it is intended to perform, what skills/knowledge it is intended to measure, and how it should measure those skills/knowledge.
These decisions should not be left to the item writer; few item writers are equipped to make such decisions. In the past, item writers worked in-house, or were mostly former employees of test publishing companies, and so were at least minimally conversant with principles governing the design and construction of assessments. Sometimes item writers were corralled to help assemble the tests and were given Xeroxed sheets containing lists of item numbers and associated data. That is no longer the case. I don't know of any test publishing company who maintains a staff of in-house item writers. Today test publishing companies commonly hire item writers who have never worked for test publishing companies and who have little experience writing items for high-stakes assessment (they may have written for curriculum and textbooks, if they have any experience at all). They may not have any classroom experience; they may not even have kids, and so the world of education--the real world of education and of what kids really are able to know and do at a given grade level--is a mystery to them. Or they develop their own ideas about what K-12 students know and can do, ideas that are as inaccurate as they are ambitious and inflated. (This is through no fault of their own, but the remedy is simple: volunteer in the classroom. Go to a school and offer to spend an hour a week in a classroom.)
Even if item writers were equipped, they shouldn't make decisions which should rightfully be made at a much higher level, by folks with greater knowledge, experience, and authority. Such decisions take time. There must be time to consider, reflect, think about it in the shower and in the car, time to return to one's colleagues and say Well, what if and how will it work if. The what-ifs must be given time to rise to the surface.
Rushing inevitably creates chaos. Whatever writers produce under slippery circumstances--when the expectations are not specified-- will fail to meet those unspecified expectations.
Assuming, however, that the big decisions have been made, and that the writers have been provided with everything they need to do a good (or excellent) job, what else can companies do to get the best from writers?
1. Take care of all housekeeping details upfront. Provide the writer with written information about the scope of work, schedule, deadlines, pay rates, and points of contact. Preferably all in one email message. Send the contract and the W-9. Tell the writer whom to invoice and how. Remove possible sources of worry. Worry is destructive to creativity and productivity.
2. Provide training. The training should be as brief as possible, and should be conducted at the commencement of the project. A training that is offered a month before writing begins is useless, because writers will have forgotten the information they learned. Materials for the training should be emailed in advance. The writers should be told whom to call if they have questions.
3. There should be a dedicated content lead available to respond to writers' questions and to provide timely guidance throughout the course of the project.
4. Give writers the chance to do it right. The content lead's ducks must be lined up and ready to waddle. There must be a clear style to follow, preferences to comply with, and so on. The directions and feedback should be clear. To be effective, feedback must be immediate. Feedback must have the purpose of informing work in progress. Consider how disheartening it is to submit 50 items and then be told that there is now a new requirement, please revise those items accordingly and resubmit.
5. Allow the writers to work as they work best. More and more companies are requiring writers to input items directly into an online authoring system. While some of these are better than others, all add time and effort on the part of the writer, thus siphoning off energy better spent on item development. For each project, writers must learn how to use a new system; they might finish the project before they become proficient. Then it's off to a new system. I often decline opportunities to work in authoring systems, because I find the levels of clickage annoying--seconds add up to minutes add up to hours over the course of a year, hours I would much rather have spent reading or looking out the window or talking to my daughters or whatever else.
6. Let the writers do the work they do best. Writers write. Now that companies are operating on principles of leanness akin to corporate anorexia, companies are expecting writers to take on the work that used to be the province of content editors and desk-top publishers. With no increase in pay and no increase in time allotted to do the work.
7. Give writers enough space to write. Some assignments are so rigid and exacting, with so many criteria of so many types, that they become impossible.
8. Allow the writers to contribute their unique knowledge, experience, and skills. Writers work for all the educational assessment, test preparation, and curriculum publishers. They have access to a depth and breadth of knowledge about what's happening in educational publishing that is denied to the folks whose only job in educational publishing has been to work at the one company at which they are currently employed. Being open to the possibility that the writers know something and giving the writers freedom beyond the stricture This is how we do it will only serve the company and ultimately, the kids.
9. Be a human and let the writer be a human. We are none of us robots. We all have strengths and weaknesses. We have the skills we shine at and the skills we don't. This is normal and the nature of being human; it's not a flaw unique to a particular writer if she has trouble juggling multiple spreadsheets (not to name any names, me). No one in this world is capable of doing everything perfectly; no one is guilty of never making a mistake. The industry used to understand that; the protocol for test publishing included many rounds of editorial review prior to submitting materials to proofreading, and then to QA.
If these principles were applied, quality would improve.
That's all I got for today. I'm off to Valencia, to the Cal Arts campus, to visit my daughter, Twin A ("A" being the initial written on the knitted cap the nurses placed on her head after her birth) who is a creative writer in the California State Summer School for the Arts program.
What I'm reading: I finished As I Lay Dying. I love Faulkner. It always takes me at least half the book to marshall my resources to focus on his writing, I find it so challenging, but once I'm in, I'm there. I have a novel by Andre Brinks next, I think.
Subscribe to:
Posts (Atom)