The Big Idea: Lauren McLaughlin
Posted on October 25, 2011 Posted by John Scalzi 42 Comments
After a small time out coinciding with my trip to Germany, The Big Idea is back, and to get us back into the swing of things, we’ve got an interesting one for you by my friend Lauren McLaughlin, whose new novel Scored imagines a world where monitoring and testing is taken to its final analysis conclusion — and the results, while perfect in their way, are also not exactly what would fill most people with joy. How did Lauren get from today to a world of “perfect” testing and surveillance? It started with a walk to the park.
A few years ago, I was living in the Hackney section of London on a street known as “murder mile.” It was an area which would later erupt in a storm of sudden vandalism during the London riots. Despite its grim reputation, it was actually a great place to live and very close to London Fields. On sunny days, I would venture out to that park to have lunch under one of the giant oak trees in the company of black crows. The walk to London Fields took me past a series of abandoned warehouses and every day there were usually one or two cars with their windows freshly smashed in. Hackney was living up to its reputation.
But one day I noticed that I hadn’t seen a smashed-in window in quite some time. Following the path of a crow alighting on something above me, I noticed the bright reflective lens of a brand new surveillance camera. Looking up and down the street, I saw about six more of them. The window-smashers had obviously seen them too and decided to change their ways.
But when I wandered down a different street–one where surveillance cameras had yet to be installed–I discovered two more cars with smashed-in windows. The criminals hadn’t changed their ways. They’d merely adapted. Here, I thought was the perfect test case for the effectiveness of surveillance in crime prevention. And the takeaway lesson was obvious: put surveillance cameras everywhere and there’ll be no more smashed-in windows.
This was a lightbulb moment for me. I already knew the arguments against ubiquitous surveillance. I’d read 1984. I opposed the Patriot Act’s warrantless wiretapping. I worried, as did so many, that England was “sleepwalking into a surveillance society.” But what I had never understood until the moment that crow landed on that camera was how seductive surveillance could be. Despite everything I knew and believed about government overreach and the potential for abuse, I wanted those cameras there. I appreciated their protective gaze as I walked to London Fields. They made me feel safe.
That is when I became truly frightened of surveillance–not because it would be foisted upon us by a domineering government or a corporate giant, but because we would invite it. It simply had too much to offer.
Shortly thereafter I moved back to the US and became aware of the debate raging about high stakes standardized testing in the schools. Again, I knew the arguments against it: an overemphasis on testing reduces class time to little more than test prep and replaces rich educational experiences with rote memorization and gamesmanship.
But there’s an upside to standardized testing too. It provides a means of evaluating teachers and schools in ways that can help those students currently underserved, to the point of betrayal, by their education. Additionally, it can provide an end run around a poor school system for those bright students who score well.
In short, standardized testing promises true meritocracy. Who isn’t in favor of that?
So what would happen if you combined ubiquitous surveillance with high stakes testing? What if instead of filling in those little bubbles on a written test, students were simply observed in their day to day lives by innocuous-looking little dome cameras so numerous you hardly noticed them any more. And what if a software program could crunch all of that data, along with information about their web habits, phone conversations, schoolwork, emails, etc. in order to produce a monthly score that represented their overall mental fitness?
Sound like a nightmare? Or the promise of true meritocracy finally delivered? In the world of Scored, there is no gaming the system. There’s no Kaplan Test Prep. No tutors for rich kids. Rich or poor, black or white, male or female, you are the sum total of your observable behaviors. How you walk, how often you swear, who you hang out with, how much time you spend on homework are all fed into the system. The software constantly learns from these observations, fine tuning its scoring algorithm until its results are indisputable. The highest scorers get into the best colleges, qualify for the best jobs, earn the most money. The lowest scorers fulfill their destiny as misfits, delinquents, and the permanently dependent. No one can argue with the accuracy of the score because there are no exceptions to the rule. Society doesn’t allow it any more.
But there’s no way we would actually allow something like this to happen, right? Surveillance cameras in parking garages are one thing. Reducing our children to a number is absurd. We’re smarter than that. We know how complex and unpredictable humans are. We’d never funnel our children into predetermined futures based on what a software program told us.
Consider this. The majority of US colleges and universities rely on the SAT to filter out students. The SAT was originally created to predict a college freshman’s academic performance. Do you know how often it accurately makes this prediction? Seventeen percent of the time. That means that four out of five times the SAT gets it wrong.
Did you know that boys routinely score higher than girls on the SAT but that girls routinely outperform boys in college? Is this fair? Is this merit?
The SAT used to stand for Scholastic Aptitude Test. But when people rightly pointed out that it did not, in fact, accurately test for scholastic aptitude, they changed it to the more generic Scholastic Assessment Test. This wasn’t much better as critics, again rightly, pointed out that it did not accurately “assess” anything other than the ability to take the test itself. Do you know what SAT stands for now?
It tests nothing. It predicts nothing. It disadvantages girls unfairly. And we’re still using it. Why? Because it promises meritocracy. It claims to take the chaotic unpredictable nature of human intellectual capacity and turn it into something neat and rankable.
And we love that, don’t we.
Scored: Amazon|Barnes & Noble|Indiebound|Powell’s
Read an excerpt (pdf link). Visit the book page here, including a trailer. Visit the author’s site. Follow her on Twitter.
I loved this post, Lauren. Can’t wait to read the book.
Thanks, Karen. I hope you like it. And if not, I hope you dislike it for really excellent reasons.
John, you are gonna bankrupt me with all these great books you suggest. My Kindle runneth over!
As far as the premise, I know kids are gonna find a way to game the system! This is an expression of their creativity. You go, kids!
And someone writes the program and decides, even in this AI system, what is important. And those geeks work for someone with their ideas of right and wrong.
But it is a good Big Ideaand I will buy this book.
I’m getting this right away. I’m going to get it for my son too, as I think he’ll really enjoy it while finalizing his college applications. :)
Wow. Great post. Sounds like a thought-provoking book, too–I’ll look for it.
Your novel premise does sound very interesting. I’ll check it out.
I find your understanding of modern testing and its relationship to scholastic achievement underwhelming. The SAT is an IQ test — not a strong one, to be sure, but an IQ test all the same, as all such pen-and-paper tests must be. IQ has the nice property of being relative easy to test; however it is by no means the only thing that matters in human achievement. Other traits — agreeableness, conformity, conscientiousness, etc. also matter a great deal. Not to mention luck.
As for your stat about the correlation of SAT with “success”: I don’t know where you got that from, (maybe here?), but there are many studies of this relationship. All show generally low correlations of all admissions criteria with “success”, usually freshman grades. But this is no indictment of the SAT: the SAT and high school grades are both predictors of “success” of roughly equal strength, and they tend to complement each other as they measure different things. And as for the strength of the test, well, you must read the footnotes (same paper above):
As for the reason they renamed the test, it was exactly because it does test “aptitude”, aka IQ aka intelligence. But many people hate the idea that some people have more aptitude than others, because they were born that way and did nothing to deserve it.
Finally, when you state that SAT rates girls lower but women outperform men in college, you are making the assumption that performance in college is the gold standard by which merit is measured. Personally, I would suggest using real-world success, not college, since education is not an end in itself to more than 1% of modern students. But in any case, for the narrow purpose of its design, there’s no reason why colleges cannot use SAT and grades and sex to determine admission. If they don’t use sex, it’s because they don’t need to — grades tend to measure agreeability, conformity, and conscientiousness highly, and these traits (which men tend to have a bit less than women) are also important at college.
Thanks guys! Pnkrokhockeymom, best of luck to your son and his college applications. I remember the stress of those days all too well. If it gets to be too much for him, here’s a blog post I wrote recently on the subject: http://bit.ly/oaEr6S.
Whoa. If the book is as good as this post, then I’m definitely reading it. Do you know anywhere I can read an excerpt?
Interesting perspective, Leonard. I think IQ is easy to test in large part because it doesn’t reflect anything very meaningful. Human intellectual capacity is vague, complex, and nearly impossible to pin down. I would personally have no problem accepting that some people “have more aptitude than others,” but I’m not convinced that the SAT or any IQ test truly measures aptitude, beyond the ability to take the test. I do, however, believe that tests like the SAT become self-fulfilling prophesies that reinforce hierarchies of dubious merit.
My husband and I were talking about this just yesterday. As a (US) society, we’ve expended a huge amount of effort in recent years to convince students that they don’t “really” want to be a plumber, they “really” want to be a lawyer. We’re trying so hard to force every child into a four-year university, we’re not even considering the possibility that not every child has the aptitude for that environment. We’re also convinced that our children are “too good” to be members of the skilled professions (carpenter, electrician, plumber, etc), we should be steering them towards the “real” professions instead (architect, electrical engineer, hydraulic engineer, etc).
My mother was an elementary school music teacher for nearly a quarter-century. She witnessed firsthand how, even in her classroom, she was forced to teach less and less about general music (simply learning songs), and more and more about mathematics (eighth notes, quarter notes, whole notes, etc) and other “three Rs skills” as they related to music. Why? Because public school districts these days are so focused on getting their share of the federal/state educational pie, they’re not really paying attention to the fact that there are *children* who need to *learn*. The educational bureaucrats have reduced every living, breathing child to a number–how well they do on standardized testing.
My brother, unfortunately, is a perfect example of what’s wrong with that approach. He’s a very intelligent man who’s been tinkering with electronics since he was a child, and who is now working for a major US computer/IT concern. But, from the time he was a child, he’s never “tested well”. He never finished college. Going by “the numbers”, that means he’s a failure, right?
Tell that to Michael Dell and Steve Jobs (RIP). They never finished college either.
Hi Sara J. I’m trying to get the first 2 chapters up on my website. In the meantime, I know you can read the first few chapters for free on a Kindle. Possibly on the Nook too?
“In short, standardized testing promises true meritocracy. Who isn’t in favor of that?”
I’m not sure about this as a logical conclusion. But as Lauren rightly points out, it is the perception of true meritocracy that really happens. My son started school this year and my only conclusion is that schooling is a start of a lifelong indoctrination process into conformity. And I’m not talking about the kid, I’m talking about how we, as parents are coerced to go along, because all the other parents are going along and sorry that we have to explain it to you but you are obviously a new parent and this is how we do it here.
Scored sounds like a really really scary book and i would love to read it.
Hi Gerhi, as the mother of a 2 year old, I am beginning to fear the onslaught of school and all that it entails for my daughter. I totally agree with you on the issue of conformity. I also fear the gradual degradation of my daughter’s natural curiosity as she is forced to squeeze her growing brain into the rigid confines of a test-obsessed educational system.
Sara J, and anyone else interested, I just posted the first 2 chapters of Scored for free on my website. Go forth and read gratis: http://bit.ly/sQO9mw
wow great premise! I’m looking forward to this now
Anecdote (not datum): Some people when “scored” appear to show remarkable talent because they are particularly attuned to answering a type of question found almost exclusively in assessment tests. If accepted/hired/promoted on that basis – which may be hugely out of proportion to actual ability – they may be as far or further adrift than the next bloke when a real-world answer is needed.
FWIW, the notion that test-taking ability might be transmuted into more widely applicable skills through the use of panopticon-style surveillance and corrective feedback is exactly the sort of harebrained conclusion I might expect from those who continue to extol the SAT, the ASFAB, and surveys such as “New Career Paths: Identifying Your Strengths” (spit).
Lauren’s take on the subject sounds much more humorous and snarky, however, so I look forward to reading this with pleasure.
Updated article to include sample chapters link.
What a great idea for a book – I’m going to Amazon! To quibble with the “SAT disadvantages girls” meme, I’m sure it does in some individual cases (ie: perhaps not getting accepted in as elite a program) the reality in colleges today is that girls have passed boys in almost every measure and a large majority of college students and graduates are female. Whatever “disadvantage” the SAT is providing doesn’t seem to be working very well!
Just finished the two-chapter offer and I’m stone hooked on what happens next. Dang it, Lauren!
I mean that in a nice way, really. Pax?
MuleFace, you bring up a good point re: boys and girls in college. Before long we’ll be looking for ways to level the playing field for the boys. I know a lot of people already feel that boys are underserved by the educational establishment. I think it’s good that we constantly question the way we educate people. It’s the only way to improve.
Nook + Wireless = downloaded. Plus got a great review at Boingboing.
Thanks, GC. Enjoy!
Oooo, have you seen any of the episodes of NBC’s new prime time drama “Person of Interest”? My bride and I have yet to miss an episode. I shall find it interesting to compare your vision in your novel with the drama’s underlying premise of ubiquitous survellance.
This is timely for me in Atlanta, caught up as we are in the cheating scandal regarding standardized testing. My understanding from the news coverage is that we aren’t the only system that is under investigation.
By trade I am a psychiatric social worker who works with incarcerated kids. By night I am a foster and adoptive parent for adolescent and teenaged boys. All but two of the 17 boys that we’ve taken into our home were involved with the juvenile justice system when we got them. Only one was still involved when he left. My oldest will complete his masters in December, my middle son is doing very well in the Air Force, another foster who keeps in touch has just won a combined athletic/academic scholarship for college. If any of these boys were judged by the averages, all would rank with the “misfits”.
The future that is proposed in this book makes me simultaneously nod, want to retch, and want to scream. I both want to read it and am not sure I can.
Have you any word on when this might be available for Kindles in Australia?
What Mr Norrish said.
They’ve recently re-introduced standardised testing in Australian schools, as part of a move to a national curriculum. As far as I can tell, they are very good at predicting how good you are at doing short answer and multiple-guess tests. As opposed to doing and understanding maths, or comprehending a piece of text. The results are blatantly aimed at parents – the scores of all schools are posted on a website so parents can worry about where their children are eligible (or obliged, in smaller towns, or where there are no private schools) to attend.
My husband had to pass GREs to get into a PhD program in the States. He has an eidetic memory, along with the academic chops, so he had his choice of schools after hitting the prep-books. It turns out, though, that GRE results are not an accurate predictor of whether you’ll enjoy being a grad student. Oh well. He’s the happiest artisan charcuiterer I know, these days; these things do work out eventually.
Shorter Big Idea: The road to hell is paved with good intentions.
Well said, Greg. If only I’d thought of that, I could have skipped the whole messy novel-writing part. Maybe next time.
Vian, how interesting that your husband went from PhD to charcuiterer. As a connoisseur of said meats, I’m glad he’s found his true calling. I’ve always found it odd that we’re expected to know these things at the ripe old age of 18. Thank goodness for happy accidents.
Michael, no word on Australia as of yet. They bought my last 2 books and published simultaneously with the US publication, so hopefully it will happen soon.
Christy, your story fascinates me. The novel I’m nearly finished writing is about juvenile detention, so I’ve been reading tons of stuff about kids who wind up there. One of the strongest points that keeps coming up in my research is how important it is for an at risk kid to have at least one good adult role model. Whether a teacher, social worker, parent, aunt, sibling or foster parent, it can make a world of difference. It seems so simple, but my guess is that it’s one of those obvious things that most of us overlook because we’re blessed with nurturing adult figures in our lives. Sounds like you’ve stepped into that role. Best of luck to your kids!
Jennifer, sorry I missed your comment earlier. Your story is familiar. I have a cousin who grew up right down the road from me. She’s whip smart, definitely sharper than me. But I was a decent test taker with a mother who instilled in me a desire to achieve academically. She found school boring and her parents weren’t as academically focused. There should have been another pathway for her, another means of tapping into her natural curiosity and interests. But that school, which was fantastic for me, basically failed her. I know how challenging it must be to reach a bewildering variety of different kids, but it’s something we should strive for. Otherwise we’re just letting untapped potential wither.
Gary, I’ve heard of Person of Interest, but I live in the UK right now, so haven’t seen it. I’m curious about the idea that the software can identify social security numbers of people involved in the upcoming event, but can’t identify how they’ll be involved.
Speaking as someone who’s had more than a little interaction with the College Board and the larger realms of standardized testing, I have only one response to this post:
Looking forward to the book, Lauren.
I’m intrigued by this Big Idea post and I look forward to reading this book. I just don’t understand one small part and am hesitant to simply nod and agree with that statement without proof. Has the SAT changed a lot since the early 1980’s, aside from becoming a test a young adult could opt to use a computer to take?
[The SAT] “disadvantages girls unfairly.”
How exactly does it do that? Can I please see some typical sample SAT questions that do so, for example?
I did try to Google ‘SAT disadvantages girls’ and ‘SAT sexist’, but found that the only relevant-seeming link or two that turned up high on the lists of results offered reasons that themselves seemed to make unfair blanket assumptions about the abilities and inclinations of both young women and young men. Maybe I need to use different search terms?
Hmm. Next time I’m in a big-box bookstore I’ll look through some present-day SAT prep manuals. Maybe what I see then will enlighten me.
Lauren: Cool! I downloaded the Kindle reader on my laptop (yes, that’s the most portable device for reading e-books I have) so I’ll check it out and see whether I want to check it out of the library once the county’s done doing its system upgrades.
Loved the first two chapters. Can’t wait for the rest.
We have a bright child in the local public schools. It is amazing how much even the school specifically for gifted kids can sometimes miss the mark. The very excuses they use to give out TONS of homework I’m sure could be cut and pasted into Lauren’s novel with little effort. The adage for every parent is a good education demands an involved educated parent. We are learning the art of polite and persistent demand.
To be fair, for the price, our local public schools have been a good deal overall. We’ve seen a lot of bright kids being shuffled to the side, but almost always because their parents were not involved enough to help them. Hard to blame a school for that, moreover I don’t know that I would be comfortable giving the government the power to correct such situations, as many of them would involved taking the child away from the parents.
Hi Elsa. Evidence for the SAT “disadvantaging” girls comes from the fact that it is supposed to predict a college freshman’s academic performance. Though girls routinely outperform boys in college, their SAT scores are lower. I’m not sure whether anyone has examined why this is the case. Still, I’d call that an SAT fail.
tolladay: My nextdoor neighbor’s oldest son is very academically gifted. He’s a sixth-grader (11-12 yo) this year, and for at least part of his academic career he’s been in gifted/talented (GT) classes. However, what’s been happening in recent years is that his parents have been told that there “aren’t enough GT students to justify” devoting even one class period to a particular GT subject.
I once dated a guy who was put in “special education” until his grandfather figured out that, unlike what the school thought, it wasn’t an issue of “he’s so slow he can’t keep up”, but instead an issue of “he’s so bright he’s bored silly”. When his grandfather insisted on having him tested, and he performed at GT levels, he was immediately put into the appropriate class(es).
If “special needs” children are routinely put into classrooms of less than a dozen, what’s so wrong with doing the same for GT kids? And what’s the point of academic testing if teachers are simply going to assume they “know” a child is slow, and treat him accordingly? Not testing at all is just as pointless as totally depending on test scores for all the answers–they’re just opposite ends of the same spectrum.
Hmmm…Yet another logic-train examination of the principle of unintended consequences. As in “it’s hard to remember when you’re up to your ass in alligators that you thought it was a good idea to drain the swamp.” A similar light-bulb moment happened to me when I saw an ad for a phone that was “smaller than any other phone on the market: so small that it’s hard to find.” There was a movie that I can’t remember the name of, that had as the villian…Ma Bell. They were plotting to surgically implant a micro-miniature phone in everyone’s head. I realized that they didn’t have to “plot” anything…just announce that it was available….and hire guards to keep the mile-long lines from mobbing up.
As to the educational aspect of this discussion, I am one of those who recognize the truth behind the statement that a test only measures your ability to pass that test. I am one of those who are REALLY good at passing tests. That ability has never helped me in the real world…ever. I never had the money to take advantage of college scholarships or loans, so my life has been a long line of dead-end jobs and missed opportunities. According to the tests I took, I was above genious. My subsequent life has had me scratching my head and quoting two of my favorite classic lines “hello, my name is Wile E. Coyote, super genious” and its bookend line “hello, my name is mud.”
Hi Lauren. I completely agree that the SAT is no predictor of success in college, business, or any other part of life except maybe a person’s ability to take more tests just like it in the next year or so.
I was just curious how the questions were biased. Back in the early ’80’s, I could well believe the verbal section could be jiggered somehow to be biased, but I’m scratching my head over how to bias a math section. Phrase the word problems in a biased way?
Or is “why” due to more than the choice or phrasing of SAT questions, and if so, can we know why? Would even surveillance or telepathy, both of which invasions of privacy I’m definitely NOT advocating, thankyouverymuch, be able to answer the question of why and how exactly the SAT does a disservice to a significant number of young women who want to get into the college of their choice, by somehow making them look less promising than they truly are? Howinheck did the test happen to be designed so it does that? Accident? Design? Maybe it’s not the SAT, but social factors, even before taking it, that might influence access to prep for an apparently pointless test?
I think there are all sorts of skills, traits, etc., a multiple choice test or surveillance can’t test that factor into life success, including persistence, social comfort levels & skills, level of hope or optimism or pessimism and their effect (enough to motivate, paralyze, influence in a helpful way or not), family and extended network situation, ability to learn new things quickly, ability to communicate things learned, creativity or ability to “think outside of the box” or make mental connections, ability to adapt, etc…. Humans are the sum of so much more than the ability to sit and take rigidly defined tests, and they routinely answer less well-defined questions than those tests present.
I am so looking forward to reading this book.
I’ll also go look for the previous, unrelated duology that explores gender. The sample chapters for both Cycler and Scored have me already caring about the characters and curious to see what happens to them. Thank you for posting them. Samples have more than once been highly effective in getting me hooked on series and non-series works by authors – Carrie Vaughn (werewolf Kitty series), Devon Monk (Magic series), and John Scalzi come to mind, among others.
As a third grade teacher, all I will say about standardized testing is that you really, really don’t want to get me started on the subject.
However, on our educational system in general, I will say this: Like with many other things, you tend to get out of it pretty much what you put in. That applies to both parents and students. Yes, over a dozen years there is a decent likelihood that you will encounter a handful of less-than-stellar teachers, and with luck you will encounter a couple of excellent ones. If you as a parent just put it on cruise control, though, and don’t hold up your end of the educational bargain, you’re doing your children a disservice. We are just now finishing the first quarter of our school year, and as always it was obvious weeks and weeks ago which kids have parents who were keeping an active eye on their school performance and which kids don’t. One guess as to which kids are having the most success–and maybe more significantly, which ones who are experiencing some struggles are still managing to keep their heads above water and not get swept away.
Hi Canyon42. I think you’re right that parental values regarding education are arguably the most important determining factor in the quality of a child’s education. Yet we continue to demand that teachers solve all of our problems, as if, on their own, teachers could provide custom-tailored education to every child, guarantee American supremacy in the global marketplace, and (this is my favorite) wipe out historically entrenched inequality. That’s a bit much to demand of teachers. If these things are as important as we all claim they are, we should recruit more than just teachers into the effort.
@Jennifer Davis Ewing: Your neighbors experiences are alas not the exception. If they are not aware, you might point them to Hoagies Gifted Education website. The site is broad and deep (and OMG, really needs an artist to organize it), but it is the best single source I have found so far for helping parents of gifted children.
@Scott Warwick: I feel your pain brother. One of the most common misunderstandings about the gifted is that because they are smart, they will be smart at everything. Nothing can be further from the truth. Being an especially good reader does not make you a mathematician, nor does it give you a better love life, or protect you from making poor business decisions or bad investments. The first long-term study ever done was based on IQ tests, and they discovered much to their surprise that there is no correlation between exceptional intelligence and success at life. For success you need to be good looking, or so I understanding, but not smart. I like to tell people that being exceptionally smart is like being exceptionally tall; its great if you are playing basketball, but sucks if you are driving a VW.
@Canyon42: Exactly. You get what you pay for, in this case “pay” meaning time you spend on your kids.
This gave me shivers! Flashbacks to my children’s time in school and the constant battle with the school “authorities”. Obviously, I must have this book!
“Master Li spat with lamentable accuracy upon a statue of K’uei-hsing, God of Examinations.”
Just finished it… what a wonderful story! Enjoyed the premise, the action, and the characters, but most of all the utterly enjoyable exploration of individualism taken to the extreme. Oh, and loved the setting too.
I think the book sounds interesting and I’ll make a note to read it, but I’m also underwhelmed by the criticism of the SAT. While a prior poster has given the “real” answer as to its alleged shortcomings, mine is much more metaphorical.
The SAT, even through its many redesigns, is a legacy app, designed to feed input into “college that was.” However, students, after being fed through this app, now attend “college that is.” The changes that have been made in college education make it more likely that girls will do well in “college that is” than they did in “college that was.” These changes have nothing to do with their innate scholastic ability (or that of boys.) Unless and until the SAT is modified in accord with “college that is,” there will continue to be a discrepancy. Unfortunately, that’s essentially impossible, because one of the fundamental differences is that “college that is” is far, far less objective and standardized than it used to be, and therefore anything which can even remotely be considered an objective and standardized test is fundamentally incompatible with it.