Review of "Who: The 'A' Method for Hiring"

The basic argument of the book is the typical social science critique, that shows up in particular in Dan Ariely’s “Predictably Irrational,” that we are much less rational and much less capable than we typically think. When it comes to hiring, for example, we say that “personnel is the key,” but then we in practice mostly ignore that advice by waiting until there’s an opening to put out a “standard” advertisement that gets only a few decent candidates that we then put through a mostly hit or miss screening and interview process predicated on a few people’s personal judgments. Because we’ve run a “process” and because we trust our own personal judgments much more than we should, we fail in the hiring process more often than we would like to admit.

To give a JBU example, I personally would like to think that we have a fairly rigorous screening and evaluation process for faculty candidates at JBU (more so than we do for academic staff, for example), and I would like to say that we’ve done a pretty good job in hiring people in the 8 years I’ve been in this role. And it turns out that we are indeed following some (much?) of the advice in this book. Nevertheless, our success rate (defined as faculty who stay in their roles for more than 3 years and who score above our teaching and OCR goals) is at best 50%. That’s pretty much what all of the studies on hiring would indicate, that the typical hiring process is about as good roughly as flipping a coin (50% success). Sigh.

So what can we do better here at JBU? Here’s the advice from this book (with accommodations for the “business speak” which almost killed the book for me).

1) “Who” matters more than “what.” In theory, this is just another example of the “personnel is the key” argument, but in practice, very few of us really believe this (we talk about “mission,” for example, way more than we do “people”), and it’s taken me 20 years in higher education and 7 years in this role to take this idea really seriously and to begin to study the issues in more depth. The authors suggest that you should shoot for candidates “who have a 90% chance of achieving a set of outcomes that only the top 10% of possible candidates could achieve.” My bar has been lower than that, and I know that my bar has typically been higher than the faculty on the search committees (primarily because of the usual pressures to hire “someone” before the year begins).

2) So if we really wanted to aim higher in our hiring process (especially our faculty hiring process), what might we do differently? The authors suggest a 4-step approach: scorecard, source, select, and sell. I’ll take them each in turn, but to get away from the usual approach of waiting to “source” until the search has already begun, I’m reversing the order of the first two items and putting “source” first.

3) “Source” just means creating and maintaining networks. For example, the Bible and Business divisions probably do this better than the rest of us, and those two divisions have scored the best on our various measures over the last five years in part as a consequence. Again, we all know the theory here, but few of us probably do the kinds of things that this book suggests.
- How many of us keep a list of possible people we’d like to consider hiring someday and then actively keep in touch with those people?
- How many of us ask our colleagues at other institutions “who are the best people in your organization” and then keep notes for later reference?
- How many of us encourage our subordinates (with pay bonuses, for instance) to feed us the names of the people in their fields who we should keep an eye on for potential hires at some point?
- How many of us actively try to develop those who work in our areas (which also means saying “no” to some others) instead of just allowing the seniority system to take its normal course?
- How many of us actually reward (financially or otherwise) those employees or those colleagues at JBU or at other institutions who’ve given us a referral that has eventually panned out?
- How many of our units have created advisory boards, one of whose main purposes it to create connections (to students, to alums, AND to possible future employees)?
- How many of us know the “connectors” in our fields who “know everyone” and how many of us have been tapping those people for possible referrals?
In short, how many of us actually do the kinds of things that we know we should if we really thought that “personnel is the key”? My own personal take-away is that I need to create a better tracking system for potential recruits to JBU and to follow up more diligently in looking for these individuals. Whenever I hear a presenter I like or see someone interesting on a list of people who’ve attended a CCCU/CIC leadership event, I need to get that person on my list and follow up with them whenever possible. For example, there were a couple people at the CCCU Forum last spring that in retrospect, I really should have said hello to and gotten to know better.

4) “Scorecards describe the mission for the position, outcomes that must be accomplished, and competencies that fit with both the culture of the company and the role.” As a mission driven institution with a fairly detailed evaluation system, we probably have some of these pieces in place, at least on the faculty end of things. We even have an actual scorecard that I fill out for each faculty candidate with various groups feeding into this system along the same lines as what we do with our formal evaluation system. That’s a relatively new effort on our part, but it’s very consistent with what this book suggests, i.e. to create a detailed set of quantifiable outcomes for each position that everyone has agreed on before the search begins and that each candidate will be judged on. I’ll be looking to see over the next few years whether our judgments during the search process turn out to be on the mark or not when people finish up their 3-year evaluations. From two years of data, I can already tell that our judgments during the search process tend to be higher than our judgments after someone’s been here awhile. With this book’s conclusions in mind, we might tighten up this faculty “scorecard” practice even more and, once we have a more consistent staff evaluation process in place, perhaps apply some of this thinking to this area as well. We’ll see.

5) “Select” basically refers to “winnowing the candidates that you have found through your sourcing process.” This is the section of the book that I learned the most from. The authors suggest a four-part “structured” interview process that seems to comport well with the arguments I’ve read in other books and articles on hiring.

- “Screening” interviews are short phone calls using a standard list of questions. The authors give lots of suggestions for what to look for in these conversations and how to follow up on points of potential interest with “what, how, and tell me more” questions. The screening process is designed to winnow down the field to no more than 3 and maybe just to 1 or 2 candidates.
1) What are your career goals?
2) What are you really good at professionally?
3) What are you not good at or not interested in professionally?
4) Who were your last five bosses, and how will they each rate your performance on a 1-10 scale when we talk to them?

- “Topgrading” interviews (pardon the abysmal business speak) are the main interviews of 2-3 hours. Again, these are very structured interviews with a small group (perhaps our search committee?) in which you look for patterns of behavior instead of just trying to “get a feel” for an individual (which is what most of our interviews do). It’s essentially a chronological walk-through of a person’s career in which you ask related to each job (or “chunks” of jobs if the person has moved around a lot) the following 5 questions. And again, the authors offer lots of suggestions for how to maneuver through each question including a sample script for the beginning of the interview.
1) What were you hired to do?
2) What accomplishments are you most proud of?
3) What were some low points during that job?
4) Who were the people you worked with? Specifically . . .
a) What was your boss’s name, and how do you spell that? What was it like working with him or her? What will he or she tell me were your biggest strengths and areas for improvement?
b) How would you rate the team you inherited on an A, B, C scale? What changes did you make? Did you hire anybody? Fire anybody? How would you rate the team when you left it on an A, B, C scale?
5) Why did you leave that job?

- “Focused” interviews are chances for wider input on specific elements of the scorecard, especially regarding cultural fit. We already do much of this by having the candidate meet with the faculty status committee (regarding the “spiritual modeling/cultural fit” outcome), a class (regarding the “teaching” outcome), and the president, VPAA, and Dean (regarding multiple outcomes but again primarily regarding “fit”). I’m not sure if we want to modify our system much to get even more “focused” feedback, but here’s what the authors might suggest if we were to consider heading more in this direction. We’d again have a structured list of questions as follows that a few small groups would each ask (for up to an hour) regarding one of the outcomes and/or a couple of competencies on the scorecard.
1) The purpose of this interview is to talk about (fill in the blank regarding the outcome and/or competencies).
2) What are your biggest accomplishments in this area during your career?
3) What are your insights into your biggest mistakes and lessons learned in this area?

- “Reference” interviews are, interestingly, done after the main interview and not before. After comparing notes among the team based on the interview day (how well does this person match the scorecard), you may decide to continue with this candidate. At that point, you need to pick the references, especially those not given to you initially by the candidate but instead those that came up during the interview process. Then you need to ask the candidate to set up the reference checks (apparently for legal and “transparency” reasons). Finally, you need to call 4-7 of these references—three past bosses, two peers, and two “subordinates” (assuming the person is in an administrative capacity). You can divide these checks between the small group running the search. Again, there’s a structured list of questions to ask. I was particularly intrigued by their tips for the “code” to look for in risky candidates because I’ve said or heard many of these things (“if . . . then” statements in particular) and hadn’t always recognized the “code.”
1) In what context did you work with the person?
2) What were the person’s biggest strengths?
3) What were the person’s biggest areas for improvement back then?
4) How would you rate his/her overall performance in that job on a 1-10 scale? What about his or her performance causes you to give that rating?
5) The person mentioned that he/she struggled with (fill in the blank) in that job. Can you tell me more about that?

6) “Selling” is something I think we have a fairly good sense of, living in rural Arkansas as we do, but we might learn something from the author’s breakdown of the things to talk about. That list includes fit, family, freedom, fortune, and fun. Okay, we can’t sell “fortune,” but the others we can. Again, there’s lots of follow up here, but notice that “fit” is the first item, even in a profit-driven context. The other thing to note is that “selling” takes place throughout the entire process, not just at the end. Basically, persistence pays off.

So, what does all of this mean for how we might function here at JBU? I’m sure some of you can see potential applications in your areas, but I’ll just focus on how we might run our faculty candidate process a bit differently in the future as a consequence of these ideas.

1) I’m going to be more proactive about networking and keeping notes about prospects.

2) I may refine the faculty scorecard process, perhaps by making that scorecard more transparent to all parties, including the candidates.

3) I’ll be more involved in the screening process than I’ve been in the past, including participating in the screening phone calls, which I’ve only done on occasion up to this point. And I’ll probably use this list of questions for those screening interviews, perhaps along with a couple JBU-specific questions.

4) For the interview day, I’d like to combine the search committee and VPAA interviews into a 2-hour “chronological review” interview more along the lines of what this book is suggesting, again by using the list of questions included here. Rob (or Dick) will probably still do a short “welcome and review the day” meeting, perhaps over breakfast, and the rest of the interviews will probably stay pretty much the same.

5) I’ll probably move the reference checks to after the interview day and follow more of the format noted here.

6) “Persistence” has never been a problem for me, so I’m not sure how much I’ll change here. 

7) I do want to follow up on some of these ideas for a possible staff hiring and evaluation process that Darrin and I are likely going to work on this year.

8) I would encourage those on the academic side of the house to consider adopting some of these ideas, as appropriate, when hiring adjuncts and staff members in your areas.

If you’re still reading, thanks for hanging in there. As usual, if you have any thoughts, feel free to send them along.