Faith-based versus Fact-based
- June 21, 2009
- 0 Comments
No, this isn’t a political posting!
I just read Steve Blank’s blog on Faith-Based versus Fact-Based Decision Making, and it resonated really well with my experiences at prior startups. As he points out, starting the company is an article of faith. The first company I worked for was started by 5 Stanford undergrads, most of whom dropped out of school to start the company. That is certainly an act of faith. However, as the company grew, there was more and more data available with which to make good decisions. Steve Blank’s post focuses primarily on the product development, customer development, and marketing aspects. But it is true of every facet of your business. And the transition from faith to fact is important. Not doing it can lead to some pretty bad decision making down the line. At the company referenced above, 10 years after its founding, veterans often referred back to faith-based decisions in the past as articles of proof that the right way to make a decision was based on faith rather than based on facts. “If we had relied on your facts, we never would have <insert action here, that gleaned millions of dollars for the company, or brought a key person into the organization>”.
Well, it is often hard to counteract this sort of discussion. But let’s take another aspect that most people can relate to: hiring. The first time you hire someone for your startup, it is an act of faith. Oh sure, you have the resume, you check references. You interview. You interview again. You have lunch, dinner. You meet for coffee. You spend a lot of time with that person. And it is still an article of faith when you hire them. Why? Because the role you’re hiring for isn’t well-defined (almost by definition in a startup). Because you don’t have enough other interviewers to compile feedback from. Because you don’t have enough candidates who are willing to work for peanuts and equity (or even for cash for a risky proposition).
It turns out, at a larger company, every time you open a new position (role), you run into this problem again. You don’t know exactly what the job description is. You don’t have anyone else to compare the candidates to in terms of “this guy is good at that job”. You don’t even have consensus on what it means to *be* good at the job.
At a previous employer we needed to hire someone to run our recruiting operation. The first person our HR director brought in, came highly recommended by an executive at another local company. Our HR director was confident they would pass the executive interviews with flying colors. But the candidate bombed across the board. I told the HR director we’d likely have to interview 10 candidates before we would figure out what we really wanted out of the job, and if we were lucky, one of those first 10 would fit the bill – but if not we’d have to be prepared to interview and additional 5-10 people at least. She was mortified at the idea of doing that kind of work for “just hiring a recruiter”. But we wanted someone who would really make the recruiting process run smoothly and help us source good candidates. Sure enough, we churned through 10 candidates… and then made our compromise, by realizing that we needed a different style of recruiting for engineering than we did for our professional services team. Our engineering hires were heavily dependent on the local networking capability of the recruiter – we needed someone with deep Austin connections who could get people who weren’t looking for a job to pick up the phone, and help us make the sale. But for professional services, we needed someone who had a process for finding people all over the country, and screening them before they got to us so that we didn’t waste too many cycles interviewing people who were obviously not a fit. (Our professional services group had a regional staffing model)
But once you get big (as my first employer did) and you’re hiring for a well-understood role, you can establish really interesting hiring machinery. Everyone goes through 4-6 interviews with specific goals. You track statistics on each interviewer, and on the folks that you hire, to correlate interviewers positively or negatively with good hires (note: you have to have a review system for this to work). You correlate job performance with GPA, previous job titles, industries, etc.
We were doing a lot of hiring of people from college. The data said that people with above a certain GPA, who also passed our rigorous interviewing process, generally did quite well. There wasn’t a lot of additional correlation of higher GPA to higher performance, surprisingly enough (some, but not much). However, the data also said that we rejected something like 99% of the folks with less than that cut-off GPA who were granted a first round interview… And worse, most of those we hired with less than that GPA didn’t perform as well as those with a higher GPA. My read on the data was simply that our GPA cutoff reflected more about commitment than smarts. After all, the people who passed our interview process were smart. So why would they get a bad GPA? Mostly, they weren’t committed during college. I’ve known people like this all my life – and most of them eventually find what moves them, get committed, and go forward with much success from that point onward (sometimes in highschool, sometimes in college, sometimes graduate school or the first or second job). But there is no way to predict what will trigger their inner mojo. And there is no reason to assume that your company is the secret ingredient that will unlock their motivation! Our recruiters, however, had it as an article of faith that some of the most outstanding employees had crummy GPAs (or even dropped out as the founders had!). They were arguing that we should continue to make these leaps of faith. But outside of the founders, every other successful anecdote with the low GPA was someone who had previous industry experience where they had demonstrated excellence. In other words, once someone has had a real job, GPA is pretty irrelevant as a predictor of performance, compared to data from their previous work history. We did NOT have a compelling case based on the facts, for college hires with low GPAs achieving success at our firm.
I think the key lesson is: when you don’t have enough data, you still have to make a decision, make the best decision you can. But when you have enough data, let that data inform your decision-making. Don’t confuse good fortune with a good process.