Every employee has an output curve. The baseline is something we are interested in having in an employee on day 1, but is not to be mistaken with lifetime employee value. Ultimately what determines lifetime employee value is in fact the growth rate, because the growth rate is a signal of the aptitude or ceiling. The single biggest hiring mistake is hiring for experience (baseline), and not growth rate (or aptitude).
As illustrated by above, even though Non-Top Talent might start off with a higher baseline, the Top Talent ends up delivering far more value over their lifetime. Therefore, the growth signal is the single most important factor in evaluating talent.
That being said, typically, we want to also verify a baseline to ensure day-1 productivity as well as sustained interest in a domain area, even though in theory the top talent could pick up any arbitrary skill in some period of time.
Sourcing
Since verifying baseline is precisely what almost everyone in sourcing does, we omit details on how to source for baseline, and focus on the growth signal, which is far less applied in sourcing.
The growth signal is essentially learning rate divided by time. Therefore, in sourcing we want to look for people who have done disproportionately more relative to others in the same amount of time. Good examples of this is:
A new grad non-CS major but has top CS knowledge
Went above and beyond classes to gain knowledge in research or entrepreneurship
Very high rate of project delivery
A few flags for common errors in sourcing:
looking at experience without looking at time in which it was achieved
over-indexing on specific technologies, degrees, or backgrounds
Interview Design
Sourcing is the first step of the recruiting pipeline, and therefore, is unlikely to be perfect, mostly since people have an incentive to market themselves on their resumes, and more broadly, it is very hard to get a serious read of anyone just by looking at a one-pager. The only way to avoid the interview is to get a hot intro from a trusted person in your network, but even then a sanity check interview is warranted.
There are a few different types of interviews, but at most top startups, there usually is the following three types:
Coding
Design
Behavioral
The WHO Method solves the behavioral read, which approximates the theory of talent as presented here. Furthermore, we leave optimizing the design interview for a point in the future, but it would apply similar principles as the coding interview design we present here.
The Coding Interview
As in sourcing, we want to 1. establish the baseline for the position, and 2. filter for a growth/aptitude signal. For new grads, we recommend the following skills as baseline:
proficiency in at least one statically typed programming language
ability to solve basic data structures and algorithms problems cleanly and fast
ability to work in teams
Additionally, for algorithms heavy positions like Perception or Autonomy, we also expect some baselines in those domain areas (at the expense of maybe a bit weaker pure CS skills).
Most interviews produce a decent signal for 1, 2, and 3., but a few flags for common mistakes:
worth avoiding over-indexing on esoteric algorithm solving ability since this is not even remotely required for being extremely great at a typical software engineering position
mistaking solution recall for problem-solving ability
if your candidate produces a solution almost too quickly then you should consider the signal on that problem to be tainted and quickly aim to move on either to another part of your problem, or introduce a newer one
almost always you should expect to 1) have an “easy” part to your problem thats solvable in 5-15 mins, and then 2) a “hard” part that you expect to work through with the candidate to evaluate communication and problem solving skills
Filtering for growth/aptitude is much trickier, so we recommend leaving the read for that to leads or engineering managers. The growth/aptitude signal, again, is whats the quality of talent relative to background:
if the person has a totally random background but does shockingly well on a CS challenge, then thats a high signal
if the person is just generally a really fast thinker, speed is a very high quality signal on the challenge
if the person is great at abstract and/or analytical thinking
they have a bewildering breadth of knowledge for their career stage
Thanks to Brian Schimpf, Jared Newman, and Nabil Enayet for their feedback.