Platforms vs. Algorithms

Everyone knows that Algorithms are hard, but few understand Platforms.

Platforms vs. Algorithms

Startups have broadly two types of work. First is building platforms. A platform is something that either enables the creation of applications, or increases the value of existing applications. For example, GFS is a platform since it enables virtually every other Google application. Any kind of infrastructure code typically tends to fall in the “platform” bucket.

The other type of work is developing various algorithms. At startups, this means workstreams such as developing tracking algorithms, consensus/replication and other distributed systems algorithms, detection algorithms in computer vision and radar processing, and motion planning algorithms.

The Platform Skillset

While it is well understood that algorithms development requires a type of specialization, the notion that platform development also requires a type of skillset or specialization is not as well known. The reason for this is that the platform skillset is harder to define precisely. 

Platform development requires a set of skills such as:

  • system/API design

  • thinking about the “code as a machine” in the sense of identifying how various factorings affect long run returns from the codebase

  • understanding a bit of human nature with respect how people interact with the code or system, and how certain behaviors can be incentivized or disincentivized to produce optimal returns

Most saliently, platform development is a long term optimization problem that requires building a repertoire of key intuitions, and if you get those basic intuitions right, then you can make high value platforms, but if you don’t, you will make a mess.

Focus

The platform skill and algorithms skill are fairly uncorrelated, although typically people have to pick one or the other since becoming very good at algorithms means not investing as much time in platform development, and vice-versa. Therefore, in practice, most engineers will fall into one of these two buckets. 

From the perspective of team leads, this might mean that leads might want to focus people on their specialization. While certainly platform engineers are expected to understand the high-levels of algorithms so they can work with algorithms engineers, and vice-versa, we want platform engineers to primarily be focused on building platforms, and algorithms engineers to be primarily focused on building algorithms. Maximum organizational flow and efficiency can be pushed out by making sure that workstreams align with people’s skillsets. Conversely, having platform engineers work on algorithms can mean faulty mathematics, and having algorithms engineers work on platforms can mean non-linear damage to the codebase since everything is not well-designed. 

Integration

Both algorithms as well as platform engineers are also expected to perform integration work. Integration essentially is about making your algorithms or platform work actually useful, since without integration, your algorithm might just run in a simulated sandbox, or your platform would be used by nobody. 

For algorithms engineers, this basically means working with algorithms consumers, such as either a customer, or an internal application developer, to make sure that their algorithms designed work for the end goal. For example, perception algorithms engineers might write some algorithms in a python notebook, but it has to work with the real world hardware as well, so there is a set of work that needs to be performed there for integration.

For platform engineers, integration usually means integrating with other applications. For instance, a GFS engineer might have to work with a BigTable engineer to make sure that GFS is supporting their needs.


Thanks to Jared Newman for helping develop these ideas.