The Future Copilot

“It’s getting harder to work in Research.” This is a common phrase we keep hearing these days. Not because computer science research has suddenly become more complex. But because we’re beginning to see a whole new kind of researcher emerging. One that can not only beat the current AI benchmarks but can also figure out a way to translate them into meaningful product outcomes. We saw that today at the OpenAI Spring Update where two researchers demoed and launched an exciting update to ChatGPT. The lines between driving cutting edge AI research and strategically leveraging that to capture the markets have become the standard within the industry now. This is more evident when we look at the shipping cycles of ChatGPT, MS Copilot, Perplexity, Midjourney, Meta AI etc.

At a time like this, where research and product priorities are essentially the same, I find it imperative to have a clear lens for both research and product partners to align on key areas we invest in. The usual approach of letting research do its thing while product circles back to then take those advances and apply them strategically no longer work. This is where the Copilot Value Pyramid becomes more valuable.

In the previous memo, we used the Copilot Value Pyramid to understand what would be the most meaningful relationship between humans and a copilot. Here’s a quick summary in case you missed it - today’s copilots are excellent at INFORMATION seeking, essentially summarizing a plethora of things in new and unique ways. The next thing in that relationship is the ability to draw critical INSIGHT so the human spends less time in finer tasks. The level above that is the copilots ability to understand goals and draw IMPLICATIONS addressing the larger context of your work. Finally, it’s capable of creating an IMPRESSION around you so it can emulate your needs and values.

The current landscape is heavily focused on ‘Generative AI’ which enables most of the INFORMATION layer to work. The best models, GPT-4 and beyond are excellent at this layer and with improvements in things like RAG and access to enterprise data they get even more powerful. Everything working in AI right now is isolated to this layer.

For the INSIGHT layer to work, we need copilots to be able to successfully do complex tasks. E.g. Drafting the quarterly business report. While almost everybody in this space is talking about ‘Agentic AI’ or agents we need to move beyond the hype. As of May 2024, the best GPT-4-based agents only achieve less than 20% success while evaluated in functional correctness environments. Simply put, the current state of the art agents simply aren’t good enough to run independently completing complex tasks. With GPT-5 being teased soon, this might be the biggest leap we make in this industry soon. Where agents begin to solve increasingly complex tasks by interacting with the world as they continuously learn and adapt.

The IMPLICATION layer of the pyramid is still far from being achieved. This is the ability of a copilot to understand the larger goal a human might have. E.g. Helping a provider improve the quality of life for cancer patients. We’re barely scratching the surface when it comes to copilots being capable or even operate at this level. Advances in Agentic AI will likely enable what I call ‘Proactive AI’ - the ability of a system to no longer have to rely on prompt or a human explicitly asking it to do things. Where a copilot understands the context of the larger goals and helps you achieve them more efficiently along the way.

Finally, the most meaningful layer in the pyramid creating a deeper relationship of Human-AI collaboration - the IMPRESSION layer. Almost all copilots today are likely to generate similar outputs based on identical prompts. I’d argue that this is terrible and will create redundant systems at an artificial scale. What we need instead is for a system to learn about you in an intimate way. Your values, your craft, your approach, your expertise, how you like to learn, how you see the world, what you consider as important, what gaps of knowledge exist in you and so on. This is what I call ‘Personal AI’ or an AI that is uniquely capable of making sense of the world around you, for you. We haven’t even begun to look at copilots through this lens and this goes far beyond the rudimentary custom instructions approach.

An elegant model like the Copilot Value Pyramid helps decipher signal from noise. A model that’s focused more on the world we’re trying to build or what the future copilot may look like and applying the advances in tech in that spirit, rather than getting distracted along the way and eating into the hype.