Michele Catasta
Amjad Masad, CEO: I'm excited to welcome Michele Catasta to the Replit team as our VP of AI. Michele joins us from Google, where he was Head of Applied Research at Google Labs and, before that, Google X, where he researched Large Language Models applied to code. Michele has a Ph.D. in Computer Science and was Research Scientist and Instructor in AI at Stanford University. This manifesto is a culmination of a conversation Michele and I have had for over a year on the future of AI and its impact on the software industry:
“People who are really serious about software should make their own hardware.” [1982]
More than 40 years later, Alan Kay’s quote still sounds as relevant as it gets – no wonder he is recognized as one of the most influential pioneers of our industry. As a matter of fact, over the past few decades software and hardware worked in conjunction to dramatically augment human intelligence. Steve Jobs celebrated Alan’s quote in his famous 2007 keynote, right after launching the very first iPhone – possibly one of Steve’s mottos, considering how Apple built its reputation and massive success story on unprecedented software/hardware integration.
At Replit, our mission is to empower the next billion software creators. Our users build software collaboratively with the power of AI, on any device, without spending a second on setup. We accomplished that by creating from the ground up a delightful developer experience. The Replit development environment allows our users to focus solely on writing software, while most of the hardware (and Cloud computing) complexities are carefully hidden away.
When it comes to software development, this recipe has paid off successfully for many years. Programming languages and frameworks have been gradually abstracted away from the hardware, by means of designing more expressive abstractions and of exposing more powerful primitives. Replit applies the same recipe to the development environment. We built a vertically-integrated software/hardware platform running completely in the Cloud, with the purpose of making software creation as frictionless as possible. No dependency downloads, no Cloud configs, no setups – Replit features are just one click away, enabling a seamless Human-Computer Symbiosis. But the story doesn’t stop here…
AI is changing the game for software development. Large Language Models (LLMs) can now write substantial portions of code, provide assistance with debugging, explain functions step-by-step, and recommend system design – all features you can already experience today on Replit Ghostwriter. These are major steps forward that we could have not imagined even a few years ago. That being said, modern IDEs have made strides towards improving developer efficiency, but they are still tailored around a developer experience that AI will inevitably disrupt.
At Replit, we are rethinking from the ground up the developer experience, with AI as a first-class citizen of the development environment. To this end, we have been working daily to build an ADI – Artificial Developer Intelligence. In time, the Replit ADI will create Agents capable of developing complex software architectures, both by generating code and by orchestrating advanced software tools developed and deployed on Replit. We have all the building blocks and the platform features in place to make this ecosystem thrive – for instance, Cycles (Replit’s platform credit system) will reward the creators of the tools orchestrated by the Replit ADI, while Bounties (Replit's creator marketplace) will engage developers in solving hard problems that our ADI still cannot tackle.
We began exploring the idea of developing an ADI because of a simple but powerful empirical observation. Much like Programming Languages (PLs) have been the lingua franca of the computing era, they are also becoming a fundamental language type on which LLMs are trained – as an example, LLMs gain stronger reasoning capabilities when trained on code. Our bet is that the utility of LLMs will grow exponentially as they become capable of creating Agents specialized on software development. On Replit, millions of users develop software applications that create value for both them and others. The productivity improvements due to Ghostwriter are already apparent, but we expect our ADI to greatly amplify the efficiency of every software developer, regardless of their experience level. Ultimately, such efficiency will unlock a new scale of value creation.
Building the Replit ADI is the most ambitious project we have ever undertaken. In spite of the meteoric rise of LLMs, we are deeply aware of the many challenges they still entail (for instance, hallucinations). Our focus on code, though, allows us to lift the LLM input/output representation from NL to PL, creating a new class of opportunities to improve the reliability of software development Agents.
In fact, contrary to Natural Language (NL), PLs have been designed with the aim of promoting structured thinking while preventing ambiguity. The idiosyncrasies of PLs have fostered a powerful ecosystem of tools and techniques to verify syntactic and semantic correctness, observe and reason about the program state, handle exceptions and recover from errors, and so forth. Replit has devoted a tremendous amount of engineering effort to support and harness a myriad of interpreters, compilers, linters, debuggers, testing frameworks, etc. As such, every time a user runs a Repl, our platform is uniquely equipped to create a real-time feedback loop between the runtime environment, the ADI and the developer.
This novel feedback loop sits at the core of the Replit ADI. We call it the “Reflect, Evaluate, Percolate and Learn” loop – a whole new meaning to REPL. The agentic behavior of our ADI won’t be limited by what recent LLMs can do, but will rather keep evolving thanks to the contributions and creativity of the developers using Replit. Already today, our first in-house ReplitLM (AKA replit-code-v1-3b) shows substantial performance improvements when further fine-tuned on public Repls. In the near future, our ADI will not only benefit from the static representation of code, but it will also Evaluate, Percolate and then Learn from all the salient actions that the Replit users perform on the platform. Imagine needing help to debug a particularly tricky portion of your codebase, and getting an immediate response from the Replit ADI which already fixed that issue literally a million times!
Our company history, since 2016, is one of steady and tangible growth rather than of chasing the hype – it’s deeply rooted in our culture. We can see the road ahead that will lead us to ADI, and we are paving the way for it – line of code by line of code. Only time will tell how thin the line is between ADI and AGI, and if one will become the foundation of the other. Our AI roadmap will remain the same, though. We are genuinely curious about AGI, and excited in playing a key role in the AI space, but we won’t get distracted by longtermistic aspirations – our laser focus is on ADI.
We have conviction that Replit will become the most powerful platform to create new software and AI tools, turning each one of our users into a 1000x developer thanks to the support of our ADI.
If we succeed in our mission, years from today experts will look back at our growth in the AI space and quote:
“People who are really serious about AI should make their own platform.” – Replit team, [2023]
