Devon Stewart
First-class pip support
We’re thrilled to announce that first-class pip support is now available on Replit! This should work out-of-the-box when importing existing repositories from GitHub. For developers who want to switch from poetry to pip: start by deleting poetry.lock, then move your dependencies over from [tool.poetry.dependencies] to requirements.txt (structure hint: flask = "^3.0.2" would become flask>=3.0.2,<4 in requirements.txt), and finally, delete the other [tool.poetry...] sections from pyproject.toml. After that, the packaging infrastructure will use pip for all future operations going forward.
Where we started
When we first chose how to support Python packaging, we went with poetry: a newer packager, offering a compelling feature set. While we stand by the value of having reproducibility and the stability that brings, we also acknowledge the significance of pip as a tool in the wider Python ecosystem.
One challenge we were trying to address in this change came in the form of following suggestions from AI assistants, StackOverflow, or blog posts that were written with the framing of pip being the “standard” packaging interface. Packages installed in this way would exist in an ephemeral state: both immediately available for use, but not yet recorded in the dependency list. This would no doubt result in frustration as a project that ran fine interactively in a Repl resulted in a ModuleNotFoundError during deployment.
How we solved it
Fortunately, since upm (Universal Package Manager) underpins the Replit Workspace package manager, the majority of the work to support pip could be done in one place. The biggest challenge was providing a nice user experience when upm add-ing a package. This came in the form of creating a pip UPM backend, as well as modifying the argument parser to be able to unpack version specifications when supplied directly on the CLI, mirroring poetry add and pip install’s CLI semantics.
To accomplish this, we wrote a requirements.txt parser, following the following standards: PEP-423, PEP-440, PEP-508, as well as checking our implementation against the PyPA specifications. Additionally, we reimagined the pip user experience to be in a better place for when a pip lockfile is supported natively. Instead of just taking a snapshot of all installed packages (as pip freeze does), we filter that list down to only the packages that we have just installed. This prevents the list of all transitive dependencies from ending up in requirements.txt with tightly constrained version ranges, making further upgrades difficult.
Plan for the future
While we are happy to share the work we’ve been doing, this is only the beginning. We aim to make it easier to onboard new package managers to UPM, as well as continuing to build out the supporting Golang functions for interoperating with the Python ecosystem. Some areas for research include: better support for PEP-517 and PEP 518, offering “migration” tooling from requirements.txt to pyproject.toml, or deeper analysis into package structure to improve upm guess.
If this work sounds interesting to you, check out our Careers page!
