Wed, Apr 28, 2021Why We Built Our Own DNS Infrastructure
This post is part of a series about the wonderful world of clusters. Check out the first post for an overview of what clusters are all about. In this post we will take a peek under the hood of our hosting infrastructure and walk through how we made hosting work in a multi-cluster world. Hosting overview If you didn't already know, you can host web servers right on Replit.com. Just create a new repl and spin up a web server using Flask, Express, or your favorite web framework. We automatically detect the web server and open a webview in the workspace. Your repls are automatically accessible via a *.repl.co domain and usually looks something like <repl name>.<user>.repl.co. In fact, this blog is hosted on a repl. On the backend, a proxy service handles proxying HTTP requests to the proper repl based on the host (domain name) of the request. This service shares a database with the container management service so that it knows which repl container to proxy HTTP requests to. Pre-Clustered World Our pre-cluster hosting setup was rather simple, we ran multiple instances of our proxy service behind a single load balancer. The load balancer had a single static IP address; both repl.co and *.repl.co had an A record with the static IP address of the load balancer. For an intro to DNS, check out howdns.works
Tue, Apr 27, 2021Why We Switched From Webpack To Vite
At Replit, our mission is to make programming more accessible. We provide people with free compute in the cloud so that they can build apps on any device. Among the most popular ways to create apps on the web today is React. Historically, however, React tooling has been slow on Replit. While the JavaScript ecosystem has produced excellent tools for professional developers, many of the most popular ones, like Create React App and Webpack, have become increasingly complex and inefficient. Fortunately, we've seen the JavaScript community recognize this problem and move to build faster and more efficient tooling, which means we can finally deliver the experience our users expect from us. This new experience is powered by Vite, a JavaScript build tool that provides a fast and lean development experience. Vite comes with a number of features including HMR, or Hot Module Replacement, a build command that bundles your tools with Rollup, and built-in support for TypeScript and JSX. Vite makes React dev fast. Like really fast. With HMR, changes you make trigger rerenders within milliseconds which makes prototyping UIs really quick. With that in mind, we decided to rewrite our React template using Vite and we were shocked to see just how much faster it was. Here's what it looks like next to our old CRA template:
Fri, Apr 23, 2021Debug, Debugger, Debuggest!—A new Collaborative Comprehension Experience
Earlier this year, we decided to close the #1 most requested entry in our Canny board, which requested to improve our previous debugger so that it could also work with multiple files in a project. This was done because it became very clear that there is a need to have better tools to aid with program comprehension built directly into Replit. But we also realized that we could go even further. Even though most of our users could be happy with a traditional debugger experience, our mission is to give people computer superpowers! So today we are announcing that we are working on a new, reimagined, collaborative debugging experience (or maybe we should call it a "program comprehension experience?"). We plan to tackle this problem from two fronts: the first one is that interactive debugging is very limiting and potentially frustrating, in the sense that it only exposes the state of the program at a single point in time. There are ways to make the program state advance, but if it advances too much, there's no going back, and the program needs to be restarted from the beginning and be placed in the same state again. This process is stressful enough that one very commonly-used alternative is to manually instrument the code to add tracing (also derogatively referred to as printf() debugging). Logging/tracing definitely has its merits and is a good practice to do overall, but it's not ideal to have to rebuild the program and re-run it just to know what it's doing. What's your theory for why programmers reach for printf-debugging more frequently than other modes (like step-debugging)? — Amjad Masad ⠕ (@amasad) April 9, 2021 All of these problems are the reason why we want to provide a time-travel debugging experience for our most popular languages, which should make the whole process a lot less stressful. The best thing is that we plan to make this such that there will be very little setup needed, other than enabling debugging on your repl! The second problem that we want to attack is that whatever we do, it must integrate well with the Replit editor, and that includes features that are mostly unique to it: collaborative code editing. This means that the debugger will also have a way for a group of people to be in this debugging experience at the same time, collaborating in understanding what the code is doing, and why it is doing so. We still have some open questions about what this is going to look like, and the kinds of problems that you are trying to solve, so we are going to reach out to some of you for interviews to gather a better understanding and design tools that will work great for everybody. Because Replit today is the editor of choice for so many people at the start of their programming journey, if we're successful, we will expand access to debugging tools and increase mindshare and hopefully get the next generation of programmers using advanced debugging techniques. In a sense, this is going to bring us several steps closer to achieving the dream of learnable programming!
Thu, Apr 22, 2021A whiteboard for every repl
Replit is a place where beginners, educators, and professionals alike can code and share their ideas collaboratively. Something that goes hand and hand with your source code is how you communicate it. We built threads, which allow you to leave contextual messages around code, and added better markdown support for READMEs and other guides. However, we were missing a visual way for people to explain and collaborate around code. Today we're excited to announce support for draw files, powered by excalidraw! Any new file with the .draw extension will become a whiteboard to sketch shapes, draw freely, and add text. These files are multiplayer by default, anyone who joins your repl will be able to sketch and diagram along with you. During this period of remote collaboration, we've used draw files to sketch out problems with candidates during interviews, to diagram how our internal systems work, and to quickly prototype user flows. How it works Thanks to the awesome open source excalidraw project, we were able to combine their library with our infrastructure to treat .draw files just like any other file in your repl. We use the same file structure under the hood, which makes importing from excalidraw a breeze (just save your work on excalidraw.com with a .draw extension and upload it to Replit).
Sun, Apr 18, 2021Announcing Secrets Management
Replit is the go-to place for a lot of people to build their applications. People build chatbots, web applications, games, and so much more. Naturally, projects like this need to maintain sensitive information (secrets). Today we're announcing a new and improved way to manage secrets. To give you an idea of how and when this is useful, let me give you some examples. To build a Discord chatbot, the programmer would have acquired a Discord API key. If the key is exposed, then anyone can come along and send messages as the bot. Before we gave a database for every repl, applications would use an external database with access credentials stored somewhere in the code. Exposing the database connection credentials is disastrous for obvious reasons. When someone builds a web application with authentication (users), the server signs cookies with a cryptographic key, ensuring that the visitors are who they say they are. Stay tuned for an announcement for a Replit authentication system, so you don't have to worry about signing cookies either. Traditionally, people purchased the hacker plan to make their repls private and keep their secrets secure. However, that meant hiding the repl and the code from the world. People wanted to also share their repls with the community. Code sharing allows our community members to get feedback, teach each other, gain customer trust, and get all the goodness that comes with open source. For this reason, in 2019, we introduced the first iteration of the secrets feature to Replit, the dotenv file (".env"). The file followed a line-based key="secret" format, which was consumed as environment variables in the code, only exposing the "key" part and keeping the secret safe. Dotenv served its purpose for our advanced users. They were able to use it to store data and configurations they want to hide. However, dotenv had many shortcomings. Unless people followed a Replit-specific tutorial or went scouring through our docs, the feature was undiscoverable. Beginners required an understanding of environment variables, and an empty free form file was overwhelming. The issues of dotenv even extended to the advanced users. The lack of a public standard for dotenv confused people when following external resources and tutorials. Another problem was that the dotenv file was not an actual file. Still, we communicated it as if it were. Some programmers would try to read it from the file system and be confused because it was not there. We did not write dotenv to the filesystem because forking relied on making a copy of the filesystem of a repl. It also gave us the flexibility of encrypting dotenv and taking other security measures to keep everyone's dotenv safe. It has become apparent to us that we needed a better solution. What made the most sense for us is to add a GUI that is easy to use for learners and keeps our advanced users productive.
Mon, Apr 12, 2021Announcing Replit Ventures
TL;DR Replit Ventures will give a grant of $2000 each to 4 startups and 1 non-profit (in Bitcoin) with a chance for more funding later on. They will have additional help with tools, mentorship, networking, and further fundraising. You can apply here. What is Replit Ventures? 💸 At Replit, our mission is to make programming more accessible. Our platform removes friction that exists when building applications. This seamless development process has enabled entrepreneurs to create prototypes, MVPs, and even launch startups all in one place. We're proud to share that there are many ambitious startups that either started or running entirely on our platform (e.g., Fig, Blubbr)- some generating thousands of dollars every month. Our users are building innovative products, and we want to help them grow. Replit Ventures is our brand-new experimental startup incubator dedicated to bringing emerging startups and ideas to market. We want to invest our time and money into the next generation of entrepreneurs. What makes Replit Ventures different? 💡
Mon, Apr 12, 2021Meet the college students who used Replit to build a startup with $1,000 in monthly revenue in 3 weeks
Hi 👋🏻, I'm Søren. I'm a computer science student based in Seattle, Washington. I've always been interested in the intersection of finance and computer science and I tend to find myself building things in the space. Before I was offered a position at Replit, I worked as a software engineer at a FinTech startup writing market-making systems for various crypto brokerages. While I was doing this, I was also working on a side-project with my close friends and co-founders Justin and Steven. This project is called Blubbr. What is Blubbr? When we first started Blubbr, we just wanted to make a trading bot. We became interested in the relatively new market of SPACs and had developed a strategy we believed would make us money. A SPAC is a company whose entire goal is to buy a private company and take them public. Before a SPAC is allowed to talk about anything to the public, they have to release a special type of form on the SEC website. We discovered that shortly after companies release these specific forms, their stock prices tend to increase. Blubbr was created based on the premise that if you bought within minutes of the form coming out, there was a good chance that you would make money. How did we build this so fast?
Sun, Apr 11, 2021Boosts
Our Hacker plan has always been a great choice if you need more powerful repls. In addition to benefits like private repls, more storage, and always on, the Hacker plan gives all of your repls 4x the CPU and RAM compared to those in our free plan. However, sometimes, even 4x is not enough. Whether you're building a new Terminal or playing Doom, you could blow past those resource limits pretty quickly. That's why today we're excited to announce Boosts! Boosts are a new addition to our Hacker plan that allow you to make your repls even faster. Boosted repls come with 4 vCPUs and 4 GB of RAM which is double the resources of standard Hacker repls (and 8x more than free repls). As of today, we're including 5 free boosts as part of the Hacker plan. However, boosts can also be purchased on a free plan using Cycles. To boost your repl, simply click on your repl's title to bring up the repl info card and click the "Add boost" button underneath the always on toggle like so:
- Thu, Apr 8, 2021
Introducing system audio
We know that games are an important part of our commitment to making programming more accessible, more creative, and more fun. Back in February when we announced a significant revamp to our graphics stack, we also promised that we would also provide system audio integration. That support is finally available today, as an opt-in feature. How to opt-in We are firm believers in the "Don't pay for what you don't use" philosophy. Since enabling system-wide audio is moderately expensive, resource-wise, it's better for this feature to be opt-in. In order to do so, all you need to do is create a secret called VNC_ENABLE_EXPERIMENTAL_AUDIO with a value of 1 and restart your repl by running kill 1 on the shell. Once that's done, a checkbox with headphones will appear in the lower right corner of the VNC output window: Due to restrictions in the browser security model, there needs to be an explicit user interaction when enabling the audio, which means that the checkbox needs to be manually toggled every time the repl is opened.
- Wed, Apr 7, 2021
Securing your graphic repls
Have you ever wished to be able to protect your graphic repls against prying eyes? Starting today, all VNC connections are automatically protected by the same token that keep your connection to the repl secure. There is nothing you need to do, since this is completely transparent. We also acknowledge that some of you want to access your repls with the stock noVNC client. For that case, we are also allowing you to access it with the runner username and a password of your choosing: All you need to do is create a secret called VNC_PASSWORD. Due to limitations in the Remote Framebuffer authentication mode that is currently being used, only the first 8 characters in the password are considered at the moment, so try to auto-generate it with your password manager (like 1Password / LastPass) to make them harder to guess. The detailed steps can be found in our official docs. Don't worry about having to type the password every time you launch it: it will be transparently filled in when you access your repl from the website. Happy graphic repling!
- Mon, Apr 5, 2021
Introducing Kaboom
Games are awesome, making games is even more awesome. I love making games, making games makes me happy, being happy makes me make better games, i hope everyone can be happy and enjoy making games! -- me, today We at replit are thinking about ways to make programming more accissible, Kaboom is an effort to try make game making more accessible and fun for both beginners and intermediate programmers. I want to discuss some core designs of both the library and the editor here. (it's strongly recommended first check out some examples on the kaboom website to get a quick taste of what kaboom looks like) Official website
- Fri, Mar 26, 2021
Welcome to the Wonderful World of Clusters
Having a solid foundation is critical for us to be able to fulfill our mission of making programming more accessible, more creative, and more fun. We did a great job of making the platform more stable during last year, but every now and then we would still run into unforeseen problems that cascaded into other parts of the platform, producing a bad experience for everyone. So back in October (just a few weeks after I joined the team) when we had 2 load-related site-wide outages within a week, we knew it was time to do a major overhaul of how our infrastructure handles traffic. And now today, we're announcing that as a side effect of that infrastructure change, Hacker repls now run in newer, more powerful machines, which means that we'll have more room to grow and experiment with more benefits for Hackers (and you might find that things feel a little bit snappier lately). This is the first of a three-part series of blogposts on how we rebuilt our infrastructure over the course of ~5 months. Introducing a new failure domain The original goal of this endeavor was simple, in theory: split the Replit infrastructure into multiple failure domains, so that when (not if) the next incident occurs, it only affects a subset of the users instead of all of them. There are multiple strategies to do so, and we chose to move Hackers to their own failure domain. But what does this split entail? Prior to this change, our backend infrastructure was relatively simple: a Google Compute Managed Instance Group of VMs that run containers, and another one that serves HTTP traffic on .repl.co domains. So in theory if we take the right-hand side of the diagram and make multiple copies of it, treating each one as a completely independent cluster, we solve the problem. Mission accomplished, right? Not quite: clients now need to know where to make their requests to, since they will now need to hit different hostnames depending on which cluster their repls are (since they are now completely independent after all). Clients that wanted to connect to their repl through the workspace were already required to contact the replit.com web server to get an auth token for the WebSocket connection anyways, so it would make sense for the web server to be the source of truth about the repl->cluster mapping. But what about hosting? All repls get a .repl.co domain, and users can CNAME them to be able to get to their app. At this point, the problem started looking very DNS-shaped, and that's kind of what we ended up doing: an external cluster resolution service, that can be used by both the Replit.com web server and individual clients trying to access their apps!
Mon, Mar 22, 2021Replit Case Study - Creating a 700+ User Web App in a Month
Hey there 👋🏻 My name is Rishabh Anand and I am an ex-resident at the National University of Singapore's (NUS) Raffles Hall of residence. It's one of the on-campus student accommodations that houses students from all levels of undergraduate education at NUS with the majority being freshmen and sophomores. RHDEVS – the software development club at Raffles – was tasked by upper management to build a convenience app for the Hall’s student body. The app itself was nothing new; it has some basic features students would potentially use in their daily hall lives: Laundry Facilities Booking for clubs Events Booking / Management Calendar and Timetable Planner
- Fri, Mar 19, 2021
Introducing the Python package cache
Figuring out how to install third-party libraries can derail people from learning to code or starting a new side project. We built the Universal Package Manager (UPM for short) to save people from having to think about package installation at all. Just import the library, press run, and UPM will install it into your repl! Every time you run a repl or a repl wakes up, UPM checks to see if it needs to download and install any dependencies. This is handy (no matter what's gone inside your repl, it will always have the dependencies it needs) but slow: UPM needs to download typically tens of megabytes worth of packages, extract them on the local filesystem, and sometimes also pre-compile them for better runtime performance. Or at least it used to be slow until yesterday when we enabled the Python package cache, so now the most popular Python packages are pre-populated in pip's cache (/home/runner/.cache/pip), so the download step is going to be mostly gone for the vast majority of Python repls! It also uses pre-built wheels as much as possible to avoid even having to pre-compile code. How does it work? We had two goals in mind when we started designing this: it should be as transparent as possible, which meant that users would still be able to add packages to the local cache while avoiding copying files around (which would have defeated the purpose of using a cache in the first place), and
Thu, Mar 11, 2021Replit Case Study - Catalyst Coding Club
Realizing the lack of exposure to STEM skills for younger students, a group of high school students started Catalyst, a club to introduce students all ages to CS and making sure it is accessible for everyone. Located in Redmond WA, Catalyst strives to produce an environment in which students can thrive and become passionate about the technological world. It is our goal to help ensure students in our community get the exposure they need to computer science and make an informed decision about their future careers. Working on projects and organizing discussions are only a couple of the ways we go about exploring computer science. Whether it is the intricacies of a Python Loop, the making of a div in HTML, or the application programming in Java, we strive to inspire genuine interest in our students over the span of a school year. With the rise of COVID-19 in the United States, many states were forced to shut down schools and resume operations online. This has been detrimental to the learning capabilities of students, especially the younger ones. Now that most resources have been moved online, schools and organizations have been looking for programs that can help aid them in teaching various concepts to students and help those especially at a younger level learn about various concepts using simulations. Our organization was many of many that needed a way of teaching coding online to our students. Transitioning to online learning as a club has been difficult for us because usually in a normal environment, we would be able to help students debug their issues in person. Adding on to that most of the issues beginners and students face is with the editor itself. We searched for resources that can help us teach these passionate students about CS for several weeks, and that is when we stumbled upon Repl.it. Repl.it is a solution that fixed most, if not all of our problems. We didn’t have to worry about editor issues thanks to the built-in editor Repl.it has that supports several languages, from Python to Java to HTML. Not only does it have a built-in editor, it also has a unique suggestion system which helps us teachers pinpoint problems in the students’ code. Repl.it is built with many features that all in all really help us teach our students and make learning CS a lot easier and complex algorithms more understandable.