Recently, we surpassed one billion tests run on the Sauce Labs platform. No matter how many times I do the math, that enormous total still seems unreal. It’s just mind-blowing. To celebrate, I thought I'd take a look back at our first test, and at what Sauce Labs looked like back then.
Our first test was run back in 2009. Exactly when in 2009 depends on which test you want to count as our official first test.
The very first test recorded in our database was run by me, on January 9th, 2009, at 2:00:14am PST. It had to wait over two minutes to get assigned to a virtual machine, and it ran for just 33 seconds. Our database doesn't record the platform, but I know it was Firefox 3.0 on Linux, because that was all we supported when we first brought the system up. It used the old Selenium RC protocol, back in the days before Selenium WebDriver. It recorded no logs, screenshots, or video, because we didn't have those features yet.
I remember running that test. My co-founders and I had been doing demos of an earlier product prototype, but it wasn't turning out to be usable by most prospective customers. In early January, my co-founder Jason Huggins started to suspect that first prototype wasn't the right approach, and he suggested the concept for the Sauce Labs service as the world knows it now. That week, while Jason took one last shot at getting a beta user going with the earlier prototype, I worked furiously to get a first version of the new service up and running, and that Thursday I was so close to finishing it that I stayed up working late into early hours of Friday to wrap it up. I ran that first test sitting in the dark at my dining room table in my chilly apartment in San Francisco. We didn't have an office yet, or funding, or salaries. But just over a month later, we would have users.
The first test run by someone other than me was started on February 13th, 2009, at 11:44:27am PST. The user was part of our limited beta program, using the service for the low, low price of providing us with feedback. That poor user, an employee at a small consultancy specializing in automated testing, had to wait almost 10 minutes for a virtual machine to start up — hey, at least it was free!
Finally, on May 27th at 2:08:56pm PDT, we had our very first test run by a paying customer, Digg. We still didn't have funding or offices. In fact, for a while in 2009 I worked from a spare desk in Digg's offices, so that I could be available to provide them the kind of dedicated-bordering-on-obsessive customer support you get when you take a chance on a tiny startup. By the time Digg came along, we'd added more platforms — their first test ran against Chrome on Linux, probably because I told them that was our fastest and most reliable choice, but I know we supported Windows XP with Internet Explorer and Safari by then. (Windows 7 wasn't released until that October! We never bothered to support Vista, and only one customer ever complained about that.) We also had most of our core features: the test recorded screenshots and video. And we were able to get their test assigned to a VM in only two seconds.
No matter which first test you prefer to count, we still ran our billionth test on March 17th, 2017. We now run more tests every day than we did in the entire year of 2009, so we blew right past a billion by any measure (in fact, 37 jobs were started in the very second we crossed the one billionth mark).
Beyond our scale and usage levels, a lot has changed since then. We now support many versions of Windows and macOS, and hundreds of versions of Chrome, Firefox, Edge, Internet Explorer, and Safari. We've added mobile testing for iOS and Android, for web and native apps, on emulators and simulators as well as real mobile devices. The Sauce team has grown to hundreds of people, and we've learned to serve the complex needs of large enterprises, to operate our own datacenters, and to integrate with a complex ecosystem of testing tools. We've advanced the careers of our employees, done well for investors, and most important to me, we've aided in the success stories of thousands of customers.
Here's to the next billion tests!