Depending on exactly who you ask, FTX started running around the beginning of May, 2019. So we’re calling this our anniversary month.
It’s weird to think that I still haven’t spent as long in crypto as I did in college, or high school, or at Jane Street. I started Alameda Research in late 2017, and each year since then has felt like a lifetime. In my past lives, four years was just a phase, a new set of goals to be collected and ultimately mostly forgotten. Now each year has brought with it a new company, new colleagues, a new city, and a new way of living.
We’ll have a series of posts up this month, taking a look at where we’ve come from and where we’re going. For now I’d like to start with one of our less important features, which we just released today. You’ll see a new login form on FTX:
It’s hard to get less sexy than login forms. No VC drools at the way ‘Email’ pops off the page, and no new customers are won with our decision to put ‘FORGOT PASSWORD’ in all caps. But it’s still a significant moment for us: previously we had been using Auth0 for authentication, and we’re now using an entirely in-house system.
One is that we just didn’t have total control over what you could do on the login form. Which seems irrelevant–you enter your username and password, obviously–until you think about all the details. Is there a button to reset your password? What does it require? What if you lost your 2FA? How do we handle brute-force attacks? What if we want ref codes, or custom subaccount logins? Each time we wanted to do one of these, we hit the same wall–what does Auth0 allow?
And as we dug more into it, we hit weirder limits as well. We couldn’t control the IP address of our login forms, and this caused it to lag out for some regions. This ended up being a significant problem, and the only real solution was to get off of Auth0.
We’ve seen this time and time again–that a third party system seems fine until you remember that you can’t change it, ever, even if you really need to. Even if the system has 99% of what you want, it’ll never get that last 1%!
Here’s another example. We had database issues for a month or so last year; we kept trying to track down the root cause but it was hard to find. The symptoms — skyrocketing database connection count, increased write latency, database load balancers rejecting lots of internal requests — all resembled that of standard database problems. But this wasn’t one of them–our database load seemed fine–and it’d appear out of nowhere.
So why did things sometimes grind down? After something like a hundred hours of solid digging, we realized that one of the databases we used allocated some particular set of information to a very small subset of its memory. All connections needed to know about this information, and it grew with every unresolved lookup, increasing exponentially. And so once in a while, it would exceed artificially its small memory limit and go to disk — this is significantly slower, which increased write latency and the number of unresolved writes at any given time, dramatically exacerbating the problem! Once we fully understood what was happening we could cover for it–we can detect when it’s making bad decisions and stop it–but we still can’t actually control exactly how it allocates memory. The only real, permanent solution is to stop using this industry-standard database system entirely, and build our own.
Over time we’ve come to believe that if there’s any system we want to fully rely on, we have to understand all parts of it–and often build them ourselves. You can see this philosophy permeate FTX: when it comes to margin and liquidations, we built a system up from scratch, and have never had a clawback. When it came to collateral, we threw out the old notions of single-collateral and isolated liquidations, allowing users to get full control over their balances–allowing a bunch of different assets to be used as margin for any of our futures and letting people share or isolate margin as they wish using subaccounts. We replaced inverse futures with linear, cross-margined, USD stablecoin settled futures. We combined all fungible stablecoins into one bucket so users could combine liquidity in them, and integrated our OTC liquidity directly into the wallet.
And in one of the truest signs of a product well built, you can see many of these ideas starting to propagate through the industry. Many exchanges are now experimenting with stablecoin cross-margined futures, mirrors of our CONVERT functionality, and more sensible margin limits. Our tokenized products are listed on tens of exchanges, and our partnerships are growing weekly.
We’ve also been quietly expanding our platform in ways that get less attention; I think that fiat is one of the most underrated parts of FTX. We’ve been frustrated for years with the state of fiat on/offramps in crypto; intrepid traders might remember getting “USD” on a platform only to realize that the only thing you can do with it is pay a 25% fee to send $50 to a South American bank account. Over the past year we’ve added support for 11 different fiat currencies, from USD to EUR to ZAR. And when we say we support a currency, we mean really support it: that we can take large deposits and withdrawals, settled promptly, with minimal fees. We don’t put “fake fiat” on FTX. And as time goes on we’re adding local channels for the currencies so you can do millions of dollars of deposits or withdrawals using fast, cheap interbank transfers.
We can already take SEPA EUR deposits, interbank USD transfers with most crypto banks, and international wires of a whole selection of other currencies. Today, we’re excited to announce one new member of the FTX fiat family: Brazilian Real (BRL). You can now trade spot and futures on BRL, and more importantly you can deposit it for large size using nearly instant domestic interbank transfers with no fees. As part of this rollout we’re happy to announce our partnerships with the BRZ stablecoin; you can deposit either tokens or fiat and be credited with both on FTX.
Finally, we’ll be giving out a free FTX x HTC phone (complete with a built in cold wallet) to one random person who deposits at least $1,000 of fiat this week!