No one has benefitted from the scaling hypothesis quite like NVIDIA. On the back of an AI boom and GPU monopoly, they’ve become the fastest scaling hardware company in history–adding $2T of value in 13 months with SaaS-like margins. 

While the H100 generation likely represents peak pricing power (new B200s have lower margins and higher COGS), an immediate lack of alternatives means they’ll continue to print cash. 

The open question is long-term (>6yrs) durability1. Hyperscalers (Google, Microsoft, Amazon, and Meta) are aggressively consolidating AI demand to become the dominant consumers of AI accelerators; while developing competitive, highly-credible chip efforts. 

Simultaneously, the sheer scale of compute needs has hit limits on capex, power availability, and infrastructure development. This is driving an enormous shift towards distributed, vertically-integrated, and co-optimized systems (chips, racks, networking, cooling, infrastructure software, power) that NVIDIA is ill-prepared to supply.

In this paradigm, NVIDIA can lose with the highest-performing GPUs; the implications will reverberate at every level of the AI stack–from fabs and semiconductors, to infrastructure, clouds, model developers, and the application layer. 

Demand Consolidation

NVIDIAs predicament has been driven by hyperscalers' consolidation of AI workloads and accelerator demand–setting the stage for toothier custom silicon and evolving infrastructure requirements. 

Already, ~50% of NVIDIA’s datacenter demand is from hyperscalers, the other half comes from a large number of startups, enterprises, VCs, and national consortiums. 

That share is set to shrink–the tidal wave of startup spending on GPUs was a transient phenomenon to secure access in a fiercely competitive market. Today, most startups simply don’t have unusual control or infrastructure requirements and are better served by the cloud. 

As such, early purchases were ill-fated; this is borne out by low utilization and abysmal ROI’s for small/short term GPU rentals (usually offered by startups that over-provisioned and are forced to rent out at a loss). This is eerily reminiscent of dot com era startups defending costly server hardware as the world moved to the cloud.

Share

More fundamentally, and contrary to early expectations, the model rollout has aggressively consolidated around a few closed source APIs. Even open source and edge models are now the domain of the hyperscalers. Custom, small-midsize models trained with unique data for specific uses have struggled (e.g. Bloomberg GPT). Scaled frontier models can be cheaper while performing and generalizing better – especially with effective RAG and widely available fine-tuning. Thus, the value proposition for most companies training proprietary models is unclear. Going forward, demand from this long tail of buyers looks shaky; considerably consolidating NVIDIA’s revenue base.

Meanwhile, the smaller independent clouds (Coreweave, Lambda, Crusoe, Runpod etc) have very uncertain futures. NVIDIA propped these businesses up with direct investments and preferential GPU allocations in order to drive fragmentation and reduce their reliance on the hyperscalers. Yet, they face long term headwinds without the product variety, infrastructure, and talent to cross-sell and establish lock-in; forcing them to sell commoditized H100 hours. NVIDIA’s production ramp has eroded scarcity and attractive margins baked into initial assumptions, while undermining the “moat” of favorable allocation. These companies are also extremely leveraged on 3rd party demand, and have been relying on GPU-secured debt + heroic fundraising to expand fast enough to reach competitive economies of scale. It’s an open question if this will work, but things look ugly. The effects of lukewarm third party demand are already visible from the high availability and declining GPU/hour costs at small clouds. 

Price cuts have reduced costs by 40%+ since last year and show no signs of stopping. This is disastrous for the durability and unit economics of independent clouds. Currently, you can rent GPUs for $1.99/hour. At those prices providers are getting <10% ROE; if prices dip below ~$1.65/hour they will be eating losses. This pain may be obfuscated2 by long depreciation schedules (4+ year useful lives for an A100?) but will come to bite eventually. When it does, NVIDIA will have few alternative outlets for demand.

In contrast, external hyperscale cloud demand has grown rapidly. It’s so high that developers (even OpenAI!) are facing chronic shortages with monthlong lead times. This is bolstered by internal demand, with 50-70% of total compute going to frontier training runs and inference for major launches (e.g. Copilot, Gemini, Meta AI). Despite $20b+ each in yearly capex, all the major clouds are at capacity. With their scale and infrastructure experience, the “big three” cloud providers are best positioned to amortize depreciation and downtime costs while offering superior flexibility, security, and reliability. As a result, AI service margins are high and customer trust is strong. 

These are the same structural advantages that drove the original transition to the cloud from on-prem. Hyperscalers will keep pressing and are poised to increase market share for AI workloads. As a result, we’ve already begun to see a major revenue consolidation for NVIDIA: in their Q2 earnings, a single cloud provider accounted for 29% of quarterly data center (a $3.9b purchase or ~130k H100s). 

In light of this, NVIDIA is almost akin to an automotive component supplier; deriving most of their revenue from four customers with deep coffers, talent, and competitive ambitions. They have no way to reverse this trend. Now, they must gamble the business on their most competent competitors' chip efforts.

Competing Silicon

NVIDIA’s dominance for powering parallel AI/ML workloads was a manageable equilibrium when GPU spending was in the hundreds of millions. This is no longer the case; Microsoft added $10B/quarter in incremental AI spend last year. Capex is now >25% of gross profit at some hyperscalers with GPUs making up half of infrastructure spend and ~80% of datacenter TCO. Also, unlike software, inference is core to the marginal cost structure.

As future training runs ($5B+) bump against spending limitations, customization, and capital efficiency will be paramount to maximally scaling compute, training the best models, and staying competitive. Simply ignoring NVIDIA and their margin stacking is no longer an option.

While independent NVIDIA alternatives have struggled, the hyperscalers have a successful track record of substituting and designing out key chip suppliers. Meta designed NVIDIA out (!) for some of their largest DLRM workloads with their ASICs, and Google did the same for key video encoding workloads from YouTube. Amazon replaced hypervisors with Nitro back in 2012 and a huge volume of intel CPUs with Graviton.

Hyperscale customers present extreme risk3 as they focus relentlessly on cost reduction, substitution, and internal chips efforts. In doing so, they (and key design partners like Marvell, Broadcom, Astera, Arista and AIchip) bring to bear massive amounts of capital and engineering talent to actively undermine NVIDIA’s core business. 

Google is a prime example of this. Their legacy workloads (search, ads, translate) made them one of the largest consumers of accelerated computing and ML inference. As such, they’ve been working on AI accelerators (TPUs) in-house since 2013. While development now follows multi-year timelines, v1 only took 15 months from kickoff to deployment – with limited budget, talent, and an architecture from a 1978 paper. Given this, TPU performance is astonishing. They’re cheaper than H100s for inference and competitive for large training runs–Gemini-Ultra, a GPT-4 Class, frontier model was trained exclusively on TPU V4 clusters (these are 2 generations old!). 

Large enterprises have switched as well. Midjourney trained exclusively on TPUs and Anthropic uses v5e for inference. Most recently, Apple chose to train its SOTA 3B parameter local model on a mix of TPU V4 and V5p instead of NVIDIA. 

The pace of improvement isn’t slowing. TPUs are in their 6th generation and investment has accelerated. The newest gen (Trillium) has doubled energy efficiency and HBM capacity while more than tripling peak performance. Google is also iterating on its software stack. They’ve transitioned away from TensorFlow to JAX – which handles compilation and low level deployment optimization. While slower than CUDA on GPUs, JAX is an increasingly viable cross-platform alternative. This is the result of improving support (e.g. libraries like Equinox), better scalability, speedups from compiling into XLA, as well as native support for autodiff and JIT compilation.

Critically, TPUs + JAX have been so effective that they’ve almost entirely replaced NVIDIA GPUs for Google’s internal AI Workloads. This includes all of Deepmind (!), Gemini training + inference, as well as YouTube DLRM. Employees literally can’t remember the last time they used NVIDIA. Nearly all remaining demand is for external GCP clients, but even here TPUs are aggressively ramping to serve more workloads and further constrain NVIDIA spend. 

It’s not just Google, Amazon is ramping up chip development–powered by its 2015 acquisition of Annapurna Labs4. They recently launched second generation Inferentia and new Trainium chips, with spending projected to ramp up to $2.5B by 2026. While Gen 1 ran Alexa’s backend, Gen 2 (with more HBM and bandwidth) is focused on LLM price/performance optimization. It’s been accompanied by the Neuron SDK – a custom compiler that completely replaces CUDA. It provides low level control and is compatible with Torch, Jax, and TensorFlow – developers particularly like its performance analysis tooling. Combined, this has proved a viable NVIDIA substitute for inference and large model training workloads. Anthropic now uses Amazon as their primary compute vendor; moving frontier model training and inference workloads to custom hardware in lieu of NVIDIA. The two are even co-developing future generations of Amazon hardware, informed by Anthropic.

Microsoft is similarly focused on ramping up internal chip development efforts5. 300 employees have drawn on past chip experience (Xbox) and a close partnership with AMD (eventually Marvell by ‘25-’26) to launch their first-gen Maia 100 accelerator and associated Cobalt 100 CPU (an ARM based, Ampere replacement). The Maia accelerator is impressive; in raw FLOPS it’s competitive with H100s, but it’s been constrained by HBM bandwidth (likely on account of their pre-LLM design and optimization for CNNs). 

On the software side, Microsoft launched the Maia SDK – a Pytorch compatible compiler that leans on OpenAI’s Triton framework to replace CUDA6. Triton is both fairly efficient and very abstracted from underlying hardware, making it generalizable and highly legible to an ML engineer. Microsoft is pushing to use Triton to standardize and simplify custom kernel development. They’ve also already rolled out a new data format (MX v1.0) to standardize quantization across providers. This would make custom and 3rd party chips seamlessly interoperable and give devs complete portability; in effect commoditizing NVIDIA. Maia is very early, but has been tested in production; serving inference for Bing Chat (now Copilot) and Github Copilot. Future use cases will include 3rd party inference (OAI), training Microsoft’s GPT4 level internal model (MAI-1), and serving Azure customers. 

Even Meta is stepping into the custom silicon game and serving meaningful production workloads. Early Meta chips were optimized to serve ranking, advertising, and recommendation workloads (DLRM) for Instagram and Reels. This has changed with the launch of their second-gen MTIA chips and a custom built software stack to support it. These V2 chips are designed to serve AI inference applications for Meta AI characters/chat on Instagram and WhatsApp; they have already been deployed to data centers and are serving use cases in production. Meta is intent on expanding the scope of these chips particularly for synthetic data generation. Llama 3.1 relied on this for supervised fine tuning and distillation to pretrain smaller models. In Zuckerberg’s view, immense inference capacity is a priority for future frontier model training. While Meta still purchases NVIDIA, the direction they’re moving is clear. 

The shift to custom silicon is especially telling because it’s happening in a market that should reinforce NVIDIAs moat. Current AI workloads are skewed towards training, even at Microsoft training is ~60% of compute demand. In theory, this should have limited the viability of custom silicon, as training is memory bound and sensitive to footprint constraints. As AI adoption accelerates, productization matures, and the revenue hole is filled7, the balance will shift to inference; increasing pressure on NVIDIA. By 2025, 70% of GPU hours will be spent on inference where custom silicon is especially competitive. Even to date, inference commoditization has been extremely aggressive. In many cases (especially for mid size models) last gen A100s offer better performance/$ than H100s, and AMD has quietly become a more competitive substitute8. In the longer term it’s even possible that CPU-based inference is optimal, particularly for small models.

Ultimately, hyperscalers' successes with developing and deploying custom chips (particularly Google’s) is deeply worrying for NVIDIA. Hyperscalers have a long history of designing out component vendors, in a world of increasing consolidation this may be existential. Many of the most competitive chip efforts were started with different mandates over a decade ago–well before the massive wave of AI demand. While accelerator design cycles are long, it’s an eminently solvable problem that simply requires commitment, capital, and scale. 

The Distributed Scale Problem

When discussing competing chip efforts, some will rebut that on a per-chip basis NVIDIAs upcoming B100 will be much faster (delivering the performance of 3 Trillium TPUs). This may be true, but it illustrates a more important issue; per-chip performance doesn’t matter; NVIDIA can make the highest-performing GPUs on earth and lose. 

Hyperscalers have the ability to vertically integrate while leveraging scale and specialization to make distributed systems that are cheaper and more performant in aggregate. For a platform agnostic supplier like NVIDIA, this is a structurally difficult trend to compete with or adapt to.

This is the result of a broader trend; with the breakdown of dennard scaling and SRAM logic shrinking, returns to transistor miniaturization have decreased and new node costs have been skyrocketing. Hardware gains (density, performance, and cost) increasingly come from system technology co-optimization; the intelligent design of interconnect, chiplets, cooling, power, racks, and data centers. Much of this has been enabled by advanced packaging (e.g. chiplets, increased on-package HBM) and larger substrates. However, even more gains have come from a conscious effort by datacenter operators, suppliers, and system integrators to tailor designs for hyperscale needs. We’ve just begun to see optimization at this layer in light of changing requirements for AI servers and data centers.

Microsoft's data center design is an example of this. They are planning their own telecom fiber network and rolling out new ColorZ pluggable optical transceivers + DSPs to support long distance (<80km) data transmission – effectively increasing peak cluster performance by connecting data centers. To support the huge data flows, Microsoft is planning developments of in package, and eventually on chip optics for switches and transceivers. Current racks with Maia chips have even more vertical integration. They use custom power distribution, high-bandwidth ethernet protocols, and dedicated in-rack “sidekicks” to enable closed loop liquid cooling; increasing chip density while decreasing interconnect requirements and cost. Importantly, these servers integrate into existing Azure datacenter infrastructure, allowing for dynamic power optimization, easier management, and greater compute capacity in existing footprints; a requirement for scaling training runs in a world limited by build outs. 

Google understands this deeply. They have intentionally moved away from big chips to swarms of smaller, cheaper ASICs with deep interconnects. At the rack level, all TPUs have identical configurations – unburdened by the need to support different environments. They also have fewer server boards that are integrated in the rack. This increases shelf density and further reduces the complexity of hardware deployment. Google also uses ICI as opposed to NVLink enabling them to connect slices of 64 TPUs very cheaply over passive copper. 

Unlike Nvidia, however, Google also customizes significantly at and beyond the data center level. TPUs are designed to run in pods of 4096 chips using custom optical switches (Apollo) in lieu of normal electronic packet switches, greatly reducing power-consumption and downtime compared to NVIDIA. This is possible because across the data center, TPUs use a unique torus-shaped network topology, as opposed to Clos; improving latency and locality. Combined with proprietary cloud networking hardware and software (Jupiter), massive clusters of TPUs can be organized across data centers. This is how Google data centers provide performance/TCO that is nearly 30% lower than competitors (primarily on account of reduced chip cost, downtime, and infra spend). 

More importantly, in conjunction with smaller batch sizes, data verification tricks, and advanced partitioning, it means Google can train frontier models like Gemini Pro 1.5 over multiple connected data centers. 

This minimizes power/size requirements for future training (and likely search + synthetic data) infrastructure. It also greatly improves hardware modularity and extends the useful life of older chips/build outs as they can run alongside cutting-edge clusters in the future to train frontier models. Other hyperscalers are going in this direction as well; Meta trained Llama 3 across two data centers and Microsoft is connecting clusters across the country for OpenAI. Infrastructure will become increasingly distributed as labs move away from synchronous training via hierarchical gradient descent. Meta, OpenAI, and Google are all actively researching ways to train separate model copies and merge branches back together–enabling truly asynchronous distributed training. 

This is a massive advantage; distributed infrastructure is the only way we continue scaling and count new OOMs. Right now compute is the biggest limitation for pushing frontier models; Mark Zuckerberg has explicitly said that building out ultra-large data centers is a binding constraint. Multi-datacenter training runs enable the use of smaller, interconnected data centers. This reduces cost and increases throughput, but it also confers substantial benefits for quickly conducting and scaling infrastructure build outs. With smaller data centers, it is far easier to create designs, procure land rights/permitting, EIA approval, suppliers/GCs and build out transmission + voltage transformers. Connected clusters could also help overcome power constraints on a grid that's at capacity. Vanishingly few utilities have enough excess capacity to keep scaling; this is doubly true in prime locations with clean energy (required for hyperscalers given net-zero commitments). The issue is so acute that Microsoft reopened 3-Mile Island (835MW) and pre-purchased fusion power from Helion; AWS purchased a 960MW nuclear plant. Regardless, individual data centers will quickly hit size limits (e.g. a 10GW cluster would be ~1% of installed US power). 

However, connected campuses and networked cross-regional clusters may be able to move away from single endpoints and draw power from multiple utilities and power sources. Interestingly, If you look at a map of hyperscale data-centers and cross reference it with power zones, you’ll see many nearby datacenters are already served by different utilities. Currently we’re limited to in-region data centers using ethernet (<40km), but plans are being made to leverage telecom fiber and connect datacenters as far as ~500km away. Microsoft is already digging and has spent over $10 billion with fiber network providers like Lumen. While this will take a few years to deploy, it will provide 1-5Pbit of bandwidth and <1sec weight exchanges; enough to make multi-region training viable.  

These regionally distributed training systems will keep scaling going and become the norm, enabling cheaper, faster infrastructure build outs while pushing peak power/compute across a single run (x-xxGWs). While distributed systems create some opportunities for nontraditional but well capitalized players, infrastructure expertise matters and it’s imperative for hyperscalers to scale along the frontier to train better models. The key long-term difference is that training will occur on “small” networked campuses of x00MW-xGW datacenters as opposed to monolithic xxGW class data centers. 

Such a world makes it extremely difficult for NVIDIA to compete on a performance or cost basis. Their lunch will be eaten as there’s no room for unspecialized platforms. They can’t deeply integrate with a cloud or customize hardware without designing out subscale customers– which would further consolidate demand, subsidize competitors and commoditize their product. This is the innovator's dilemma that makes NVIDIA ill-suited to serve cross cluster or exascale use cases. The evidence of this is already reflected in surprising weaknesses when building massive, maximally performant systems.

For instance, NVIDIA’s Infiniband networking infrastructure is clearly not meant to support 100K+ GPU clusters. By design, every GPU is only connected to a single NIC, and every packet has to be delivered in the same order. Since models are so large and weights are distributed, a single failed NIC, GPU, or optical transceiver can take down an entire server, forcing data to be retransmitted (this compounds as new racks make nodes larger). Even with frequent checkpointing this drags down MFU by multiple percentage points. Moreover, while Nvidia is driving the transition to 800 Gbps networking (useful for all customers), they’ve been caught flat footed for a massive, hyperscale-led shift in datacom. We are likely to see increasing use of pluggable ZR transceivers to support the ultra-high bandwidth fiber connections needed to connect data centers. As part of this we may also get new DSPs and supporting telecom hardware (Amplifiers, Multiplexers, Transponders etc)–undermining NVIDIAs networking stack while benefiting providers like Coherent, Lumentum, Inphi, Cisco, and Nokia.

Unfortunately for NVIDIA, blind spots extend to the infrastructure software stack. One of the most pressing issues is fault tolerance – key for ensuring reliability and high utilization when dealing with many points of failure in a training run. For subscale customers this isn’t a problem, but with really large cluster sizes NVIDIAs network design and hardware choices provide no built in protection. Despite this, NVIDIA still doesn't have a compelling proprietary library to handle fault tolerant training; subscale clients rely on open source frameworks. The major hyperscalers now rely on more robust internal solutions. For instance, Google uses proprietary software called Pathways; it does a fantastic job of covering more edge cases and types of faults than other solutions (While being able to operate flexibly with synchronous and asynchronous data flows). Pathways is also very good at detecting and fixing nearly invisible GPU memory issues that slip past ECC; NVIDIAs diagnostic tool (DCGM) is considerably less reliable. NVIDIA also struggles to provide competitive partitioning and cluster management software. Their BaseCommand system (built on Kubernetes) is designed to be cross platform and work with heterogeneous systems. However, once again, hyperscalers have better proprietary, integrated solutions like Borg (Google, also the basis of Kubernetes) and Singularity (Microsoft) which better handle scaled VM/container management, transparent migration, and GPU workload scheduling/management. Google even has a custom sharder called MegaScaler for synchronously partitioning workloads across TPU pods within and beyond a single campus. 

Hyperscalers' core advantage is their vertical integration across the semiconductor, infrastructure, and model layers–which enables improved system level understanding, observability, co-optimization and failure analysis. NVIDIA has made some strides with Blackwell and GB200 servers to confront structural issues stemming from this. 

On the hardware level, they installed rack wide copper-connected backplanes and required liquid cooling to improve density, energy efficiency and TCO. They’re also rolling out a new ethernet-based, datacenter-scale networking solution called Spectrum-X (both the absolute and relative success to Broadcom will be worth watching closely). On the software side, they’re aggressively updating their DCGM software and pushing a dedicated engine for reliability, availability and serviceability (RAS). This is meant to help mitigate persistent NIC/Infiniband failures by using sensor level chip data to preemptively alert operators to failures. 

However, all things considered, this isn’t enough. As discussed, better versions of nearly all key infrastructure software exist internally, and on the hardware side NVIDIA is behind the curve; they will struggle to adapt to massive change whereas the hyperscalers have been scaled and vertically integrated for a long time. Cooling is a great example of this. While NVIDIA just mandated liquid, Google did so back in 2018 with TPU v3 deployments. Today, they use 2x less water/kWh than Microsoft’s NVIDIA datacenters and have a PUE (power usage effectiveness) of 1.1, compared to >1.4. This means NVIDIA’s decision to keep small customers happy and drag their feet on integration cost them 30% of effective power and 50% of footprint at every hyperscale data center!

No doubt, NVIDIA is aware of this and they are trying to push up the value chain (DGX Cloud, NGC etc). Yet current efforts betray a narrow scope of vision; primarily optimizing at the data center level when the future is moving towards large campuses and clouds. While Jensen has demonstrated a willingness to play for the future (see the prescient Mellanox acquisition), NVIDIA today faces unprecedented secular challenges that have killed a long line of high-flying predecessors. Uneasy lies the head that wears the crown.

Thanks to: Lachy Groom, Jack Whitaker, Lauren Reeder, Gavin Uberti, Divyahans Gupta, Philip Clark, John Luttig, Tyler Cowen, Shahin Farshchi, Jannik Schilling, Bridget Harris, Harry Elliott, Luke Farritor, Jacob Rintamaki, Joe Semrai, Coen Armstrong, Trevor Chow, and Catherine Wu

Disclaimer: All views are exclusively my own and do not represent those of any of past or present employers. Everything in this article is based on public information. I hold no positions in Nvidia (NVDA). I am long Marvell (MRVL), Google (GOOG), Broadcom (AVGO), Vertiv (VRT), Micron (MU), Fabrinet (FN), Coherent (COHR), Talen Energy (TLN), ACM Research (ACMR), and Credo (CRDO)

[Note: Originally published in the Stanford Review]

In landmark rulings today and earlier this month, Kalshi, a U.S.-regulated prediction market platform, won a nearly year-long court battle against the Commodity Futures Trading Commission (CFTC) to offer commercial election markets. A federal judge ruled in Kalshi’s favor, an astonishing decision that finally allows the company to offer event contracts on the outcomes of presidential elections and control of Congress—the first of their kind in the U.S. in nearly a century. 

In winning this case, more was at stake than any individual company: It’s a landmark and turnaround in the broader struggle for prediction markets to be recognized for their value in forecasting and decision-making, unburdened by regulatory overreach and misunderstanding.

Prediction markets (also known as event markets or contracts) rely on collective intelligence to generate accurate forecasts. Participants buy and sell contracts based on their expectations for future events, where the price of a contract reflects the probability of that outcome. For example, a contract on whether Barack Obama will win an election may be trading YES at 59c, implying a 59% probability of winning. If Barack Obama wins the election, the contract will pay out $1, otherwise it’s worth nothing.

With enough participants, this can form a real-time market with significant informational value as it aggregates the beliefs of participants. The ideal market is general enough to attract broad interest and matter, but unusual enough not to be easily replicable with existing instruments. Most volume on prediction markets has been focused on political events like elections, but prediction markets can forecast any resolvable questions on uncertain future events. Common categories include tech, sports, business, and weather. To date the largest prediction market with the most volume is Polymarket (>$1b since inception), founded in 2020.

Not all prediction markets are public-facing. Companies like Google and Hewlett-Packard have used internal prediction markets to predict the success of projects and products more accurately than conventional methods. These internal markets leverage information and insights from employees, oftentimes breaking down cross-team barriers and bureaucracy, making them powerful tools for improving corporate decision-making.

Oftentimes with surprisingly little liquidity, prediction markets prove highly accurate and robust in the face of interference. The "wisdom of crowds" phenomenon ensures that these markets frequently outperform traditional forecasting tools like polls. For instance, the University of Iowa operates the Iowa Electronic Markets (IEM) as a research project. The IEM has consistently offered better predictions than opinion polls, particularly in political races; out of 964 polls for presidential elections from 1988 to 2004, the IEM was closer to the actual election outcome 74% of the time, oftentimes as far as 100 days before the election. 

The success of prediction markets stems from their incentive structure—participants are motivated to gather and act on the best available information. Moreover, markets by their very nature aggregate diverse information from participants who have various pieces of knowledge and insights. As a result, market prices adjust in real-time as new data emerges, providing a dynamic, crowd-sourced probability of future events. 

Most prediction markets (like Kalshi) are real money markets, but some notable markets are “play-money”, using in-app/website tokens with no real value. Even with weak incentives and relatively small userbases, markets like Metaculus and Manifold can be quite accurate. This information value is significant enough that people check prediction markets for real-time updates on events; some (e.g. The Base Rate Times, Polymarket, Kalshi) are building out media coverage and newsletters based on this data.

It’s worth noting prediction markets provide value beyond just forecasting—they also enhance market efficiency by allowing traders to hedge against major real-world events, such as wars, natural disasters, or political shifts. For example, companies with exposure to geopolitical risks could use prediction markets to hedge against the outcomes of international conflicts or sanctions by betting directly on questions based on those events.

For instance, companies with exposure to geopolitical risks could use prediction markets to mitigate the impact of an international conflict or sanction by placing bets on the outcomes of such events using a prediction market. Investors manage such risk today by constructing baskets of traditional financial instruments. This process is indirect and imprecise, it’s also difficult, expensive and bespoke—greatly limiting access. Prediction markets make it easy to isolate specific risks and are a valuable complement to conventional hedging strategies, helping to contain costs and filling a vital need in the status quo.

Despite their utility, prediction markets have faced extremely harsh treatment from the CFTC. To date, enforcement actions have been highly punitive and arbitrary; limiting reach, chilling innovation, and making new ventures difficult to back. The CFTC hasn’t given clear guidance, and current approaches ignore the fact that prediction markets have a substantial function as tools for information aggregation, not just speculation.

PredictIt is a prime example of CFTC overreach. Founded in 2014, it was run for nearly a decade as a nonprofit academic project that allowed participants to trade on political outcomes. It operated on a 2014 no-action letter granted by the CFTC, allowing it to operate with substantial constraints. In doing so it successfully served more than 175,000 traders and 29,000 contracts, while giving citizens new ways to express their views and providing a real-time research laboratory for political scientists. Then without warning in August 2022, the commission revoked its no-action letter and ordered a shutdown in 6 months. 

No specific reasons were given, and there was no process for appeal. The CFTC’s decision was arbitrary without clear evidence of harm. It deprived academics and political forecasters of a valuable tool for real-time analysis. Fortunately, a group of PredictIt traders and providers successfully won an injunction (Clarke v. CFTC) in 2023 after a series of appeals, but the level of arrogance displayed is astonishing. The commission said it owed no explanation and wasn’t subject to review of any kind, including by the courts. While PredictIt can now keep operating as a small, tightly regulated prediction market, the enforcement action created a chilling effect. 

Polymarket was also the victim of similar enforcement. They use the Polygon blockchain to operate decentralized, global, and real-time event markets that are fee-free. (This innovation has become one of the most compelling use cases for consumer crypto). With little warning, the CFTC classified Polymarket as a swap execution facility, retroactively fining them $1.4M and banning them in the US, nearly killing the company.

Kalshi’s enforcement actions similarly demonstrate the arbitrary and punitive approach to regulation the commission has taken. Despite proactive engagement on Kalshi’s part to seek clarity and achieve compliance, the commision voted 3-2 in an attempt to ban ALL election event trading; under a provision giving them the authority to regulate gaming. This included sports, yet the ban would leave sports betting apps like FanDuel and DraftKings in place (no doubt thanks to their extensive lobbying efforts). 

Comparisons to gaming and gambling are fundamentally flawed. Gambling is based purely on chance, such as betting on roulette. Prediction markets, by contrast, are driven by informed decision-making. Far from being speculative games of chance, prediction markets reward skill and research, with participants incentivized to gather information that's reflective of real-world data and events.

Another key argument the CFTC raises against prediction markets—particularly those involving elections—is that they could undermine the integrity of democratic processes. The CFTC has expressed concerns that large financial incentives might influence voter behavior or even the outcomes themselves. However, these concerns are unfounded. Liquid election betting markets, such as Polymarket (officially banned in the US thanks to the CFTC) have proven highly resistant to manipulation; that’s why they are so accurate. 

One remarkable example of this robustness is when a coordinated group of traders spent upwards of $2.5M attempting to drive up the odds of a Kamala Harris win in Polymarket's main 2024 election market. They failed spectacularly and were unable to change odds for more than a few hours. 

Legal event contracts in the US require bans on manipulation and insider trading to protect users. Platforms have strong incentives to do this regardless, but bringing grey-markets into the fold can improve the user experience by mandating standards that improve consistency and integrity. Other countries, like the UK, have successfully operated prediction markets for decades without undermining election sanctity. On the contrary, such markets have increased trust by providing clear, accurate, and reliable sources of truth.

Ultimately, prediction markets have the potential to become indispensable tools, not just for traders but for society as a whole. As new platforms continue to innovate, policymakers must reevaluate the regulatory framework to foster innovation and drive adoption. Congress and the courts should either formally carve out exemptions for the space (as was done with sports betting in 2018), or rein in the CFTC.

Kalshi’s recent victory in the space is resounding, the regulatory tides have finally begun to shift for prediction markets. With a legal green light, event contracts can thrive free from overreach, while offering powerful tools for reshaping the future of decision-making.

[Note: Originally published in the Stanford Review]

In the sitcom that is Silicon Valley, the debate surrounding artificial intelligence has reached a fever pitch, and nothing seems to divide people more strongly than the future of open-source AI. Depending on where in the valley you work, open-source is either a futile endeavor or God's greatest gift to mankind. The truth likely lies somewhere in the middle. 

The argument for open source’s futility is based on the scaling hypothesis, the idea that better AI models will be larger and necessitate more data, compute, and ultimately, cost. For reference, the amount of compute used in frontier models has been increasing ~4x a year, and GPT-4 cost over $100 million to train. Major AI lab CEOs are predicting $1 billion and even $10 billion training runs. Only a select number of closed-source model providers (Google, Microsoft, etc.) will be able to underwrite these ever-increasing costs. Additionally, many closed-sourced providers can train models more efficiently because of their differentiated negotiating power, access to compute, and scaled cloud infrastructure. While costs are rapidly decreasing, the rate should slow if fewer organizations can effectively commoditize the frontier by open-sourcing. 

It’s also possible that only closed source will be able to forge the data partnerships needed to train at the cutting edge in the first place—the data needed to create high-quality AI can’t simply (or legally!) be scraped from the web, it must be bought from third parties (e.g., Reddit data). It’s unclear if open-source consortiums can muster the coordination to compete here. 

The open-source providers that can train at the frontier (Meta) would ultimately be forced to “close” or risk hemorrhaging their balance sheets as the commodification of their models makes them unable to amortize the spend for their training runs. Closed frontier models will have more pricing power, and the cost of lagging edge models will decrease primarily with the cost of compute/inference. 

Ultimately, there is a significant chance that cutting-edge foundation models end up closed and exclusively in the hands of a few well-resourced enterprises. A large chunk of value, and even more of the capture, will come from closed-source frontier models that continuously eat their way up the value chain.

This sounds like a damning indictment of open-source—it needn’t be. Larger and more costly models necessitate higher variable costs (inference) passed along to customers. If you spend $10 billion on a training run, it must be paid for. Current frontier models demonstrate this reality, GPT-4 costs 20x as much per token compared to GPT 3.5. Moreover, the additional performance doesn’t matter for many use cases, and developers will opt for the cheaper model. 

As model performance improves and costs rise, this tradeoff will become increasingly acute and cost sensitivity will increase. Microsoft’s Copilot product, for example, already uses a mix of GPT-4 and smaller, cheaper Phi models. 

Multiple models are the direction we are headed towards—Satya Nadella, CEO of Microsoft, has commented on his excitement for cheaper, custom “models-as-a-service.” Ultimately, the frontier will be expensive and not uniquely useful for many high-volume, narrowly defined applications—this is where open source can compete. 

The massive amount of developer talent and optimization in open-source will enable the creation of highly specialized and inference-optimized models. For any niche that doesn’t require maximally performant models, the open-source community will be able to create fine-tuned models that push the performance/cost frontier for a given task. Moreover, the ability of the open source community to cover a wide variety of use cases, get close to the end user, and iterate fast to develop optimal models is unmatched—major providers simply can’t cover and iterate with the necessary pace on all fronts.

This kind of dynamic already exists in another major technology: semiconductors. Currently, expensive, cutting-edge chips (the two-nanometer ones that go in iPhones) are dominated by Taiwan Semiconductor Manufacturing Corporation (TSMC). Historically, as research costs and intensity in the industry increased, the number of cutting-edge fabrication plants decreased. Just ten years ago, we had three such manufacturers. Now, TSMC stands alone. This is logical: new plants cost billions of dollars, and machines must run 24/7/365 to make up costs. The cutting-edge fabs have captured the most value/profit; TSMC is a half-trillion-dollar company. That said, while TSMC has a sizable revenue share, it represents only a small percentage of total semiconductor volume. 

Most chips, around 95%, are cheap and serve legacy needs like those for cars and industrial equipment. These chips are built on mature, deprecated nodes (18 nanometers and larger) and are all highly specialized. Texas Instruments alone has over 50,000 different semiconductor products. Despite the variety, lagging edge chips remain cheap, widely manufactured commodities. The parallels to AI are striking. Already, Azure provides access to 1700 distinct models—the vast majority of which are not at the frontier. High-performance, low-volume products capture the most value, attention, and investment. Yet, the long tail of cheap, lagging edge and specialized products will ultimately account for much of the volume and core functionality.

Interestingly enough, US dependence on these mature Chinese chips appears to be an emerging national security issue in its own right. Similar concerns show up in the open-source debate. Many argue against open source, citing national security fears and US-China competition. There is merit to this—better models are useful in training better models. That said, for now, the quality of models coming out of strategic competitors like China is competitive with US open source. If performance gaps were to widen, the conversation would change. But, at the present moment, bans seem premature. This appears especially true given the scale of existing IP theft. 

American adversaries can’t stop state actors from stealing model weights—they may already have access. Banning open source on national security grounds would do little to slow our rivals but would kneecap a nascent (and valuable) domestic industry. Let’s not make that mistake.