Startup and VC Ecosystem Updates | Issue# 15 [July 5, 2026]
Last updated on July 6, 2026
What?
In February 2026, at the India AI Impact Summit in New Delhi, US private equity giant Blackstone committed to lead a $1.2 Bn financing into a three-year-old Mumbai startup called Neysa, to deploy over 20,000 GPUs across India. Days earlier, half a world away, OpenAI had unveiled its first custom inference chip – Jalapeño, co-designed with Broadcom, built for one purpose – running models cheaply at scale. Two events, two continents, one underlying shift: the centre of gravity in AI has moved from training Large Language Models (LLMs) to running them, and the entire economics of compute is being redrawn around that fact.
For India, this shift is the whole story. It decides which parts of the AI stack the country can realistically win, which it cannot, and where VCs should actually place their bets. Indian AI startups raised $676 Mn in H1 2026, over 4x the year-ago figure — yet that entire half-year haul is dwarfed by a single global round, OpenAI’s $112 Bn and Anthropic’s $65 Bn in the same window. At that scale, competing at the frontier-model layer is arithmetically out of reach – which is precisely why the rational bet lies elsewhere in the stack.
The reasons are aplenty and stack up quickly. A frontier model costs billions to train, and needs exactly the cutting-edge chips that export controls keep out of India’s reach. Worse, the rewards go almost entirely to whoever gets to scale first — and that race is already crowded with players holding deeper pockets, more data, and a multi-year head start. Building a frontier model is a bet of enormous, patient capital on a prize India would reach late and out-gunned. The software and services layers, by contrast, turn a profit far sooner, on a fraction of the money.
Our thesis, stated upfront: India may not be likely to win either frontier race – neither the hardware (NVIDIA) nor the frontier models (OpenAI, Anthropic) – and it does not need to. The durable value – and the venture returns – sit in the layers between and above: the software that serves and governs AI, and the asset-light silicon design that feeds it – not in out-building NVIDIA or out-training OpenAI. Let us work through why, level by level, comparing where the world is against where India stands.
The Global Picture: Compute is now an Inference-and-Energy Problem
For two years, the AI story was about training the LLMs – who could assemble the most GPUs to build the biggest model. That era seems to be closing. Inference – the act of running a trained model to serve an answer – now consumes roughly two-thirds of all AI compute, and the AI industry is re-architecting around serving rather than experimenting.
Three consequences follow. First, the bottleneck has become supply, not demand. NVIDIA CEO Jensen Huang has pointed to around $500 Bn in Blackwell and Rubin revenue visibility through end-2026, with the constraint being how fast chips, memory, and packaging can be produced – not whether anyone wants them. Memory alone is on track to make up around 30% of hyperscaler (giant cloud providers, such as Amazon, Microsoft, and Google) AI infrastructure spend in 2026, up from roughly 8% three years ago. Per a Jefferies report, 8.9 GW of global data-centre capacity came online in 2025 against ~21.1 GW of demand — a ~12 GW shortfall — with the crunch expected to deepen as hyperscalers pour $770 Bn into the sector in 2026.
Second, power has become the wall. A single server rack now draws more than 120 kW – enough that air-cooling no longer works, forcing a wholesale move to liquid cooling and purpose-built, gigawatt-scale “AI factories.” Hyperscalers are signing nuclear and dedicated-power deals simply to keep the lights on. Compute is now as much an energy problem as a silicon one.
Third, and most strategically, custom silicon is quietly eroding NVIDIA’s grip on inference. NVIDIA still holds over 90% of the accelerator market and dominates training. But custom ASICs – Google’s TPU, Amazon’s Trainium, Microsoft’s Maia, Meta’s MTIA – are, by TrendForce estimates, growing at 44.6% a year against 16.1% for merchant GPUs. The logic is simple: when a hyperscaler designs its own inference chip, it stops paying NVIDIA’s margin and tunes the silicon precisely to its workload. Owning the chip means owning the economics. Amazon claims up to 50% inference savings versus NVIDIA GPUs on Trainium. OpenAI’s Broadcom-built Jalapeño chip is the newest entrant in exactly this game.
Two caveats keep this honest. Most of these custom chips are captive – one cannot rent a TPU or a Trainium unless you are inside Google or Amazon; for everyone else, NVIDIA remains the only real option. And NVIDIA’s true moat is not the chip but CUDA – the two-decade software layer that every AI framework runs on. Switching chips means rewriting that very layer. Precisely, the AI infrastructure battle is shifting from raw performance to performance-per-watt and software lock-in.
Where India Stands on Compute: Operator, not Designer
Against that backdrop, India has made a clear and, in our view, correct choice: it is building the capacity to operate AI compute, not to design the chips. The IndiaAI Mission, a ₹10,371.92 Cr programme, has assembled a shared national GPU pool that crossed 34,000 GPUs in mid-2026 and targets 100,000 by year-end, offered at ₹115–150 per GPU-hour – roughly 42% below market, with eligible projects getting a further discount. On top of this sit private neo-clouds: Neysa, now majority-owned by Blackstone and deploying ~30% of India’s high-end compute; Yotta, Jio, E2E Networks, and others. Blackstone’s own estimate is that India has fewer than 60,000 GPUs deployed today, scaling potentially thirtyfold to over two million in the coming years.
What India is not doing is designing frontier accelerators (the specialised chips — GPUs and ASICs — that run AI workloads). It imports essentially all of them. And here the constraint is not just capability but access: US export controls cap high-performance GPU supply to Indian firms, while US hyperscalers face no such limit, and Indian data centres were historically never built for 100kW-plus AI racks (cabinets housing stacked GPU servers in a data centre). This is precisely the structural gap we flagged in our earlier piece on the DeepSeek moment – and it has not closed. The dependence is now openly framed as a strategic vulnerability — IESA’s president argues GPU allocation is a sovereign-security issue and India must manufacture its own compute, as export controls create a tiered market where hyperscalers and sovereign programmes are served first and smaller enterprises wait in a queue.
But the same DeepSeek episode pointed to India’s escape route. Frugal training proved that frontier-grade capability does not strictly require unlimited GPUs – clever architecture, such as Mixture-of-Experts (MoE), can substitute for brute compute. This is why India’s realistic lane is inference, and why it is a genuine advantage rather than a consolation prize. Inference runs on lower-end and older GPUs, runs continuously at steady utilization, and rewards exactly what India has in abundance: cheap power, cheap land, cheap operations (the cost of running and maintaining data centres — power, cooling, staffing), and a data-residency mandate that forces regulated workloads to stay onshore. Training needs the newest, most export-restricted chips and frontier research talent – where India is blocked. Inference needs cheap, reliable, sovereign operations – where India competes.
The Missing Layer: Serving Software
There is, however, a gap in India’s stack that is worth naming plainly, because it is also the clearest opportunity. India has built the GPU-cloud layer, but that layer is infrastructure-led. What it has not produced is the independent serving and inference-optimization software layer – the companies that sit between the model and the metal and make inference cheaper and faster through quantization, batching, KV cache management, smart routing, and scheduling. Globally this is a thriving category: Baseten, BentoML, Fireworks, Together AI, Modal, Runpod. In India, the equivalent barely exists as a standalone product. Neysa, Yotta and Krutrim all offer serving features bundled into their clouds, but the pure-play software company – the one that makes other people’s compute run better – is missing.
This matters because that software layer is where the margin lives. Renting GPUs is a commodity business competed down to razor-thin margins; the software that squeezes meaningfully more (say, 30%) throughput out of the same hardware is defensible and asset-light. It is, in short, the most venture-attractive slice of the compute stack, and it is sitting open. This is no longer theoretical. With new-generation GPU lead times stretching to 36–52 weeks and some orders slipping into 2027, Indian AI companies are already turning software optimization into their competitive edge — CoRover routes only the heaviest workloads to LLMs, handling nearly 80% of tasks without GPU-heavy inference, while Murf and Nurix schedule training and segment inference to squeeze more from small fleets. In fact, the market already bears testimony to the same: H1 2026 funding data shows thin application wrappers struggling to raise while capital concentrates in infrastructure, sovereign compute, and vertical AI — see more on the numbers here.
Enterprise AI: The Gap is no Longer the Model
If compute is where India operates, the enterprise is where India deploys – and here the global story has flipped in a way that plays directly to India’s strengths. The defining enterprise-AI statistic of 2026 is not the most promising: roughly 80% of enterprise applications now embed an AI agent, yet only about 31% of organizations have one genuinely in production. The most-cited figure, from Forrester and Anaconda research, is that 88% of agent pilots never reach production. MIT’s NANDA initiative put it even more starkly: 95% of enterprise GenAI pilots deliver no measurable P&L impact. The gap is now a market in itself.
The scramble to close it the gap already on. Microsoft just launched Frontier Company, a $2.5 Bn business embedding 6,000 engineers inside customers purely to move AI from pilot to production, with AWS committing a further $1 Bn to the same problem. The signal is unambiguous: the pilot-to-production gap is real enough that hyperscalers are now spending billions to close it — which is precisely the layer India should be building.
The crucial insight is why. The failure is not the model. Models improve every year while the production-conversion rate barely moves – proof that the bottleneck sits outside the model. The blockers are operational: data quality, integration with real enterprise systems, change management, unclear ownership. And above all, evaluation – the single largest blocker, cited by 64% of leaders. Evaluation refers to the systematic testing of whether an AI system’s outputs are correct, safe, and reliable enough for production. The problem is rarely “the model is wrong”; it is “we cannot tell in advance when it will be wrong, and our tests don’t catch it.” This is precisely why the 12% who succeed share a consistent profile: vertical rather than general-purpose agents, a named “agent owner” with budget and a measurable target, and automated evaluation run on every change. The ROI, where it lands, is real – a median payback of around five months, with banking and insurance leading adoption.
India’s position here is unusually strong on one side and unusually thin on the other. On the strong side, India is the deployment and services engine of global enterprise AI: its GCCs and IT-services majors – TCS, Infosys, Wipro, now paired with OpenAI and Anthropic – are where a vast share of the world’s enterprise AI is actually integrated. The clearest budget magnet is application-based agentic AI — voice and workflow agents for regulated industries — where Indian players like Uniphore, Gnani, Yellow.ai and Observe.ai already have a foothold. The thin side is the same shape as the compute gap: India under-builds the trust, evaluation and governance layer – the very thing blocking that 88%. There is no Indian Langfuse or Braintrust at scale. For India’s VCs, this is the opening, not the shortfall: the evaluation and governance layer for regulated enterprises is an under-contested, software-margin category waiting to be built.
That gap is finally beginning to fill. In June 2026, Pramaana Labs – founded by IIT-Madras alumni – raised a $27 Mn seed led by Khosla Ventures to build a formal-verification layer that wraps a model with a deterministic proof engine and either returns a machine-checkable proof or refuses to answer, aimed precisely at high-stakes domains like tax, law and healthcare. It is the hardest, deepest version of the eval thesis – “probably right” to “provably right.” This implies that the broader monitoring and observability layer, the everyday eval tooling for regulated enterprises, and agent infrastructure remain wide open.
The Demand Side: The World’s Second-Largest Consumer, Sparse on Builders
One mismatch underlines everything above. India is the second-largest consumption market for AI on earth – among the top user bases for OpenAI and the global labs, and the reason Sarvam’s consumer reach runs to over 2 Mn interactions and 10 Mn API inference calls on a daily basis. Yet it produces relatively few builders of the underlying stack – the inference-optimization and serving software, and the evaluation layer. That gap between how much AI India uses and how little of the machinery it makes is not a footnote. It is the argument. Every consumer app running on someone else’s model and someone else’s inference layer is a reminder of where the value is quietly accruing – and where India has room to build.
Sarvam itself shows the constructive version of this. In June 2026 it became India’s newest AI unicorn with a $234 Mn round at a $1.5 Bn valuation, led strategically by HCLTech. Notably, its revenue comes overwhelmingly from conversational agents and enterprise deployment, not from the model itself – and Krutrim, which once chased frontier ambitions, has pivoted to cloud and profitability. The lesson repeats at every level: in India, infrastructure and applications monetise; frontier for its own sake does not.
Thoughts
Step back and the picture is coherent. Globally, value in AI is migrating in two directions at once – from training toward inference, and from raw FLOPs toward power and software. India cannot reverse the first shift — nor should it; its move is to ride it. It does not make the chips, and under current export controls, cannot. However, it is unusually well-positioned on the second. Its advantages – cheap sovereign compute, a captive regulated-data market, the world’s deepest enterprise-deployment workforce, and a frugal-engineering instinct – all point to the same place. That place is definitely not the silicon, but the layers on top of it.
This is why we keep returning to one line: India should not try to out-build NVIDIA on frontier accelerators – that is sovereign-capital, decade-horizon work, best left to the Reliances, Adanis and Tatas and the government, not to venture funds. The capital flows are indicative: investors are backing AI infrastructure, sovereign compute, Indian-language models and vertical AI, while thin application wrappers on someone else’s model struggle to raise — two-thirds of investors say the IndiaAI Mission directly shaped their thesis. Where venture capital actually wins is narrower and sharper: the serving and inference-optimization software that makes compute cheaper; the evaluation and governance layer that unblocks enterprise deployment; and, on the hardware side, asset-light fabless chip design rather than fabs. These are the slices where Indian IP can reach a global exit within a fund’s lifetime.
The uncomfortable corollary is that India’s biggest AI opportunity is also its least glamorous. It is not a sovereign frontier model or an Indian NVIDIA. It is the unbuilt software sitting between the model and the metal, and between the demo and production. The country that runs the bulk of the world’s AI deployment has not yet built the tools that make that deployment work. That gap is not a weakness to apologize for or regret – it is the single clearest place for the next generation of Indian AI companies to build, and for Indian VCs to back them.
India’s AI funding quadrupled in a single half-year, yet still raises the harder question — is it enough to compete globally? That question runs well beyond AI. For the wider deep-tech map – semiconductors, quantum, EV cells, robotics, space – of which AI is one piece, watch out for our upcoming companion State of Indian Deep-Tech 2026.
Thank you for reading through! I genuinely hope you found the content useful. Feel free to reach out to us at ankanatwork@gmail.com and share your feedback and thoughts to help us make it better for you next time.
Acronyms used in the blog that have not been defined earlier: (a) Venture Capital (VC), (b) Artificial Intelligence (AI), (c) United States (US), (d) Billion (Bn), (e) Graphics Processing Unit (GPU), (f) Chief Executive Officer (CEO), (g) Gigawatt (GW), (h) kilowatt (kW), (i) Application-Specific Integrated Circuit (ASIC), (j) Tensor Processing Unit (TPU), (k) Meta Training and Inference Accelerator (MTIA), (l) Compute Unified Device Architecture (CUDA), (m) Crore (Cr), (n) India Electronics and Semiconductor Association (IESA), (o) Key-Value (KV), (p) Massachusetts Institute of Technology (MIT), (q) Networked AI Agents in Decentralized Architecture (NANDA), (r) Generative AI (GenAI), (s) Profit and Loss (P&L), (t) Amazon Web Services (AWS), (u) Global Capability Centers (GCCs), (v) Information Technology (IT), (w) Tata Consultancy Services (TCS), (x) Indian Institute of Technology Madras (IIT Madras), (y) Million (Mn), (z) HCL Technologies Limited (HCLTech), (aa) Floating-Point Operations (FLOPs), and (bb) Intellectual Property (IP).