Building systems that use AI for a local market is very different than building systems that should scale globally. For AI embedded in a system, global availability matters as much as for a website. And just like websites, the same AI can be fast in one place and painfully slow in another.
Hi, my name is Tom Smykowski. I'm a Staff Full-Stack Engineer who has scaled SaaS platforms to millions of users, and I write about AI-assisted development and production engineering. On this blog I share practical insights on building systems that work at scale.
For websites we use CDNs and other techniques to serve content from the closest point to the user as possible. That's why data centers in different regions make so much sense.
So when you want to deliver a good experience to the user whether he's from Mumbai, Frankfurt or San Jose, you have to make sure the AI service you use is fast and available in that region.
To illustrate what is the problem I've put up a page that shows what is the speed of different models from various places in the world. For example GPT-5.6 Luna in my tests was quite fast from Montreal, Stockholm, or Mumbai, but slower from Frankfurt and was painfully slow from Seoul and Singapore.
To measure speed of the AI we can use multiple metrics: Time To First Token for streaming, Time To First Byte for non-streaming requests, network latency, and inference speed. Multiple things affect the latency, and for the user, it doesn't matter what causes it, because these times affect their experience of the system.