As long as consumption cannot be traced back to its source, Generative AI usage remains an expense without a basis for control. The path to robust control involves a central control point that makes consumption and costs visible across providers. This allows budgets to be broken down granularly by users, teams, models, says Tony van den Berge at Cloudflare.
Generative AI has rapidly evolved from individual pilot projects into a widely used infrastructure across development, sales, marketing and support. As the number of use cases grows, so does the consumption of model accesses – and with it, the cost burden.
In many companies, the management of these expenses is lagging behind the pace of adoption. This is due less to a lack of budget discipline than to the way in which access to language models is organised.
Teams often access providers’ models via a shared API key, with billing aggregated at account level and in tokens. From this perspective, it is impossible to determine which user, team or automated process is generating the cost. Effective cost control therefore first requires a reliable mapping of consumption to its source.
The financial implications of this gap become clear when compared with other types of expenditure.
Personnel, cloud and licence costs are budgeted and allocated to individual cost centres, allowing their value to be assessed and their growth to be limited. In many organisations, this foundation is completely missing when it comes to AI consumption.
Without an overview of actual AI expenditure, the value contribution of an AI investment cannot be calculated, and without allocation to teams and processes, expenditure cannot be specifically limited.
Added to this is a typical behavioural pattern: in the absence of a budget, transparency and a rationale for model selection, users generally opt for the most powerful model available, even where a simpler and cheaper option would deliver the same result.
A potentially obvious safeguard falls short when it comes to AI expenditure: whilst a rate limit caps the number of requests within a time window and is suitable for mitigating individual misdirected processes or peak loads, the number of requests is only a rough approximation of the resulting costs, as models and context lengths differ in price by orders of magnitude.
A single request to a top-tier model with a long context can prove more expensive than hundreds of requests to a compact model. Billing in tokens is of little help either, as tokens are not a manageable control variable for budget managers.
What is therefore required is a control mechanism that records the cumulative expenditure in monetary terms and operates independently of the sheer volume of requests.
Role of gateways
A viable solution starts at the point where the requests leave the company. If access to the model providers is routed via a central intermediary layer, a so-called AI gateway, all requests pass through a defined control point before reaching the respective provider.
At this point, consumption and costs can be recorded, logged and controlled across all providers. This creates the conditions for defining budgets not in abstract tokens, but in monetary terms.
Such budget control must possess several characteristics to be robust in a corporate environment. It is advisable to set limits with fine gradations, by model, by provider and by freely definable dimensions such as user, team or application. Billing periods should be configurable on a daily, weekly and monthly basis, with the option of a fixed or rolling reset.
Cumulative consumption is calculated per request based on model prices and compared in real time against the stored limit. If a budget reaches its limit, two responses are advisable: blocking further requests as a hard cap, or automatically redirecting to a more cost-effective model so that the workflow is not interrupted.
The values recorded in this way are based on the number of tokens and the model price, and provide a reliable estimate of the cost; the final billing remains the responsibility of the respective provider.
Integrating gateways to identities
Granular budgets only realise their full potential when usage can be reliably attributed to an identity. Budgets based on attributes from the calling application are only as trustworthy as the reporting application itself. For verified and automatic allocation, the identity of the requester must be established at the checkpoint level.
This is achieved by integrating the gateway with the company’s existing identity management system. When a user authenticates via the company-wide identity provider, their identity can be read from the issued token and appended to every request as secure metadata. On this basis, consumption per person, the breakdown by teams, and cost allocation across the entire organisation become visible in one place.
Budgets and policies can then be linked to the identity provider’s maintained groups, for example by granting a team access to high-performance models whilst other departments are directed to more compact models.
Cloudflare, for instance, maps this verified allocation via the connection between its AI Gateway and the Cloudflare Access service, which retrieves the identity from the authentication token and assigns it to each request.
This approach takes on particular significance with automated actors. CI/CD pipelines and autonomous agents account for a growing share of AI consumption without a human user behind them. If each of these services is assigned a named identity via its own service account, its consumption can be identified individually and, if necessary, specifically limited without affecting other processes.
Optimising value
However, whilst a set budget limits expenditure, it does not yet fully exploit the potential; the next step in development therefore lies in the task-appropriate allocation of models.
A summarisation or a simple classification step runs on a smaller and cheaper model without any significant loss of quality, whilst a complex refactoring or analysis task justifies the use of the most powerful model available.
Intelligent routing analyses the request and automatically assigns it to the model that delivers the best result at the lowest cost. This shifts the focus from merely limiting expenditure to optimising it.
With user-specific consumption tracking, the issue moves beyond a purely financial framework and becomes a matter of governance. An analysis that attributes AI consumption to individual persons raises data protection concerns and, in Germany, issues of employee participation. Data protection officers and the works council should therefore be involved at an early stage before any personalised analysis is rolled out into production.
A phased approach is also advisable. An initially high limit set in observation mode reveals actual usage patterns before binding limits take effect. For security managers, another factor is crucial: blanket bans on individual AI services merely shift their use to uncontrolled channels.
A phased control system based on visibility and guidelines is more effective than a blanket ban and integrates cost control into the overarching safeguards for AI usage.
The proliferation of Generative AI has brought the question of cost-effectiveness to the fore.
As long as consumption cannot be traced back to its source, AI usage remains an expense without a basis for control. The path to robust control involves a central control point that makes consumption and costs visible across providers, budgets in monetary terms that can be broken down granularly by users, teams and models, and verified attribution based on corporate identity.
With a model selection tailored to the task at hand, this evolves into a control mechanism that both limits and optimises expenditure. In this way, AI consumption becomes a regularly budgeted and traceable item, just as is the case for any other resource within the company.
