Introduction
When choosing Foundry model deployment options, start with a model that meets the application's quality and capability requirements, then reject deployment paths that violate production constraints. Microsoft recommends Serverless API whenever possible; the selected model determines the available deployment path. Managed compute is currently public preview, has no service-level agreement (SLA), and is not recommended for production workloads in the verified deployment overview.
Make the decision in this order: model fit, release eligibility, processing geography, access and safety controls, then quota and billing. This prevents an attractive model or serving price from overriding a requirement that makes the deployment unusable. The outcome should be a supported deployment choice with evidence and unresolved conditions recorded, rather than a guess based on the word “serverless” or the availability of a graphics processing unit (GPU).
Architecture Overview
The current overview distinguishes two options. Serverless API deployments belong to a Foundry resource. Managed compute deployments belong to a Foundry project and use dedicated GPU capacity managed by Foundry. This is a Foundry deployment comparison; Azure Machine Learning managed online endpoint procedures are not interchangeable instructions for either option.
Within Serverless API, Microsoft lists Standard, provisioned throughput, batch, and other deployment types. Do not use “Serverless API” as a synonym for only consumption-based Standard serving. Provisioned throughput units (PTUs) and batch remain choices within that family, subject to the selected model's support.
The resource/project distinction matters for the deployment record. Record the actual hosting resource, project context where applicable, model/version, deployment name and selected type. Avoid relying on a portal display label alone when another team must identify what was approved.
Core Components
Model fit and supported serving path
Treat the model catalog as a discovery starting point. Recommend defining a small representative evaluation set before selecting a candidate: normal requests, difficult domain requests, expected structured results, and unacceptable responses. Identify any required modalities or tool interactions, then verify them for the actual candidate. A model family name or general benchmark score is insufficient evidence for an application decision.
Next, establish which serving path that candidate supports. Do not assume that a desirable model can simply be moved between deployment families. If a required feature is absent from its supported path, the decision might be to choose a different model rather than to force an unsupported architecture.
Production eligibility and controls
Apply the managed compute preview restriction before approving a production design. Its current public preview also lacks built-in content filtering in the overview's comparison. Record any required safety capability explicitly; do not infer identical controls from a common client interface.
For an exploratory evaluation, a team might accept a preview after reviewing the terms and limitations. That decision is separate from production approval. Write down which environment is authorized and which restrictions would require revisiting the choice. A successful test response does not establish a service commitment.
Geography, quota and billing
Serverless API offers regional, data-zone and global processing options; current managed compute supports global deployment. Managed compute billing is hourly per accelerator SKU. Its quota is separate from Azure virtual machine quota and granted by accelerator SKU and region. Serverless API billing uses token consumption or PTUs, depending on the selected type.
Keep three questions separate in the review: where the resource is created, where inference may be processed, and which quota or billing unit applies. Verify the actual selected offering against each question. A quota location should not be treated as evidence that processing is confined to that location.
How the Decision Fits Together
Use a sequence of gates rather than a weighted score that lets a low price compensate for a prohibited capability or geography:
| Gate | Evidence to gather | Decision if the requirement is not met |
|---|---|---|
| Model suitability | Representative quality results and required capability support | Reconsider the candidate |
| Release eligibility | Current feature status, service terms and production support | Keep the option exploratory or reject it for the release |
| Processing location | Processing scope of the exact deployment type | Select another supported type or candidate |
| Access and safety | Required authentication, network and safety behavior | Resolve the missing control before approval |
| Capacity and quota | Applicable quota, availability checks and realistic workload observations | Hold deployment approval pending evidence |
| Economics | Current billing unit, projected use and retained-resource costs | Rework the workload or deployment choice |
These are recommended review gates, not additional Foundry product features. Mark a gate as unresolved when evidence is missing; do not silently count uncertainty as a pass.
For access requirements, carry forward the principal and private-connectivity design from AI-300: Securing Foundry Identities and Private Connectivity. Then qualify the chosen deployment against that design. “Private networking supported” is a capability statement, while a successful request from the intended application host is evidence about an implementation.
Design Considerations
Match the serving type to the work
Consider an illustrative customer-support application and a separate overnight document-processing workload. For the interactive application, recommend measuring response time, request shape, error rate and quality under representative demand before choosing its serving type. Evaluate a supported Standard option against that evidence; investigate provisioned throughput when capacity planning warrants it. Do not conclude that PTUs are cheaper merely because traffic is important or predictable.
For the overnight workload, ask whether asynchronous completion meets its business deadline, then verify batch support for the selected model and deployment type. Keep it a separate decision from interactive serving even if both workloads use the same model. Combining their requirements prematurely can obscure which constraint drives cost or capacity.
Suppose a third candidate uses managed compute and performs well in evaluation. If the application requires a production SLA and approved regional processing, the current preview/global conditions prevent treating that quality result as production approval. Record the candidate's strengths, then reconsider the supported model or serving option. This is an illustrative constraint analysis, not an observed deployment or model recommendation.
Review financial units before comparing estimates
Recommend documenting the billing unit beside each estimate and naming the assumptions behind expected utilization. A token-based estimate and an hourly accelerator estimate describe different usage patterns. Include the duration that test resources remain deployed and identify who will remove resources after the evaluation.
Avoid a universal token-to-GPU or token-to-PTU conversion. Use current offering details and measurements for the chosen workload. Detailed PTU sizing and capacity verification belong in AI-300: Sizing Provisioned Throughput and Checking Capacity; this article establishes when that investigation is needed.
Preserve a reviewable selection record
Before handoff, record the candidate model/version, supported deployment family and type, hosting identifier, processing geography, feature status, required controls, applicable quota, billing basis and decision owner. Attach the evidence that resolved each gate. Include a clear reason for rejected candidates so later reviews can distinguish a quality failure from an operational incompatibility.
Repeat the selection review when a requirement or available capability changes. Model migration mechanics and upgrade policy are separate operations; they should not be improvised from this initial-choice framework.
Common Mistakes
- Selecting a model before defining a meaningful application evaluation.
- Treating Standard, provisioned throughput and Serverless API as three unrelated deployment families.
- Approving managed compute for production without carrying its current preview and SLA restriction into the decision.
- Inferring processing location from a resource name, quota location or dedicated-capacity description.
- Assuming quota from one service applies to a different deployment offering.
- Comparing prices without recording whether the unit represents usage, provisioned capacity or elapsed deployment time.
- Copying an Azure Machine Learning endpoint workflow into a Foundry deployment decision.
Certification Relevance
For AI-300 learning, explain how a model can be suitable yet its available deployment path unsuitable for a release. Be able to identify the next evidence needed for geography, production support, safety, quota or economics. The useful outcome is a defensible choice and an explicit reason for rejecting alternatives, rather than a memorized claim that one serving type always wins.
Conclusion
Choose Foundry deployments by applying operational constraints to a supported model path. Keep Serverless API's deployment types distinct, preserve managed compute's current preview restrictions, and verify geography, quota and financial assumptions. A clear selection record makes the choice testable before detailed capacity work or a production rollout begins.