Deployment options and configuration parameters

Exam objective (Domain 1, 40–45%): Identify AI model components and configurations — identify appropriate model deployment options and configuration parameters. Related: choosing a model (u04), deploying and using the playground (u12), the chat client in code (u13), client data governance (u21), quotas and cost (u23).

1. The pain: the pilot goes live on a Monday

Priya's Service Desk assistant runs on a gpt-5-mini deployment she created with default settings. Then the pilot widens:

• Monday 09:00, the timesheet deadline. 600 staff ask questions at once. Half get HTTP 429 — rate limit exceeded.
• Risk & Legal read the deployment details: type: Global Standard. "So a German client's name in a prompt can be processed in a US datacenter?"
• Finance wants 40,000 archived invoices summarised by month-end. Nobody needs the answers within seconds — but the pilot deployment would bill them at full price and compete with live users for quota.
• Users: answers are sometimes rambling, sometimes different each time.

All four are the same topic: how a model is deployed and how each request is configured. The model was the right one. The deployment and the parameters were not.

2. What a deployment is

A deployment makes a catalog model callable at an endpoint under a deployment name. When you deploy, you decide:

1. Model and version — pin a version or let it auto-upgrade.
2. Deployment type — where requests are processed and how you pay (section 3).
3. Rate limit / capacity — tokens per minute (TPM) for pay-per-token types, or provisioned throughput units (PTUs) for provisioned types (section 5).
4. Safety settings such as the content filter (u06).

The portal's Deploy button with default settings usually chooses Global Standard. That is a reasonable start — and a decision you should make on purpose for client data.

Serverless API vs managed compute

As of September 2026, Microsoft's deployment overview names two deployment options:

Older material and some portal screens use other names (for example "standard deployment" for the serverless option). For the exam: pay per token, Microsoft-managed vs you pay for GPU hours. Microsoft also offers a preview "instant access" mode that calls some models by name without creating a deployment — useful for trying a model, not a production pattern.

Exam trap — managed compute is not "cheaper by default". GPU hours bill while the deployment is running, whether or not anyone calls it. For spiky or low traffic, per-token serverless is usually cheaper.

3. Deployment types: where data is processed, how you pay

Every deployment type answers two questions: where may prompts and responses be processed, and how do you pay. Microsoft's data-privacy documentation adds the constant: data stored at rest (for example uploaded files) stays in the customer-designated geography whatever the type.

Three families to remember:

• Standard (pay-as-you-go): Global → Data Zone → (regional) Standard, from widest to narrowest processing boundary.
• Provisioned: you reserve capacity in PTUs and pay for it whether you use it or not; in return, throughput and latency are predictable.
• Batch: you submit a file of requests, results arrive within a 24-hour target at about half price; batch has a separate enqueued-token quota, so it does not starve your live traffic.

Not every model supports every type in every region — check the model card and Microsoft's model-by-region tables (u04).

Exam trap — Data Zone vs Standard. "Data must be processed within the EU" → Data Zone Standard in an EU region (or regional Standard, if available and throughput is enough). "Within the resource's own geography only" → Standard or Regional Provisioned. Global types may process anywhere.

Exam trap — batch is not "slow standard". Batch is for jobs that can wait up to a day, at a discount, with its own quota. It is wrong for anything a user waits for.

4. Configuration parameters: shaping each request

A deployment fixes where and how much. Parameters in each request shape what comes back. In the Foundry playground they appear as model settings; in code they are arguments to responses.create(...) (Responses API) or chat.completions.create(...) (Chat Completions API). Parameter availability depends on the model and API.

Frequency and presence penalties, stop sequences and seed belong to the Chat Completions API; the Responses API used in the AI-901 labs exposes a smaller set (temperature, top P, max output tokens, reasoning settings).

Reasoning models are different. Microsoft's reasoning-model documentation lists parameters they do not support, including temperature, top_p, presence_penalty, frequency_penalty and max_tokens. You steer them with reasoning effort and cap them with max_output_tokens / max_completion_tokens, which counts reasoning tokens plus visible output. Set the cap too low and the model can spend it all thinking and return little or nothing.

Exam trap — max tokens vs context window. Max output tokens is a per-request cap you choose. The context window is the model's fixed maximum for input + output (u03).

Exam trap — temperature vs reasoning effort. "More creative / more varied" → temperature (non-reasoning models). "Think harder on a complex problem" → reasoning effort (reasoning models).

In code (Responses API, via the Foundry SDK from u01):

# policy Q&A on a chat model: focused and short
response = client.responses.create(
    model=os.environ["MODEL_DEPLOYMENT"],
    instructions="You are the ENGAGE Service Desk assistant. Answer in at most 5 bullets.",
    input="Who approves time on a client engagement?",
    temperature=0.2,
    max_output_tokens=300,
)

# complex analysis on a reasoning model: no temperature, set effort instead
response = client.responses.create(
    model=os.environ["REASONING_DEPLOYMENT"],
    input="Compare fixed-fee vs time-and-materials for this engagement: ...",
    reasoning={"effort": "medium"},
    max_output_tokens=4000,   # includes reasoning tokens
)

5. Quotas: TPM and RPM

• Tokens per minute (TPM) is assigned per subscription, per region, per model, per deployment type. Creating a deployment with 50K TPM takes 50K from that pool; you can split the pool across deployments.
• Requests per minute (RPM) comes with TPM: quota is allocated in capacity units, each carrying a fixed amount of TPM and RPM that varies by model.
• The rate limiter estimates a request's tokens when it arrives — prompt size plus the max output tokens you asked for. An oversized max_output_tokens "uses up" quota even if the answers are short. Microsoft notes this estimate is not the same as the billed token count.
• Over the limit you get 429 responses. Clients should retry with backoff; the fix is more TPM, a smaller max_output_tokens, fewer concurrent calls, spreading load — or a different deployment type.
• Provisioned deployments are sized in PTUs instead of TPM; batch has its own enqueued-token quota.

6. Aldervane's fixes

7. The cost

• Global is cheapest, narrow boundaries cost choice. Global Standard has the lowest price and highest quota; Data Zone and regional types may cost more, get new models later and offer less throughput.
• Provisioned bills whether you use it or not. PTUs make sense for steady, predictable load; for spiky internal tools, pay-per-token is usually cheaper.
• Batch trades time for money. Half price, but results within a 24-hour target — sometimes later.
• Managed compute bills GPU hours while running; scale-to-zero helps, idle capacity still costs.
• Max output tokens is a cost and quota lever. A high cap wastes quota via the rate-limit estimate and allows long, expensive answers.
• Reasoning effort is a cost dial. Higher effort → more reasoning tokens, billed as output → higher bill and latency.
• Compliance boundary. Choosing Global for client data may breach a contract even if the answers are perfect. For Aldervane, the deployment type is a Risk & Legal decision, not a developer default.

8. Exam traps — summary

• Default/general, cheapest, highest quota → Global Standard.
• Process within EU/US data zone → Data Zone Standard (or Data Zone Provisioned/Batch).
• Process within one geography → Standard / Regional Provisioned.
• Guaranteed throughput, predictable latency → Provisioned (PTUs).
• Large async job, 24-hour target, ~50% cheaper, separate quota → Batch.
• Evaluate a fine-tuned model cheaply, 24-hour lifetime → Developer.
• Data at rest stays in your geography for every type; the type governs processing.
• Consistent/factual → low temperature; creative → higher. Change temperature or top P.
• Response length and cost cap → max output tokens; repetition → frequency penalty; new topics → presence penalty; cut off at a marker → stop sequence.
• Reasoning models: no temperature/top P/penalties; use reasoning effort; the output cap includes reasoning tokens.
• 429 = rate limit (TPM/RPM); large max_output_tokens counts against the estimate.

9. Remember the pictures

Redraw the rings — GLOBAL, DATA ZONE, REGION, with AT REST locked in the centre — and the TEMPERATURE dial from LOW/FACTS to HIGH/IDEAS. Add a small clock labelled "batch 24h" and a coin labelled "PTU = reserved", and you have this whole objective.

Sources: Microsoft Learn — Understanding deployment types in Microsoft Foundry Models; Deployment options for Microsoft Foundry Models; Data, privacy, and security for Azure Direct Models; Azure OpenAI quota management; Azure OpenAI reasoning models; Azure OpenAI REST API reference (chat completions); Microsoft Foundry Playgrounds; module Get started with generative AI and agents in Azure.