Skip to content
Tutorials Center
My learning

Topics

Lesson 35 of 57 ยท AI and Machine Learning

Azure AI Services and Azure OpenAI

Learn Azure AI Services (language, vision, speech, translator, Document Intelligence) and Azure OpenAI: resources, endpoints, keys versus Entra ID, deployments, quotas and token pricing.

Azure offers two families of ready-made AI you can call over HTTPS without training anything yourself. Azure AI Services (earlier called Cognitive Services) are task-specific APIs: translate this text, read the text in this image, turn this audio into a transcript, pull the fields out of this invoice. Azure OpenAI gives you large language models such as the GPT family through the same security, networking and billing model as the rest of Azure. Both are consumption services: you create a resource, get an endpoint, send a request and pay for what you use.

This lesson explains the services, how an endpoint and a deployment differ, the two ways to authenticate, and how token-based pricing works. A real call needs a subscription and a resource, so the Azure commands and HTTP requests are read-only. The runnable examples are JavaScript stand-ins with invented prices and a fake endpoint that mimics the real rules (keys, deployments, quotas and 429 Too Many Requests).

The service families

Service What it does Typical input and output
Language Sentiment, key phrases, entity recognition, summarization, question answering Text in, JSON with labels and scores out
Vision Image analysis, captions, tags, reading printed or handwritten text (OCR) Image in, JSON out
Speech Speech to text, text to speech, speech translation Audio in, text out (and the reverse)
Translator Text translation and language detection Text in, translated text out
Document Intelligence Extracts fields, tables and layout from forms, receipts and invoices (formerly Form Recognizer) PDF or image in, structured fields out
Azure OpenAI Chat and text generation, embeddings, image generation, speech models Messages in, generated content out

Microsoft renames and regroups these products regularly (the umbrella for building AI apps is now branded Microsoft Foundry), so treat the names above as stable ideas and check the current docs for the exact portal labels. The shapes of the APIs change slowly: a JSON request, a JSON response, an endpoint and a credential.

Resources, endpoints and regions

You create a resource in a resource group, exactly like a storage account (see Resource Groups and Azure Resource Manager). The resource has an endpoint URL such as https://my-ai-resource.cognitiveservices.azure.com/ and gives you two keys. You can create a multi-service resource that covers several AI services with one key, or single-service resources when you want separate billing or access per service.

Region matters for three reasons: not every model or feature exists in every region, your data is processed in the region you pick (important for privacy rules), and quotas are tracked per region. Create the resource near your users and near your other Azure services.

Terminal
az group create --name rg-ai --location westeurope

# A multi-service AI Services resource (read-only: it creates a billable resource)
az cognitiveservices account create \
  --name my-ai-resource --resource-group rg-ai \
  --kind CognitiveServices --sku S0 --location westeurope \
  --custom-domain my-ai-resource --yes

az cognitiveservices account show --name my-ai-resource --resource-group rg-ai \
  --query properties.endpoint --output tsv

The --custom-domain option gives the resource its own subdomain. You need one to use Microsoft Entra ID tokens instead of keys.

Keys versus Microsoft Entra ID

There are two ways to prove who you are, and the choice is a security decision, not just a convenience:

  • Keys. Send a secret in a request header (Ocp-Apim-Subscription-Key for most AI Services, api-key for Azure OpenAI). Easy to try, but anyone who has the key has full access, you must rotate it, and it ends up in config files and logs.
  • Microsoft Entra ID. Your app has an identity and requests a short-lived bearer token; you grant it a role such as Cognitive Services User or Cognitive Services OpenAI User. No secret is stored, access is per identity and every call can be audited. On Azure, a managed identity gives the app that identity for free (see Managed Identities and Service Principals).

The best habit is keyless: use a managed identity in Azure and your own developer sign-in locally. If you must use a key, keep it in Azure Key Vault and disable local (key) authentication on the resource once nothing needs it.

Calling the service

Every AI Services call is an HTTPS request. This translation request is read-only; a real call needs a real key:

Terminal
curl -X POST "https://api.cognitive.microsofttranslator.com/translate?api-version=3.0&to=de" \
  -H "Ocp-Apim-Subscription-Key: $TRANSLATOR_KEY" \
  -H "Ocp-Apim-Subscription-Region: westeurope" \
  -H "Content-Type: application/json" \
  -d '[{"Text": "Good morning"}]'

The response is a JSON array with one translation per input text.

Azure OpenAI: deployments and tokens

Azure OpenAI adds one idea you do not see in the other services: the deployment. A model such as a GPT version is not called directly. You first create a deployment of that model in your resource, give it a name of your choice and a capacity (quota), and then you call the deployment by name. This lets you pin a model version, run two versions side by side and give each application its own limit.

Terminal
az cognitiveservices account create \
  --name my-openai --resource-group rg-ai \
  --kind OpenAI --sku S0 --location swedencentral \
  --custom-domain my-openai --yes

az cognitiveservices account deployment create \
  --name my-openai --resource-group rg-ai \
  --deployment-name chat-small \
  --model-name <model-name> --model-version "<version>" --model-format OpenAI \
  --sku-name Standard --sku-capacity 10

Model names and versions are placeholders because the catalog changes; list what your region offers in the portal. A chat request then targets the deployment name:

Terminal
curl -X POST "https://my-openai.openai.azure.com/openai/deployments/chat-small/chat/completions?api-version=<api-version>" \
  -H "api-key: $OPENAI_KEY" \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Explain a resource group in one sentence."}],"max_tokens":60}'

Language models read and write tokens, small chunks of text (roughly three quarters of an English word). You are billed per token, with different prices for input (prompt) tokens and output (generated) tokens, and every request must fit the model's context window. Long prompts and long answers both cost money, so cap max_tokens and keep prompts short.

Estimating cost: a stand-in

The stand-in below turns a monthly usage forecast into a bill using invented prices per million tokens. Real prices depend on the model and region and change often; use the Azure pricing calculator for numbers you plan a budget on.

JS Estimate a monthly token bill

Press Run (or Ctrl + Enter) to see the console output here.

No output.

The table shows the two levers you control: pick the smallest model that is good enough, and trim the prompt, because input tokens often dominate.

Quotas, throttling and retries

Every deployment has a quota expressed in tokens per minute (and a matching requests-per-minute limit). When you exceed it the service answers HTTP 429 with a Retry-After header. Production code must treat 429 as normal: wait, then retry with exponential backoff and a little random jitter, and give up after a few attempts. The same applies to transient 5xx errors. The official SDKs retry for you, but you should still set sensible timeouts and surface failures to the user.

A fake endpoint with the real rules: a stand-in

This stand-in is not Azure. It checks a credential, looks up a deployment by name, enforces a small per-minute quota and returns 429 with retryAfter; the client retries with backoff. It uses a simulated clock so it runs instantly.

JS Keys, deployments and 429 retries

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Read the output from top to bottom: a bad credential is a 401 that retrying will never fix, a wrong deployment name is a 404 (a very common mistake, because the deployment name is your choice, not the model name), and the burst shows 429 followed by a wait that honors Retry-After.

Responsible use and cost control

  • Content filtering is on by default for Azure OpenAI and can block or annotate harmful content in prompts and answers.
  • Data privacy: prompts and outputs sent to Azure OpenAI are not used to train the base models, and you can use private endpoints to keep traffic off the public internet (see Private Endpoints, VPN and ExpressRoute).
  • Cost control: set a budget alert (see Cost Management), cap max_tokens, cache repeated answers, and monitor token usage in Azure Monitor.

Text analytics and opinion mining

Azure AI Language (formerly the Text Analytics part of Cognitive Services) analyzes text without any model training. Typical features: language detection, key-phrase extraction, named-entity recognition, PII detection, summarization, and sentiment analysis. Sentiment analysis labels a document and each sentence as positive, neutral, negative or mixed and returns confidence scores that add up to 1.

Opinion mining goes deeper. It is an option on sentiment analysis (show_opinion_mining=True in the Python SDK) that finds the targets people talk about and the assessments they make about each one. In "The room was spotless but the wifi was terrible", plain sentiment says "mixed"; opinion mining reports target room with assessment spotless (positive) and target wifi with assessment terrible (negative). That is what a product team needs from thousands of reviews: which aspects customers love and which they hate. Benefits are accuracy at scale, support for many languages, and no machine-learning expertise; for custom categories you can train a custom text classifier in the same service.

A real call (read-only here, because it needs a resource and key or Entra sign-in; see Azure AI Services for authentication):

Python
from azure.ai.textanalytics import TextAnalyticsClient
from azure.identity import DefaultAzureCredential

client = TextAnalyticsClient("https://<resource>.cognitiveservices.azure.com/",
                             DefaultAzureCredential())
docs = ["The room was spotless but the wifi was terrible."]
for doc in client.analyze_sentiment(docs, show_opinion_mining=True):
    print(doc.sentiment, doc.confidence_scores)
    for sentence in doc.sentences:
        for opinion in sentence.mined_opinions:
            print(opinion.target.text, [a.text for a in opinion.assessments])

You can also run these services in a container, such as the sentiment or language-detection images, on your own web app host or Kubernetes cluster. Containers keep data on your side of the network, cut latency, and fit environments with strict connectivity rules, but you still need an Azure resource for billing and must keep the image current.

The stand-in below mimics the opinion-mining idea with a tiny hand-written word list. It is a toy for seeing the output shape, not a real model.

JS A toy opinion miner

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What is an Azure OpenAI deployment?

    You deploy a model into your resource and give the deployment a name; requests are sent to that name, and quota is assigned per deployment.
  2. 2. Which authentication option avoids storing a secret in your app?

    Entra ID issues short-lived tokens to an identity, so there is no long-lived secret to leak or rotate.
  3. 3. Your call returns 429 with a Retry-After header. What should the client do?

    429 means you exceeded the quota. Honoring Retry-After and backing off is the correct response.
  4. 4. Which change most often lowers an Azure OpenAI bill?

    Cost is tokens multiplied by the model's per-token price, so a smaller model and shorter prompts directly reduce it.

Summary

  • Azure AI Services are task-specific APIs (language, vision, speech, translator, Document Intelligence); Azure OpenAI offers large language models.
  • You create a resource, call its endpoint over HTTPS, and pay per use.
  • Azure OpenAI requires a deployment, and you call the deployment name, not the model name.
  • Prefer Microsoft Entra ID and managed identities over keys; keep unavoidable keys in Key Vault.
  • Billing is per token with separate input and output prices; quotas return 429, which clients must handle with backoff.

Next: Azure Machine Learning and IoT Hub.