You have probably already used Microsoft Copilot, ChatGPT or GitHub Copilot, and you might be amazed by their AI-powered code suggestions and productivity enhancements. What if you could integrate AI into your own products and services? In this blog post I’ll scratch the surface of some of the capabilities that are available in Microsoft Azure.
Azure AI Vision
Azure AI Vision is a service you can use for image recognition and to extract information from your images. Image recognition is available as a REST method provided in the SDK for C#, Python and other languages. A use case is to use AI Vision to recognize the objects that are present in an image. For example persons, animals and cars. You can then create a new image and draw around recognized objects along with captions for the objects. In addition to detecting objects, the service can also be trained to detect and recognize faces in the image. These are just a small amount of what Azure AI Vision can be used for. For more information see Azure AI Vision with OCR and AI | Microsoft Azure
To get started with testing Azure AI Vision, you will need an Azure subscription with access to Azure AI Services.
Subscriptions with access to Azure AI have the possibility to provision a Azure AI resource.
For .NET development, you can use the Azure Image Analysis client library, available from NuGet, that connects to your AIServicesEndpoint.
<PackageReference Include="Azure.AI.Vision.ImageAnalysis" Version="1.0.0-beta.1" />
Example on using the ImageAnalysisClient to open an image as a filestream, and then analyze the image:
using FileStream stream = new FileStream(imageFile, FileMode.Open);
ImageAnalysisResult result = client.Analyze(
BinaryData.FromStream(stream),
VisualFeatures.Caption |
VisualFeatures.DenseCaptions |
VisualFeatures.Objects |
VisualFeatures.Tags |
VisualFeatures.People);
Complete example on Microsoft Learn
Using the ImageAnalysisClient to analyze an image, you can receive captions and tags from the analysis. This is a sample run on one of my photos from an urban street area:
Analyzing images/cars.jpg
Caption:
"a street with cars and trucks on it", Confidence 0,61
Dense Captions:
Caption: 'a street with cars and trucks on it', Confidence: 0,61
Caption: 'a black suv on a street', Confidence: 0,71
Caption: 'a black truck on a street', Confidence: 0,73
Caption: 'a black delivery truck on the street', Confidence: 0,76
Caption: 'a black car parked on the street', Confidence: 0,65
Caption: 'a tree with green leaves', Confidence: 0,68
Caption: 'a white truck on the road', Confidence: 0,75
Caption: 'a blurry image of a car driving on a street', Confidence: 0,76
Tags:
'building', Confidence: 1,00
'outdoor', Confidence: 1,00
'land vehicle', Confidence: 0,99
'vehicle', Confidence: 0,98
'road', Confidence: 0,98
'car', Confidence: 0,97
'street', Confidence: 0,93
'wheel', Confidence: 0,93
'sky', Confidence: 0,93
'downtown', Confidence: 0,92
'mixed-use', Confidence: 0,91
'metropolitan area', Confidence: 0,91
'skyscraper', Confidence: 0,90
'traffic', Confidence: 0,90
'commercial building', Confidence: 0,89
'urban area', Confidence: 0,88
'city', Confidence: 0,87
'neighbourhood', Confidence: 0,87
'metropolis', Confidence: 0,86
'thoroughfare', Confidence: 0,84
Together with the captions, and tags, the ImageAnalysisClient also provides a confidence score.
Document Intelligence Center
Azure Document Intelligence Center is another AI service available in Azure. It comes with a REST API and SDK you can integrate into your solutions. Some of the capabilities of the service is to extract text from documents. It can be used to analyze both printed and handwritten text. In addition to the SDK and REST API, you can also use Document Intelligence Studio. This is a visual tool for exploring and integrating features from Document Intelligence Center with your applications.
Example usage for integration with Document Intelligence Center : An app for travel expenses that extracts text from uploaded receipts.
This is an example using one of the samples in Document Intelligence Studio to analyze a receipt:
Receipt
Result
Azure AI Search
With Azure AI Search you can index and query information from different sources. For example from documents, databases and also from images. Images can be searched using image analysis.
Learn more about Azure AI Search
Summary
This was a quick introduction to Azure AI services. Later on I plan to write a more deep dive into the services. I hope this article inspired you to start explore Azure AI services yourself.