Images are more than just pixels; they convey stories, emotions, and information. Imagine building an e-commerce platform where users can find products by uploading photos or creating a recommendation system that suggests visually similar items. Perhaps you’re working on a content moderation pipeline to filter out inappropriate images. Whatever your use case, image search plays a pivotal role.
The primary goal of this article is to provide step-by-step guidance on leveraging Azure AI Search for image-based search scenarios. We’ll explore image vectorization (transforming images into meaningful vectors) and querying (finding relevant images based on user input).
Azure AI Search (formerly known as “Azure Cognitive Search”) is an AI-powered information retrieval platform. It helps developers build rich search experiences and generative AI apps that combine large language models with enterprise data.
Key features:
Vector Search: Enables searching based on vectors (representations) of text and images.
Full Text Search: Allows searching within text content.
Hybrid Search: Combines both vector and full-text search capabilities.
Rich Indexing: Supports data chunking, vectorization, lexical analysis, and content extraction.
Query Syntax: Provides powerful query options for various scenarios.
Integration with Azure Services: Works seamlessly with other Azure services like Azure AI Vision and Azure OpenAI.
Search Explorer is a tool that allows users to search for images on an Azure AI Search index. Here’s how it works:
Accepts text, vectors, and images as query inputs.
You can drag or select an image into the search area.
Vectorizes the image and sends the vector as a query input to the search engine.
It helps you find visually similar images based on the embedded vectors.
Image vectorization transforms images into numerical representations (vectors) that capture their essential features. In Azure AI Search, a vectorizer converts images into vectors during indexing and query execution. The embedding model (such as Azure OpenAI) generates these vectors. These vectors enable efficient similarity search, allowing you to find visually similar images based on their embedded features.
Querying with Vectors: When you search for images, you provide a query image (or its vector representation). The system compares this query vector with the vectors of indexed images. Similarity is determined by measuring the distance (cosine similarity) between vectors. The closer the vectors, the more visually similar the images. This approach allows for content-based image retrieval and recommendation systems.
Prerequisites:
Azure Subscription: Users need an Azure subscription (they can create one for free).
Azure AI Services Multiservice Account:
Set up an account with access to Azure AI Vision multimodal embeddings (available in specific regions). Regions supporting multimodal embeddings include EastUS, WestUS, WestUS2, NorthEurope, WestEurope, FranceCentral, SwedenCentral, SwitzerlandNorth, SoutheastAsia, KoreaCentral, AustraliaEast, and JapanEast.
Azure AI Search: Users must have an Azure AI Search service for indexing and queries.
The service tier determines the number of blobs that can be indexed.
Azure Blob Storage:
Use Azure Blob Storage (standard performance, general-purpose v2) to store image files as blobs.
Avoid using Azure Data Lake Storage Gen2.
Public Access:
Ensure all resources have public access enabled for successful wizard execution.
Afterward, consider enabling firewalls and private endpoints for security.
Steps to Search for Images by Using Azure AI Search and Search Explorer
STEP 1: Use image files (e.g., JPGs) from the azure-search-sample-data repository or other sources.
STEP 2: Navigate to the Azure Storage account, in the Azure portal.
STEP 3: On the left panel select Containers, under the Data Storage section.
STEP 4: Click on "+ Container" to create a new container. Enter the Name of the container and then click "Create".
STEP 5: After creating the container, upload the images.
STEP 6: Now, go to the Azure AI search service.
STEP 7: Create a new service, if you don't have an Azure AI Search service.
STEP 8: Select "Import and vectorized data", on the overview page.
STEP 9: On the Set up your data connection section, select the "Azure Blob Storage" option.
Specify:
Subscription
Blob storage account
Blob container
And then click Next.
STEP 10: Select the "AI Vision vectorization" model, in the Vectorize your text section. This version is not available in many regions. You need to make sure that Azure AI Search and Azure AI Multiservice accounts are in the same supported region.
Select "Next".
STEP 11: In Vectorize your images section, select the "Vectorize images" checkbox and select "AI Vision vectorization".
Now, check the box for:
Use same AI service selected for text vectorization.
Extract text from images
Use same AI service selected for images vectorization.
Select "Next".
STEP 12: In the Advanced Settings page, there is a schedule indexing option. Here you can specify a run schedule for the indexer.
Under the Schedule indexing option, select "Once". Select "Next".
STEP 13: In the Review Your Configuration section, specify a prefix for the object. Select "Create".
Using Search Explorer
STEP 1: In the Azure portal, go to “Search Management > Indexes” and select the created index. Switch to “Search Explorer” (the first tab).
Select “Image view”. Drag and drop an image from the local folder or use the file browser to select one.
Click “Search” to run the query.
Best Practices for Optimizing Image Search Performance
Best Practice 1: Choose the Right Image Format:
The format of an image affects its performance and file size. Use modern formats like WebP or JPEG 2000 for better compression and quality. Avoid large, uncompressed formats like BMP or TIFF.
Best Practice 2: Compress Images for Faster Loading:
Compressed images load faster, improving user experience. Use tools to compress images without sacrificing quality. Consider using lossless compression for critical images.
Best Practice 3: Use Relevant Images:
Select images that align with your content and context. Relevant images enhance user engagement and convey information effectively.
Best Practice 4: Properly Name Images and Use Alt Tags:
Give images descriptive file names (e.g., “red-sports-car.jpg” instead of “img123.jpg”). Add alt text (alternative text) to describe the image content for accessibility and SEO.
Best Practice 5: Implement Image Lazy Loading:
Load images only when they come into the user’s viewport. Use the loading="lazy" attribute in HTML to improve page load times.
Best Practice 6: Add Image Structured Data:
Use schema markup (such as JSON-LD) to provide context about images. Structured data helps search engines understand image content and context.
Conclusion
In this article, we’ve covered the following key points:
Image Vectorization:
We learned how to transform images into meaningful vectors using the wizard.
These vectors enable efficient similarity search and content-based recommendations.
Search Explorer:
We explored the Search Explorer tool within the Azure portal.
We retrieved visually similar images based on their embedded vectors by providing an image query.
Remember that image vectors are supported in stable API versions, but the wizard and vectorizers are in preview.
Comments