Azure Storage Queue
Azure Storage Queue is a type of message queuing services provided by Azure that provides queue storage infrastructure for a REST-based interface, within and between different applications and services.
Azure Queue storage service enables storing large numbers of messages that can be accessed from anywhere via authenticated calls using HTTP or HTTPS.
Some key conditions and metrics to monitor in Azure Storage Queue are;
Availability of Storage Queue
Number of messages in the queue
Total requests (Write, Read and Delete Requests)
Usage:
It primarily helps in the assured delivery of system messages between and within applications. Queues can be used to offload background and non-interactive workloads that ultimately help with scaling applications and in managing sudden traffic bursts. Azure Storage Queue enables storing messages until the system is available to process queues further.
Azure Queue storage allows application components to communicate with each other by decoupling system components. These components can communicate in the cloud, on the desktop, on-premises, or on mobile devices without depending on each other. If a component crashes, Azure Storage Queue can buffer requests until it is back up and able to process requests again.
Azure Storage provides a number of client libraries that interact with Azure Storage to provide support for application component failures. A single queue message can be up to 64 KB in size, and a queue can contain millions of messages, up to the total capacity limit of a storage account.
Azure Service Bus
Azure Service Bus is a cloud-based technology for messaging and communicating between different applications and devices. Azure Service Bus avails ‘message brokers‘ to conduct the processing of messages and ‘messaging stores‘ to cache the messages. It helps enhance the availability by constructing topologies with a routing system.
Before using Azure Service Bus, one should know its capabilities to be sure of it. The various features of Azure Service Bus are as follows:
Sessions Messaging – This attribute of Service Bus contributes to handling the unbounded message sequences. Request-Response and FIFO are the two patterns where Session messaging is enabled.
Dead-lettering – It is a process that helps in holding the messages that have not to be processed or delivered. DLQ (Dead-Letter-Queue) inspect them to take further action.
Scheduled Delivery – Messages can be scheduled to be delivered at a particular time into the Queue or the topic.
Partitioned – Azure Service Bus enables partition in queue/topic that partition messages across message brokers and stores. Through partition, the transaction is also supported.
Filtering – Subscription in Azure Service Bus can set a filter on the type of message it wants to receive. Service Bus makes use of three types of filters: SQL, Boolean and Correlation filters. Learn more about Azure Service Bus filters.
Batching – When a batch is enabled, it groups the messages into batches to reduce the number of operations. For example: when multiple messages are sent simultaneously, the Azure Service Bus group them under a single batch and send in fewer operations.
Deferral of Message – When a receiver is not ready to receive specific messages due to some circumstances, then those messages are sent back to queue/topic and remain there.
Auto Forwarding – This feature helps automatically send messages from one queue/subscription to another under the same namespace.
Azure Service Bus consists of Queue and Topic, explained below in detail.
Azure Service Bus Queue
Service Queue works on First In, First Out (FIFO) model. The communication between the application occurs via a broker, i.e. Queue. The message sender sends the message to this intermediary (queue), then these messages are then pulled from the queue by the Message Receiver. Message sender doesn’t wait for the confirmation from the receiver; it keeps on processing messages as the queue delivers the message to the receiver in the same order as added to the Queue.
Queue helps when one needs to communicate between web app and Azure application. Further, on-premise cloud and hybrid solution’s hosted apps can make a connection with each other. In addition to that, distributed application’s components can communicate with various organisation.
Queue helps in achieving the application scalability and ensure resilience to the application architecture. The load-levelling feature helps the receiver and the sender convey messages at different rates. Application components decoupling is achieved with the queue, which means the sender and the receiver need not communicate simultaneously.
Difference Between Azure Storage Queue vs Service Bus Queue
1. Message Size
Azure Storage Queues can be used when there is a need to store messages larger than the size of 80 GB. This is because the maximum size of a Service Bus Queue is 80 GB. Once this limit is reached, the queue will start to discard the incoming messages throwing an exception.
2. Polling
Service Bus Queues can be used when the application must receive messages without polling the queue.
3. Server-side Logs
Azure Storage Queues provide logging of all the occurred transactions in the Storage Queue which can be used for analytics or audit purpose.
4. Duplicate Detection
Azure Service Bus Queues provide a property called requires Duplicate Detection. When this property is enabled, all the duplicate messages within the provided duplicate detection time frame window will be discarded. The new message will be considered as a duplicate when there is another message already in the queue with the same message id.
5. Session Handling
If the application requires to process the messages within a session separately, Service Bus Queues can be used.
6. FIFO Handling
Service Bus Queue maintains the FIFO principle which cannot be guaranteed in the Storage Queues. This is due to the visibility time out feature of the messages in a Storage Queue. This causes the visibility time out of the expired message to be placed after a message, which was originally enqueued after it.
7. Failure Handling
If the application requires load balancing, failure tolerance, and increased scalability then Azure Storage Queues are the best choice.
8. Dead-letter
Dead-letter messages are unwanted or error messages due to an error that is stored separately in the Service Bus Queue or Topic Subscription. If the downstream application goes down, it is possible that the messages in the Service Bus Queue would get dead-lettered exceeding the Time to Live. This makes sure that none of the active listeners will try to process unwanted messages. It also provides us a way to view the unprocessed messages or unwanted messages.
In certain cases, the messages in the Service Bus Queue will be automatically dead-lettered. Some of such reasons are:
Messages sent to session enabled queue without session id will be dead-lettered
Messages that are received beyond the provided max delivery count of the queue will be dead-lettered
Messages with header size more than 64KB will be dead-lettered.
Apart from the above-mentioned scenarios, the business application can also dead-letter messages by providing a custom dead-letter reason. For example, in an e-commerce scenario, while processing an order the message can be dead-lettered due to custom reasons like “Item Out of Stock”, “Invalid details” and so on.
9. Transaction Support
Local transactions in the context of a single queue are supported in Service Bus Queues
10. Flexible Access Control System
Both the Storage Queues and Service Bus Queues provide flexible and reliable delegated access control mechanisms. Users can provide access to the Namespace and Storage Account level or at the entity level.
11. Scalability
When it comes to scalability, Storage Queues are more preferred as the Storage Queue can store up to 200 TB of messages. It is also possible to create an unlimited number of Storage Queues in a Storage Account.
12. Reduced Messaging Operation Cost
Messaging operation cost can be reduced using the Receive and Delete mode of Service Bus Queues. When the messages in the Service Bus Queue are received in Receive and Delete mode, they are removed from the queue.
13. Auto Forwarding of Messages
The auto-forward capability provided by Service Bus Queues, allows us to configure another destination queue to which all the messages received by the source queue will be forwarded. This mechanism can be used to achieve security.
14. Naming Conventions
Allowed characters in Storage Queues names are lowercase alphabets, numbers, and hyphens with length ranging from 3 to 63. Name of a Service Bus Queue can be up to 260 characters long containing letters, numbers, periods, hyphens and underscores.
15. Maximum Message Size
The maximum message size in Storage Queues is 64 KB. If the messages are base64 encoded, then the maximum message size is 48 KB. Large message sizes are supported by combining queues with blobs, through which messages up to 200 GB can be enqueued as single data. The maximum message size of Service Bus messages is 256 KB in Standard Tier and 1 MB in Premium Tier. The maximum size of the message header is 64 KB
16. Maximum Number of Queues in Service Bus
A maximum of 10,000 queues can be created within a single Service Bus Namespace. If there is a need to create more than 10,000 queues within a single Service Bus Namespace, Azure support team can be contacted.
17. Poison Messages in Storage Queues
Whenever the messages in a Storage Queue are retrieved more than the specified Dequeue Count, the messages are then moved to the configured dead-letter queue.
The Tech Platform
Comments