Azure Service Bus

Azure Service Bus enables your applications to interact in several different ways such as: 1) letting applications send and receive messages through a simple queue, 2) a queue with a publish-and-subscribe mechanism when an ordinary queue isn’t enough, 3) no connection between applications, and queues are required. 

Service Bus is a multi-tenant cloud service, which means that the service is shared by multiple users. Each user, such as an application developer, creates a namespace, then defines the communication mechanisms needed within that namespace. Within a namespace, you can use one or more instances of three different communication mechanisms, each of which connects applications in a different way. The choices are:

Queues, which allow one-directional communication. Each queue acts as an intermediary (a broker) that stores sent messages until they are received. Each message is received by a single recipient.
Topics, which provide one-directional communication using subscriptions-a single topic can have multiple subscriptions. Like a queue, a topic acts as a broker, but each subscription can optionally use a filter to receive only messages that match specific criteria.
Relays, which provide bi-directional communication. Unlike queues and topics, a relay doesn’t store in-flight messages; it’s not a broker. Instead, it just passes them on to the destination application.
When you create a queue, topic, or relay, you give it a name. Combined with whatever you called your namespace, this name creates a unique identifier for the object. Applications can provide this name to Service Bus, then use that queue, topic, or relay to communicate with each other.

To use any of these objects in the relay scenario, Windows applications can use Windows Communication Foundation (WCF). This service is known as WCF Relay. For queues and topics, Windows applications can use Service Bus-defined messaging APIs. To make these objects easier to use from non-Windows applications, Microsoft provides SDKs for Java, Node.js, and other languages. You can also access queues and topics using REST APIs over HTTP(s).

It’s important to understand that even though Service Bus itself runs in the cloud, applications that use it can run anywhere. You can use Service Bus to connect applications running on Azure. You can also use it to connect an application running on Azure or another cloud platform with an on-premises application or with tablets and phones. It’s even possible to connect household appliances, sensors, and other devices to a central application, or to connect these devices to each other. Service Bus is a communication mechanism in the cloud that’s accessible from pretty much anywhere.

How to load data to SQL Azure

You can import data to SQL Azure and your data could exist in various sources such as SQL Server, Oracle, Excel, Access, flat files and others. Your data could exist in various locations. A location might be a data center, behind a corporate firewall, on a home network, or even in Windows Azure.

The following tools are commonly used for bulk upload:

BCP: This is a utility available with the SQL command line utilities that is designed for high performing bulk upload to a single SQL Server/Azure database.

SSIS: This is a powerful tool when operating on multiple heterogeneous data sources and destinations. 

Import & Export Data: A simple wizard that does not offer the wide range of configuration that SSIS provides, but is very handy for schema migration and smaller data uploads.

SSMS: This tool has the option of generating SQL Azure schema and data migration scripts. It is very useful for schema migration, but is not recommended for large data uploads.

Bulk Copy API: In the case where you need to build your own tool for maximum flexibility of programming, you could use the Bulk Copy API. This API is highly efficient and provides bulk performance similar to BCP.

Azure Scheduler versus Quartz Scheduler

Let’s dig deeper on Quarts Scheduler to learn why it should not be used in Azure for scheduling jobs.

Adding the quartz library to your app does not begin to ready your application to schedule jobs. You have to write your implementation of the job interface, then you have to construct large xml configuration files or add code to your application to build new instances of JobDetails and then schedule them using Schedule instance from a ScheduleFactory. What happens when you need to make a change to a job’s schedule? Temporarily disable a job? Change the parameters bound to a job? All these require a build/test/deploy cycle which is impractical for any organization.

Quartz does not meet the following enterprise scheduler requirements:

  • No out of the box support for multiple execution nodes 
  • No administration UI 
  • No monitoring
  • No alerts
  • No strong mechanisms for dealing with errors/failures and recovery

Azure and AMPQ

What is AMQP ?

Traditionally, message-oriented middleware products have used proprietary protocols for communication between client applications and brokers. Connecting messaging brokers from different vendors is tricky. This typically requires application-level bridging to move messages from one system to another and to translate between their proprietary message formats. This is a common requirement; for example, when you must provide a new unified interface to older disparate systems, or integrate IT systems following a merger.

The main goal of developing the Advanced Message Queuing Protocol (AMQP) was to create an open-standard messaging protocol that made it possible to build message-based applications using components built using different languages, frameworks, and operating systems, all using best-of-breed components from a range of suppliers.

Both the Azure Service Bus cloud service and on-premises Service Bus for Windows Server (Service Bus 1.1) support the AMPQ. This protocol enables you to build cross-platform, hybrid applications. You can construct applications using components that are built using different languages and frameworks, and that run on different operating systems. All these components can connect to Service Bus and seamlessly exchange structured business messages efficiently and at full fidelity.

Azure Webjobs

The main advantage of Azure WEbjobs are:

  • They work great with message queues
  • They are free
  • They deserialise your messages without you perform any actions
  • They auto-deploy from GitHub with the website
  • You can easily track them via the management portal
  • Config is a breeze
  • They’re resilient to errors
  • They are extremely simple to test
  • They are parallelised and they auto scale-out with your site

 

Design and Implement an Azure Storage Strategy

Azure Storage and Azure SQL Database both play an important role in the Microsoft Azure Platform-as-a-Service (PaaS) strategy for storage. Azure Storage enables storage and retrieval of large amounts of unstructured data. You can store content files such as documents and media in the Blob service, use the Table service for NoSQL data, use the Queue service for reliable messages, and use the File service for Server Message Block (SMB) file share scenarios. Azure SQL Database provides classic relational database features as part of an elastic scale service.

Uploading a blob
You can upload files to blob storage using many approaches, including the following:

1- Using the AzCopy tool 
2- Directly using the Storage API and writing HTTP requests
3- Using the Storage Client Library, which wraps the Storage API into a language and platform-specific library 
4- Using Windows PowerShell cmdlets 

To upload a blob using AzCopy, complete the following steps:

1- Download AZCopy 
2- Open a command prompt and navigate to C:\Program Files (x86)\Microsoft SDKs\Azure\AzCopy.
3- Create a text file in a folder that is easy to get to. Insert some random text in it.
4- In the command window, type a command that looks like this: AzCopy /Source:c:\…../Dest:https://myaccount.blob.core.windows.net/mycontainer2 /DestKey:key /Pattern:*.txt.
Press Enter to issue the command to transfer the file.

Reading data
You can anonymously read blob storage content directly using a browser if public access to blobs is enabled. The URL to your blob content takes this format:

https://<your account name>.blob.core.windows.net/<your container name>/<your path and filename>

Or you can read blobs using Visual Studio.  You can also use Server Manager in Visual Studio to view the contents of your blob containers and upload or download files. Navigate to the blob storage account that you want to use.
Double-click the blob storage account to open a window showing a list of blobs and providing functionality to upload or download blobs.

Priority Queue Design Pattern in Azure

Challenge

Applications can delegate specific tasks to other services, for example, to perform background processing or to integrate with other applications or services. In the cloud, a message queue is typically used to delegate tasks to background processing. In many cases the order requests are received in by a service isn’t important. In some cases, though, it’s necessary to prioritize specific requests. These requests should be processed earlier than lower priority requests that were sent previously by the application.

Solution

A queue is usually a first-in, first-out (FIFO) structure, and consumers typically receive messages in the same order that they were posted to the queue. However, some message queues support priority messaging. The application posting a message can assign a priority and the messages in the queue are automatically reordered so that those with a higher priority will be received before those with a lower priority. 

Competing Consumers Design Pattern in Azure

Challenge

An application running in the cloud is expected to handle a large number of requests. Rather than process each request synchronously, a common technique is for the application to pass them through a messaging system to another service (a consumer service) that handles them asynchronously. This strategy helps to ensure that the business logic in the application isn’t blocked while the requests are being processed.

The number of requests can vary significantly over time for many reasons. A sudden increase in user activity or aggregated requests coming from multiple tenants can cause an unpredictable workload. At peak hours a system might need to process many hundreds of requests per second, while at other times the number could be very small. Additionally, the nature of the work performed to handle these requests might be highly variable. 

Solution

Use a message queue to implement the communication channel between the application and the instances of the consumer service. The application posts requests in the form of messages to the queue, and the consumer service instances receive messages from the queue and process them. This approach enables the same pool of consumer service instances to handle messages from any instance of the application. 

Comparing Azure Container Service, Azure Service Fabric and Azure Functions

Azure Container Service: If you are looking to deploy your application in Linux environment and are comfortable with an orchestrator such as Swarm, Kubernetes or DC/OS, use ACS. A typical 3 tier application (such as a web front end, a caching layer, a API layer and a database layer) can be easily containerized with 1 single dockerfile (or docker-compose file). It can be continuously decomposed into smallerservices gradually. This approach provides an immediate benefit of portability of such an application. Containers is Open technology and there is great community support around containers.

Azure Service Fabric: If an application must have its state saved locally, then use Service Fabric. It is also a good choice if you are looking to deploy application in Windows server ecosystem(Linux support is in the works as well). 

Azure Functions: If an application needs an HTTP endpoint for a potentially long running  process without getting into a elaborate programming model, Azure Functions is a good option. They can be developed as an extension of an existing application. They support routing based endpoints similar to an API. They support AAD and other authentication, SSL, Custom Domain, RBAC, etc. They have a good CI/CD support as well. They are in the process of integrating .Net Core support. So if you are looking to get a simple application model and don’t want to get into setting up/managing underlying infrastructure, Azure Functions is good choice.

Advantages of Microservices Developed using Azure Service Fabric

Microservices developed using Service Fabric have many advantages such as:

  • Hyper Scale — Application developed using Service Fabric can be independently created and deployed without any dependencies. The services can be auto-scaled based on CPU Consumption, Memory Usage etc., Service Fabric can help with maximizing resource utilization with features such as load balancing, partitions, and replications across all nodes in the cluster.
  • Partitioning — Stateful services can be partitioned across multiple nodes in a cluster. The partitions are re-balanced regularly to ensure resource availability to each service deployed on the cluster.
  • Rolling Upgrades — Services deployed on service fabric platform can be updated in stages with minimum downtime. The update domains are used to divide the nodes in the cluster into logical groups which are updated one at a time. When a service needs to be upgraded (new version deployed), Service Fabric can ensure that newer version can be rolled out one node at at time, thereby ensuring that there is no downtime.
  • High Density — Service Fabric offers native support for Microservices. Each service hosted on the Service Fabric will be logically isolated and can be managed without affecting other services. This ensures that a relatively high number of Microservices can be deployed to a node, to maximize resource utilization. This can significantly reduce the costs associated with hosting applications.
  • Fault Tolerance — Microservices deployed on Service Fabric can leverage the support of automatic fault tolerance. When Service Fabric detects a fault on an instance of a microservice, it can automatically spin up the new instance of the microservice on a healthy node within the cluster to ensure availability. This process is completely automated and requires no additional effort from the teams managing the clusters or developing microservices.
  • Reverse Proxy — When you provision a Service Fabric cluster, you have an option of installing Reverse Proxy on each of the nodes on the cluster. It performs the service resolution on the client’s behalf and forwards the request to the correct node which contains the application. In majority of the cases, services running on the Service Fabric run only on the subset of the nodes. 

Copyright © All Rights Reserved - C# Learners