My Azure Study Note - 15 - Messaging- Service Bus, Event Hubs

Service Bus

• Fully managed, full blown message queueing service

• Durable

• Supports point-to-point (Queue) and pub/sub (Topic) scenarios

• Compatible with AMQP protocol

• Compatible with JMS 2.0 API (Premium only)

 Advanced features:

• Message sessions (guarantees FIFO)

• Dead-letter queue

• Scheduled delivery

• Transactions

• Duplicate detection

• And more…

Availability:

• SLA: 99.9%

• Can be configured for geo-disaster recovery 

Event Hubs

• Big Data streaming platform and event ingestion service

• Note: No “messaging” in the description

• Basically a managed Kafka implementation

• Can receive millions of events per second

Event Producers

- Components generating the events

- Can be done by anyone with the client /

HTTP client

- Simple connection and API

Partition

- Single event stream

- Can think of it as a single queue

- Guarantees order

- Limited availability

- Better to spread messages across

partitions to improve availability

- …but then order is not guaranteed

- Max 32 partitions on a single Event Hubs

Consumer Group

- Logical group of receivers, belong to the

same application

- Example:

- Receivers for processing telemetry

= Consumer Group

- Receivers for storing the telemetry

= Consumer Group

- Event receiving is done via AMQP

protocol 

SLA:

• Basic and Standard tier: 99.95%

• Dedicated: 99.99% 

Event Hubs Throughput Units

• Throughput is measured in Throughput Units (TU)

• 1 TU =

• Ingress (Input) – 1MB / sec or 1000 events / sec

• Egress (Output) – 2MB / sec or 4096 events / sec

• Prepurchased, billed by the hour 

 

 

My Azure Study Note - 14 - Messaging- Event Grid

Event Grid

• Allows building event-based architectures

• Publishes events to interested parties

• No queue / no order

• Strong integration with many Azure services

• Cost effective, simple pricing

• No tiers, HA is built in

 

SLA:

• 99.99%

• Max event size:

• 1MB

Event Grid

• Performance:

• 10,000,000 events / sec

• 5,000 events / sec / topic

• Latency:

• Subsecond end-to-end latency in the 99th percentile 

Event Grid Pricing

• Based on:

• Number of operations

• First 100K operations are free 

 Create Event Grid

The event will be triggered when some item will be added to the storage account and write to the cosmos db

Install package- Microsoft.Azure.Webjobs.Extensions.EventGrid

Copy the connection strings

ProcessOrderCosmos.cs

using Microsoft.Azure.WebJobs;

using Microsoft.Azure.WebJobs.Extensions.Http;

using Microsoft.AspNetCore.Http;

using Microsoft.Extensions.Logging;

using System.Threading.Tasks;

using System;

using System.IO;

using Newtonsoft.Json;

using System.Collections.Generic;

using Microsoft.Azure.WebJobs.Extensions.EventGrid;

using Microsoft.Azure.EventGrid.Models;

using Microsoft.Azure.Storage.Blob;

namespace AzureCourse.Function

{

    public static class CosmosOrderFunction

    {

        [FunctionName("ProcessOrderCosmos")]

        public static void Run(

            [EventGridTrigger]EventGridEvent eventGridEvent,

            [Blob("neworders", FileAccess.Write, Connection = "StorageConnectionString")] CloudBlobContainer container,

            [CosmosDB(databaseName: "readit-orders", collectionName: "orders", ConnectionStringSetting = "CosmosDBConnection")]out Order order,

            ILogger log)            

        {        

            order=null;

            try  {    

                log.LogInformation("Function started");

                log.LogInformation($"Event details: Topic: {eventGridEvent.Topic}");

                log.LogInformation($"Event data: {eventGridEvent.Data.ToString()}");

                string eventBody = eventGridEvent.Data.ToString(); 

                log.LogInformation("Deserializing to StorageBlobCreatedEventData...");

                var storageData=JsonConvert.DeserializeObject<StorageBlobCreatedEventData>(eventBody);  

                log.LogInformation("Done");

                log.LogInformation("Get the name of the new blob...");

                var blobName=Path.GetFileName(storageData.Url);

                log.LogInformation($"Name of file: {blobName}");

                log.LogInformation("Get blob from storage...");

                var blockBlob=container.GetBlockBlobReference(blobName);

                var orderText=blockBlob.DownloadText();

                log.LogInformation("Done");

                log.LogInformation($"Order text: {orderText}");

                

                order=JsonConvert.DeserializeObject<Order>(orderText);  

            }

            catch (Exception ex)  {

                log.LogError(ex, "Error in function");

            }            

        }

    }   

}

Create Event Subscription

 Now Upload a file azure storage account->Container, check the function app log and check the cosmos DB

Content of the uploaded json file as below

My Azure Study Note - 13 - Messaging, Storage Queue

Messaging in Azure

Azure has 4 fully managed messaging services

1. Storage Queue
2. Service Bus
3. Events Grid
4. Event Hubs

Storage Queue

• Part of Azure Storage Account

• The simplest queue implementation

• Create queue -> Send Message -> Receive message

• No special pricing for queue, included in Storage Account

• Same for availability

Performance:

• Requests (1KB msg)

• 20K msgs / sec / account

• 2K msgs / sec / queue

• Max msg size: 64KB 

Development:

• Client libraries for many development languages

• .NET, Java, Python, NodeJS, C++, PHP, Ruby

• Extremely simple object model

Create a Storage Queue

Step 1: Create a storage account

Step 2 : Click on Queues

Step 3 : Add a queue

Step 4. Add message

Step 5. Read Message from queue

Step 6. Copy connection string and paste it to the above connection string, run the programme

Step 7. Refresh the queue in azure and you can see nothing in the queue.

Write some new messages but you can see nothing appears in azure because as soon as we will write the message, it will be continiously pulling by the programme.

My Azure Study Note - 12 - Cosmos DB

 Azure Cosmos DB is a fully managed NoSQL database for modern app development. 

Create Cosmos DB

Create DB

Create Container

Add Item

Use Query

Connection String/Keys

Add Connection in Function

My Azure Study Note - 11 - Configure SQL

 

Create the DB

Install the extension for VS Code

Copy the Connection string from azure and change the PASSWORD

Add the Connection

Error Message while trying to set the connection

Allow the IP

Allow Firewall

Connection Success

Change COnnection string in the application and run the below commands(code first) to generate tables

Add the IP address of the VM where the application is running