Overview

Information in this article is related to using CloudMonix for monitoring Azure Stream Analytics jobs.


The article covers the following topics:

  • common use cases where CloudMonix can help with monitoring and automation

  • what is needed to connect to and monitor Azure Stream Analytics jobs

  • what metrics CloudMonix tracks, visualizes and monitors


Why use CloudMonix for Azure Stream Analytics?

Popular usages of CloudMonix include the following examples:

  • Monitoring Uptime and Availability

  • Monitoring jobs status, inputs and outputs state

  • Alerting on outages, runtime errors, failed jobs, high utilization, etc.

Configuration

Azure Stream Analytics monitoring can be configured either via Setup Wizard or by using the “Add New” button in the dashboard. It’s highly recommended to use Setup Wizard when configuring permissions for the first time, as that will simplify authorization. Learn more about authorizing with Setup Wizard here.

Metrics

Every diagnostic data point that CloudMonix retrieves from the monitored resource is considered a metric in CloudMonix. Refer to the Metrics article to learn more about metrics in general.


CloudMonix provides a default template for monitoring Azure Stream Analytics jobs:

 

The metrics can be added, removed and customized in the Metrics tab of the Azure Cloud Role resource configuration dialog.


Built-in Metrics


ResourceStatus

Identifies the last state of the monitored job. This is a critical metric that is captured for most types of resources that CloudMonix tracks. It is used for Uptime reports and should not be removed.

  • Data Type: string

  • Possible values: Ready, Stopped, Down, Unknown

  • Included in sample profile:  yes, tracked as a metric called Status

  • Included in default alerts: yes, as an alert called ResourceOutage

Statuses are determined according to the following rules:


  • Ready - Azure Managemet API reports the job status as Running

  • Stopped - Azure Managemet API reports the job status as Stopped

  • Down - Azure Managemet API reports the job status as Failed

  • Unknown - in all other cases


AzureStreamAnalyticsInputs

Tracks the state of Azure Stream Analytics job inputs.

  • Data Type: Data Type: object with the following properties:

  • Name (string): The name of the input of Stream Analytics Job.

  • Code (string): An opaque diagnostic code.

  • Message (string): A human-readable message describing the condition in detail.

  • Since (DateTime): A UTC timestamp of when the condition started. Customers should be able to find a corresponding event in the ops log around this time.

  • Included in sample profile: yes, tracked as a metric called JobInputs

  • Included in default alerts: no


AzureStreamAnalyticsOutputs

Tracks the state of Azure Stream Analytics job outputs.

  • Data Type: Data Type: object with the following properties:

  • Name (string): The name of the output of Stream Analytics Job.

  • Code (string): An opaque diagnostic code.

  • Message (string): A human-readable message describing the condition in detail.

  • Since (DateTime): A UTC timestamp of when the condition started. Customers should be able to find a corresponding event in the ops log around this time.

  • Included in sample profile: yes, tracked as a metric called JobOutputs

  • Included in default alerts: no


AzureStreamAnalyticsMetric

Tracks a number of metrics accessed using Azure Insights API.

  • Data Type: double

  • Requires specifying Metric definition, i.e. selecting a specific value to track, such as Input Event Bytes (Total) or Runtime Errors (Total).

  • Included in sample profile: yes, tracked as metrics:

    • EventsInput: the number of Input Events/minute

    • EventsOutput: number of Output events/minute

    • BytesInputEvent: data ingress/minute

    • EventsLateInput: late-input events/minute

    • ErrorsConversion: number of data conversion errors/minute

    • ErrorsRuntime: average number of run-time errors/minute raised during running of the ASA job

    • EventsOutOfOrder: number of out-of-order events/minute

    • FunctionEvents: number of function events/minute

    • FunctionRequests: number of function requests/minute

    • FunctionFailedRequests: number of failed function requests/minute

    • Utilization: Streaming Units Utilization

  • Included in default alerts: yes

    • Conversion Errors Detected (Warning): ErrorsConversion > 0: Raises an alert when conversion errors have been detected in the last few minutes.

    • Failed Function Requests Detected (Warning): FunctionFailedRequests > 0: Raises an alert when failed function requests have been detected in the last few minutes.

    • High Utilization (Warning): Utilization > 80: Raises an alert when utilization of over 80% has been detected and sustained for at least 5 minutes.

    • Runtime Errors Detected (Warning): ErrorsRuntime > 0: Raises an alert when runtime errors have been detected in the last few minutes.


AzureStreamAnalyticsOperations

Tracks the log of Azure management operations on the monitored resource.

  • Data Type: object with the following properties:

  • Name (string): Operation name.

  • Category (string): Event category.

  • Description (string): Event description.

  • Caller (string): Caller.

  • EventName (string): The event name.This value should not be confused with operation name.

  • Level (string): Event level.

  • Status (string): The event status. Possible values include: Started, Succeeded, Failed.

  • SubStatus (string): The event sub status. Most of the time, when included, this captures the HTTP status code.

  • ExtendedInfo (string):The values of all properties of the EventData object displayed as Key-Value pairs, where keys are property names.

  • EventTimestamp (DateTime): The occurrence time of event.

  • Included in sample profile: yes, tracked as a metric called Operations

  • Included in default alerts: no


AzureStreamAnalyticsJobStatus

Indicates the current execution state of the job as reported by Azure, e.g. Created.

  • Data Type: string

  • Included in sample profile: yes, tracked as a metric called JobStatus

  • Included in default alerts: no


AzureStreamAnalyticsLastOutputEventMinutesAgo

The last output event time of the job.

  • Data Type: double

  • Included in sample profile: no

  • Included in default alerts: no


Alerts


Users can create alerts based on changes in any value tracked by CloudMonix (including custom metrics). Each resource template includes alerts which are suitable for a given resource. The predefined alerts for Azure Stream Analytics are listed in the Metrics section. Refer to the Alerts article to learn more about alerts in general.


Alerts are available during the Trial period or in Professional and Ultimate plans only.