One of the big challenges with maintaining production flows in Power Automate was the lack of read only access. This meant to read a flow log you would need full edit access, not good for production environments using Service Accounts.
Fortunately Microsoft heard us and just launch flow log integration with Application Insights. App Insights is the Azure standard for all logging, so it's incredibly powerful, additionally it can be setup with notification alerts and used as a data source for Power BI dashboards. It's not free (small memory cost and for alerts) and requires someone with Azure experience and permissions to create one.
Couple of call outs:
- It's not the same as the logs, as you do not see in flow data (the inputs and outputs, just the actions)
- The Global Power Platform Admin who creates the connection requires edit access to the app insight instance
- Each connection is between one environment and one app, but you can create multiple connections to each app, so you could have all your production flows linked to one App, all Dev another. Likewise you could link one environment to 2 apps, one for all Prod, one for the particular Dev,Test,Prod stack.
- In preview it works on any environment, once out of preview it will only work on Managed Environments (no surprise there lol)
Setup
Setup is nice and easy, you need to be a Global Power Platform Admin, then you select following menus:
Analytics - Data Export - App Insights - New Data Export
Select Power Automate and runs, triggers, actions
Select required environment
Select subscription, resource group, from there you should see the list of available App Insights, select the required one
You should then see list of all your connections, you can add more with the new data export from the top.
App Insights
App Insights can be found in your Azure Portal
There is so much you can do with App Insights, like live data, alerts, and more. But I'm just going to look at the log query, as this is in my opinion the most useful.
There are couple key commands that I use:
let- for variables let myEnvironmentId = 'ca99eaab-99a9-ef99-9b99-99ded999999f';
requests dependencies traces exceptions customMetrics - the table for what log you need, for Flows we just use requests and dependencies
| where - standard filter the results by | where customDimensions ['environmentId'] == myEnvironmentId
| extend - brings nested keys/fields to the root | extend Data = todynamic(tostring(customDimensions.Data))
| project - selects required keys/fields for the report | project timestamp ,id ,DisplayName = Data.FlowDisplayName
Request and Dependency Structure
There are a lot of keys/fields returned from the 2 tables, but to keep it simple I will identify the key ones:
Requests
| Key/Field | Description | Node |
|---|---|---|
| timestamp | date / time of run | |
| id | GUID for app insights | |
| success | boolean if run was successful | |
| name | flow id | |
| duration | time for flow to complete | |
| itemType | request or dependency | |
| customDimensions | main node of data | |
| Data | node of data | customDimensions |
| FlowDisplayName | flow name | Data |
| RunId | Run ID | Data |
| tags | node of data | Data |
| createdBy | flow owner id | tags |
| environmentId | environment id | customDimensions |
| error | node of data | customDimensions |
| code | error description | error |
| message | error reason | error |
| signalCategory | log type (cloud/desktop) | customDimensions |
Dependencies
| Key/Field | Description | Node |
|---|---|---|
| timestamp | date / time of run | |
| id | GUID for app insights | |
| resourceId | Flow ID | customDimensions |
| success | boolean if action was successful | |
| name | action name | |
| duration | time for flow to complete | |
| itemType | request or dependency | |
| RunId | Run ID | Data |
| customDimensions | main node of data | |
| Data | node of data | customDimensions |
| FlowDisplayName | flow name | Data |
| operation_ParentId | Run ID | Data |
| actionType | action type | Data |
| tags | node of data | Data |
| createdBy | flow owner id | tags |
| environmentId | environment id | customDimensions |
| error | node of data | customDimensions |
| code | error description | error |
| message | error reason | error |
| signalCategory | log type (cloud/desktop) | customDimensions |
Relationships
Queries
You can see there are infinite different queries you can run, but I thought I would show the 3 that I have started to use:
Flow Runs
let myEnvironmentId = 'ca84eccb-78a7-ef84-9b20-84ded785680f';
requests
| where customDimensions ['resourceProvider'] == 'Cloud Flow'
| where customDimensions ['signalCategory'] == 'Cloud flow runs'
| where customDimensions ['environmentId'] == myEnvironmentId
| extend Data = todynamic(tostring(customDimensions.Data))
| extend Error = todynamic(tostring(customDimensions.error))
| project timestamp
,id
,DisplayName = Data.FlowDisplayName
,name
,RunID = Data.OriginRunId
,ErrorCode = Error.code
,ErrorMessage = Error.message
,success
,customDimensions
This query brings up all my flow runs (I use the time range in App Insights to narrow down time range). I also add | where success== False when I want just failed runs.
Flow Action Performance
let myEnvironmentId = 'ca99eaab-99a9-ef99-9b99-99ded999999f';
let myFlowId = '90911457-7dd5-453b-afb3-8f8f3374c599';
dependencies
| extend Data = todynamic(tostring(customDimensions.Data))
| extend DisplayName = Data.FlowDisplayName
| extend RunID = Data.OriginRunId
| extend Error = todynamic(tostring(customDimensions.error))
| extend ErrorCode = Error.code
| extend ErrorMessage = Error.message
| where name == 'Fail'
| where customDimensions.resourceId == myFlowId
| where customDimensions ['environmentId'] == myEnvironmentId
| project timestamp
,id
,DisplayName
,name
,operation_ParentId
,ErrorCode
,ErrorMessage
,success
,customDimensions
This query allows me to look at how a specific action has performed over a set time span, helping debug potential issues.
Flow Detail
let myEnvironmentId = 'ca99eaab-99a9-ef99-9b99-99ded999999f';
let myEnvironmentId = 'ca84eccb-78a7-ef84-9b20-84ded785680f';
let queryId='08585075362661487682209269813CU165';
(requests |union dependencies)
| extend Data = todynamic(tostring(customDimensions.Data))
| extend DisplayName = Data.FlowDisplayName
| extend RunID = Data.OriginRunId
| extend Error = todynamic(tostring(customDimensions.error))
| extend ErrorCode = Error.code
| extend ErrorMessage = Error.message
| where operation_ParentId == queryId or RunID == queryId
| where customDimensions ['environmentId'] == myEnvironmentId
| project timestamp
,id
,DisplayName
,name
,RunID
,operation_ParentId
,ErrorCode
,ErrorMessage
,success
,customDimensions
The above query allows me to view a specific flow run, with the request and dependencies union'd together.
What's Next
So this is a good starting point to track flow performance and identify failed runs. But what is really cool is you can easily add the App Insights data to Power BI:
Just click new dataset and it will open Power BI and create the connection and allow you to create reports with live data.
Below is auto generated, so you can do a lot more with some effort.
Top comments (9)
Brilliant post.
I enabled this against some Dev flows to test what data it logged. I found a few issues where a flow had errors, and the error node was either null, or had an error message in it which didn't match the error message in the flow.
Would be curious to know how you have found the accuracy of the error data.
Not going to lie I had lots of issues, mainly with missing runs or actions, though not seen any missing errors yet. I've reached out to Microsoft and they are investigating and promises resolution soon (even though in preview). There are so many different computations there was bound to be bugs, fingers crossed bugs will drop over time
Yeah 100% can't expect preview stuff to work straight away.
How did you reach out to MS? Is there a feedback type forum or are you raising it via some sort of company support channel?
I use the support channel for non preview as they normally don't do much for preview (except thank you for the feedback). Luckily have a contact through our Microsoft business partner.
Are you aware of blogs or release notes feeds from MS which would tell us whenever the make improvements to things like this?
Here is a new discussion I have raised in the Power Automate community to see if anyone or MS has any pointers
powerusers.microsoft.com/t5/Genera...
That's very strange, my flow id and name match and I get the right error in the dependency error. If we are both having lots of errors hopefully a few more will comment/like your discussion too, helping motivate Microsoft
Managed environment only (at some point)?
Booooo 😁
Love the detailing steps..