It was a wonderful experience sharing insights on Azure Open AI Application Landing Zone and architecture patterns for deploying Azure Open AI using Azure API management at the Toronto AI meetup earlier this week. We conducted a thorough review of all deployment patterns aligned with Azure’s Well Architected Framework WAF, and detailed a specific use case for implementing a charge-back model, logging, and security for enterprise-grade deployment.
In previous post we talked about a great feature of Azure Storage Account when it comes to cost management.
In this post I would like to propose an enhancement to blob inventory and share my experience with this feature.
Lately, a client assigned me to collaborate with a 3rd party vendor to review all their data assets. Initially, the task appeared straightforward, but it became complex when handling over 500 storage accounts and massive data volumes, such as hundreds of petabytes!
Here is how you can create a blob inventory rule:
This interface, does not accept a custom path for inventory report, for instance if you are working with a container and like to get the report for that particular container, the report gets generated to the directory within your container with year/month/day/Rule-Name structure. This is great until you work with a 3rd party vendor where you don’t want to give them full access to all 500 storage accounts in production.
In scenarios like this it would be great to have some flexibility to export the report to a separate storage account and container.
I would like to propose this to the azure storage account team and if you see this feature please vote for this feature to see this sooner than later.
Recently with one of my clients, I’ve been tasked to work with a 3rd party vendor to go through all data assets for my client. at first it seems an easy task but not when you deal with more than 500 storage accounts and huge data volume e.g. hundreds of petabytes!
In this post I would like to explain one great feature in azure storage account and in another post will share some required feature that needs to be added to this to make life easier in such scenarios
Azure Blob Storage Inventory is a functionality within the Azure Storage Account that gives a comprehensive view of the items in your blob storage. It generates a CSV file that catalogs all your blobs along with their related metadata. Depending on your setup, this file is produced daily or weekly and is saved in a designated blob storage container.
It is under Data Management section of you azure storage account on Azure Portal.
The Inventory is beneficial for a variety of applications such as tracking data growth over periods, data analysis, confirming encryption status, audit inspections, among others. It facilitates effective management and examination of your blob storage, particularly when dealing with substantial data volumes.
The inventory report incorporates information such as the blob’s name, the name of the blob container, the type and tier of the blob, the last time it was modified, when it was created, its content length, and other blob attributes. It offers high levels of customization, enabling you to add or remove specific blobs or containers as per your needs.
After creating your first inventory rule, you will have your inventory report file generated for each storage account. You can then process all of these files in a warehouse and generate nice report about your data assets using power bi.
In this post, I’m thrilled to present Azure Verified Modules (AVM), a groundbreaking venture that the AVM Team, Community, and Product Groups have been tirelessly developing. This initiative is set to revolutionize the deployment of Infrastructure-as-Code modules.
Welcome to my new blog. I decided to change the old layout to something new and different. I still need to migrate the old posts here which will do very soon.
Ehsan Eskandari 