Somehow Microsoft build, Software companies, and cloud providers’ annual developer meetings are back. Even the virtual nature of the event is now an annual tradition, but I hope it can be abolished next year. But one of the traditions that hasn’t been abandoned is to take advantage of Build’s opportunity to announce new developments at the forefront of data and analytics. This year, Microsoft didn’t make any breakthrough news in itself, but it does make a number of announcements about new features and new service layers for BI and database products in the cloud.

Streaming data and aggregation

First of all, Microsoft Power BI announcement. These include Power BI features. data flow (Cloud-based incarnation Power queryHandles streaming data sources that begin with). Azure IoT hub And Azure event hubI haven’t actually used this feature yet, but the team promised to make streaming data and real-time analytics just like batch data, and traditional analytics is only for BI specialists and data engineers. Not for business users.Also, Event Hubs Apache KafkaCompatibility mode raises the question of whether this feature will one day work on open source streaming event platforms.

Also interesting is a feature called automatic aggregation. To understand this feature, you need to know the basics. Aggregation function That is the basis. The basic functionality allows you to pre-compute measure aggregates, which gives you excellent performance, especially for data sources to which Power BI is connected. Direct query, Fetch data from the backend data source.

Aggregates allow Power BI to avoid going to the back end for general summary data and reserve that approach instead of a relatively small set of detailed data. The problem with aggregation is that when you model the data that Power BI queries, you need to design them. The good news here is that the auto-aggregate feature automatically builds these based on the query patterns actually observed, and the algorithm improves as we collect more such observations.

Previews of both streaming dataflows and auto-aggregates will begin in July. Power BI PremiumPremium is currently available for an additional $ 10 per user per month. Power BI Pro, This allows these features to be relevant to more customers. Meanwhile, another premium feature, the Power BI Deployment Pipeline, has a developer-oriented automation API that enables tools such as: Azure DevOps, GitHub, And Azure pipeline Automate the deployment of Power BI assets. The automation API is now available.

This is a lot of Premium-only features, and the Premium-Per-User price level can mean more of those features. Meanwhile, Microsoft has released a very cool new feature that allows you to embed Power BI reports. Jupiter Notebook as an open source Python package. It’s also compatible with Power BI Pro, including the version available as a free trial. Below is an image of a Jupyter notebook with Power BI built in.

Cosmos DB: serverless, cached, encrypted, free (r)

On the database side of the world, Microsoft has a lot of news. Cosmos DB, And various bits of MySQL and Postgres.

First, Cosmos DB, a large multi-model NoSQL database managed by Microsoft, takes the history of cloud natives one step further. Serverless An option to general availability for all of Cosmos’s many APIs. The serverless version of Cosmos DB is suitable for what Microsoft calls a “spiked traffic pattern” because it can be used without provisioning an explicitly sized cluster. Even for customers who are accustomed to sizing clusters, the serverless option basically provides maintenance-free automatic sizing. After all, the serverless belief is to delegate the responsibility of distributing the resources you need, when you need them, to the service.

Cosmos also has many features that can be appreciated by traditional relational database experts. These include integrated caching, role-based access control (RBAC), and a feature borrowed from Always Encrypted. SQL server And Azure SQL database Fully encrypt the data in your application before it is stored in the database. The integrated cash service is of particular interest because it not only improves performance, but also saves on the surface costs by reducing overall database reads. According to Microsoft, caching reduces the cost of read-heavy workloads and improves performance by up to 96% and 300%, respectively.

Cosmos DB is also expanding its free tier. The new offering provides developers with 1,000 RU / s (request units per second) provisioning throughput and 25 GB of free storage each month. And speaking of free, Microsoft has been offering the Cosmos DB emulator for some time. This allows developers to run code against a working Cosmos DB instance without paying for cloud services. The problem was that the emulator only works on Windows machines, but Microsoft today announced a Linux version of the emulator that also works on the Mac. Similar to Cosmos DB Serverless, extended free tiers are generally available as well as RBAC. Meanwhile, the integrated cache, Always Encrypted, and Linux emulator are in preview.

OSS RDBMS, PDQ

Return to the world of relational databases and get free for 12 months Azure database for PostgreSQL And Azure Database for MySQL Flexible ServerWill be available in June and will be offered free for up to 750 hours per month for the first 12 months with a free Azure account. further, Azure Database for PostgreSQL-Hyperscale (Citus)The new base layer in the preview stage provides entry-level functionality based on a single-node deployment. What’s called “hyperscale” based on a single node looks awkward to say the least, but think of it as a working endpoint for developing and testing code in the same way as a Cosmos DB emulator. Is probably the best. The difference is that Microsoft offers the option to add worker nodes and scale them up later so they can be put into production in-place.

Builds may be used by product teams to launch major new versions or releases of a product. At other times, it is used as a convenient startup time frame for small features. In the data world, this year’s build looks like the latter case, but stacking up all the details creates quite a few new features and features. In some cases, that’s the best way to do it.

Microsoft is a customer of Blue Badge Insights, Brust’s advisory firm. He is also the Microsoft Data Platform MVP.