Engineering, Design and Architecture

Despite the odds, Serilog and Application Insights are a fairly common combination. Let's dive in and find out if such partnership is well justified. Introduction Serilog  is an extremely popular structured logging library, with powerful DSL and serialization features. The  benefits of structured logging  are well known an widely appreciated, so if you're not convinced yet, do spend some time to read up on the topic. Application Insights  is a very popular APM SaaS offering from Microsoft on Azure, especially in the .NET world. Motivation Now, you might wonder -  why put those two together ? After all, not all great tech plays together well. And I completely agree with that. In fact, when starting a greenfield application, Elastic Search seems to be a better choice for storing and searching structured logs data. One of the obvious benefits would be the data ingestion pipeline speed. But logs are only part of the story, however. When we look at the AP...

Elastic Stack is quite capable of running blazing fast queries against your data. However, the more data you have, the more time it will take to query it. Most of the times, however, Elastic Stack will not be the mechanism you use for long time data retention. Consequently you will have to get rid of the old data. Let's find out how. Brief overview of steps required Pick a naming convention Create index lifecycle management policy Create an index template that connects the alias and the ilm policy Create an index  Associate an  alias  with the index Let's dig into the details. Pick a naming convention Depending on the size of your company, your topology, team and org structure, you will probably have a Elastic Stack deployment that is shared between several teams and used by multiple services (distributed tracing benefits).  Namespaces is a useful notion that we can leverage while naming ELK objects, even though the notion itself is not dir...

APM is a necessity in the world of micro services, but knowing that we need it , raises the next question: which of the many available products to choose and how ? Let's try and find out. Summary The following post tries to evaluate two APM alternatives from a perspective of a .NET Core application. As it turns out the feature set is quite different, so there are several trade offs to make. Disclaimer This overview is done under the following architectural constraints: APM agent is hosted in-process of the application The application is built on ASP.NET Core. The information is as of end of June 2019, both platforms are undergoing active development, so things will change! Key characteristics Evaluating and comparing each and every feature would of course be a complete and exhaustive approach. It would also be exhausting to read, and pretty expensive to conduct. Who would want to pay for that? Usually, for every investigation there is a time box, a limit on the bu...

Aplication Performance Monitoring in Elastic Stack Elastic APM is a separate component that takes input from APM agents and puts data to Elasticsearch, guide Elastic APM Agent for .NET is a library that basically runs the apm agent in-process, so there is no need to install it on the host machine. You might argue that nobody does the "installs" nowadays, and everything is built on containers, so we just run on docker docker pull docker . elastic . co / apm / apm - server : 7.0 . 1 or take a Helm chart  I would agree, using APM in-process and out-of-process results in a different architecture, with it's pros and cons, which we will talk about in another post. Apm-agent-dotnet writes data to APM Server, and not into Elasticsearch directly. Apm-agent-dotnet uses two mechanisms to provide automatic instrumentation: ASP.NET Core middleware and in particular ApmMiddleware DiagnosticSource  and in particular AspNetCoreDiagnosticListener, EfCoreDiagnosticLi...

EXEC : error : DIA SDK is missing at "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\DIA SDK". Make sure you selected the correct dependencies when installing Visual Studio. [C:\w\corefx\src\Native\build-native.proj]   Visual Studio Express does not include the DIA SDK. You need Visual Studio 2017 or 2019 (Community is free).   See: https://github.com/dotnet/corefx/blob/master/Documentation/building/windows-instructions.md#required-software C:\w\corefx\src\Native\build-native.proj(50,5): error MSB3073: The command ""C:\w\corefx\src\Native\build-native.cmd" x64 Debug Windows_NT outconfig netcoreapp-Windows_NT-Debug-x64" exited with code 1. Build FAILED. EXEC : error : DIA SDK is missing at "C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\DIA SDK". Make sure you selected the correct dependencies when installing Visual Studio. [C:\w\corefx\src\Native\build-native.proj] C:\w\corefx\src\Native\build-native.proj(50,5...