Developing a Modern .NET Core Console Application: (1) Project Introduction and Usage Details of Dependency Injection/Configuration/Logging

Introduction

Recently, I've had quite a few scenarios requiring the development of small tools. Last time, I developed a Hive export tool using the Go language, and the experience was pretty good. It's just that I can't quite get used to Go's syntax, so this time I'm continuing to try developing small tools with C#.

The functionality of this small tool is simple: database data migration. However, that's not the main focus. The main point is to document the development process of a console tool that's more suited to the .NET Core ecosystem. 😃

In this article, I've defined a "modern console application development experience": the ability to elegantly integrate various components like a web application, which .NET Core's provided tools can achieve. I used components from the Microsoft.Extensions.* series, including dependency injection, configuration, and logging, and supplemented them with third-party components for features like environment variable reading and debugging.

The small tool in this article is very simple and designed for non-professional users who don't need command-line knowledge. Therefore, all functionality is controlled through a configuration file. If you want to develop a traditional CLI tool, you can use the System.CommandLine library.

Dependencies

The dependencies used in this project are as follows

<ItemGroup>
  <PackageReference Include="dotenv.net" Version="3.1.3" />
  <PackageReference Include="Dumpify" Version="0.6.0" />
  <PackageReference Include="FreeSql" Version="3.2.802" />
  <PackageReference Include="FreeSql.Provider.Dameng" Version="3.2.802" />
  <PackageReference Include="Microsoft.Extensions.Configuration" Version="7.0.0" />
  <PackageReference Include="Microsoft.Extensions.Configuration.Binder" Version="7.0.4" />
  <PackageReference Include="Microsoft.Extensions.Configuration.EnvironmentVariables" Version="7.0.0" />
  <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="7.0.0" />
  <PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="7.0.0" />
  <PackageReference Include="Microsoft.Extensions.Logging" Version="7.0.0" />
  <PackageReference Include="Microsoft.Extensions.Logging.Console" Version="7.0.0" />
  <PackageReference Include="Microsoft.Extensions.Options" Version="7.0.1" />
  <PackageReference Include="Serilog" Version="3.0.1" />
  <PackageReference Include="Serilog.Extensions.Logging" Version="7.0.0" />
  <PackageReference Include="Serilog.Extensions.Logging.File" Version="3.0.0" />
</ItemGroup>

Although it's a small console utility, I have set up a simple project skeleton for a smoother development experience.

Configuration

I initially wanted to use dotenv

Having extensively used dotenv when writing Python and Go, I find it very convenient

dotenv

Using it in C# is also simple. Install the dotenv.net library.

Execute DotEnv.Load(); to read the configuration from the .env file into the environment variables.

After that, you can directly load from the environment variables, for example, using the Environment.GetEnvironmentVariable() method.

Microsoft.Extensions.Configuration

For those who have used AspNetCore, this component should be familiar.

Originally, I planned to use dotenv for configuration, but in the end, I opted for JSON files combined with this configuration component. The reason is simple: it's convenient and easy to use.

After installing the necessary dependencies, execute the following code to initialize it:

var configBuilder = new ConfigurationBuilder();
configBuilder.AddEnvironmentVariables();
configBuilder.SetBasePath(Environment.CurrentDirectory);
configBuilder.AddJsonFile("appsettings.json", optional: false, reloadOnChange: false);
var config = configBuilder.Build();

This way, you get an IConfigurationRoot object.

Writing Configuration Files

The familiar appsettings.json—for anyone writing AspNetCore: DNA, moved!

{
  "Logging": {
    "LogLevel": {
      "Default": "Debug"
    }
  },
  "ConnectionStrings": {
    "Default": "server=host;port=1234;user=user;password=pwd;database=db;poolsize=5"
  },
  "DmTableMigration": {
    "Schema": "schema",
    "DbLink": "link_test",
    "Fake": true,
    "ExcludeTables": ["table1", "table2"]
  }
}

Define Strong-Typed Configuration Entity

For a better development experience, we use strong-typed configuration (强类型配置).

Create a new AppSettings.cs file.

public class AppSettings {
  public string Schema { get; set; }
  public string DbLink { get; set; }
  public bool Fake { get; set; }
  public List<string> ExcludeTables { get; set; } = new();
}

Registration Options

Here, the Microsoft.Extensions.Configuration.Binder library is used to implement configuration binding. When used in conjunction with IOptionsMonitor<T> or IOptionsSnapshot<T> for configuration injection, it enables configuration hot reload.

services.AddOptions().Configure<AppSettings>(e => config.GetSection("DmTableMigration").Bind(e));

In the initialization configuration above (configBuilder.AddJsonFile("appsettings.json", optional: false, reloadOnChange: false);), you can set reloadOnChange to true to enable automatic loading when the configuration file is modified.

If hot reload is not needed, the registration can be simplified:

services.Configure<AppSettings>("DmTableMigration");

In this case, the configuration is read at program startup, and subsequent configuration changes will not take effect. When injecting, only IOptions<T> can be used.

Injection Configuration

When injecting, write it like this

private readonly AppSettings _settings = options.Value;

ctor(IOptions<AppSettings> options) {
  _settings = options.Value;
}

ctor stands for constructor method (构造方法).

Logs

Logs are an essential part of any application.

I use the Microsoft.Extensions.Logging logging framework, but since its official providers do not support writing to files, I combined it with Serilog to log to files. It’s also possible to implement a custom file-writing provider—I’ll get around to that when I have time.

PS: Recommended logging components for the .NET Core platform include NLog and Serilog. I find Serilog more convenient, while NLog insists on XML configuration, which reminds me of the XML-dominated nightmares in Spring. No thanks.

Serilog Configuration

Just configure it directly in your code.

Log.Logger = new LoggerConfiguration()
  .MinimumLevel.Information()
  .WriteTo.File("logs/migration-logs.log")
  .CreateLogger();

Logging Configuration

Output logs to both console and Serilog

Serilog is also configured to write logs to files

services.AddLogging(builder => {
  builder.AddConfiguration(config.GetSection("Logging"));
  builder.AddConsole();
  builder.AddSerilog(dispose: true);
});

Dependency Injection

Using Microsoft.Extensions.DependencyInjection for dependency injection

AutoFac is also an option, reportedly offering more features. I haven't used it yet, but I plan to give it a try when I have time.

Registering Services

var services = new ServiceCollection();
services.AddLogging(builder => {
    builder.AddConfiguration(config.GetSection("Logging"));
    builder.AddConsole();
    builder.AddSerilog(dispose: true);
});
services.AddSingleton(fsql);
services.AddOptions().Configure<AppSettings>(e => config.GetSection("DmTableMigration").Bind(e));
services.AddScoped<MigrationService>();

Using Services

Services registered in the IoC container can be retrieved for use. Refer to the following code.

await using (var sp = services.BuildServiceProvider()) {
    var migrationService = sp.GetRequiredService<MigrationService>();
    migrationService.Run();
}

Services have different lifecycles. For example, scoped services can be injected within a scope created using the following code.

await using (var sp = services.BuildServiceProvider()) {
    using (var scope = sp.CreateScope()) {
        var spScope = scope.ServiceProvider;
        var service = spScope.GetRequiredService<MigrationService>();
    }
}

For other usage methods regarding dependency injection, please refer to the official documentation.

Debugging Tools

I also recommend the debugging tool Dumpify

It's very convenient to use. After installing the nuget package, simply append .Dump() to any object to output its structure.

I'm currently finding this tool quite useful~

Compilation & Publishing

For such a simple utility, I prefer to include the publishing configuration directly within the project settings.

For this particular tool, my publishing approach is: SingleFile + partial Trimmed (包含运行时的 SingleFile + partial Trimmed).

In practice, the packaged output is around 22MB. After compressing with zip, the final size is approximately 9MB, which is a reasonably good size control.

Edit the .csproj file with the following configuration:

<PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <PublishSingleFile>true</PublishSingleFile>
    <PublishTrimmed>true</PublishTrimmed>
    <TrimMode>partial</TrimMode>
    <PublishRelease>true</PublishRelease>
</PropertyGroup>

During trimming, I also encountered a minor issue. The default TrimMode is full, which minimizes the published program size as much as possible. In that case, the compiled output was about 17MB. However, I ran into problems with configuration binding and JSON serialization, so I switched to the partial mode. After that, the program ran smoothly.

Manual Binding Configuration (手动绑定配置)

Reference: https://github.com/dotnet/runtime/discussions/59230

Manually binding the configuration items is a bit troublesome, but it works.

services.AddSingleton<MyConfigurationModel>(provider => {
    var c = provider
        .GetRequiredService<IConfiguration>()
        .GetRequiredSection("MyConfiguration");
    var config = new MyConfigurationModel
        c["Url"] ?? "https://default",
        c["UserName"] ?? ""
    );
    config.Validate();  // member method to validate parameters
    return config;
});

PS: It's too much trouble; I'm not going to bother.

About AOT

Regarding the recently popular .NET 8 AOT solution, I have tried it as well, but the results were not ideal. Firstly, this small tool is built on a dependency injection framework, and AOT is inherently not friendly towards technologies like dependency injection that rely on reflection. So, when I tried using AOT, I found that the first step—configuration loading—already posed a problem.

After resolving the configuration loading issue (through manual binding), I encountered another challenge with JSON serialization, which is also based on reflection and difficult to handle.

I didn’t want to spend too much time developing this small tool, so I didn’t delve deeper into it. However, AOT seems to be a minor trending topic in the near future, and I may take the time to explore it further.

By the way, if you want to publish with AOT, you only need to add the following configuration:

<PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net8.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <PublishAot>true</PublishAot>
</PropertyGroup>

For more configuration details, please refer to the MSBuild configuration in the reference materials.

JSON Serialization in AOT and Trimmed Modes

AOT mode cannot use reflection, so simple JSON serialization is not possible. Instead, you need to use the JsonTypeInfo or JsonSerializerOptions approach.

The object to be serialized is this result class:

public class MigrationResult {
    public DateTime DateTime { get; set; } = System.DateTime.Now;
    public List<string> DeleteFailedTables { get; set; } = new();
    public List<string> MigrateFailedTables { get; set; } = new();
}

First, add a new subclass of JsonSerializerContext.

[JsonSourceGenerationOptions(WriteIndented = true)]
[JsonSerializable(typeof(MigrationResult))]
internal partial class SourceGenerationContext : JsonSerializerContext {
    
}

Using the JsonTypeInfo approach:

JsonSerializer.Serialize(result, SourceGenerationContext.Default.MigrationResult)

Using the JsonSerializerContext approach:

JsonSerializer.Serialize(result, typeof(MigrationResult), SourceGenerationContext.Default)

Both are similar and work fine.

For details, please refer to the documentation on Source Generation.

Miscellaneous

Retrieve All Tables of a Schema in DM Database

Obtain them from the all_objects view (or table?).

PS: Domestic databases like DM (达梦) have many pitfalls. Of course, Oracle is the same.

logger.LogInformation("获取Table列表");

var list = fsql.Ado.Query<Dictionary<string, object>>(
    $"SELECT OBJECT_NAME FROM all_objects WHERE owner='{_settings.Schema}' AND object_type='TABLE'");

var tableList = list.Select(e => e["OBJECT_NAME"].ToString() ?? "")
    .Where(e => !string.IsNullOrWhiteSpace(e))
    .Where(e => !_settings.ExcludeTables.Contains(e))
    .ToList();

logger.LogInformation("Table列表：{List}", string.Join(",", tableList));

C# New Syntax: Primary Constructor

This should be the correct name—Primary Constructor

When a class has only one parameterized constructor, this syntax can be used to simplify code.

Original code

public class MigrationService {
    AppSettings _settings;
    IFreeSql _fsql;
    ILogger<MigrationService> _logger;
    
    MigrationService(IFreeSql fsql, IOptions<AppSettings> options, ILogger<MigrationService> logger) {
        _settings = options.Value;
        _fsql = fsql;
        _logger = logger;
    }
}

New syntax

public class MigrationService(IFreeSql fsql, IOptions<AppSettings> options, ILogger<MigrationService> logger) {
    private readonly AppSettings _settings = options.Value;
}

Summary

Due to time and length constraints, this article can only briefly introduce the development approach for a "modern console application." During future exploration, there may be additional content added, and I will continue to supplement this article on my blog. If you are reading this on a platform other than Blog Garden or StarBlog, you can "view the original" to see the latest version of this article.

References

• MSBuild configuration - https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props
• JSON serialization - https://learn.microsoft.com/en-us/dotnet/api/system.text.json.jsonserializer.serialize
• SourceGeneration - https://learn.microsoft.com/en-us/dotnet/standard/serialization/system-text-json/source-generation