Asp-Net-Core Development Notes: Implementing Dynamic Audit Log Functionality

Introduction

Recently, I've been writing Go and Python a lot and haven't touched C# in a while. Returning to C# code actually gave me a sense of nostalgia~

Let's get back to the main topic.

In this era of rapid digitalization, every operation can have a profound impact on business, whether it's a simple data query or a modification to system configuration. In this context, audit logs are not just a means of following best practices but also a critical tool for ensuring data security, enhancing system transparency, and promoting clear accountability. By meticulously recording who did what to the system and when, audit logs help organizations track user activity, analyze system issues, and even provide necessary clues for investigation when security incidents occur.

There are various methods to implement audit logs, but how to efficiently integrate this functionality without disrupting the main business logic is a major challenge developers face. This article focuses on how to draw on the design philosophy of Aspect-Oriented Programming (AOP) to implement dynamic audit logging in ASP.NET Core applications with minimal code intrusion. AOP allows us to declaratively enhance code functionality through predefined patterns, such as logging, performance statistics, and security control, without modifying the actual business logic code.

This article will guide readers from understanding the concept to concrete implementation, and finally to data persistence processing, specifically how to use MongoDB, a powerful NoSQL database, to persist audit log data. Whether you are a newcomer just getting started with ASP.NET Core or a seasoned developer looking to add audit functionality to an existing project, this article will provide comprehensive guidance from theory to practice. Through this article, you will learn how to design and implement a flexible and scalable audit log system while maintaining minimal interference with the main business logic.

Let's begin this journey, step by step exploring how to integrate an efficient and flexible audit log mechanism into ASP.NET Core applications, leveraging AOP design philosophy to achieve highly decoupled and dynamically enhanced system functionality.

Audit Log Fundamentals

Definition and Usage

Audit logs help track user operations, data change records, and security analysis of the system.

Common types of audit logs include:

• Operational Auditing (操作审计): Records all user operations on the system, such as login, logout, and CRUD (create, read, update, delete) actions on data.
• Data Auditing (数据审计): Records details of data changes, such as capturing values before and after modifications.
• Security Auditing (安全审计): Records security-related events, such as failed login attempts and permission changes.
• Performance Auditing (性能审计): Records performance data for key operations, helping to analyze system bottlenecks.

The code in this article uses operational auditing as an example.

Model Definition & Key Information

Audit logs are a critical part of system security and administration, helping us understand what happened within the system, when it occurred, and who triggered it. To achieve this, audit log records need to include several key components.

• EventId is the unique identifier for each audit record. Just as each person has a unique ID number, each audit log entry has a distinctive EventId. This allows us to easily locate and reference specific audit events.
• EventType describes the type of event that occurred. This tells us what the record is about—whether it's a user login, a data modification, a permission change, etc. By examining the EventType, we can quickly grasp the core information of the record without needing to delve into the details.
• UserId is the identifier of the user who triggered the event. Recording the UserId in audit logs is crucial because it helps us track who is responsible for what actions. If an issue or improper behavior is detected, we can use the UserId to identify the responsible party.

Design Audit Log Model

AuditLog Class

Create a new AuditLog.cs class. Each field has comments, so I won't go into further detail.

public class AuditLog {
  /// <summary>
  /// 事件唯一标识
  /// </summary>
  public string EventId { get; set; }

  /// <summary>
  /// 事件类型（例如：登录、登出、数据修改等）
  /// </summary>
  public string EventType { get; set; }

  /// <summary>
  /// 执行操作的用户标识
  /// </summary>
  public string UserId { get; set; }

  /// <summary>
  /// 执行操作的用户名
  /// </summary>
  public string Username { get; set; }

  /// <summary>
  /// 事件发生的时间戳
  /// </summary>
  public DateTime Timestamp { get; set; }

  /// <summary>
  /// 用户的IP地址
  /// </summary>
  public string? IPAddress { get; set; }

  /// <summary>
  /// 被操作的实体名称
  /// </summary>
  public string EntityName { get; set; }

  /// <summary>
  /// 被操作的实体标识
  /// </summary>
  public string EntityId { get; set; }

  /// <summary>
  /// 修改前的数据，可根据实际情况以JSON格式存储
  /// </summary>
  public string? OriginalValues { get; set; }

  /// <summary>
  /// 修改后的数据，可根据实际情况以JSON格式存储
  /// </summary>
  public string? CurrentValues { get; set; }

  /// <summary>
  /// 具体的更改内容，可根据实际情况以JSON格式存储
  /// </summary>
  public string? Changes { get; set; }

  /// <summary>
  /// 事件描述
  /// </summary>
  public string? Description { get; set; }
}

Capture Audit Logs

IAuditLogService Interface

First, write an interface to handle audit logs. Using an interface helps keep the code clean and reusable, while also making it easier to extend or modify the audit logging logic in the future.

For simplicity, here we have only written a logging method for now.

public interface IAuditLogService {
  Task LogAsync(AuditLog auditLog);
}

Then, register it in the dependency injection container (assuming the implementing class is named AuditLogService):

builder.Services.AddScope<IAuditLogService, AuditLogService>();

This design maintains clarity and simplicity in the code, while also providing enough flexibility for potential future changes in requirements (such as altering the storage method for audit logs or adding new audit fields).

The specific implementation will be covered in the subsequent data persistence section.

Action Filter Approach

In ASP.NET Core, Action filters provide a powerful mechanism that allows us to insert custom logic before and after the execution of controller actions.

We can automatically capture user operations and data changes without modifying the existing business logic code. This approach fully utilizes the AOP (Aspect-Oriented Programming) concept, achieving minimal code intrusion.

Creating the AuditLogAttribute Class

Let's look at the code directly. By inheriting from the ActionFilterAttribute class, we can implement an Action filter attribute. In this design, EventType and EntityName are specified manually, while other attributes can be obtained through various methods.

public class AuditLogAttribute : ActionFilterAttribute {
  public string EventType { get; set; }
  public string EntityName { get; set; }

  public override async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next) {
    var sp = context.HttpContext.RequestServices;
    var ctxItems = context.HttpContext.Items;

    try {
      var authService = sp.GetRequiredService<AuthService>();

      // Before action execution
      var executedContext = await next();

      // After action execution

      // Get the current user's identity information
      var user = await authService.GetUserFromJwt(executedContext.HttpContext.User);

      // Construct the AuditLog object
      var auditLog = new AuditLog {
        EventId = Guid.NewGuid().ToString(),
        EventType = this.EventType,
        UserId = user.UserId,
        Username = user.Username,
        Timestamp = DateTime.UtcNow,
        IPAddress = GetIpAddress(executedContext.HttpContext),
        EntityName = this.EntityName,
        EntityId = ctxItems["AuditLog_EntityId"]?.ToString() ?? "",
        OriginalValues = ctxItems["AuditLog_OriginalValues"]?.ToString(),
        CurrentValues = ctxItems["AuditLog_CurrentValues"]?.ToString(),
        Changes = ctxItems["AuditLog_Changes"]?.ToString(),
        Description = $"Operation Type: {this.EventType}, Entity Name: {this.EntityName}",
      };

      var auditService = sp.GetRequiredService<IAuditLogService>();
      await auditService.LogAsync(auditLog);
    } catch (Exception ex) {
      var logger = sp.GetRequiredService<ILogger<AuditLogAttribute>>();
      logger.LogError(ex, "An error occurred while logging audit information.");
    }
  }
}

Considerations

• Exception Handling: Since logging should not affect the execution of the main business process, exception handling logic needs to be added. This ensures that even if an error occurs during logging, it does not interfere with normal business logic.
• Performance Issues: Although audit logs are recorded in an asynchronous method, if the logging process is slow, it may slightly increase response time. Consider using batch processing, caching, or writing to the database asynchronously. Alternatively, offload the logging logic to background tasks or message queues.

Obtaining the IP Address

You can get the IP address using the HttpContext.Connection.RemoteIpAddress property. However, if the application is deployed behind a proxy server (e.g., using a load balancer), the IP address obtained directly might be the proxy server's address, not the client's real IP address.

Therefore, I've encapsulated the GetIpAddress method.

private string? GetIpAddress(HttpContext httpContext) {
  // First check the X-Forwarded-For header (when deployed behind a proxy)
  var forwardedFor = httpContext.Request.Headers["X-Forwarded-For"].FirstOrDefault();
  if (!string.IsNullOrWhiteSpace(forwardedFor)) {
    return forwardedFor.Split(',').FirstOrDefault(); // May contain multiple IP addresses
  }

  // If no X-Forwarded-For header, or if you need the direct connection's remote IP address
  return httpContext.Connection.RemoteIpAddress?.ToString();
}

The code first attempts to retrieve the IP address from the X-Forwarded-For request header, which is a standard HTTP header used to identify the client's original IP address when the request passes through HTTP proxies or load balancers. If the request does not pass through a proxy, or if you want the proxy server's address, it falls back to using HttpContext.Connection.RemoteIpAddress.

The X-Forwarded-For header may contain multiple IP addresses (if the request passes through multiple proxies), so the code uses Split(',') to handle this scenario and only takes the first IP address as the client's real IP address.

Usage

After encapsulation, using this auditing functionality is very convenient. You can implement auditing simply by adding a line of code to your interface.

[AuditLog(EventType = nameof(SetSubTaskFeedback), EntityName = nameof(SubTask))]
[HttpPost("sub-tasks/{subId}/set-feedback")]
public async Task<ApiResponse> SetSubTaskFeedback(string subId, [FromBody] SubTaskFeedbackDto dto) {}

Manual Recording Method

While using Action filters is an efficient automated approach, there are situations where more fine-grained control over audit log recording is required. In such cases, you can only modify the interface code and incorporate audit log recording within the business logic.

Although this method requires direct modification of the business code, it provides maximum flexibility and control.

There's nothing special about this code—simply call the LogAsync method of IAuditLogService within the interface to record audit logs.

Sharing Data Through HttpContext

Some parameters are difficult to obtain automatically within an ActionFilter, as they are often related to business logic. In such cases, HttpContext serves as an ideal bridge.

We can store temporary data, such as a snapshot of the data before an operation, in HttpContext.Items. Then, we can access this data in filters to complete the audit log recording. This approach not only maintains code decoupling but also allows us to flexibly share data across different parts of the application.

HttpContext.Items is a key-value pair collection that can be used to share data within the lifetime of a single request.

Thus, the code in the interface would be:

HttpContext.Items["AuditLog_OriginalValues"] = item.FeedbackId;
HttpContext.Items["AuditLog_CurrentValues"] = dto.FeedbackId;
HttpContext.Items["AuditLog_Changes"] = $"更新反馈结果 {item.FeedbackId} -> {dto.FeedbackId}";

Notes

• Ensure that the keys used in the business logic and AuditLogAttribute (such as AuditLog_OriginalValues) are unique and consistent to avoid potential conflicts. It is best to encapsulate these constants within a class for better management.
• If the business logic is abstracted to the service layer, you need to inject IHttpContextAccessor to access HttpContext. This service can be registered via services.AddHttpContextAccessor().

Log Persistence (日志持久化)

Effective persistence of audit logs is crucial for ensuring long-term security and compliance.

Choosing a Storage Solution

When selecting the most suitable storage solution, multiple factors need to be considered, such as the importance of the data, query frequency, cost, and the complexity of maintenance.

Relational Database (RDS)

Relational databases, such as MySQL, PostgreSQL, etc., are widely recognized for their stability and maturity. They provide strict data integrity guarantees and powerful capabilities for complex queries, making them suitable for audit logs that require complex analysis and reporting.

• Advantages: Structured data, support for complex queries, mature management tools.
• Disadvantages: Relatively higher cost, may require complex architecture to support large-scale data.

NoSQL Database

NoSQL databases, such as MongoDB, Cassandra, etc., offer flexible data models and good horizontal scalability, making them suitable for audit logs with highly variable structures or massive data volumes.

• Advantages: High scalability, flexible data model, fast write speeds.
• Disadvantages: Relatively limited query functionality, weaker data consistency model.

File System

Directly writing audit logs to the file system is the most straightforward storage method, suitable for scenarios where log volume is not extremely high or query requirements are not demanding.

• Advantages: Simple implementation, low cost, easy to migrate.
• Disadvantages: Inconvenient for querying and analysis, difficult to manage a large number of log files, limited scalability.

Each storage solution has its applicable scenarios. Therefore, the choice of which solution to use should be made by comprehensively considering specific needs and available resources. For an audit log system that requires fast writes and high scalability, a NoSQL database is a good choice.

Therefore, this article chose MongoDB to record logs.

Choosing MongoDB as the storage solution for audit logs is not only because of its high performance and scalability but also because it supports a flexible document data model, making it simple to store unstructured or semi-structured audit data.

Implementing AuditLogMongoService

Using MongoDB in C# is very straightforward.

First, you need to add the MongoDB.Driver NuGet package

dotnet add MongoDB.Driver

Let's jump into the code,

public class AuditLogMongoService : IAuditLogService {
  private readonly IMongoCollection<AuditLog> _auditLogs;

  public AuditLogMongoService(string connectionString, string databaseName) {
    var client = new MongoClient(connectionString);
    var database = client.GetDatabase(databaseName);
    _auditLogs = database.GetCollection<AuditLog>("audit_logs");
  }

  public async Task LogAsync(AuditLog auditLog) {
    await _auditLogs.InsertOneAsync(auditLog);
  }
}

Prepare Connection Strings & Register Services

To avoid hardcoding, place connection strings in the configuration file (appsettings.json).

"ConnectionStrings": {
  "Redis": "redis:6379",
  "MongoDB": "mongodb://username:password@path-to-mongo:27017"
}

Register Services

builder.Services.AddSingleton<IAuditLogService>(sp => new AuditLogMongoService(builder.Configuration.GetConnectionString("MongoDB"), "db_name"));

Done~

Deploying MongoDB

Here's how to deploy MongoDB. I'm using Docker here, which is very convenient.

version: '3.1'

services:

  mongo:
    image: mongo:4.4.6
    restart: always
    volumes:
      - ./data:/data/db
    environment:
      MONGO_INITDB_ROOT_USERNAME: username
      MONGO_INITDB_ROOT_PASSWORD: password
    ports:
      - 27017:27017

  mongo-express:
    image: mongo-express
    restart: always
    environment:
      ME_CONFIG_MONGODB_ADMINUSERNAME: username
      ME_CONFIG_MONGODB_ADMINPASSWORD: password
      ME_CONFIG_MONGODB_URL: mongodb://username:password@mongo:27017/
    ports:
      - 8081:8081

Using docker-compose to orchestrate the deployment, mapping ports 27017 and 8081.

You can access the mongo-express web service on port 8081.

How to View Logs

• Use MongoDB Compass software to view data
• Use the mongo-express service to view data on the web

Summary

Although it's a relatively simple function, using AOP to implement it still feels quite satisfying. I have to admit that AspNetCore's functionality is indeed rich~