Implementing round-robin API key calls using C# and MemoryCache to improve concurrency

Article Information

Title: Techniques for Using C# and MemoryCache Component to Rotate API Keys for Improved Concurrency

Abstract: This article explores how to utilize the MemoryCache component in C#, combined with concurrency programming techniques, to rotate through multiple API keys for enhancing system concurrency. With detailed explanations and example code, readers will learn how to effectively manage API key usage limits and optimize system performance.

Foreword

The use case requires using an API that has a rate limit of 5 requests per APIKey per second.

Initially, the simplest (but least efficient) approach was to wait 200 milliseconds after each call to avoid exceeding this limit.

However, this was too inefficient. Since we have many APIKeys available, we switched to concurrent requests instead. Let's do it!

This article serves as a brief note on this.

Approach

Store each APIKey's usage in memory.

C#'s MemoryCache component provides a key-value structure where expiration times can be set for each value.

I store the APIKey as the key, with the value being the number of times it has been used, and set an expiration time of 1 second.

This way, I only need to check if an APIKey's usage count is less than 5. If it is, I use it; if it exceeds 5, I read the configuration to get a new APIKey.

Using fluent-console

fluent-console is a C# Console application template I previously developed, providing a "modern console application development experience" scaffolding. It elegantly integrates various components like a web application, including dependency injection, configuration, logging, and other features.

Project repository: https://github.com/Deali-Axy/fluent-dotnet-console

This article will utilize components such as MemoryCache, making this template quite convenient. First, create a project using this template.

# Install Template
dotnet new install FluentConsole.Templates

创建项目

dotnet new flu-cli -n MyProject

Prepare Configuration Files

Prepare APIKeys in the configuration file

Edit the project's appsettings.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Debug"
    }
  },
  "AppSettings": {
    "Name": "The name of this app is fluent console demo",
    "Boolean": true,
    "DemoList": [
      "item1",
      "item2",
      "item3"
    ],
    "ApiKeys": [
      "apikey-xxx",
      "apikey-xxx",
      "apikey-xxx",
      "apikey-xxx",
      "apikey-xxx",
      "apikey-xxx",
      "apikey-xxx",
      "apikey-xxx"
    ]
  }
}

The fluent-console template has already handled the configuration-related logic, you can use it directly later.

Configuration Service

Edit the Program.cs file

Add the required services

services.AddMemoryCache();
services.AddScoped<ApiService>();

Next, write the code in ApiService

ApiService

Create the ApiService.cs file in the Services folder

First, inject the dependencies

using Flurl;
using Flurl.Http;
using Microsoft.Extensions.Caching.Memory;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Options;

namespace MyProject.Services;

public class ApiService {
  private readonly ILogger<ApiService> _logger;
  private readonly AppSettings _settings;
  private readonly IMemoryCache _cache;

  public ApiService(IOptions<AppSettings> appOptions, IMemoryCache cache, ILogger<ApiService> logger) {
    _cache = cache;
    _logger = logger;
    _settings = appOptions.Value;
  }
}

Encapsulating Key Management

I've written a method here to retrieve an available APIKey.

Since concurrent execution needs to be considered, the _cache object is locked.

The approach is simple as introduced above，so I'll translate it directly into code，and I've included very clear comments.

private string? GetNextApiKey() {
  lock (_cache) {
    foreach (var key in _settings.ApiKeys) {
      if (_cache.TryGetValue(key, out int count)) {
        // If this API Key exists in the cache，check its call count.
        // If it has reached the call limit，stop using this API Key and move on to the next one.
        if (count >= 5) {
          continue;
        }

        // If the call count hasn't reached the limit，increment the count and return this API Key.
        _cache.Set(key, count + 1, DateTimeOffset.Now.AddSeconds(1));
        return key;
      }

      // If the API Key is not in the cache，add it to the cache and return it.
      _cache.Set(key, 1, DateTimeOffset.Now.AddSeconds(1));
      return key;
    }
  }

  return null; // All API Keys have been used up.
}

Modify Properties

Since it was originally a single-threaded version, I directly used ApiKey to read the fixed APIKey configuration.

Now, I directly changed the original single ApiKey property to call the GetNextApiKey method.

private string? ApiKey => GetNextApiKey();

Method for Requesting API

The adjustment is minor, you only need to add one check.

The GetNextApiKey mentioned above returns null when no available APIKey is obtained.

Simply check if it's null; if there's no APIKey, wait for 1 second.

public async Task<ApiResponse> RequestData(string somedata) {
  var key = ApiKey;
  
  // All API Keys are exhausted
  if (key == null) {
    await Task.Delay(1000);
    return await RequestData(somedata);
  }

  var url = "https://api.dealiaxy.com"
    .AppendPathSegment("one")
    .AppendPathSegment("service")
    .AppendPathSegment("v1")
    .SetQueryParams(new {
      key, somedata,
    });

  _logger.LogDebug("API request: {url}", url);
  var resp = await url.GetJsonAsync<ApiResponse>();

  return resp;
}

Here, I used the Flurl library to implement URL building + network requests, and it's truly excellent!

Parallel API Calls

Here I wrote a closure, though a lambda could also be used.

Build a list of tasks, then use await Task.WhenAll to wait for all tasks to complete.

Finally, save the results to a JSON file.

public async Task RequestParallel() {
  // Prepare some data here, around tens of thousands will do
  var data = new string[];

  var results = new List<ResultData>();

  // Write a closure to call the API
  async Task MakeApiCall(int index) {
    var item = data[index];

    var resp = await RequestData(item);
    _logger.LogInformation("Calling API, data: {data}, status: {status}, message: {message}", item, resp.status, resp.message);
    results.Add(resp.data);
  }

  var tasks = new List<Task>();
  for (var index = 0; index < data.Count; index++) {
    var i = index; // Due to the closure, a new variable is needed in the loop to avoid issues
    tasks.Add(Task.Run(() => MakeApiCall(i)));
  }

  _logger.LogInformation("There are {count} data items, starting request", data.Count);
  await Task.WhenAll(tasks);

  _logger.LogInformation("Done, writing to file");
  await File.WriteAllTextAsync("results.json", JsonSerializer.Serialize(results));
}

Done!

Displaying Progress

A new problem emerged at this point.

With so much data, even with parallel execution, it would take some time.

Displaying the progress became an urgent requirement.

C# has a built-in IProgress interface, but it only allows setting a total and then directly updating the current progress. Although the MakeApiCall method has an index representing the task's sequence number, it's out of order during concurrent execution, so obviously, this index cannot be used to update the progress.

At this point, the only option was to add an int progress variable and increment it by +1 for each task.

It's really cumbersome. I'll just use the ShellProgressBar component. I used it before when writing a crawler in C#. For more details, see this article: C#<strong>爬虫</strong>开发小结

This component has a Tick mode that can accomplish this function.

Let's go to the code. Just executing bar.Tick in each task is enough, very convenient 👍

public async Task RequestParallel() {
  // Prepare some data here, tens of thousands should be enough
  var data = new string[];

  var results = new List<ResultData>();

  var bar = new ProgressBar(data.Count, "正在执行");

  // Write a closure to call the API
  async Task MakeApiCall(int index) {
    var item = data[index];

    var resp = await RequestData(item);
    _logger.LogInformation("调用接口，数据: {data}, status: {status}, message: {message}", item, resp.status, resp.message);
    results.Add(resp.data);

    // Update progress
    bar.Tick();
  }

  var tasks = new List<Task>();
  for (var index = 0; index < data.Count; index++) {
    var i = index; // Due to closure, a new variable needs to be created inside the loop to avoid issues
    tasks.Add(Task.Run(() => MakeApiCall(i)));
  }

  _logger.LogInformation("共有 {count} 个数据，开始请求", data.Count);
  await Task.WhenAll(tasks);

  _logger.LogInformation("搞定，写入文件");
  await File.WriteAllTextAsync("results.json", JsonSerializer.Serialize(results));
}

Good, good, good, now that's comfortable.