Several Common Issues When Using System.Text.Json

Preface

Previously, .NET Core did not have a built-in JSON library, so everyone used Newtonsoft's JSON library, and it was indeed quite nice to use. But since the official team released a standard library, it's more convenient and more worth using more often—at least you don't have to install the Newtonsoft.Json library via NuGet every time.

However, there are some issues in daily development use. This article records the solutions; discussion is welcome~

(There's a little easter egg at the end of the article)

Character Encoding Issues

By default, when System.Text.Json serializes, it escapes all non-ASCII characters, which often causes some of our non-ASCII characters to become forms like \uxxxx. This is not very friendly in many scenarios. We can configure character encoding to solve the escaping problem.

Example:

var testObj=new {
  Name = "测试",
  Value = 123
};

var json = JsonSerializer.Serialize(testObj);
Console.WriteLine(json);

Output

{"Name":"\u6D4B\u8BD5","Value":123}

When we serialize, we can specify a JsonSerializeOptions, and this JsonSerializeOptions has an Encoder we can use to configure the supported character encodings. Unsupported ones will be escaped, and by default only ASCII characters are supported.

So the solution is as follows:

var json = JsonSerializer.Serialize(testObj, new JsonSerializerOptions()
{
    Encoder = JavaScriptEncoder.Create(UnicodeRanges.All)
})
Console.WriteLine(json);

Output result

{"Name":"测试","Value":123}

Character Escaping Issues

For some text that contains HTML tags, even if all character sets are specified, they will still be escaped; this is for security reasons. If you feel escaping is unnecessary, you can also configure it by using JavaScriptEncoder.UnsafeRelaxedJsonEscaping.

Sample code

var testObj = new {
    Name = "测试",
    Value = 123,
    Code = "<p>test</p>"
};

var json = JsonSerializer.Serialize(testObj, new JsonSerializerOptions {
    Encoder = JavaScriptEncoder.Create(UnicodeRanges.All)
});
Console.WriteLine(json);

Output

{"Name":"测试","Value":123,"Code":"\u003Cp\u003Etest\u003C/p\u003E"}

As you can see, the HTML code was escaped, which clearly doesn't work

Solution

var json = JsonSerializer.Serialize(testObj, new JsonSerializerOptions {
    Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping
});

Output result

{"Name":"测试","Value":123,"Code":"<p>test</p>"}

Done!

Object Nesting Recursion Problem

I discussed this problem in detail in a previous article. At that time I used Newtonsoft.Json to solve it. But after I published that article on Blog Garden, an expert pointed out that the .NetCore standard library System.Text.Json also has a way to solve this problem, so I'll record it here as well~

First, create a few entity classes

internal class EntityBase {
    public string Id { get; set; }
}
internal class CrawlTask : EntityBase {
    /// <summary>
    /// 爬虫名称
    /// </summary>
    public string Name { get; set; }

    /// <summary>
    /// 创建这个爬虫的用户
    /// </summary>
    public User User { get; set; }

    /// <summary>
    /// 用户ID
    /// </summary>
    public string? UserId { get; set; }
}
internal class User : EntityBase {
    /// <summary>
    /// 用户名
    /// </summary>
    public string Name { get; set; }

    /// <summary>
    /// 用户创建的爬虫
    /// </summary>
    public List<CrawlTask> CrawlTasks { get; set; }
}

Then use mock data to reproduce the problem

//模拟数据
var crawlTask = new CrawlTask { Name = "爬虫名称", UserId= "0f3d4b2f-3b4e-4d08-8f4c-0009a316f041" };
var user = new User { Name = "用户名", CrawlTasks = new List<CrawlTask> { crawlTask } };
crawlTask.User = user;

// 输出
var json2 = JsonSerializer.Serialize(crawlTask);
Console.WriteLine(json2);

Output result: it throws an error directly

Unhandled exception. System.Text.Json.JsonException: A possible object cycle was detected. This can either be due to a cycle or if the object depth is larger tha
n the maximum allowed depth of 64. Consider using ReferenceHandler.Preserve on JsonSerializerOptions to support cycles. Path: $.User.CrawlTasks.User.CrawlTasks.U
ser.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.Us
er.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.User.CrawlTasks.Name.
...

We all know this is the object nesting recursion problem

So next, here's the solution right away

var json2 = JsonSerializer.Serialize(crawlTask,new JsonSerializerOptions {
    Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping,
    WriteIndented = true,
    ReferenceHandler = ReferenceHandler.IgnoreCycles
});
Console.WriteLine(json2);

The ReferenceHandler.IgnoreCycles approach was newly added in .NET 6, and it can achieve an effect similar to ReferenceLoopHandling.Ignore in Newtonsoft.Json.

The final output looks like this

{
  "Name": "爬虫名称",
  "User": {
    "Name": "用户名",
    "CrawlTasks": [
      null
    ],
    "Id": null
  },
  "UserId": "0f3d4b2f-3b4e-4d08-8f4c-0009a316f041",
  "Id": null
}

As you can see, the property that caused the nested recursion became null

However, this still differs a bit from the effect implemented by Newtonsoft.Json

In my previous article, the effect implemented by Newtonsoft.Json was

{
    "name": "test crawl123",
    "user": {
        "name": "string",
        "crawlTasks": null,
        "id": "0f3d4b2f-3b4e-4d08-8f4c-0009a316f041"
    },
    "userId": "0f3d4b2f-3b4e-4d08-8f4c-0009a316f041",
    "id": "4d52d83b-f3ec-47c6-ab26-e241c09c14d1"
}

As you can see, the crawlTask.user.crawlTasks property differs: System.Text.Json produces an array with a null object inside, while Newtonsoft.Json sets this property directly to null

By comparison, I prefer Newtonsoft.Json's implementation, because when parsing on the frontend you can clearly get a null object, rather than an array containing a null object (a bit of a tongue twister……

Afterword

To be honest, JSON handling is more convenient in dynamic languages like Python

For issues like those above, in Python you just add an ensure_ascii parameter (though C# isn't hard either)

For example

import json

test_obj = {
    "name": "测试",
    "value": 123,
    "code": "<p>test</p>"
}
print(json.dumps(test_obj, ensure_ascii=False))

Sometimes I also like to add an indent parameter, so the output JSON string looks nicer

json.dumps(test_obj, ensure_ascii=False, indent=2)

Output result

{
  "Name": "测试",
  "Value": 123,
  "Code": "<p>test</p>"
}

References

• #yyds Useful Content Roundup# Character Encoding in System.Text.Json: https://blog.51cto.com/u_11739124/4571054