System.Text.Json in action with practical example in VB.net

XML is almost 25 years old and it is now a format that is widely-known to the point that also non tech-savvy people have at least a vague notion of it. The format was originally meant to encode documents in a format that is, of course, machine-readable, but that is friendly enough to also be human-readable.
It has been criticised for its complexity and the difficulty to exchange data structure between application.

For this reason, the format known as JavaScript Object Notation (JSON) came to prominence as a less verbose (albeit less human-readable as well) alternative for data interchange.
In this article I am going to show the creation of a JSON data structure to be used to programmatically create (or modify) an item on a WikiBase instance using the WikiBase API with VB.net.

JSON in .NET

Before NET Core 3.0 and .NET Framework 4.7.2 using JSON in .NET was much more complicated than, say, XML. The choices were to use the somewhat limited JavaScriptSerializer class, that was specifically designed for AJAX-enabled applications, or move to the popular Json.net library.
Starting with the above-mentioned versions, however, .NET finally started to offer a serious implementation of classes intended for JSON manipulation.
This has been announced in the .NET Blog, in a post called Try the new System.Text.Json APIs that I suggest you read if you like to know more about the implementation.
In the rest of this article I will show you a practical use of this new component.

The problem

For a project of mine, I need to massively create and edit items into a WikiBase-enabled MediaWiki installation. WikiBase is the extension that powers WikiData, but can be used on custom wiki site as well.
Any WikiBase-equipped site (WikiData included, of course, if you are wondering) allows interaction through WikiBase-specific API.
To programmatically access and edit MediaWiki content I already used the excellent DotNetWikiBot framework so I decided to expand it, using its already existing infrastructure to add WikiBase-specific functions (work is still underway on this project, but I will eventually release it as opensource, ideally integrating it into existing code).

Data into WikiBase are easily accessible as a JSON formatted data by accessing the EntityData special page like, for instance:

https://www.wikidata.org/wiki/Special:EntityData/Q96482610.json

When using the API, JSON data are of paramount importance to create or edit entities or part of them. Thus, handling JSON into my .NET application was crucial.

My first task was to create a set of Properties into the WikiBase instance, so I needed to use the wbeditentity module following the example

Create a new property containing the json data, return full entity structure
api.php?action=wbeditentity&new=property&data={"labels":{"en-gb":{"language":"en-gb","value":"Propertylabel"}},"descriptions":{"en-gb":{"language":"en-gb","value":"Propertydescription"}},"datatype":"string"}

The data parameter containing the JSON data.

The solution

My application is a form-based one as I need to easily interact with the interface and be able to read and store results in various controls.
So first of all I needed to add the System.Text.Json (in contrast, this library is already part of the .NET Core 3.0) via the NuGet package.

This library allows object serialization, but this is too complex for my needs, as I would prefer manipulating the DOM without the need of creating corresponding classes in my code. While the solution might seem less elegant than having the full JSON structure created as .NET structure in classes and then simply serialize them into JSON with a single command, the reality is that the sheer code needed to create all the classes would have been much more than the one needed to directly handling the JSON DOM.
That being said, it remains the fact that JSON serializer is very useful and might be object of a future article.

First steps

So the first steps to me were to familiarize with the Utf8JsonWriter method so I started converting the C♯ example to the following VB.net example:

Pay special attention at the Writer.Flush() method that can easily be overlooked, but that is critical for the stream to be filled with actual data.
As this worked as intended, I could proceed with the final version of the subroutine

The practical code

In order to create a Property in Wikibase you need to provide JSON data as a parameter to the API call to set the labels (in the proper language), the related descriptions and [datatype](https://www.wikidata.org/wiki/Help:Data_type) defining the type of the data the property will contain.

An example would be:

{
  "datatype": "string",
  "labels": {
    "en": {
      "language": "en",
      "value": "Label"
    },
    "it": {
      "language": "it",
      "value": "Etichetta"
    }
  },
  "descriptions": {
    "en": {
      "language": "en",
      "value": "Description"
    },
    "it": {
      "language": "it",
      "value": "Descrizione"
    }
  }
}

With labels and descriptions in English and Italian.

I started by creating an helper class like this:

Class LabelsWithDescription
    Public ReadOnly Property Culture As String
    Public ReadOnly Property Label As String
    Public ReadOnly Property Description As String

    Sub New(Culture As String, Label As String, Description As String)
        Me.Culture = Culture
        Me.Label = Label
        Me.Description = Description
    End Sub
End Class

Please note that this class cannot be serialized as it is: it holds all the necessary information, but not in the right structure. So it will only help creating the structure. As mentioned above, creating the proper structure in VB and then serialize it would have been more complicated and time consuming in comparison to writing the DOM directly. At least this is the case for the current example, of course this depends on the situation and the reader must evaluate this on a per-case basis.

Let's start to initialize the objects:

Sub New(IndentOutput As Boolean)
    Dim WriterOption As JsonWriterOptions

    With WriterOption
        .Indented = IndentOutput
    End With

    MemStream = New System.IO.MemoryStream
    JSONWriter = New Utf8JsonWriter(MemStream, WriterOption)
End Sub

We are creating here

• a Utf8JsonWriter that actually format and writes data
• its WriterOption (needed to chose if indent the cose for readability, or minify-it with no spaces) and
• the MemoryStream holding result.

An now, finally, the function doing the job:

Function PrepareNewItemData(ItemLabels As List(Of LabelsWithDescription), DataType As String) As String
    With JSONWriter
        .WriteStartObject()
        .WriteString("datatype", DataType)

        .WriteStartObject("labels")
        For Each Label As LabelsWithDescription In ItemLabels
            .WriteStartObject(Label.Culture)
            .WriteString("language", Label.Culture)
            .WriteString("value", Label.Label)
            .WriteEndObject()
        Next
        .WriteEndObject()

        .WriteStartObject("descriptions")
        For Each Label As LabelsWithDescription In ItemLabels
            .WriteStartObject(Label.Culture)
            .WriteString("language", Label.Culture)
            .WriteString("value", Label.Description)
            .WriteEndObject()
        Next
        .WriteEndObject()

        .WriteEndObject()
        .Flush()
    End With
    Return Encoding.UTF8.GetString(MemStream.ToArray())
End Function

The first couple of commands basically start writing

{
  "datatype": "string",

The tricky parts comes afterwards where the method .WriteStartObject("labels") output the fragment

  "labels": {

in the JSON structure, allowing other content to be added to this "node".
The two For Each basically uses the same strategy to create an Object with the name of the language and the related Labels, in the first cycle, and Descriptions, in the second cycle.

The final .WriteEndObject() basically closes the whole data fragment and the very important .Flush() commits actual data to the stream that can be converted to string with .GetString.

Now we can put everything to work with:

Dim JSONTest As New JSONEditor(False)
Dim LabelsList As New List(Of JSONEditor.LabelsWithDescription)

LabelsList.Add(New JSONEditor.LabelsWithDescription("en", "Label", "Description"))
LabelsList.Add(New JSONEditor.LabelsWithDescription("it", "Etichetta", "Descrizione"))

MessageBox.Show(JSONTest.PrepareNewItemData(LabelsList, "string"))

The output will be

{"datatype":"string","labels":{"en":{"language":"en","value":"Label"},"it":{"language":"it","value":"Etichetta"}},"descriptions":{"en":{"language":"en","value":"Description"},"it":{"language":"it","value":"Descrizione"}}}

our intended JSON payload as above, but in minified format.

Conclusion

System.Text.Json is intended to be a easy-to-use, fast and integrated alternative to third-party JSON editors. This first adventure using it was a bit bumpy mainly because of the lack of detailed documentation and real examples on the internet, basically, I guess, because it is very new and not yet much used.
Nevertheless it seems to me using is is really worth the minor pain of searching and experimenting.
With this little story of mine I hope having helped other users to get start with it a little faster than I originally did.