Tsonnet #11 - Habemus comment!

Welcome to the Tsonnet series!

If you're just joining, you can check out how it all started in the first post of the series.

In the previous post, we explored concatenating strings a bit more:

Tsonnet #10 - There and back again… concatenating strings

Hercules Lemke Merscher ・ Feb 14

#tsonnet #jsonnet #compiler

An essential feature of any programming language is the option of adding comments to the source code.

How can we add memes to our code without comments?! :P

Jokes aside, let's add comments to Tsonnet.

Let's start with the TDD approach, adding the sample Jsonnet files and the cram tests:

diff --git a/samples/comments/comments.jsonnet b/samples/comments/comments.jsonnet
new file mode 100644
index 0000000..f040fa0
--- /dev/null
+++ b/samples/comments/comments.jsonnet
@@ -0,0 +1,6 @@
+"this is a string" // Oh, rly?! what a surprise!
+/*
+Now...
+Tell me something I don't know!
+¬¬
+*/
diff --git a/samples/comments/unterminated_block.jsonnet b/samples/comments/unterminated_block.jsonnet
new file mode 100644
index 0000000..464823f
--- /dev/null
+++ b/samples/comments/unterminated_block.jsonnet
@@ -0,0 +1,12 @@
+"this is code" /*
+This is a block comment
+.
+.
+.
+isn't
+going
+to
+end
+?
+?
+?
\ No newline at end of file
diff --git a/test/cram/comments.t b/test/cram/comments.t
new file mode 100644
index 0000000..884e99a
--- /dev/null
+++ b/test/cram/comments.t
@@ -0,0 +1,6 @@
+  $ tsonnet ../../samples/comments/comments.jsonnet
+  "this is a string"
+
+  $ tsonnet ../../samples/comments/unterminated_block.jsonnet
+  Unterminated block comment
+  [1]

We're going to test inline and multi-line comments at once, while also handling the unterminated block error for multi-line comments.

Let's look at the lexing implementation:

diff --git a/lib/lexer.mll b/lib/lexer.mll
index bebf28a..9245a14 100644
--- a/lib/lexer.mll
+++ b/lib/lexer.mll
@@ -16,11 +16,14 @@ let null = "null"
 let bool = "true" | "false"
 let letter = ['a'-'z' 'A'-'Z']
 let id = (letter | '_') (letter | digit | '_')*
+let inline_comment = "//" [^ '\n']* newline

 rule read =
   parse
   | white { read lexbuf }
   | newline { new_line lexbuf; read lexbuf }
+  | inline_comment { read lexbuf }
+  | "/*" { block_comment lexbuf }
   | int { INT (int_of_string (Lexing.lexeme lexbuf)) }
   | float { FLOAT (float_of_string (Lexing.lexeme lexbuf)) }
   | null { NULL }
@@ -55,3 +58,9 @@ and read_string buf =
     }
   | _ { raise (SyntaxError ("Illegal string character: " ^ Lexing.lexeme lexbuf)) }
   | eof { raise (SyntaxError ("String is not terminated")) }
+and block_comment =
+  parse
+  | "*/" { read lexbuf }
+  | newline { block_comment lexbuf }
+  | _ { block_comment lexbuf }
+  | eof { raise (SyntaxError ("Unterminated block comment")) }

The inline_comment pattern takes care of swallowing inline comments. For multi-line comments, we need to match the start /*, keep reading until we find the end */, and raise an error if the block is not properly terminated.

And there you have it! We've successfully added support for both inline and multi-line comments to Tsonnet.

---

Thanks for reading Bit Maybe Wise! /* TODO: Subscribe to get more Tsonnet posts (and yes, this TODO won't be ignored like the ones in our code) */

Photo by Ágatha Depiné on Unsplash