The Death of Comments

No this isn’t what you expect, comments are useful but we are as developers at all levels bad at them. Could it be that the future of documentation code comments and tests amalgamated through AI?

I’ve seen GPT4s code interpreter understand, describe and test my code and it’s only a matter of time before we move to a workflow and pipeline that takes a wholistic perspective.

Such a tool would be a guidebook of the code with unit tests right in the code, it would make source maps and plug into dev tools, it would be the unified guide to your codebase.

I think this is the death of comments but what do you think?

Comments

[Ben Sinclair]

I think AI will be fine at describing the what but not the why.
I also think that this lack will go on to encourage better separation and naming of weird business logic functions.

But I'm an optimist at heart.

[Ben Halpern]

I agree about this. But I also think that AI will be able to replace lots of forms of docs and comments, and we should anticipate this.

But the "why" comments should still be a thing, and it's up to people to get this right — which is a big maybe as to whether they will 😄

[Adam Crockett 🌀]

I would love to write a tool that could do this but getting traction I suspect it will take a team.

Are you familiar with Gherkin test language it’s sort of plain English

I think a tool that can scrape an entire codebase and wholisticly generate such English like tests would be the first component.

I think it should assess the health of the source and identify pain points, cheep tech debt etc. finally the tool must run decentralised owned by the org and never share data with any third party.

Anyway I feel a side side side project coming on 😂

[Adam Crockett 🌀]

What chatGPT4 said:

While Gherkin's traditional line-by-line structure is designed for parsers like Cucumber to read and execute tests, if you write in a more natural paragraph-based format, I can certainly help you interpret and convert it back to a standard Gherkin format.

However, note that software like Cucumber won't directly interpret freeform paragraphs as executable tests.

For example, if you provided:

"Imagine I have a calculator. On this calculator, I type in 50, followed by 70. After that, I hit the 'add' button. What I expect to see next is the number 120 displayed on the screen."

I could help translate this into:

Scenario: Adding two numbers together
    Given I have a calculator
    And I type in 50
    And I also type in 70
    When I hit the add button
    Then I should see 120 on the screen

So, while I can assist with interpreting more natural text, it's important to use standard Gherkin when you actually want to run the tests with BDD tools.

[Adam Crockett 🌀]

I am endeavouring to find out what it can do with my new tool Amalgam … what does it do? No idea 😂

[Thomas Schoffelen]

I don’t think GPT/LLMs could write/replace the comments in code. The whole reason to have comments is to add context that can’t be gleaned from the code itself. If you find yourself writing comments to explain what the code does, you should try to rewrite it in a cleaner manner.

That is not to say GPT couldn’t be helpful in documenting the code overall - especially listing out specific behaviours in a way non-technical end-users can understand. But in most cases this will be written long before a line of code is written, so most of this content would already exist.

[Adam Crockett 🌀]

For the moment yes I agree but I am talking more about the relationship we have with comments, it could be that in a decade we have AI assistants to co -author and add that context. 😊

[Adam Crockett 🌀]

So I started writing amalgam and it’s interesting, in rust I can request to openai to write some gherkin tests from the code of a sample project but it strikes me that if the criteria for the feature was misunderstood by the developer then you end up with tests for the wrong feature, the tests will always pass as they are always correct so I think tests need to be updated on commit only and also would need to compare the gherkin with the ticket that created them to see if the feature is actually doing the expected.

Some kind of new test is emerging here and it’s exciting!