xfe.li

My last day using GitHub Copilot

2022-10-18 • Félix Dorn
This article is over 1 years old. I may think differently or it may be outdated.

For as long as I’ve been using the tool, I’ve always set aside the dilemmas that come along its use. Mostly for two reasons:

  • I believed that the tool increased my productivity by a huge, yet unknown, coefficient

  • I did not want to lose this newly found productivity

After using it for a month or so, I did my due diligence, sat down, and thought of all the ways Copilot was a bad thing.

  • Where does the code come from?

  • Is it legal? It’s in a grey zone for now, who can tell for sure if using copyrighted content to train AIs will be considered fair-use.

  • Do I trust Microsoft? Do I need to trust Microsoft?

  • "Prompt Engineering" &mdash has the word "engineering" really lost all meaning? — is a skill that may radically change / be outdated soon.

  • etc

All of these are reasonable questions to ask — at least in my opinion —, but there is no easy answer, apart perhaps for the “trusting Microsoft” one where I could confidently say no, but GitHub is less Microsoft-y than most things Microsoft acquired / built, maybe they deserve to be trusted: ever since they got acquired, good thing happened to the software ( surprisingly!)

Anyway, because these questions are not answerable easily, I looked at my untested hypothesis: “GitHub Copilot makes me code faster”.

Let’s test that. I took some pen and paper, wrote:

1Boilerplate
2Y:
3N:
4Logic
5Y:
6N:

… and started counting and coding.

It turns a lot of Go is boilerplate and it gets that right almost always (about 90% during a one-hour session):

1x, err := Func()
2if err != nil { // Added by GitHub Copilot
3return err
4}

This, by the way, requires to, every time:

  • wait a bit

  • get a correct if err ≠ nil suggestion

  • new line, tab, wait a bit

  • get a correct return err suggestion

  • new line, tab, wait a bit

  • get a correct }01

As opposed to iferr\t, using Goland.

You may think that Copilot is better at writing big chunks of code. But here too, Copilot fails badly as well, my reaction to completions goes two ways, either "absolutely not" or “sweet, this works”, then “hold up, this code doesn’t work / is vulnerable to race conditions / isn’t what I want / etc”.

I spend more time writing short comments, deleting chunks of code and asking Copilot to rewrite it that I’d be better of writing the code myself.

You may argue that I’m simply bad at "prompt engineering". Maybe.

Regardless, I convinced myself to stop using Copilot, if you’re not convinced too, here are a few links for you:


01

You might question my use of Copilot here. “Why did I wait for the } ?”, same reason I go back in my terminal history on a remote server to find a short command. If someone else (or old me) already did the work, why bother?