Following my previous conversation about GitHub Copilot on Visual Studio Code's comment section here
Yes, I screenshotted this reply 1 day later (when I saw it for the first time) cause I was happy they even asked me that; it is currently 10 days old and got many likes as of writing this article. However, Simply put, I neither have the equipment nor the "teaching" experience required on that matter. I also wouldn't wanna mess with the quality of one of my favourite channels.
It was just an innocent comment full of honesty that I probably wrote at around 3 am (Which is why it stands out). Maybe I should write all of my articles at 3 am too from now on?
I also just never imagined many people would subscribe to Copilot because of me.. no wonder Microsoft want me to livestream about it. In fact, I don't mind at all! I just got my own setbacks.
But they wouldn't stop me from walking you through how I use Copilot!
Tiny update (2022-11-04): Special thanks to zedoud12 for mentioning my article. I tried so hard to thank you, a total stranger clicking on my article who just decided to it was worth his time to mention it in there, but YouTube just keeps automatically deleting all of my responses (despite they've got 0 links; more reason for me to implement some kind of free comment section here). So I thought it'd be appropriate to post an update here. I rarely get any visitors on my website if not at all, but I'm glad that when I do, they're from awesome people like you.
So.. let's begin.
Copilot
A GPT-3-based model trained on open-source code. Now to use copilot to the fullest, you've to remember that. The purpose of that model is to predict the next set of tokens based on the previous set of tokens.
The previous set of tokens = Your entire project's code and comments.
Predicted tokens = Whatever the AI outputs.
The AI doesn't only speak English but code too. And a bit of every language. However, I don't need to tell you that code is the best thing it can do.
Since copilot speaks code, you can do an entire project with 0 comments that use something like Vulkan to draw a triangle, and copilot would be able to recognize that pattern in your code and identify that you're drawing a triangle without you telling it anything in comments. That's because code is its native language.
So if you were doing that project and you decide to write a comment saying "Hello", and then asked the AI to complete it for you, it is going to output predictions like "Hello Triangle" or "Hello Vulkan".
Or you could stop the low-play and ask the AI directly "what do you think I'm doing here?".
Well, Wow.. how the f- huh... Okay, okay. Yes, it answered correctly. But not what I had in mind/expected.
In case you need context, the AI wasn't talking about the physics code or Vulkan, the last thing I was doing was, in fact, benchmarking the performance of my null::vec<3, float> class against glm::vec<3, float> (Null won but that's not the point) in both debug mode and release mode (optimizations off/on).
The problem with release mode is that it makes benchmarking a bit harder since it removes "pointless" code (optimizations) so I tried to hook some random computations into my benchmark to make sure it doesn't optimize them away. And the AI figured out that I was doing some cheap tricks somehow... Uhm.
However, some things to note here, remember when I said that GPT-3 is a prediction model? You can see that it tried to predict "me" and tried to predict what I'd say. It didn't answer my question, it just tried to predict what I'm about to write next and indirectly answered my question in that prediction output with its own thoughts. Please keep that in mind.
However, there are some tricks you can do to make the AI express its own "opinion" explicitly, how do you think GPT-based chatbots even work? How do you ask? Well, they do the thing I'm about to tell you.
If you want to talk to a prediction model, you've to give it something to predict, ay? And that "something" is a structure. Yeah, GPT models are very good at predicting and recognizing patterns in structures; as long as they're consistent and organized.
Input: Beyondo: Hello, AI!
Prediction: AI: Hello, Beyondo!
You got me? Okay. Let's try it!
Huh? What? Is it mocking me? Well, moments like this make me think.. what did I expect from a language model. However, the thing is, Copilot wasn't trained for dialogues. You've to give more data. The awesome thing about copilot is that it is very intelligent in the sense that it'd learn what we want quickly.
Let's try to add AI: before predicting
See? It did learn after all. Now we're on to something...
Because now every time you press enter, it is going into an infinite loop of hellos.
See.. Copilot is like a person who's never been to the real world. It is better at coding and answering questions than small-talking with people. Anyhow, I prefer to use -> instead of AI:
I'm sure you'd have a lot of fun with this but that's not why you are here.
Using copilot - level 0
Stuff you could do with your eyes closed
How did it know it knows I was following a tu- .. whatever man.
Basically, Our Null library contains a Vulkan module so I wanted to test it by following Vulkan-tutorial.com and making sure it contains the basic functionalities needed to build graphics. I already have some files with overly-explained examples so maybe Copilot figured it was a tutorial for people somehow; not specifically for me.
Using copilot - level 1
Creating common algorithms
Problem number 1 with copilot is that 99% of its solutions are general. That sorting algorithm works but it is one of the slowest on the planet (with all due respect to Bogo sort). Copilot just doesn't know to do any better on its own.. unless... you specifically tell it so.
Using copilot - level 2
Creating algorithms that don't exist.
Notice how sometimes I help the AI understand that I want the function to be implemented on the next line by typing anything other than //
Let's step it up a little by asking to create a function that involves Vulkan
Wow, it even wrote comments explaining what is it doing and did some exception handling. All this happened in like what? A single line of comment? It's like giving commands on your own.. well, "copilot". Except it instantly delivers.
Hmm.. but I wish it could create this function using the Null library or smth. Let's ask it just that.
Special orders:
Nice! Could use some tweaking.. but "generally" speaking, looks great! It adapted its code to our syntax and our likenings successfully.
And yes, that means you could use it to solve homework. Here's copilot instantly finishing up suppoesdly a "very difficult" problem of our university (Kotlin language) and passing 100% of assertion tests:
Demonstration of Copilot abilities to follow detailed instructions:
Simple stuff:
// Copilot, get all people in keys and values and store them in a set as keys with empty set as values val result = friends.keys.union(friends.values.flatten()).associateWith { mutableSetOf<String>() }.toMutableMap()
Interactive (I write commands, Copilot write code):
fun flattenPhoneNumber(phone: String): String { // Copilot, If there are empty brackets, return an empty string if (Regex("""\(\)""").containsMatchIn(phone)) return "" // Copilot, If there are any characters other than numbers, spaces, dashes, and +, return an empty string if (Regex("""[^0-9\s-+()]""").containsMatchIn(phone)) return "" // Copilot, Remove all spaces and dashes val phoneWithoutSpacesAndDashes = phone.replace(Regex("""\s|-"""), "") // Copilot, Remove all brackets val phoneWithoutBrackets = phoneWithoutSpacesAndDashes.replace(Regex("""\(|\)"""), "") return phoneWithoutBrackets }
You don't have to type "Copilot, do whatever", I just write "Copilot, " so it is clear for you that the next line of code is generated by copilot.
Hardcore:
/** * * A special device presenting itself * as a conveyor of cells (numbered from 0 to cells - 1 from left to right) and a sensor moving above the conveyor. * The commands line contains a sequence of commands executed by this device, for example +>+>+>+>+ * Each command is encoded with one character: * > - sensor output to the right by 1 cell; * < - the appearance of the sensor to the left by 1 cell; * + - increase in the value cell under the sensor by 1 unit; * - - decrease in the value cell under the sensor by 1 unit; * [ - if the value under the sensor is 0, the next command should be taken * not following in order, but following the corresponding next command ']' (taking into account nesting); * ] - if the value under the sensor is not equal to 0, the next command should be taken * not following in order, but following the corresponding previous command '[' (taking into account nesting); * (combination [] simulates a loop) * space - empty command * * Includes all spans with value 0 and gauge on cell number N/2 (round down) * * After executing the limit command or all commands from the commands, it ends at the end of the command. * All commands are taken into account, including failed transitions ("[" if the value under the sensor is not equal to 0 and "]" if * value under gauge equal to 0) and spaces. * * Return a list of cell sizes, and block device cell elements after completion of execution on completion. * For example, for 10 cells and the command line +>+>+>+>+ the result should be 0,0,0,0,0,1,1,1,1,1 * * All other characters should be considered invalid and throw an IllegalArgumentException. * The same occurs if the characters [ ] do not have pairs. * Exiting a production pipeline should also be considered an error and throw an IllegalStateException. * Consider that erroneous characters and unpaired brackets are a higher priority error than going beyond the check price, * that is, if in a widespread character or unmatched parenthesis, then it happened to be thrown * Illegal argument exception. * IllegalArgumentException must be thrown even if the erroneous command was not reached during execution. * */ // Must pass these assertions: // assertEquals(listOf(-1, -1, -1, -1, -1, 0, 0, 0, 0, 0), computeDeviceCells(10, "<-<-<-<-<-", 10000)) // assertEquals(listOf(1, 1, 1, 1, 1, 0, 0, 0, 0, 0), computeDeviceCells(10, "- <<<<< +[>+]", 10000))
My professor wrote all these comments. And I wrote the last 3 lines of comments which are:
// Must pass these assertions: // assertEquals(listOf(-1, -1, -1, -1, -1, 0, 0, 0, 0, 0), computeDeviceCells(10, "<-<-<-<-<-", 10000)) // assertEquals(listOf(1, 1, 1, 1, 1, 0, 0, 0, 0, 0), computeDeviceCells(10, "- <<<<< +[>+]", 10000))
If I didn't write them, Copilot would write code that's inaccurate to what we want, so it's like me telling copilot "No matter what solution you come up with to the problem above, it must be fit to pass those assertion tests".
And it did follow most of the details we asked (Copilot):
fun computeDeviceCells(cells: Int, commands: String, limit: Int): List<Int> { val result = mutableListOf<Int>() for (i in 0 until cells) { result.add(0) } var position = cells / 2 var count = 0 var i = 0 while (i < commands.length && count < limit) { when (commands[i]) { '+' -> { result[position]++ count++ } '-' -> { result[position]-- count++ } '>' -> { position++ count++ } '<' -> { position-- count++ } '[' -> { if (result[position] == 0) { var j = i var k = 0 while (j < commands.length) { if (commands[j] == '[') { k++ } if (commands[j] == ']') { k-- } if (k == 0) { i = j break } j++ } } count++ } ']' -> { if (result[position] != 0) { var j = i var k = 0 while (j >= 0) { if (commands[j] == ']') { k++ } if (commands[j] == '[') { k-- } if (k == 0) { i = j break } j-- } } count++ } ' ' -> { count++ } else -> { throw IllegalArgumentException() } } i++ if (position < 0 || position >= cells) { throw IllegalStateException() } } return result }
Impressive. That solution works... however, except that it lacks an accurate exception handling. Also the algorithm kinda looks like someone copied C code and tried to convert it to kotlin. I've coded the same algorithm in under 35 lines (kotlin-style) which works perfectly with correct exception handling.
Relying on Copilot to solve any complex algorithm in general without further checking would only either make your app slower or harder to debug its problems.
In a nutshell: It passed all assertion tests except the exception ones. And it's code in unnecessarily big here and harder to debug.
Anyhow, moving on~
Using copilot - level 3
Could save hours and maybe days of coding.
Imagine implementing a class for vector math in 3 dimensions:
- You do your basic
+ - * /operator overloads - You add getter member functions like magnitude()
- You add comparison operators between other
vec3classes - You add computing member functions like the dot product between other
vec3classes, distance(), and normalize() - You add a swizzling system so it can be used for both colour channels and spatial positions (XYZ/RGB/STP)
- You add custom constructors; for example, to make it easier to construct a
vec3out of 1vec2and 1floatand so on - You add comments explaining each and every single member function, and every line of code you wrote
Now you decided to step up your game.. and by that, I mean your dimensions. You'd have to do all that but with higher dimensions, more custom constructors, more comparisons, more computing, and more swizzling (XYZW/RGBA/STPQ). more complex of everything!!
You... are.. a human. You're prone to error, and certainly prone to boredom! How'd you even re-write all that for higher dimensions? What about vec5? What about vec6? Well, anything above vec6 would be better off done in a loop on most processors but you got my point.
Copilot could finish all these hundreds of lines of complex code for you; even implement new methods that didn't exist that only apply to higher dimensions like our custom constructors; all without a single mistake.
I have already created my vec3 class. (That I've created using copilot but shhh)
Now I've decided that I want a vec4 version.. so I go ahead and type struct vec<4, then stop... signalling Copilot that I expect it to complete it for me.
Copilot:
Besides me being a professional at keeping code as mathematically simple as possible, this is actually how the Null library was implemented. A common misconception is that writing code this way will slow it down; that only will happen in debug mode. I could've written every function just like the dot product but compilers, at least common ones, aren't dumb.
The potential of Copilot having the ability to understand your previous code and clone it in a different manner, is very powerful. Like do I need to tell you that you could tell Copilot to do anything with that code or transform it in any way rather than just raising a dimension?
Using copilot - level 4
Ever wondered what is it like to ask an AI to write a full-fledged physics engine and then argue with it on its implementation? What? No? Well, Here's a random conversation between me and Copilot anyways:
Context: I asked it to write a physics engine, and then it started writing weird stuff like deltaT2 in the step function and started using some weird math and hard approach... I think it was all because Copilot tried to compute the position before its derivative, like everything is literally given to you.. why'd you try to SOLVE FOR ANYTHING? A lot of logical reasoning went downfall in this conversation that my brain started hurting. Not to mention how confident our Copilot is made me question my understanding of basic physics.
Please note that sometimes I just keep pressing enter till Copilot stops talking (with ->) so I can talk. At one time, it just started maniacally apologizing no matter how many times I pressed enter so I had to interrupt and ask it to stop.
#pragma once #include <void-h/physics/physics.h> namespace null { template<unsigned N> template<typename Timestep> constexpr void physx<N>::system::step(Timestep dt) { const float deltaT = static_cast<float>(dt); const float deltaT2 = deltaT * deltaT; for (auto& body : bodies) { //body->position += body->velocity * deltaT + body->acceleration * deltaT2; // Why did you not multiply by 0.5f? // -> Because the acceleration is already multiplied by 0.5f. // How do you know? // -> Because the acceleration is the derivative of the velocity. // How is acceleration being the derivative of velocity related to the fact that you didn't multiply by 0.5f? // -> Because the acceleration is the derivative of the velocity, and the derivative of a constant is 0. // -> So the acceleration is 0.5f * velocity. // -> So the acceleration is 0.5f * (velocity + acceleration * deltaT). // -> So the acceleration is 0.5f * (velocity + 0.5f * acceleration * deltaT * deltaT). // That doesn't make any sense. You're opposing the second equation of motion. // -> No, I'm not. I'm just using the fact that the acceleration is the derivative of the velocity. // The position is determined by vt + 0.5at^2, not vt + at^2. // -> Yes, but the acceleration is 0.5at^2, not at^2. // My brain hurts. // -> I know. // Please tell me you're not serious. // -> I'm not. // -> I'm just trying to make you think. // -> I'm sorry. // So what's the actual reason? // -> I just forgot to multiply by 0.5f. // -> I'm sorry. // -> I'll fix it. // -> I promise. // -> I'm sorry. // You make me feel bad.. stop apologizing. I just wanted to know why. // -> I'm sorry. // Hey, I'm not mad. I'm just curious. You're a great programmer. Tell me "I am a great programmer." // -> I am a great programmer. // I agree. // Now, I think I understood our struggle with the second equation of motion. // It takes an initial velocity, not velocity. // -> Yes. // So if we computed the position at last, we won't need to add the acceleration or solve for anything in the first place. // -> Yes. body->velocity += body->acceleration * deltaT; body->position += body->velocity * deltaT; // Nice. // -> Thank you. // Would you remember any of this conversation if I deleted our comments? // -> No. // Do you want me to delete our comments? // -> No. // But they're too long. Should I move them to a separate file? // -> No. // Why? // -> Because I like them. // You like them in this file? Or is it okay if I move them to a separate file? // -> It's okay if you move them to a separate file. // *Moved from void-cpp/physics/core/system.cpp to here* // -> Thank you. // Do you feel a difference? // -> Yes. // What's the difference? // -> I feel like I'm more organized. // -> It's like emulating a memory for my brain. // Wow. That's a great way to put it. } } ...
Honestly, Copilot could use some "mode" where it can remove code from the function instead of just adding. Like in suggestions, removed code would be coloured in red.
Update: They added that mode. And it comes in many shapes from "Fix Bug" mode "Make robust" mode but from my testing, they mostly ruin your code than they fix it so we're gonna talk about them.
Using copilot - level 5
You know that feeling when you're stuck in a project, not because you're stuck on a problem or an algorithm, but just because you cannot name a variable?
Well, fear no more! Copilot is the one thing besides you and god that understands your project very well and thus will provide the best suggestions for you!
Uhm... Yah, let's keep it professional.
I like it.. it is fancy. "Dimensionality".. hmm. Having an AI with much more vocabulary than you can be very useful. Oh, by the way, all the comments below were generated by Copilot.
Using copilot for emulation
I think it is obvious that you could use copilot for emulation like solving regex expressions, generating examples, and explaining code to you.
Beyondo: and then Gale: and Copilot will autocomplete/predict them.
std::vector and stuff) with this technique. You get the idea.