ChatGPT + Code Interpreter = Magic – @AndrewMayne
tl;dr: OpenAI is testing the power to run code and use third-party plugins in ChatGPT.
OpenAI has introduced that we’re creating plugins for ChatGPT that can prolong its capabilities. [Link] Plugins vary from third-party instruments like WolframAlpha and OpenTable, to our shopping plugin and Code Interpreter that may generate code, run code, add and obtain information starting from csv knowledge to photographs and consider the output all inside the ChatGPT interface.
At present the Code Interpreter runs Python with a small number of libraries. I’ve been enjoying with Code Interpreter and it’s been numerous enjoyable to see what it could do even with fundamental libraries.
Beside producing code, Code Interpreter (CI) can analyze the output and use it in one other perform. This implies that you could string collectively totally different sections of code, taking the output of 1 and feeding it to a different. The Pac-Man gif above was made by having CI use an algorithm to generate a maze, convert the maze into blocks, use an algorithm to search out the exit, make it seem like Pac-Man after which generate a gif.
Beforehand once I used ChatGPT to create code this concerned taking the output and placing it into one other atmosphere to check it. Now you are able to do numerous growth inside ChatGPT with out leaving the UI. If you add that to its means to research knowledge, equivalent to spreadsheets, after which doing features like graphing, you have got a really highly effective device for each coding and analysis.
Listed here are some examples of random experiments I ran enjoying round with ChatGPT’s Code Interpreter plugin.
Sound
Once I realized that ChatGPT understands numerous mathematical knowledge and CI can generate downloadable information, I requested ChatGPT to generate a Shepard tone (a sound phantasm that looks like it’s consistently getting greater in pitch.) ChatGPT confirmed the tone in a graph after which was capable of output it as a wav utilizing CI:
Shepard tone
Pushing the sound functionality slightly additional, I requested ChatGPT to generate a session of Conway’s Recreation of Life after which flip it into music. When the primary go sounded slightly too random, I requested it to make it extra pleasing. ChatGPT utilized a pentatonic scale to it and this was the end result:
Recreation of Life in Tune
Imagining a whole album of musical math (Fibonacci sequences, Mandelbrot units, Mobile Automata and extra!) I requested ChatGPT to generate an album cowl for the Recreation of Life music. Because it’s not related to DALL-E, I requested it to generate the picture from a body of the simulation.
Recreation of Life body (as album artwork)
QR Codes
Trying on the picture made me understand that ChatGPT might most likely generate a functioning QR code picture. I gave it a easy one and right here is the end result:
QR code
Trying on the Recreation of Life and QR code subsequent to one another had me questioning…might I get ChatGPT to create a Recreation of Life simulation that resulted in a QR code? (I informed it to cheat by working backwards from the QR code…) Sure, it could:
Recreation of Life QR code
Photographs
Within the weblog submit https://andrewmayneblog.wordpress.com/2023/03/23/chatgpt-code-interpreter-magic/ on ChatGPT plugins we present some cool use of Python libraries to do picture manipulation. This made me curious to see what else you can do with fundamental libraries.
Utilizing OpenCV and a easy face discovering algorithm (Haar Cascade Classifier) , CI was capable of create a perform to search out my face on this picture:
Face discovering with OpenCV
Since it could open and browse a file, that additionally implies that it might in idea use that knowledge to generate one thing new. So I requested ChatGPT to take a picture and convert it to an ASCII picture:
Uploaded picture
Transformed to ASCII
Generative Animation
The truth that CI might generate knowledge and output it into totally different kinds together with animated gifs had me questioning how far you can push this. First I requested it to make a bouncing block:
Bouncing block
Then a snowstorm:
Snowstorm
Subsequent a session of Recreation of Life (that is what impressed the QR code experiment):
[Game of life]
I attempted to get it to do that with a vectorized dice. That is as shut as I received earlier than making an attempt one thing else:
[Rotating cube]
OCR
It seems that Optical Character Recognition is without doubt one of the libraries in CI. I made a decision to check this out with a display screen seize from a web site. Positive sufficient, it really works. The power to mix OCR with a very succesful language mannequin introduces some fascinating potentialities for restoring older paperwork.
Calendar invitations
Though CI doesn’t have entry to the ics library, it could generate a calendar invite in plain textual content after which have ChatGPT reserve it as an .ics file.
Drawing
Whereas there’s been some fascinating examples of GPT-4 producing SVGs, I used to be curious to see what ChatGPT and IC might do with easy block pictures. I requested it to attract a cat and it created one thing cat-like. Simply to verify it wasn’t doing that from reminiscence, I requested for a blue high hat and pipe. Whereas not precisely the highest hat I had in thoughts – nonetheless a hat and blue.
A believable cat
A believable cat with a hat and a pipe
A cat within the hat with a pipe
Creating machine studying fashions (type of)
Whereas ChatGPT can’t load any machine studying libraries into CI (presently), we are able to do some fundamental statistical textual content era utilizing n-grams. On this instance I gave it one among my books, requested it to create a prediction algorithm after which predict the subsequent phrase in a textual content sequence. The outcomes gained’t have GPT-4 fearful anytime quickly, however have been nonetheless fascinating to see.
Subsequent phrase prediction
Simulation
I made a decision to revisit CI’s means to generate animated gifs and algorithms and had it make a maze, then have a dot discover its means out of the maze…then make the entire thing seem like Pac-Man:
Pac-Man simulator
Utilizing the Matplotlib library I used to be capable of get ChatGPT to make a simulation of the orbits of the inside planets and reserve it as a gif.
Orbits of planets
I then questioned if I might get it to simulate a sport of checkers. I managed to maneuver the items, however was too well mannered to take any of the opposite gamers’ checkers:
Checkers
What about chess? First I needed to get ChatGPT to create its personal chess library (a quite simple chess library of decreased and never fairly authorized strikes…) and use unicode for the items, after that, it was capable of simulate the beginning of a sport and reserve it as a .gif:
Chess
Right here’s the dialog I had with ChatGPT to make the chess simulation. As you’ll be able to see, I simply gave just a few directions and ChatGPT and CI did the remaining:
Wrapping up
These are simply a few of my explorations with ChatGPT and the Code Interpreter plugin. As extra individuals get entry I’m certain we’re going to see some actually mind-blowing purposes.
Printed by
Printed