Pairing with GPT-4 · Fly
GPT-4 will be useful for newbie and senior Ruby builders, nevertheless it does have limitations. It will not write your entire software program for you, however it should level you in a helpful course, particularly in the event you desire studying by doing.
Let’s take a look at how effectively GPT-4 pairing works by choosing a straightforward, however much less well-known venture with some edge circumstances: downloading a RubyGem, parsing the docs by way of YARN, and dumping them right into a Sqlite database.
The Initial Prompt
Create a Ruby script for me that downloads a RubyGem, parses the docs via YARD, and dumps them into a Sqlite database that’s named
#{gem-name}-#{gem-version}.db
. The database should also include information from the Gem spec.
GPT-4 usually starts its responses with an overview of what it’s about to do, then shows how it solves the problem, followed by recapping and summing up the solution.
GPT-4 Response
To create a Ruby script that downloads a RubyGem, parses the documentation using YARD, and stores the data in a SQLite database named with the gem name and version, you will need to follow these steps:
- Install the required gems:
gem install yard
gem install sqlite3
gem install bundler
- Create a new file named
gem_doc_parser.rb
and paste the following code:
require 'yard'
require 'sqlite3'
require 'bundler'
def download_and_parse_gem(gem_name, gem_version)
gem_path = "#{gem_name}-#{gem_version}"
# Download the gem
Bundler::CLI.start(["gem", gem_name, "--version", gem_version])
# Parse the gem documentation using YARD
YARD::Registry.clear
YARD::Parser::SourceParser.parse(gem_path)
end
def create_database(gem_name, gem_version)
db = SQLite3::Database.new("#{gem_name}-#{gem_version}.db")
# Create tables for storing gem spec and documentation
db.execute <<-SQL
CREATE TABLE IF NOT EXISTS gem_spec (
id INTEGER PRIMARY KEY,
name TEXT,
version TEXT
);
SQL
db.execute <<-SQL
CREATE TABLE IF NOT EXISTS gem_docs (
id INTEGER PRIMARY KEY,
path TEXT,
type TEXT,
doc TEXT
);
SQL
db
end
def store_gem_spec(db, gem_name, gem_version)
db.execute("INSERT INTO gem_spec (name, version) VALUES (?, ?)", [gem_name, gem_version])
end
def store_docs(db)
YARD::Registry.all.each do |object|
path = object.path
type = object.type
doc = object.docstring.to_raw
db.execute("INSERT INTO gem_docs (path, type, doc) VALUES (?, ?, ?)", [path, type, doc])
end
end
def main
puts "Enter the gem name:"
gem_name = gets.chomp
puts "Enter the gem version:"
gem_version = gets.chomp
download_and_parse_gem(gem_name, gem_version)
db = create_database(gem_name, gem_version)
store_gem_spec(db, gem_name, gem_version)
store_docs(db)
puts "Documentation has been stored in #{gem_name}-#{gem_version}.db"
end
main
This script will perform the following tasks:
- Download and parse the gem documentation using YARD
- Create a SQLite database named with the gem name and version
- Create tables for storing gem spec and documentation
- Store the gem spec (name and version) in the database
- Store the documentation in the database
To run the script, simply execute ruby gem_doc_parser.rb
and follow the prompts to enter the gem name and version. The documentation will be stored in the specified SQLite database.
Let’s See if It Works!
We have GPT-4’s solution. Now its time to copy the code above into gem_doc_parser.rb
and run it.
Ignoring rbs-2.8.2 because its extensions are not built. Try: gem pristine rbs --version 2.8.2
Enter the gem name:
rake
Enter the gem version:
13.0.6
gem_doc_parser.rb:9:in `download_and_parse_gem': uninitialized constant Bundler::CLI (NameError)
Bundler::CLI.start(["gem", gem_name, "--version", gem_version])
^^^^^
from gem_doc_parser.rb:61:in `main'
from gem_doc_parser.rb:70:in `<main>'
Uh oh! We already run into our first error. Let’s paste that output into GPT-4 and see what it tells us.
That didn’t work! I got this error…
GPT-4 Response
I apologize for the confusion. I misunderstood your initial request. You want to download and parse a specific RubyGem, not create a new gem.
Please follow these steps to update the script:
- Replace the
download_and_parse_gem
method with the following:
def download_and_parse_gem(gem_name, gem_version)
gem_path = Bundler::CLI::Common.select_spec(gem_name, gem_version).full_gem_path
# Parse the gem documentation using YARD
YARD::Registry.clear
YARD::Parser::SourceParser.parse(gem_path)
end
- Install the
bundler
gem by running:
- Update your Gemfile to include the gem you want to download and parse:
source 'https://rubygems.org'
gem 'rake', '13.0.6'
- Install the specified gem with Bundler:
After making these updates, you should be able to run the script with ruby gem_doc_parser.rb
. The script will parse the documentation for the gem specified in the Gemfile and store it in a SQLite database.
Will GPT-4’s Fix Work?
Ok, let’s do this one more time just for fun to see if it runs.
Ignoring rbs-2.8.2 because its extensions are not built. Try: gem pristine rbs --version 2.8.2
Enter the gem name:
rake
Enter the gem version:
13.0.6
gem_doc_parser.rb:6:in `download_and_parse_gem': uninitialized constant Bundler::CLI (NameError)
gem_path = Bundler::CLI::Common.select_spec(gem_name, gem_version).full_gem_path
^^^^^^^^
from gem_doc_parser.rb:58:in `main'
from gem_doc_parser.rb:67:in `<main>'
Nope!
This is where GPT-4 can be problematic for people who might be new to software development that are trying to solve specific problems right away. If you’re using it this way, you’ll find that GPT-4 will give you a plausible answer, but it usually won’t work on the first try.
That’s OK though, let’s get more specific about how we want this to run.
More Experience Helps GPT-4 Give Better Results
If you’ve built applications before, you can give GPT-4 a more specific prompt and you’ll get back an answer that’s more inline with what you want.
That doesn’t work either. Let’s try this instead: instead of using Bundler to install the gem, just download the .gem file directly from RubyGems, save it to a temporary path, then have Yard document that one gem and dump it into a sqlite file.
Put all of this into one class that I can call from Ruby code. Then let’s call that class from the CLI so that looks something like this:
$ ruby gem-to-sqlite rake 13.0.6
This type of prompt is something that feels more like an experienced developer talking to a developer who might just be starting out. It gives us back some more reasonable code.
require 'yard'
require 'sqlite3'
require 'net/http'
require 'tmpdir'
require 'rubygems/package'
class GemToSqlite
def initialize(gem_name, gem_version)
@gem_name = gem_name
@gem_version = gem_version
end
def download_gem
gem_url = "https://rubygems.org/downloads/#{@gem_name}-#{@gem_version}.gem"
uri = URI(gem_url)
file_path = File.join(Dir.tmpdir, "#{@gem_name}-#{@gem_version}.gem")
File.write(file_path, Net::HTTP.get(uri))
file_path
end
def extract_gem(file_path)
spec = nil
Gem::Package.new(file_path).spec.files.each do |file|
YARD::Registry.clear
YARD::Parser::SourceParser.parse_string(file)
spec = Gem::Package.new(file_path).spec
end
spec
end
def create_database
db = SQLite3::Database.new("#{@gem_name}-#{@gem_version}.db")
db.execute <<-SQL
CREATE TABLE IF NOT EXISTS gem_spec (
id INTEGER PRIMARY KEY,
name TEXT,
version TEXT
);
SQL
db.execute <<-SQL
CREATE TABLE IF NOT EXISTS gem_docs (
id INTEGER PRIMARY KEY,
path TEXT,
type TEXT,
doc TEXT
);
SQL
db
end
def store_gem_spec(db, spec)
db.execute("INSERT INTO gem_spec (name, version) VALUES (?, ?)", [@gem_name, @gem_version])
end
def store_docs(db)
YARD::Registry.all.each do |object|
path = object.path
type = object.type
doc = object.docstring.to_raw
db.execute("INSERT INTO gem_docs (path, type, doc) VALUES (?, ?, ?)", [path, type, doc])
end
end
def run
gem_path = download_gem
spec = extract_gem(gem_path)
db = create_database
store_gem_spec(db, spec)
store_docs(db)
puts "Documentation has been stored in #{@gem_name}-#{@gem_version}.db"
end
end
When I run it I get another error:
fly-apps/gpt4 → ruby gem_to_sqlite.rb rake 13.0.6
Ignoring rbs-2.8.2 because its extensions are not built. Try: gem pristine rbs --version 2.8.2
[warn]: Syntax error in `(stdin)`:(1,8): no .<digit> floating literal anymore; put 0 before dot
[warn]: ParserSyntaxError: syntax error in `(stdin)`:(1,8): no .<digit> floating literal anymore; put 0 before dot
[warn]: Stack trace:
/Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/yard-0.9.28/lib/yard/parser/ruby/ruby_parser.rb:601:in `on_parse_error'
/Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/yard-0.9.28/lib/yard/parser/ruby/ruby_parser.rb:56:in `parse'
/Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/yard-0.9.28/lib/yard/parser/ruby/ruby_parser.rb:56:in `parse'
/Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/yard-0.9.28/lib/yard/parser/ruby/ruby_parser.rb:17:in `parse'
/Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/yard-0.9.28/lib/yard/parser/source_parser.rb:442:in `parse'
/Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/yard-0.9.28/lib/yard/parser/source_parser.rb:124:in `parse_string'
It gives me back this fix:
GPT-4 Response
Apologies for the issues you’ve encountered. It appears the error is related to the YARD parser when parsing the gem. I’ve updated the code to address this issue.
Update the extract_gem
method in the gem_to_sqlite.rb
file with the following code:
def extract_gem(file_path)
spec = nil
gem_dir = File.join(Dir.tmpdir, "#{@gem_name}-#{@gem_version}")
# Extract the gem to a temporary directory
Gem::Package.new(file_path).extract_files(gem_dir)
spec = Gem::Package.new(file_path).spec
# Parse the gem documentation using YARD
YARD::Registry.clear
Dir.glob(File.join(gem_dir, '**', '*.rb')).each do |file|
YARD::Parser::SourceParser.parse(file)
end
spec
end
An Aside: Inline the Gem Bundle
I also remembered that I want to share this code for this post, so I asked GPT-4 to move the Gems into an inline bundler.
Could you move the require statements up top into an inline bundler block?
GPT-4 Response
Sure! You can use an inline Bundler block to manage gem dependencies. Update the gem_to_sqlite.rb
file with the following code:
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'yard', '~> 0.9.28'
gem 'sqlite3', '~> 1.4'
end
require 'yard'
require 'sqlite3'
require 'net/http'
require 'tmpdir'
require 'rubygems/package'
## ... (rest of the GemToSqlite class definition)
It’s impressive that GPT-4 is able to give me a reasonable response for an “aside”. Let’s see if that distracted it because I ran into another bug running this.
Another Bug
I ran into another bug! Check it out:
fly-apps/gpt4 → ruby gem_to_sqlite-2.rb
[warn]: in YARD::Handlers::Ruby::VisibilityHandler: Undocumentable statement, cannot determine method name
in file '/var/folders/_n/58_fmw2j26sc63pr7lcvcyfh0000gn/T/rake-13.0.6/lib/rake/dsl_definition.rb':23:
23: private(*FileUtils.instance_methods(false))
[warn]: in YARD::Handlers::Ruby::VisibilityHandler: Undocumentable statement, cannot determine method name
in file '/var/folders/_n/58_fmw2j26sc63pr7lcvcyfh0000gn/T/rake-13.0.6/lib/rake/dsl_definition.rb':24:
24: private(*FileUtilsExt.instance_methods(false))
/Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/sqlite3-1.6.1-arm64-darwin/lib/sqlite3/statement.rb:41:in `bind_param': can't prepare Symbol (RuntimeError)
from /Users/bradgessler/.rbenv/versions/3.2.0/lib/ruby/gems/3.2.0/gems/sqlite3-1.6.1-arm64-darwin/lib/sqlite3/statement.rb:41:in `block in bind_params'
GPT-4 Response
I apologize for the oversight. It seems that the error is related to the insertion of data into the SQLite database. The issue comes from the YARD object type being a Symbol, which SQLite doesn’t support. To fix this, we need to convert the object type to a String before inserting it into the database.
Update the store_docs
method in the gem_to_sqlite.rb
file with the following code:
def store_docs(db)
YARD::Registry.all.each do |object|
path = object.path
type = object.type.to_s
doc = object.docstring.to_raw
db.execute("INSERT INTO gem_docs (path, type, doc) VALUES (?, ?, ?)", [path, type, doc])
end
end
The only change made is converting the object type to a string using to_s
:
With this change, you should be able to run the script without any issues:
./gem-to-sqlite rake 13.0.6
Let’s try it again!
fly-apps/gpt4 → ruby gem_to_sqlite-2.rb
[warn]: in YARD::Handlers::Ruby::VisibilityHandler: Undocumentable statement, cannot determine method name
in file '/var/folders/_n/58_fmw2j26sc63pr7lcvcyfh0000gn/T/rake-13.0.6/lib/rake/dsl_definition.rb':23:
23: private(*FileUtils.instance_methods(false))
[warn]: in YARD::Handlers::Ruby::VisibilityHandler: Undocumentable statement, cannot determine method name
in file '/var/folders/_n/58_fmw2j26sc63pr7lcvcyfh0000gn/T/rake-13.0.6/lib/rake/dsl_definition.rb':24:
24: private(*FileUtilsExt.instance_methods(false))
Documentation has been stored in rake-13.0.6.db
Success! Let’s open up the sqlite database and make sure it has what we expect.
sqlite> select * from gem_spec;
1|rake|13.0.6
sqlite> select * from gem_docs;
1|RDoc|module|
2|RDoc::Page|module|
3|RDoc::Page::FONTS|constant|
4|RDoc::Page::STYLE|constant|
5|RDoc::Page::XHTML_PREAMBLE|constant|
6|RDoc::Page::HEADER|constant|
7|RDoc::Page::FILE_PAGE|constant|
8|RDoc::Page::CLASS_PAGE|constant|
9|RDoc::Page::METHOD_LIST|constant|
10|RDoc::Page::FOOTER|constant|
11|RDoc::Page::BODY|constant|
12|RDoc::Page::SRC_PAGE|constant|Source code ##########################
13|RDoc::Page::FR_INDEX_BODY|constant|Index ################################
14|RDoc::Page::FILE_INDEX|constant|
15|RDoc::Page::CLASS_INDEX|constant|
16|RDoc::Page::METHOD_INDEX|constant|
17|RDoc::Page::INDEX|constant|
18|Rake|module|--
Copyright 2003-2010 by Jim Weirich (jim.weirich@gmail.com)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to
deal in the Software without restriction, including without limitation the
rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
sell copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
IN THE SOFTWARE.
++
19|Rake::CommandLineOptionError|constant|
Not bad! We can see our gem name and version in the gem_spec
table and all of the docs in gem_docs
.
Let’s Have GPT-4 Clean This Up for a Demo
Since I need to bundle all of this up in a way that I can share it, I’m going to ask GPT-4 to do the work.
It works! I need to get all of this into one file so that I can share it via a Github Gist. Could you consolidate into one file, remove any unnecessary code, and include in code comments up top how to run the script? Make sure its a
.rb
file and let’s tell people to run it viaruby gem-to-sqlite.rb rake 13.0.6
(for example)
GPT-4 Response
Sure! Here’s the consolidated version of the script as a single file named gem-to-sqlite.rb
. I’ve also added comments at the beginning of the file with instructions on how to run it.
# gem-to-sqlite.rb
#
# Usage: ruby gem-to-sqlite.rb <gem_name> <gem_version>
# Example: ruby gem-to-sqlite.rb rake 13.0.6
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'yard', '~> 0.9.28'
gem 'sqlite3', '~> 1.4'
end
require 'yard'
require 'sqlite3'
require 'net/http'
require 'tmpdir'
require 'rubygems/package'
class GemToSqlite
def initialize(gem_name, gem_version)
@gem_name = gem_name
@gem_version = gem_version
end
def download_gem
gem_url = "https://rubygems.org/downloads/#{@gem_name}-#{@gem_version}.gem"
uri = URI(gem_url)
file_path = File.join(Dir.tmpdir, "#{@gem_name}-#{@gem_version}.gem")
File.write(file_path, Net::HTTP.get(uri))
file_path
end
def extract_gem(file_path)
spec = nil
gem_dir = File.join(Dir.tmpdir, "#{@gem_name}-#{@gem_version}")
# Extract the gem to a temporary directory
Gem::Package.new(file_path).extract_files(gem_dir)
spec = Gem::Package.new(file_path).spec
# Parse the gem documentation using YARD
YARD::Registry.clear
Dir.glob(File.join(gem_dir, '**', '*.rb')).each do |file|
YARD::Parser::SourceParser.parse(file)
end
spec
end
def create_database
db = SQLite3::Database.new("#{@gem_name}-#{@gem_version}.db")
db.execute <<-SQL
CREATE TABLE IF NOT EXISTS gem_spec (
id INTEGER PRIMARY KEY,
name TEXT,
version TEXT
);
SQL
db.execute <<-SQL
CREATE TABLE IF NOT EXISTS gem_docs (
id INTEGER PRIMARY KEY,
path TEXT,
type TEXT,
doc TEXT
);
SQL
db
end
def store_gem_spec(db, spec)
db.execute("INSERT INTO gem_spec (name, version) VALUES (?, ?)", [@gem_name, @gem_version])
end
def store_docs(db)
YARD::Registry.all.each do |object|
path = object.path
type = object.type.to_s
doc = object.docstring.to_raw
db.execute("INSERT INTO gem_docs (path, type, doc) VALUES (?, ?, ?)", [path, type, doc])
end
end
def run
gem_path = download_gem
spec = extract_gem(gem_path)
db = create_database
store_gem_spec(db, spec)
store_docs(db)
puts "Documentation has been stored in #{@gem_name}-#{@gem_version}.db"
end
end
if ARGV.length != 2
puts "Usage: ruby gem-to-sqlite.rb <gem_name> <gem_version>"
exit 1
end
gem_name = ARGV[0]
gem_version = ARGV[1]
gem_to_sqlite = GemToSqlite.new(gem_name, gem_version)
gem_to_sqlite.run
That’s pretty darn good, and the gist is available at Github.
GPT-4 Likes to Fake It Until It Makes It (or You Lose Your Patience)
Now I’m going to ask it a question that will probably break it using questionably imprecise language:
How could I use an instance of
YARD::Registry
instead of the.all
singleton?
GPT-4 Response
You can create a new instance of YARD::Registry
and use it to parse and store the documentation. First, create an instance variable @registry
in the initialize
method:
def initialize(gem_name, gem_version)
@gem_name = gem_name
@gem_version = gem_version
@registry = YARD::Registry.new
end
Bzzt wrong! YARD::Registry
is a module. GPT-4 doesn’t necessarily know what the actual Ruby code is behind what it’s telling you—it seems to be matching patterns on something that seems like it should be a class, but it’s not.
Let’s Make a Version for Squirrels ????️
Time to get silly! We made this script for squirrels, so let’s ask GPT to rename a few things to make it more undqerstable to squirrels.
Let’s rename everything to something that would make a squirrel really happy
# squirrel_gem_nut_collector.rb
#
# Usage: ruby squirrel_gem_nut_collector.rb <gem_name> <gem_version>
# Example: ruby squirrel_gem_nut_collector.rb rake 13.0.6
require 'bundler/inline'
gemfile do
source 'https://rubygems.org'
gem 'yard', '~> 0.9.28'
gem 'sqlite3', '~> 1.4'
end
require 'yard'
require 'sqlite3'
require 'net/http'
require 'tmpdir'
require 'rubygems/package'
class SquirrelGemNutCollector
def initialize(nutty_gem_name, crunchy_gem_version)
@nutty_gem_name = nutty_gem_name
@crunchy_gem_version = crunchy_gem_version
@nut_registry = YARD::Registry.new
end
def nut_fetcher
nut_url = "https://rubygems.org/downloads/#{@nutty_gem_name}-#{@crunchy_gem_version}.gem"
uri = URI(nut_url)
temp_nut_path = File.join(Dir.tmpdir, "#{@nutty_gem_name}-#{@crunchy_gem_version}.gem")
File.write(temp_nut_path, Net::HTTP.get(uri))
temp_nut_path
end
def crack_nut(file_path)
nut_spec = nil
cracked_nut_dir = File.join(Dir.tmpdir, "#{@nutty_gem_name}-#{@crunchy_gem_version}")
# Crack open the gem nut and release the code
Gem::Package.new(file_path).extract_files(cracked_nut_dir)
nut_spec = Gem::Package.new(file_path).spec
# Parse the code with the power of YARD
@nut_registry.clear
Dir.glob(File.join(cracked_nut_dir, '**', '*.rb')).each do |file|
YARD::Parser::SourceParser.parse(file, @nut_registry)
end
nut_spec
end
def nut_storage
nut_db = SQLite3::Database.new("#{@nutty_gem_name}-#{@crunchy_gem_version}.db")
nut_db.execute <<-SQL
CREATE TABLE IF NOT EXISTS nut_specs (
id INTEGER PRIMARY KEY,
name TEXT,
version TEXT
);
SQL
nut_db.execute <<-SQL
CREATE TABLE IF NOT EXISTS nut_docs (
id INTEGER PRIMARY KEY,
path TEXT,
type TEXT,
doc TEXT
);
SQL
nut_db
end
def store_nut_specs(nut_db, spec)
nut_db.execute("INSERT INTO nut_specs (name, version) VALUES (?, ?)", [@nutty_gem_name, @crunchy_gem_version])
end
def fill_nut_storage_with_magic(nut_db)
@nut_registry.all.each do |object|
nutty_path = object.path
crunchy_type = object.type.to_s
docs_of_wisdom = object.docstring.to_raw
nut_db.execute("INSERT INTO nut_docs (path, type, doc) VALUES (?, ?, ?)", [nutty_path, crunchy_type, docs_of_wisdom])
end
end
def collect_all_the_nuts
path_to_nut = nut_fetcher
nut_spec = crack_nut(path_to_nut)
nut_db = nut_storage
store_nut_specs(nut_db, nut_spec)
fill_nut_storage_with_magic(nut_db)
end
end
if ARGV.size
????
I’ll spare you these prompts, but I can assure you they were hilarious.
- How about for a old timey prospector from the gold rush?
- Could you give me a cool hip hop version?
- Do one for using words a 4 year old would understand
Overall GPT-4 Is a Useful Tool
To sum it up, here’s what you should keep in mind when you add GPT-4 to your set of tools:
You Still Need to Know What You’re Doing
If you’re a senior developer, giving GPT-4 more specific prompts results in better responses. It can reason with you through architectural problems or write you code samples; however, many times it doesn’t know the right answer but confidently tells you something anyway.
If you’re a beginner, GPT-4 will give you responses that you might be able to fix and get running, but it’s not the most concise way of doing it. Often times though the code won’t work and you might not know how to ask it to fix it.
It’s Great at Jumping Contexts
GPT-4 shines when spanning different contexts. If comfortable in one programming language or framework and need to learn a new one, GPT-4 can help explain it to you and relate it back to concepts that you’re accustomed to. In a lot of ways, GPT-4 is like looking into a mirror.
It Makes Up Seemingly Plausible Answers That Could Be Wrong.
When I asked GPT-4 how to create an instance of a YARD::Registry
, instead of using the singleton, it told me YARD::Registry.new
. That seems right when thinking about the structure of idiomatic Ruby code, but YARD is not, so GPT-4 has a harder time answering the question.
It Get’s Stuck When the “contexts” Get Too Large, Like if You’re Asking for Code From 5 Different Files
If you give GPT-5 too broad of a problem to solve, it can get lost and forget things. If you ask it to solve a problem for you like “how do I create server-side anchor tags using Turbo and Hotwire”, it will give you an initial answer that seems right across 5+ files. When you start fixing errors in those files, GPT-4 will loose track of some of the files and start giving you suggestions for fixing completely different code.