Apricot Weblog – ChatGPT vs. Bard: A sensible comparability
Apricot is an “AI-powered” product in just a few methods. Most instantly, the Apricot app makes use of a big language mannequin (LLM) service for on-demand content material summaries. Behind the scenes, I take advantage of LLM chat bots to put in writing drafts of each new code and advertising and marketing content material, like LinkedIn posts and Tweets.
Up to now, I’ve used ChatGPT for this (GPT-3.5 for the API, GPT-4 within the chat window), however Google made Bard generally available this week and upgraded it to make use of its new PaLM 2 LLM and the hype males got here out in predictable pressure.
Lots of the examples in these hype tweets don’t stand as much as scrutiny, by way of being significant for on a regular basis use.
-
Positive, Bard can export code on to Google Colab, however Ctrl-C, Ctrl-V right into a pocket book isn’t that laborious both.
-
Sure, Bard can get yesterday’s inventory returns, however so can Yahoo Finance.
-
Typically the examples simply don’t replicate. I requested Bard for particular website positioning suggestions for Apricot’s touchdown web page and it gave generic solutions. Positive, however nothing particular.
-
Voice prompts are in all probability very useful to some folks, however not a deciding issue for many.
I don’t wish to fall behind the pack, although, so let’s put Bard to the take a look at vs. ChatGPT for my precise Apricot-related use circumstances.
This isn’t a scientific research, clearly. As soon as upon a time, I loved doing managed, in-depth, technical comparisons of ML fashions, however these days are previous. On this submit, I’m going to take about an hour to discover just a few use circumstances, decide, and transfer on to the remainder of my lengthy to-do record.
-
OpenAI ChatGPT, with the GPT-4 mannequin, within the chat window at https://chat.openai.com/. I pay about $20/month for this.
-
Google Bard, within the chat window at https://bard.google.com/. I’m unsure what the mannequin is beneath the hood, so minus 1 level for Google. However it’s free, so plus 1 level and we’re again to even.
1. Summarize long-form content material
Benn Stancil writes a terrific knowledge science newsletter however his headlines are tremendous imprecise. Take Stancil’s submit The new philosophers for instance—I don’t know what’s its about from the title alone. Let’s ask our AI assistants to summarize the content material, so we all know if it’s value our time to learn.
My immediate to every system is similar because the Apricot App’s immediate for on-demand user-facing summaries:
Summarize the next content material in a single sentence: “““{textual content}”””
ChatGPT says: 
Bard says:
Nicely, dangle on a sec…Bard truncates the enter, and it’s not apparent the place. Let’s hope Stancil didn’t save the punchline for the very finish…Kinda shocking, to be sincere, when other tools are expanding the context window dramatically.
At any fee, Bard’s reply is:
Facet word: the View different drafts possibility is intriguing however I’m unsure it is sensible, from a product perspective. Wouldn’t it’s higher to iterate with a brand new immediate if I don’t just like the response? I don’t have the persistence to learn three solutions each time I take advantage of the device.
Winner: ChatGPT
Bard’s abstract is too terse; it conveys little or no info. ChatGPT, however, captures explicitly that the following evolution of the info administration instruments might be about AI particularly and could possibly be a elementary, discontinuous change.
2. Write a Python operate
Customers coming to Apricot from an RSS reader can import their feeds in bulk with an OPML file. After I got down to construct this characteristic, I didn’t know a lot about OPML so I requested ChatGPT to put in writing the core parsing code.
Write a Python operate to parse an OPML file.
ChatGPT’s reply:
import xml.etree.ElementTree as ET
def parse_opml(file_path):
# Parse the OPML file
tree = ET.parse(file_path)
root = tree.getroot()
# OPML should have a 'body' child, which contains 'outline' elements
body = root.find('body')
outlines = []
for outline in body.findall('outline'):
# Each outline should have 'text' attribute
text = outline.get('text', '')
# Append it to our list
outlines.append(text)
return outlinesBard’s code:
def parse_opml(filename):
"""Parses an OPML file.
Args:
filename: The path to the OPML file.
Returns:
A list of dictionaries, where each dictionary represents an outline item.
"""
with open(filename, 'r') as f:
xml = f.read()
# Parse the XML into a tree.
tree = ET.fromstring(xml)
# Get the outline items.
outline_items = []
for outline_item in tree.findall('outline'):
outline_item_dict = {}
outline_item_dict['title'] = outline_item.find('title').text
outline_item_dict['url'] = outline_item.find('xmlUrl').text
outline_items.append(outline_item_dict)
return outline_itemsOK, this is not the biggest thing, but why on earth does Bard use indentation with 2 spaces instead of the standard 4? This drove me up the wall, so I tried all sorts of prompts to get Bard to use indentation with 4 spaces but they all failed. There must be some sort of post-processing thing going on, but that’s strange because Google’s official style guide says to make use of 4 areas.
Shifting on.
Frankly, each of those code snippets are fairly dangerous. ChatGPT’s code solely extracts the textual content discipline of every entry within the OPML file, however there’s sometimes different essential stuff like URLs.
Bard’s code appears to be like higher at first look as a result of it does attempt to extract a number of fields (title and xmlUrl) from every entry. However it fails to run as a result of it makes use of the ElementTree class with out importing it first. As soon as that’s fastened, it runs however returns an empty record as a result of it fails to know that the findall technique solely searches direct youngsters of the present node, however the person define components are nested two ranges deep.
It’s no shock that the code isn’t excellent on the primary move; what issues is how fast and straightforward it’s to iterate to a great model. For ChatGPT, I prolonged the dialog to say
Please extract all fields from every
definefactor of the enter file right into a Pythondict.
and it up to date the code to a chic, right reply.
For Bard, I replied
Please replace the code to parse
definecomponents within the physique of the enter doc.
and it did repair the recognized drawback. However the brand new code nonetheless doesn’t work as a result of it fails to acknowledge that for OPML define components, the info is saved as tag attributes, not encapsulated textual content. Sigh. ????
Winner: ChatGPT
3. Social media advertising and marketing techniques
As a bootstrapped, solo mission, Apricot has relied thus far on social media and content material advertising and marketing. It’s about time to make a Show Hacker News submit, which—achieved effectively—can generate a giant spike in site visitors and sign-ups. Let’s ask our AI assistants to the way to make the submit profitable.
Bard first this time:
And now ChatGPT, with the identical immediate:
Not like the code problem, this time each solutions really feel fairly good! ????
Each LLMs embrace tips on each content material and logistics, as requested, each appear to know the Hacker Information tradition, and each give good, “right” recommendation. My solely grievance is that each are very generic. Examine to this post on Indie Hackers, which provides a really particular recipe for posting efficiently on Hacker Information.
Winner: tie
4. Write a personalized Tweet
Persevering with in the identical vein, I usually use ChatGPT to put in writing first drafts of Apricot advertising and marketing content material. The distinction with this problem is that the content material must be about Apricot particularly or perhaps a specific weblog submit subject. This time I’ll use a little bit of immediate engineering to get the tone I would like.
Let’s do Bard first once more:
Yikes! I didn’t assume this might be a very delicate request. Let’s click on that Regenerate drafts button just a few instances…
And now ChatGPT:
Appears nice, but it surely doesn’t match inside the character restrict! I needed to take away the hashtags to make it match.
Winner: tie
I like ChatGPT’s emojis and hashtags however Bard’s text-only model is a good response to my steering of a “direct, no-spam” model. Bard refused to reply in any respect at first, however ChatGPT’s response was too lengthy. Each tweets are stay, so you possibly can vote your self. ????
ChatGPT.
ChatGPT beat Bard fairly convincingly in summarization and Python coding. That’s value $20/month to me, at the least for now, even if they tied in Tweet and advertising and marketing techniques technology.
I’m blissful there’s competitors and I think Bard will enhance shortly—I sit up for re-doing this take a look at once more quickly!