Is This the Way forward for Dwelling Automation?

ChatGPT plugins are surprisingly easy. They’re simply customary Net APIs – no totally different from every other API you may write or name. My specific plugin is de facto primary. It has simply two endpoints.

One returns an inventory of lights. The opposite toggles a lightweight on and off.

I’ve arrange my little demo with 5 lights:

• Kitchen
• Bed room
• Eating desk lamp
• Lavatory
• Lounge

It’s value watching the video because it’s fairly cool. We simply say to ChatGPT, “let there be gentle,” and it switches on all of the lights linked to my Raspberry Pi.

What’s actually spectacular is that the one factor ChatGPT has to work with are just a few hints describing the plugin in free textual content and the OpenAPI yaml file. From that, it figures out that it must first get the listing of lights, after which it must loop by way of the lights and toggle them on.

Now, I say “discovered” however I’m not going to assert that it’s considering or reasoning in regards to the world – that will be an enormous stretch. However I’m going to assert that it’s fairly unbelievable.

We are able to even run by way of a very easy residence automation state of affairs.

• I’ve simply woken up – it activates the bed room gentle
• Time to bathe – it activates the toilet gentle and turns off the bed room gentle
• Time for breakfast – it activates the kitchen gentle and turns off the toilet gentle
• I’m off to work – it turns off all of the lights
• I’m again from work with takeout – it activates the eating desk gentle
• Time to chill out – it activates the lounge gentle
• Time for mattress – it activates the bed room gentle

I then tried one thing a bit totally different. I pretended to be one other gadget within the bed room that might detect sounds and informed ChatGPT that I might hear loud night breathing. It then turned off the bed room gentle and wished me goodnight.

Now clearly, it is a actually simple demonstration, however to me, the probabilities are fairly mind-boggling.

I might see giant language fashions being an attention-grabbing approach to join a complete bunch of various programs and providers collectively. There are a ton of poorly designed APIs on the market, and integrating them is a nightmare.

Wouldn’t or not it’s humorous if we ended up with computer systems speaking to one another utilizing human language as a substitute of APIs?

One of many issues that I actually loved about this challenge (other than the truth that it labored!) was that I didn’t have to put in writing a lot code in any respect – I used to be capable of get ChatGPT to generate all of the code.

That is the immediate I used:

Write me a Python Flask API to run on a Raspberry Pi and management some lights hooked up to GPIO pins.

I've the next lights:

Kitchen: pin 6
Bed room: pin 13
Eating desk: pin 19
Lavatory: pin 26
Lounge: pin 5

I need the next endpoints:

get lights - returns the listing of lights together with their present state
put up toggle_light - switches a lightweight on or off

I requested it to make a Flask API – I used this as I’m most aware of it so I might examine what ChatGPT outputs. As at all times with these Giant Language Fashions, it’s undoubtedly a case of “Belief, however confirm”.

I informed it that it was working on a Raspberry Pi and that there have been lights linked to GPIO pins.

And I gave it the listing of lights together with the pin numbers.

I then gave it a really obscure spec for the endpoints I wished.

With that primary spec, I used to be capable of get ChatGPT to generate the code for me. I added on a bunch of bells and whistles afterwards, however the core of the API was generated by ChatGPT.

from flask import Flask, request, jsonify
import RPi.GPIO as GPIO

app = Flask(__name__)

# GPIO setup
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)

# Outline your lights
lights = {
    "Kitchen": {"pin": 6, "state": GPIO.LOW},
    "Bed room": {"pin": 13, "state": GPIO.LOW},
    "Eating desk": {"pin": 19, "state": GPIO.LOW},
    "Lavatory": {"pin": 26, "state": GPIO.LOW},
    "Lounge": {"pin": 5, "state": GPIO.LOW},
}

# Setup pins
for gentle in lights.values():
    GPIO.setup(gentle['pin'], GPIO.OUT)
    GPIO.output(gentle['pin'], GPIO.LOW)


@app.route('/lights', strategies=['GET'])
def get_lights():
    return jsonify(lights)


@app.route('/toggle_light', strategies=['POST'])
def toggle_light():
    knowledge = request.get_json()
    gentle = knowledge.get('gentle')
    
    if gentle not in lights:
        return {"error": "Gentle not discovered"}, 404

    # Toggle gentle
    lights[light]['state'] = GPIO.HIGH if lights[light]['state'] == GPIO.LOW else GPIO.LOW
    GPIO.output(lights[light]['pin'], lights[light]['state'])

    return {"success": f"{gentle} toggled"}, 200


if __name__ == "__main__":
    app.run(host='0.0.0.0', port=5000)

The opposite factor that’s wanted for a ChatGPT plugin is an OpenAPI or Swagger file – these are tedious to put in writing – and it has been identified to me that if I’d used FastAPI, I wouldn’t have wanted to put in writing one in any respect. However one of many very nice issues you are able to do with ChatGPT is paste in your Flask code, and it’ll generate an OpenAPI file for you.

I’m simply working this plugin domestically, I don’t have any plans to publish it as that will require much more plumbing. For growing native plugins, it’s essential to have a server working on localhost. Thankfully, we are able to use the wonderful VSCode help for distant growth. I’m connecting by way of to the Pi from my desktop machine, and after I launch the server, VSCode tunnels by way of to the Raspberry Pi and exposes the server on my native machine.

ChatGPT appears to be like on the /.well-known/ai-plugin.json and makes use of that to get the title, description, brand, and OpenAPI file. That’s all it has to work with to determine what to do with the API.

We are able to get some perception into the way it works underneath the hood. We are able to simply ask ChatGPT to point out us its considering. If we paste within the OpenAPI yaml file and ask it to elucidate tips on how to flip lights on and off, we get this:

We are able to even get it to generate a sequence diagram for us:

It’s fairly wonderful. Now go and watch the video: