When AI goes wrong
Very self-assured AIs can produce code that looks right, code that it’ll argue very strongly is right, but is in fact very wrong.
We are going to kick off using Google’s Bard to generate some code for us. We will then copy this code into
VS Code and attempt to run it. Any code you write should be tested thoroughly and this is no different.
Bard, ChatGPT and other similar systems are trained on huge data sets, but the sources are not necessarily correct. These systems are also able to use chunks of information (code in this context) from multiple sources. Because of this, some parts of the code sample generated could be correct while others are incorrect.
So, let’s get started. First of all, open your web browser and perform a search for Google Bard. Log in using your Google account and spend some time getting acquainted. Start by asking some simple questions, such as the colour of the sky and the temperature at which water freezes, before moving on to more complicated questions, such as why water freezes at zero Celsius.
Now ask Bard to generate some Python code that can be used to forecast the weather. You can see the code that was generated when we asked that question in the first screenshot (below-left), while the second screenshot (above) describes how the code works.
Before we do any coding ourselves, open your browser once again and search for OpenWeatherMap. When this has opened, sign up for an account and generate an API key. Keep a copy of this somewhere handy for later usage.
Open VS Code and create and save a file called weatherForecast.py in a relevant directory. VS Code will use the file extension to enable the Python syntax highlighting mode and ensure everything is in order to run the code. Take the code that has been generated by Bard and copy it into your new Python file. Within the def get_weather_forecast(city): function definition, replace YOUR_API_KEY with the API key generated before creating the file.
Now that’s been completed, let’s try running the file. Open the Run menu and select Run Without Debugging. At the bottom of VS Code, you will see a terminal, which contains some errors. In this case, the errors are as such:
Current weather in San Francisco: Traceback (most recent call last):
File “/home/matt/Documents/Programming/AI/
weatherForecast.py”, line 33, in
File “/home/matt/Documents/Programming/AI/ weatherForecast.py”, line 23, in main print(“* Temperature: {}°F”.
format(forecast[“current”][“temp”])) KeyError: ‘current’
Debugging this code, the first thing we notice is an informational message stating that the weather being searched for is in San Francisco. This looks like an intentional message and can be tracked back to the code, where the line states print(“Current weather in {}:”.format(city)) . Next we can see that there are errors on lines 33 and 23. Line 33 is the final line of the code,
which calls the main function, so this tells us there is an issue within the main function. The second line tells us the line within the main function that contains the error. In line 23, we can see that the error occurs when trying to print out the current temperature. The final line states that there is a KeyError, which suggests that the data structure, which contains the information from the API, does not contain an entry called current.
As a brief aside, the data returned from the API is in JSON format and can be accessed in the same way as a dictionary. Dictionaries in Python contain key:value information. For example, a dictionary could contain the colour names and the corresponding RGB colours: colours = {‘red’: (255,0,0), ‘green’, (0,255,0),
‘blue’:(0,0,255)}
Using this sample, to return the RGB values for the colour green, we would reference colours[‘green’] . If we were to reference colours[‘pink’] , we would receive a KeyError.
We have determined that the data that’s returned from the API has a different format from the one that the code is expecting. There are a few ways of tackling this sort of problem: we could read the API specs, pepper the code with print statements, use the built-in debugging features or a mixture of all three.
First of all, we will have a look at using the debug features. In VS Code, navigate to the line of code that says if forecast: and press F9. This should draw a small red circle to the left of the line number. Now, navigate to Run, select Start Debugging and then select Python File. VS Code
runs the code and on the left-hand side of the screen, you will see variables that are currently in scope. Navigate through this variable section and investigate the format of the data. These tools can be especially useful when investigating variable changes within a loop. Setting a breakpoint in the loop allows the code to stop on the first iteration of the loop. Pressing F5 allows the code to continue until the next loop iteration. There are other options as well, which enable us to investigate what variable changes occur from line to line.
Peppering the code with print statements is another option, but this does mean more tidying up at the end of the process. Combining the Python Rich library with this form of debugging is powerful because it allows the data structure to be nicely formatted. A two-part series on using this library started in LXF303.
Looking at the data format (see screenshot, aboveleft), it can be seen that the dictionary contains a key called list. This returns a list of forecast values. The forecast information in the list is detailed by the dt
value, which is the date and time, using the format of the number of seconds in the Unix epoch (since 1st January 1970). This is telling us that we need to replace the reference to current in the original code with the first entry in the list.
The updated code (see screenshot, below)
returns information when it is run. The Google Bard
explanation states that the code can be run with a city name as an argument. This will not work as the code has not been written to add this functionality. Using some simple logic and debugging tips allows a very useful code sample from Bard to be used successfully. These techniques can be employed when checking the functionality of any code, including your own weather forecasting script, because it is highly unlikely that the same code sample will be generated again.