Tag Archives: wordpress

Coding with an LLM Sidekick

Posted on by

I fell down a rabbit hole recently which lead me to think about my experiences in the nascent field of “prompt engineering.”12

As a thought experiment, I was thinking about what I’ve managed to accomplish working with an LLM, the challenges along the way, and perhaps even where I can see the frayed edges of its current limitations.

After several starts and stops trying to hire someone to assist with a website I own, I turned to the idea of getting help from an LLM. 3 4  After all, some of them were touted as being able to actually draft code, right?  Besides, if the first step in even hiring a developer is just being able to describe what you need, and the first step of getting an LLM to generate some code is defining what I need, then…

There's no way this is going to work, right?

There’s no way this is going to work, right?

  1. 1. Task 1:  Pie Chart WordPress Plugin

    1. I started off with a simple and easy to define task.  My original plugin was a quick and dirty bit of code, so if ChatGPT could create a WordPress plugin, there was a chance it could do something simple like this.
    2. My first attempt was a wildly spectacular, but highly educational, failure.  A brief description of the plugin’s function was enough to get a WordPress plugin template file with very little functionality.  Then came the arduous LLM wrangling, my asking it for refinements, it losing track of the conversation, and the endless sincere heartfelt apologies from ChatGPT about forgetting really basic pieces of information along the way.  Some changes were minor, but changing the names of variables, functions, the plugin, switching API’s, forgetting requirements, etc.  It was constant whack-a-mole that spanned nearly 90 pages of text.
    3. My next attempt was more focused.  I created a framework for discussions, provided more context, goals, descriptions of workflow, and resources for examples.  The result was a lot better, with portions of largely functional code.  However, the LLM kept forgetting things, renaming variables, files, directories, etc.
    4. Next I created the directory structure and blank placeholder files, zipped these, and uploaded them as an attachment for the LLM to review – along with a description of the contents and the above additional context.  This was even better than before, but after a certain depth of conversation no amount of reminding could bring the LLM around to the core of the conversation.
    5. My thinking was that after a certain level of conversation, the LLM was not going to be able to synthesize all of nuance of our conversations plus the content of the code drafted.  To get around this I would begin a conversation, make a little progress, then ask it to summarize the project, the current status, and a plan for completion – which was fed into an entirely new conversation.  This way, Conversation N was able to provide a succinct and complete description which Conversation N+1 could use as a jumping off point.  My thinking was that the LLM would be best positioned to create a summary that would be useful to another LLM.
    6. This process of minor “restarts” in the conversation was one of the most successful and powerful techniques I’ve employed to combat LLM hallucinations and forgetfulness.

  2. 2. Task 2:  Blog Post Series Plugin

    1. After rewriting the above pie chart plugin using an LLM, I turned my attention to a slightly more complicated plugin.  The pie chart plugin is really just a single file which turns a shortcode with a little bit of data into a nice looking pie chart.  There’s no options page, no cross post interaction, database queries or anything.  It was really just a test to see if an LLM could really draft a basic piece of working code.
    2. The series plugin is still a reasonably simple piece of code, but it has several additional feature which require a settings page, saving settings, custom database queries, and organizing information across multiple pages.  It’s also one of the most used plugins on this website.
    3. I figured I would try feeding the LLM a description of my plugin, all the code in a directory structure, and then my initial “base” prompt which explains our roles, needs, resources, and scaffolding for a discussion.  I asked the LLM to summarize the function and features of the plugin, which it did quite nicely.  I added a few additional features I had previously worked on and asked it to incorporate this into the description.  Asking the LLM to simply “build this WordPress plugin” was met with a “you need to hire a developer” recommendation.  However, asking it to propose a workflow for building a plugin with these features was successful.  I was provided with a roadmap for building5 my plugin.
    4. This system worked reasonably well, allowing me to compartmentalize the steps, backtrack, retrace, revise code, working on a section, then another, sometimes going back to a prior sections at the LLM’s direction.  The LLM still tended to get lost, renamed variables/paths/directories/filenames, but it was less pronounced than before.  I did find it harder to use the “summarize and restart” strategy when dealing with a multi-step code development system.  However, it was still workable since I could upload all the code produced so far.
    5. The result was a new plugin, with better functionality than what I’d written myself 10 years before.  Here, the new strategy of having the LLM break the project into sections and providing a roadmap was particularly helpful.
  3. Strategy:  Conversational Scaffolding
    1. I mentioned “conversational scaffolding” and “frameworks” for discussing things with the LLM above.  This was an overarching and evolving strategy I use to help focus the LLM on the goals, keep it on track, and hopefully help it provide meaningful and useful replies.  The full text of my “prompt framework” file is too large to include here, but I’m happy to provide the highlights.
    2. Personas.  I assigned the LLM three distinct personas with differing backgrounds, strengths, and goals.  Their personas were defined in reference to one another, so the first would activate, the second would then review and interact with the first, after this process completed the third would be activated, perhaps interact with the first two, then it would move on.  I would say this process was rather successful.
    3. Myself.  I would describe myself, my goals, level of expertise, etc.  I found that I if I referred to myself as an expert, the LLM would not be as likely to offer me code proposals – but if I described myself as a newbie, it would recommend I hire a developer rather than tackle such a complex problem myself.
    4. Rules for Conversation.  These are a collection of 12 rules (at last count) which helped myself and the LLM interact.  The high points are:
      1. Answer Numbering, Answer Format, Eliminate Guesswork, Organize Assumptions, Conversational Review, Complex Answers, Context Refresher, Problem Solving Approach, File Structure, @Rules, and Personas.
      2. Each of these items were followed by a few sentences explaining something about how the LLM should be expecting to receive information and react.  My favorite of these was the rule “@Rules” which directed the LLM to begin it’s response by reviewing the Rules and following them.
    5. Knowledge.  There are a number of programming languages and technical topics I’m interested in and have used an LLM to address.  To this point, I solicited a list of useful resources from the LLM and started including a “Knowledge” section where I listed dozens of the most important resources for the languages and API’s I most commonly use.
    6. By beginning each prompt with the above “framework” (~10k of text) and following it up with a short description of my project or a file to consider, I found I was able to jump right into the project without having to provide additional significant background information.
  4. Task 3:  “Project Drift”
    1. This is a considerably more complicated task I will simply refer to as “Project Drift.”  This isn’t a real codename since the developer base is all of exactly one dude, but I don’t want to name the location/website for a variety of reasons.  In any case, Project Drift involves multiple user interfaces, numerous settings, database queries, data sanitation and validation procedures, administrator functions, and numerous other facets.  All of the above tasks and attempts were basically part of the run-up to this (ongoing) project.
    2. Using the LLM’s ability to open and read a ZIP file, as well as propose code, has been invaluable.  This in conjunction with my prompt framework allows me to get the LLM up to speed after a micro-restart – and it’s summarization procedures help me get back in the mindset after I’ve stepped away from the project for a few days.
    3. Since this project isn’t done yet, I can only give a progress report.  It’s going very well.  Much of the heavy lifting, scaffolding of the code, can be assembled for me, tedious database queries and chunks of code provided.  There are still large areas where the LLM is unable to be very helpful – and that relates to pinpointing a bug in the code (or between code sections).  This still requires a knowledgeable hand at the helm.
    4. As a solo-coder, having the assistance of another “persona” to keep me on track with a given section of code has been helpful.  I have only assigned three personas, but I could see adding a few more to fulfill different roles.

I would estimate Project Drift is roughly 30-50% complete, but this is still an incredible amount of progress in a very short time.  I would also estimate it has cut the amount of my development time by 90% (but on the easiest and most tedious stuff).

Software Development with LLMs
  1. Series Plugin Test for Illustrative Purposes Only
  2. ChatGPT WordPress Plugins
  3. Coding with an LLM Sidekick
  1. I know, it feels pretentious, doesn’t it? [↩]
  2. I’ve got the same knee-jerk reaction to “visionary,” “thought leader,” “polymath,” and “futurist.” [↩]
  3. Don’t get me wrong, some of the developers I’d hired simply disappeared while other relationships didn’t work out due to timing.  I don’t think anyone was malicious, just… busy, really. [↩]
  4. Still, the job needs to be done. [↩]
  5. Re-building? [↩]

ChatGPT WordPress Plugins

Posted on by

This is kinda bananas.  Years ago I wrote a plugin to solve a problem I had.  I wanted a simple WordPress plugin where I could insert a shortcode into a blog post, specify a series title, and have it automatically search up all the other blog posts that used the same shortcode and series title, and then insert a nice looking list of blog posts in that series in chronological order.

It was one of my first plugins, still available on WordPress.org – just hidden since it hasn’t been updated in almost a decade.  It still works to this very day, if occasionally a little buggy.  After several WordPress versions, it no longer properly displays the series title, which is a real shame.

On a whim, I tried using ChatGPT to generate some plugins.

Here’s an example of my old plugin and the new ChatGPT written plugin (in this order):

Default Series Title

See how bad that was? It completely mangled the title.

Edit:  Since publishing this post, I realized that I would have to choose between

  1. Leaving the old defunct plugin in place just to make a point about how it didn’t stand the test of a decade’s worth of WordPress updates, but then also leaving broken series titles sprinkled through my back catalog of blog posts.
  2. Go back through nearly 10 years of blog posts12 to change them over to the new plugin shortcode.
  3. Disable the old plugin, but have the new plugin work with the old shortcode as well as it’s own new shortcode, at the cost of losing an example of how bad the old plugin performed.

I went with option 3.  Just take my word for it, it looked bad.

He makes a valid point

He makes a valid point

Now for the ChatGPT version:

Software Development with ChatGPT
  1. ChatGPT WordPress Plugins

It took me about an hour to whip up a working WordPress plugin with the same core functionality.  I would break down the time I spent as follows:

Time Spent Creating Series Plugin with ChatGPT

But, that’s not all!  You see, as I was writing this blog post, I realized it would be fun to include a pie chart to indicate the time I’d spent on this.  Unfortunately, the plugin I had written to do exactly this many years ago has apparently completely given up the ghost.  Thus, before I proceeded to this very sentence, I used ChatGPT to create a plugin for displaying custom pie charts!

Time Spent Creating Pie Chart Plugin with ChatGPT

Obviously, this plugin took a lot longer.  The first few versions were having all kinds of problems between the HTML Canvas code and trying to figure out how to make sure the javascript was not loading too early or too late.  In the end, I just asked it whether it was capable of even creating a pie chart – and it gave me a piece of workable javascript.  I told it to refactor the plugin using this same javascript, and then it was a matter of fine tuning the result.

If you don’t know anything about writing WordPress plugins, you could probably use ChatGPT to create a very simple plugin.  However, once it got slightly more complicated, it would likely require some troubleshooting to figure out what was happening.  In the series plugin it took me a while to root through the WordPress functions to figure out that apparently ChatGPT was trying to use a function in a way that simply did not work.  I explained to ChatGPT that that particular function could not operate in that way, explained how the data it was feeding into that function needed to be first modified, and then asked it to refactor the code.  From that point forward, it started to look a lot better.  There were some additional quirks – like putting more than one series title in a single post would only display one.  I suspect these problems of ChatGPT taking a shortcut to generate code, hardcoding certain variables and names, not considering that it might need to operate more than once on the page, may be difficult for it to anticipate and address.  Without some degree of WordPress development knowledge, I think a novice user armed only with ChatGPT would need to do a lot of refactoring, asking the program to generate the plugin all over from scratch many times, before arriving at a workable result.  Then again, a million monkeys at typewriters, right?

I think ChatGPT could be great for creating relatively simple plugins like a series plugin, a pie chart plugin, or even a table of contents plugin.  However, having seen how much time it cut out of the development process, I think it would be interesting to try developing an A/B testing plugin or more complicated plugin.

I think the next task to see if I can get it to generate QMK code for a keyboard, Arduino code, Raspberry Pi code, or a chrome extension.

I can already see some ways to improve both of the ChatGPT generated plugins used in this blog post.  My series plugin included two arrows at the bottom so the reader could navigate to the prior or next post in the series.  And I think it would be great if the chart plugin had a feature where I could specify the units, so the magnitude data would be included with the labels.  I may try getting it to shoehorn these updates later…

If you see these reflected in the charts above, I must have already done it.  :)

Software Development with LLMs
  1. Series Plugin Test for Illustrative Purposes Only
  2. ChatGPT WordPress Plugins
  3. Coding with an LLM Sidekick
  1. NGL, I can really be a lot some times. [↩]
  2. Um, you’ve probably gathered that. [↩]

Series Plugin Test for Illustrative Purposes Only

Posted on by

The only purpose for this post is to serve as a reference for a more interesting and useful post.

Software Development with LLMs
  1. Series Plugin Test for Illustrative Purposes Only
  2. ChatGPT WordPress Plugins
  3. Coding with an LLM Sidekick
Software Development with LLMs
  1. Series Plugin Test for Illustrative Purposes Only
  2. ChatGPT WordPress Plugins
  3. Coding with an LLM Sidekick

[custom_pdf_generator visitor_data=“John Doe”]

 

How to add a custom button to the WordPress Visual TinyMCE Editor

Posted on by
Adding a button

Adding a button

The other day I found a great post that explained how to add a custom button to the WordPress Visual TinyMCE editor.1 It was fairly simple and concise and even includes a zip file of all the parts you would need to install a button into WordPress’s visual TinyMCE editor.  I’m not at all sure I could do a better job than that guy, so I’ll just direct you to the link above.  :)

Default Series Title
  1. Photo courtesy of Marco Bernardini [↩]

How to add a custom button to the WordPress editor

Posted on by

While posting about my DrawBot adventures, I’ve become increasingly annoyed with my own post series WordPress plugin.  If only it had a simple button in the WordPress editor to drop the shortcode for an existing series, it would instantly become so much better.  Thus inspired, I delved into WordPress.org and sought out the answers.  What I discovered is that while it’s not particularly difficult to add a button to the WordPress text editor, it didn’t seem to be very well documented.  I’m still working on how to add a graphical button to the WordPress TinyMCE visual editor, so stay tuned for that.  I was ultimately able to glean the necessary information from the WordPress “Quicktags API.  However, for your sake1 I’m distilling what I’ve learned right here.

First, I’ll give you an overview of what we’re going to do.  If you’re looking to add a button to the WordPress editor, it’s probably because you’re working on a plugin.  Thus, I’ll assume you have a working knowledge of plugin development.   Your plugin will need to include, at a bare minimum, one action hook and one function which performs the double duty of creating the button and giving the button some kind of ability.

// This action adds a script to the script footer on administrative pages
add_action( 'admin_print_footer_scripts', 'MBSS_quicktags', 100 );

// This is the function that is run at the end of the administrative pages in the script footer
function MBSS_quicktags()
     {
     // Optional database interactions
     // I used this section in my plugin to run a query on the database and return information that would be used in the button
     ?>

     <script type="text/javascript">
     // Creates button in the text editor
     QTags.addButton( 'MBSS_tag_id', 'Add Series', MBSS_add_series_js );
     // Function to append the series shortcode to the end of the post
     function MBSS_add_series_js()
          {
          editor = document.getElementById('content');
          editor.value = editor.value + "\n" + "EXAMPLE TEXT TO APPEND TO END OF POST";
          }
     </script>
     <?php
     }

And there you have it! One action hook and one function which adds some javascript magic to your WordPress editor in the form of a simple text editor button!

Default Series Title
  1. And, frankly, for my own future use [↩]

Apparently WordPress.org serves up even more delicious download stats

Posted on by
Lovin' that source code

Lovin’ that source code

While looking at the source code on my Simple Series plugin page, I began to wonder how WordPress.org was serving up all those nifty little stats into the downloads graph. ((Photo courtesy of Windell Oskay)) What I discovered is that there’s a little bit of jQuery magic going on that pulls download data from a WordPress.org API, formats it for niftiness, and inserts it into the dynamically draw SVG graphic of the downloads.

In case you’re interested, the API call looks like this:

http://api.wordpress.org/stats/plugin/1.0/downloads.php?slug=simple-series&limit=700&callback=?

I’m now wondering how I can use that data to show something interesting in my OCD Plugin Stats plugin.1

Default Series Title
  1. I should probably rename the meta plugin. [↩]

Simple Series WordPress Plugin Update

Posted on by
Making cereals

Making cereals

This weekend, almost an entire year from the last update, I made some revisions to my Simple Series plugin for WordPress.1 This is one of the plugins that I use the most often for WordPress.  It’s right up there, in terms of frequency, with the WP-Footnotes plugin that lets me insert footnotes.2

As much as I loved this plugin, it was always a little bit of  pain to use.  In order for two posts to be associated together in the same series, the title must be listed exactly the same in each post.  So, to make sure that I was getting it right, I would open an old post in the relevant series, copy the exact plugin shortcode with the exact title, and paste it into the new post.

With this update you can click the “Add Series” button in the TinyMCE text editor for a post and insert any series title you’ve previously used.  It’s not yet as elegant as I would like, but it makes a few button clicks out of a what used to be an annoying process.3 I’ve also added the ability to have a series between pages, posts, or any combination of the two kinds of content types.

Default Series Title
  1. Photo courtesy of Tavo [↩]
  2. Like this! [↩]
  3. Described in the prior paragraph [↩]

CompFight plugin, with 100% more awesome

Posted on by
Heart

Heart

I cannot express to you how much I just love this new CompFight plugin. 1  Snagging images off of CompFight/Flickr and dropping them into a post is so freaking easy now.  This is definitely going to become one of my stock-plugins for a fresh WordPress installation.  I’m happy to say that I contributed a small bit of code to this very very awesome plugin.  Since that comment, my modification of their code was merged into the main plugin.  I’ve also added a few small tweaks to my version of this plugin.  By modifying the javascript file very slightly, my copy of this plugin also:

  • Adds a caption, that includes the same text as the original photo on Flickr
  • Centers the image, using WordPress’s tags
  • Makes the photo credit part of the text.  I like to include the photo credit using slightly different language.  At some point I’ll get around to modifying the plugin so that I can save my format as a setting.

Admittedly, these implementations are just a little bit buggy – I just hacked those bits in without really doing any serious testing on them.  Once I have kicked the tires on this code a little, I’ll post it to the plugin’s page.  If you want to take a look at it before then, just drop me a line.

Sometimes the right post just needs the right picture – I’ve actually had a lack of a good photo hold up a post before.  I’m happy

  1. Photo Credit: seyed mostafa zamani via Compfight [↩]

Another way to perform A/B testing in WordPress

Posted on by

I ran across this article on how to do “split testing” also known as “A/B testing” in WordPress.  Basically, this process uses a Google Website Optimizer Plugin and Google’s Analytics tools to handle all the heavy lifting.  Once you create the control, test, and goal pages you would configure your Google Analytics account to look for trends in which of the two pages, the control or the test, performs better.

I’m not crazy about this method which relies on Google Analytics for two main reasons:

  1. Most importantly, it does not appear to swap control/test content for users.  Rather, it appears to simply observe which of the two concurrent pages works better.  This literally requires twice the marketing effort, since you would have to publicize both links to see which works better.  I prefer the ShrimpTest method which swaps out specific content in a single page.  This way all of your efforts can be concentrated onto one single page.
  2. This Google Analytics method is too complex and requires too much coordination between the website and Google.  Now, I love me my analytics.  I like seeing the ebb and flow of visitors, downloads, etc – and few people do it better than Google.  But, with A/B testing I just want to know as quickly as possible whether A or B is better.  Nothing else matters.  I want that knowledge as soon as possible so that I can move on to A/B testing something else.  With ShrimpTest I just need to configure the plugin, make the control and test content, and wait for a result.

I’m getting ready to launch a new product on this other website that I run.  I’ve already got ShrimpTest configured and ready to roll.  I cannot wait to see what happens.