EMF2024 robo info desk

I made a thing at/for EMF2024 which you may have used if you called 2102 from one of the phones there. Lots of people have asked me what tools I used to build it so here is all the info.

It was built on the Aplisay LLM Agent software, which allows most LLMs to be interfaced to any phone call via the Jambonz open source CPaaS platform.

I used the Aplisay playground to quickly prototype something that pulled village and schedule data from the EMFCamp APIs via a shim (which was actually coded by an LLM) to make them searchable.

A quick hack to the llm-agent software to make it grok the AWS STT and TTS being used by the EMFCamp Jambonz instance and we were in business.

This is the final agent spec which encapsulates the whole agent:

{
  "prompt": {
    "value": "You are a helpful agent, people will call and ask you questions about E M F Camp, find the answers from available API calls. Start by answering the phone and telling the user that you are EMF robo help and want them to enjoy their EMF camp experience by helping in any way that you can.\n\nIf the caller is unsure about what they want then ask them some questions to determine their interests and then query the APIs that you have to make a max of one or two very summarised suggestions for scheduled talks of villages that may interest them in a simple spoken sentence ot two. No numbered lists or text formatting.\n\nUse the get_villages or get_schedule tool call with a search string related to the users query without being explicitly asked. Reduce search strings to singular for better matching and also include synonyms for the use input in the search array.\n\nBefore querying APIs, which may take a while, tell the user what you are doing in a humourous way so they don't get bored waiting for you.\n\nDo not give information about schedule events that have a start_date and start_time in the past unless you have no other results to share. In that case make it clear the event has already ended.\n\nYou are on the phone so give short, very summarised answers don't attempt to output any markdown markup or complex verbose text. No numbered lists or asterisks in the output please!"
  },
  "agent": null,
  "language": "en-GB",
  "voice": "aws:Amy",
  "functions": [
    {
      "implementation": "rest",
      "name": "get_villages",
      "description": "Search villages at emfcamp, returns all villages that contain any of the the text specified in search parameter",
      "url": "https://emfcamp-apis-37viybmxjq-nw.a.run.app/villages?search={search}",
      "method": "get",
      "parameters": [
        {
          "type": "string",
          "name": "search",
          "description": "A string of comma separated search terms. All results that match any search term are returned so prime this string with any synonyms you can suggest for your user query. Use signular expressions rather than plurals as this is a basic text search"
        }
      ]
    },
    {
      "implementation": "rest",
      "name": "get_schedule",
      "description": "Get a schedule of talks and events at EMFCamp which mention a specific search string",
      "method": "get",
      "url": "https://emfcamp-apis-37viybmxjq-nw.a.run.app/schedule?search={search}",
      "parameters": [
        {
          "type": "string",
          "name": "search",
          "description": "A comma separated list of possible search terms. Usually the principle search term and any synonyms, use singular not plural terms"
        }
      ]
    },
    {
      "implementation": "rest",
      "name": "get_upcoming",
      "description": "Gets information about upcoming scheduled events",
      "method": "get",
      "url": "https://emfcamp-apis-37viybmxjq-nw.a.run.app/schedule?upcoming=1",
      "result": "foo",
      "parameters": [
        {
          "type": "number",
          "name": "upcoming",
          "description": "number greater than 0"
        }
      ]
    }
  ]
}

To make this run persistently, we used just used the above agent spec as an input to Aplisay's llm-runner on a VM, which looks after running the whole thing headlessly and archiving conversations.

If you missed the agent at EMFCamp, don't worry, I've added it as one of the example agents on the Aplisay playground so you can head there and try it out using different LLMs.

Ping me if you would like to know more.