The GPT services out there use something called ‘tools’.
They get presented to the model and the model can ‘call’ a tool with arguments, which can then extract some data and input it into the context for the model to continue.
I found out, the models which can run on a normal PC (or even a Laptop) are okay, but not super great. (around or a bit worse than ChatGpt3)
The good stuff (e.g. Nous-Capybara 31B or the Mistral/Mixtral ones) needs some more memory and compute.
big reason why i switched to kopia, borg just doesnt cut it anymore…