Set up an AI provider

One explanation card: API keys are added once for the whole app, and each project chooses a provider, model and prompt for each of its AI functions (roles).

1. How it fits together: keys for the app, models and prompts per project role

   API keys are added once, for the whole app. Each project then chooses a provider, an AI model and a prompt for each of its AI functions, called roles.

   

The app-wide LLM Configuration screen: active providers, then the providers you could add. A key is pasted into a masked field, tested with the provider and saved, and the provider moves to the active list without ever showing the key. A second provider is added, favourite models are starred, and the key-file location and Integrations are pointed out.

1. Click LLM Configuration and read what is active

   Click LLM Configuration. The top of the screen lists what is active: foundation model providers, platform providers, GPU rental platforms and model software platforms. vLLM needs no key, but it only works once you have started your own model server, for example on RunPod from AV.

   

2. Scroll to Not-yet-configured foundation model providers

   Further down, Not-yet-configured foundation model providers lists the ones you could add, each with Add Key.... Platform providers and GPU rental follow below.

   

3. Click Add Key... and paste your key (the field is masked)

   Click Add Key... beside the provider and paste the key from your account with it. The field is masked.

   

4. Click Test key

   Click Test key. AV checks the key with the provider before you save it.

   

5. Click Save key: the provider shows Configured

   Click Save key. The provider moves to Active foundation model providers. The row says Configured, and never shows the key.

   

6. Add a key for a second provider the same way

   Add a key for each provider you want to use. Here, a second provider as well.

   

7. Under Favorite models, click All and star the AI models you use

   Under Favorite models, click All and tap the star beside the AI models you use. ★ Favorites then lists just those, and so does the AI model list when you choose one for a project.

   

8. Open Configuration and key files to see where keys are kept

   Configuration and key files shows where AV keeps your keys, in a password-protected key file, and the provider and model lists it uses.

   

9. Use Integrations for your own servers and rented GPUs

   Servers of your own and GPUs rented on RunPod are set up under Integrations, with their own keys and tokens.

   

A new project with no models refuses to translate and says where to choose one, sending nothing. In the project's LLM Configuration, All roles at once gives every role the same provider and model; one role is then given a second provider and another a different prompt. Re-running All roles at once asks before replacing them.

1. In a new project, press Ctrl+Shift+M: AV names the role with no model and sends nothing

   In a new project every role is Not Set. Translate a segment with Ctrl+Shift+M and AV stops, names the role, and says where to choose an AI model. Nothing is sent.

   

2. Open the project's LLM Configuration

   Close the message and the editor, and open the project's LLM Configuration. All roles at once is at the top; below it, each role has its own provider, model and prompt.

   

3. Under All roles at once, click Choose...

   Click Choose... under All roles at once. The providers you added keys for are offered, with vLLM.

   

4. Choose the provider, then the AI model (★ Favorites shows your starred ones)

   Choose the provider, then the AI model. ★ Favorites shows the AI models you starred.

   

5. Check that every role has the AI model (saved as you go)

   Every role now uses the AI model you chose. Changes are saved as you make them, so there is no Save button. Do this in every project.

   

6. On one role, click Change Provider to give it a different model

   To give one role a different model, click its Change Provider. Here Single Segment Translation uses the second provider; the other roles keep theirs.

   

7. Click Change Prompt to choose another prompt

   Each role starts with the Default prompt. To use another, click Change Prompt. Prompts of your own are listed after Default.

   

8. Check the role now uses the prompt (here Formal register)

   Section Translation now uses Formal register. Default is always there to go back to.

   

9. Use All roles at once again: AV asks before replacing every role

   All roles at once replaces every role's model, including ones you changed singly, so AV asks first.