Package dev.gmitch215.bytebox.binding

Interface Ai

All Superinterfaces:
org.teavm.jso.JSObject
public interface Ai extends org.teavm.jso.JSObject
The Workers AI binding. Declared with ai(), named AI.

Inference runs on Cloudflare's GPUs rather than in the isolate, so this always crosses the network and always blocks. There is no local emulation: a test either fakes this binding or points it at the real service. 

 TSObject input = TSObject.object();
 input.set("prompt", TSObject.of("why is the sky blue?"));
 String answer = env.ai().run("@cf/meta/llama-3.1-8b-instruct", input).get("response").asString();
Since:
1.0.0

  • Method Details

    • run

      default TSObject run(String model, TSObject input)
      Runs a model.
      Parameters:
      model - the model name, such as @cf/meta/llama-3.1-8b-instruct
      input - the model's input, whose shape depends on the model
      Returns:
      the model's output

    • run

      default TSObject run(String model, TSObject input, TSObject options)
      Runs a model with options such as gateway or returnRawResponse.
      Parameters:
      model - the model name
      input - the model's input
      options - the options
      Returns:
      the model's output

    • embed

      default TSObject embed(String model, String text)
      Embeds text with an embedding model, which is the common case worth naming.
      Parameters:
      model - the embedding model
      text - the text
      Returns:
      the response, whose data holds one vector per input

    • runModel

      org.teavm.jso.core.JSPromise<TSObject> runModel(String model, TSObject input)

    • runModel

      org.teavm.jso.core.JSPromise<TSObject> runModel(String model, TSObject input, TSObject options)