aux4/decision-gemma

Local decision provider for aux4/decision. It uses EmbeddingGemma 2 to rank choice options and a local Gemma 4 E4B chat server for noul yes/no estimates. No hosted model API or API key is used.

Installation

aux4 aux4 pkger install aux4/decision-gemma

Installation also installs aux4/decision and aux4/embedding.

Quick Start

choice runs locally with EmbeddingGemma 2 after aux4/embedding has downloaded its model on first use:

aux4 decision ask "Is this message spam?" --provider gemma --type choice \
  --options "spam=unsolicited or deceptive bulk email" \
  --options "not-spam=legitimate personal or transactional email" \
  --state "Congratulations, claim your prize now by clicking this link"

To use noul, start a local OpenAI-compatible chat server with a Gemma 4 E4B instruct model, then:

aux4 decision ask "Is this message spam?" --provider gemma --type noul \
  --state "Congratulations, claim your prize now by clicking this link" \
  --threshold 0.7

The command prints an estimate from 0.0000 to 1.0000. The exit code is 0 when the estimate meets the threshold, and 1 when it falls below.

Configuration

The same aux4/config fields work with decision ask and decision run. Use run to define a reusable question set and evaluate one or multiple samples in one command:

config:
  spam:
    provider: gemma
    model: mlx-community/gemma-4-e4b-it-bf16
    baseUrl: http://127.0.0.1:8381
    questions:
      isSpam:
        type: noul
        instructions: Is this email unsolicited or deceptive bulk email?
      category:
        type: choice
        instructions: Choose the best email category.
        options:
          spam: unsolicited or deceptive bulk email
          personal: legitimate personal message
          transactional: expected account, receipt, or delivery update
    state:
      - "Congratulations, claim your prize now by clicking this link"
      - "Your package is out for delivery today"

aux4 decision run --config spam

The questions map and repeated samples are passed through by aux4/config; no separate setup command is needed. ask also accepts repeated --options and repeated --state flags, or reads one sample from stdin when --state is omitted.

Provider Behavior

| Question type | Local model | Behavior | |---|---|---| | choice | EmbeddingGemma 2 | Ranks each option description against the question and sample using cosine similarity. scores are softmax-normalized similarity estimates. | | noul | Gemma 4 E4B instruct | Asks the local chat model for Yes/No and derives a score from its token logprobs. | | score | — | Not supported. The command returns an explicit error. |

Every response declares "provider": "gemma" and "scale": "estimate". These values are not calibrated probabilities. The choice score map is based on similarity, and the noul score is derived from a local model's token distribution. Validate any threshold on labeled examples for the messages, categories, and model you use. For safety-sensitive decisions, review low-confidence cases and retain a human override.

For noul, the provider requires both Yes and No candidates in the chat server's token logprobs. If the server omits logprobs or one candidate, the command fails instead of guessing a confidence. The command does not use --apiKey; the chat server must be reachable from the same machine.

Samples longer than 12,000 UTF-8 bytes are truncated to their leading portion before inference. For long messages, provide a representative excerpt or split the input before calling the decision command.

Choice uses a single batched aux4 embedding embed call for all samples and option descriptions in one ask or run. The first embedding call may install its runtime and download the model (about 314 MB with the embedding package's default q8 setting). Later calls use the local cache.

Options

| Variable | Default | Description | |---|---|---| | provider | bm25 | Set to gemma to select this provider. | | type | noul for ask | choice or noul. | | question | — | Short classification question or instruction. | | options | — | Repeated choice labels, optionally label=description. In run, use a map of labels to descriptions. | | state | stdin | Sample text. Repeat for multiple samples; stdin is one sample when omitted. | | questions | — | Named question map for run, normally sourced from config.yaml. | | model | mlx-community/gemma-4-e4b-it-bf16 | Local Gemma chat model ID. Used by noul. | | baseUrl | http://127.0.0.1:8381 | Base URL of the local OpenAI-compatible chat server. Used by noul. | | threshold | — | Optional noul estimate threshold from 0 to 1; exits 0 at or above it and 1 below it. | | top | 0 | For ask, print the top N ranked options with their scores. For run, limit each choice answer's scores map. |

Environment

AUX4_DECISION_GEMMA_MODEL — default local chat model ID.
AUX4_DECISION_GEMMA_BASE_URL — default local chat server base URL.
AUX4_DECISION_GEMMA_EMBED_BIN — override the aux4 executable used to call aux4 embedding embed.