Installing fahmek and auto-gad

fahmek hides bank and identity numbers before a chat message leaves the company, and auto-gad fixes French and English spelling and simple grammar. This guide installs both on one machine in about 30 minutes, connects them to a LiteLLM gateway the customer already runs, and prepares production.

French version: docs/fr/installation.md.

Why it exists#

The problem. Staff at banks, insurers and telecom operators paste real customer details into AI chat: a RIB to check, a CIN to fill a form, a phone number to call back. Each message then goes to an outside AI provider, often abroad, and the company loses control of that data.

What fahmek does. It sits inside the company's AI gateway (LiteLLM) and replaces each identifier with a label before the message leaves. The model sees «MA_RIB_1» instead of the account number. When the answer comes back, the gateway puts the real values back, so the employee reads a normal reply.

• Finds 7 Moroccan identifiers (RIB, IBAN, phone, CIN, CNSS, CNSS affiliation, ICE), plus email addresses and bank card numbers. ICE numbers are reported but left visible.
• Reads MSA Arabic, Darija in Arabic or Latin letters, French and mixed messages, including Arabic-Indic digits.
• Checks RIB keys and card numbers with their check digits, so random numbers are not masked.
• If fahmek is down, the strict policies stop the message rather than let it out unchecked.

What auto-gad adds. A spelling and grammar helper for typed questions. It fixes French and English mistakes (je veu → je veux, a la banque → à la banque) so the model understands the question better. It never touches Darija or Arabic, and it never blocks a message.

Why customers can say yes.

• Runs entirely on their own servers: no cloud service, no call to the vendor, no internet needed after installation.
• Plugs into the LiteLLM gateway they already run, even when it is managed only from LiteLLM's web UI. Chat apps and staff habits do not change.
• Adds a few milliseconds per message (under 15 ms for masking, about 20–50 ms for corrections).
• Helps meet Law 09-08 obligations on personal data sent abroad. It is a technical control, not a certification or legal advice.

What it does not do. It does not find names, addresses or dates of birth. It does not check images, audio, or new personal data the model invents in its answers. Accuracy on real customer messages has not been measured yet: see Known limits.

fahmek comes without an AI model. The customer keeps their provider (Anthropic, OpenAI, an internal server…). The demo setup has only a test backend that sends each message back, enough to see masking work.

How a message travels#

1. The employee types in the chatbot (Open WebUI) and presses Enter.
2. LiteLLM, the gateway, asks fahmek to find identifiers. fahmek swaps each one for a label like «MA_PHONE_1» and keeps the real value in Redis, in memory only, for a few minutes.
3. LiteLLM asks auto-gad to fix spelling and grammar. auto-gad asks LanguageTool, a spelling engine on the same machine.
4. LiteLLM sends the cleaned, masked message to the AI model.
5. The answer comes back. LiteLLM asks fahmek for the real values and puts them back in place of the labels.
6. The employee reads a normal answer with the real number in it.

Choose your path#

Before you start#

• A Linux computer or virtual machine, Ubuntu 22.04 or 24.04, with at least 4 CPU cores, 4 GB of memory (8 GB is better) and 10 GB of free disk (20 GB is better). The package's images are for Intel or AMD processors (x86-64); for an ARM machine, ask the vendor for the ARM package (see below).
• An internet connection for the first installation (several GB of downloads). After that, fahmek needs no internet.
• A user account that can use sudo.
• fahmek itself, from the vendor: the package file fahmek-0.1.0.tar.gz, or a temporary access token to download it with Git (Step 2).
• The license file fahmek.license, from the vendor.
• Optional: an AI provider key to try a real model (Step 3).

How to type the commands. Open a Terminal (on Ubuntu: Ctrl+Alt+T). Each grey box is a command: copy it, click in the terminal, paste with Ctrl+Shift+V, press Enter, and wait for the prompt (the line ending in $) before the next one. Lines starting with # are notes; the computer ignores them.

Check the machine is big enough:

nproc     # number of CPU cores: 4 or more
free -h   # "Mem" total: 4G or more
df -h ~   # "Avail": 10G or more
uname -m  # x86_64 (aarch64 = ARM: ask for the ARM package)

Intel/AMD or ARM. fahmek runs as Linux containers, and Docker runs Linux containers on any operating system: directly on Linux, on Windows through Docker Desktop and WSL2, on a Mac through Docker Desktop. What matters is the processor: a container image is built for one processor family and does not start on the other (exec format error).

The package's images are amd64 unless the vendor built an ARM package (its file name then ends in -arm64). With option B of Step 2, the images are built on the machine itself and always match it.

Step 1 — Install Docker and the small tools#

Docker runs each part of fahmek in its own sealed box (a container). Skip this step if docker compose version already prints a version.

1. Install the small tools (git downloads the code, python3 makes passwords, make gives shortcuts):

   sudo apt update && sudo apt install -y git python3 make curl

   Copy

2. Install Docker with its official installer:

   curl -fsSL https://get.docker.com | sudo sh

   Copy

3. Allow your user to run Docker without sudo:

   sudo usermod -aG docker $USER

   Copy

4. Log out and log back in (or restart the machine). Without this, the next command says permission denied.

5. Check that Docker works:

   docker run --rm hello-world
   docker compose version

   Copy

   You should see Hello from Docker! and then a line like Docker Compose version v2.x.

Step 2 — Get fahmek#

The vendor gives you one of two things. Both end with fahmek in a folder named fahmek in your home folder.

Option A — the package file (fahmek-0.1.0.tar.gz, about 600 MB, by download link or USB stick). It holds fahmek's four ready-made parts, the settings files and these guides. Say it is in your Downloads folder:

mkdir -p ~/fahmek
tar xzf ~/Downloads/fahmek-0.1.0.tar.gz --strip-components=1 -C ~/fahmek
cd ~/fahmek
for f in fahmek-image.tar fahmek-litellm-image.tar fahmek-demo-stub-image.tar fahmek-autogad-image.tar; do docker load -i "$f"; done
ls docker-compose.*.yml

Each docker load ends with a line like Loaded image: fahmek:0.1.0.

Option B — Git, with a temporary token from the vendor.

cd ~
git clone https://github.com/Anas10x/fahmek.git
cd fahmek
ls docker-compose.*.yml

Git asks for a username and a password: type the username from the vendor's message, then paste the access token as the password (nothing shows while you paste; press Enter). The token expires after a few days; you need it only to download.

Either way, the last command lists five files: autogad, chat, demo, litellm-ui and services.

Every command from now on runs in this folder. In a new terminal, type cd ~/fahmek first.

Step 3 — Write the settings file#

All settings live in one small file named .env in the fahmek folder. Paste this whole box at once: it creates the file, makes two random passwords and protects the file.

cat > .env <<EOF
# Shared secret between fahmek, auto-gad and the gateway (required, no default)
FAHMEK_API_KEY=$(python3 -c 'import secrets; print(secrets.token_hex(32))')
# Password apps use to call LiteLLM (the chatbot uses it too)
LITELLM_MASTER_KEY=sk-$(python3 -c 'import secrets; print(secrets.token_hex(16))')
# Masking policy: bank-pilot masks the most and blocks the call if fahmek is down
FAHMEK_POLICY_ID=bank-pilot
# 1 = readable trace of each message in the gateway logs (demos only)
FAHMEK_TRACE=1
EOF
chmod 600 .env

Look at it with cat .env: four settings with long random values.

Optional: connect a real AI model. Without this, the test backend sends each message back as the answer, which is enough for every check in this guide. To use a real model, add three lines. For Claude (Anthropic):

cat >> .env <<'EOF'
DEMO_MODEL=anthropic/claude-opus-5-5
DEMO_API_BASE=https://api.anthropic.com
DEMO_API_KEY=paste-your-anthropic-api-key-here
EOF

The key comes from the Anthropic Console (platform.claude.com) and needs API credits; a claude.ai chat subscription does not include API access. For OpenAI or any OpenAI-compatible server (an internal model server, for example), use DEMO_MODEL=openai/<model name>, DEMO_API_BASE=https://<server>/v1 and that server's key. If fahmek is already running, apply with $C up -d (Step 5).

Keep .env private: anyone who reads it can call the gateway and spend your provider credits. Never email it or put it in Git.

Step 4 — Put the license file in place#

The license is a small text file that says who may use fahmek and until when. Without it, fahmek refuses to mask, and the gateway then stops every chat message rather than send it unchecked.

1. Copy the file you received onto the machine. Say it is in Downloads.

2. Put it in a folder named license, under the exact name fahmek.license:

   mkdir -p license
   cp ~/Downloads/*.license license/fahmek.license
   ls license

   Copy

   The last command must print fahmek.license and nothing else.

You never restart anything to change a license later: fahmek reads this file again every minute. Step 6 checks that it is accepted.

Step 5 — Start everything#

fahmek comes as three pieces that stack together: the core (fahmek, Redis, LiteLLM, test backend), the chatbot, and auto-gad with LanguageTool. Save the three file names once in a shortcut called $C:

echo 'C="docker compose -f docker-compose.demo.yml -f docker-compose.chat.yml -f docker-compose.autogad.yml"' >> ~/.bashrc
source ~/.bashrc
echo $C

The last line must print the long docker compose ... command. From now on, $C means "all of fahmek", in every terminal.

Download the outside parts (Redis, LanguageTool, the chatbot). The box tries again by itself if the connection drops:

for i in 1 2 3 4 5; do $C pull --ignore-buildable && break; echo "retry $i"; sleep 10; done

Then start. With the package (option A), fahmek's parts are already loaded:

$C up -d --remove-orphans

With Git (option B), build them first, in the same command:

$C up --build -d --remove-orphans

Be patient. The first time takes 10 to 60 minutes, depending on the internet speed. Later starts take seconds. When the prompt comes back, wait one more minute: LanguageTool loads its dictionaries.

Check that everything runs:

$C ps

You should see seven lines (fahmek, redis, litellm, stub-provider, open-webui, autogad, languagetool), each Up or running. Exited or Restarting means something is wrong: see Troubleshooting.

Step 6 — Check that each piece answers#

First load the passwords from .env into this terminal (repeat in any new terminal):

set -a; . ./.env; set +a

Then paste each check and compare with the good answer.

Last, ask auto-gad to fix a sentence with one grammar mistake (veu) and two spelling mistakes (numero, telefone):

curl -s -H "X-Fahmek-Key: $FAHMEK_API_KEY" -H 'Content-Type: application/json' \
  localhost:8200/v1/correct -d '{"text":"je veu changer mon numero de telefone"}'

The answer starts with {"text":"je veux changer mon numéro de téléphone".

If check 2 shows missing, invalid or expired, its reason field says why: go back to Step 4. If check 4 shows false, or the sentence comes back unchanged, wait 30 seconds and try again: LanguageTool is still loading.

Step 7 — Try it in the chatbot#

1. Open a second terminal and watch the trace. It shows, for each message, what fahmek hid, what auto-gad fixed and exactly what the model receives. Leave it open:

   cd ~/fahmek && make trace

   Copy

   If fahmek is in another folder, start from that folder (for example cd ~/airs-lab/fahmek && make trace). Ctrl+C stops watching; the stack keeps running.

2. Open the chatbot. In a browser on the same machine, go to http://localhost:3000. From another computer, use the machine's address instead of localhost (find it with hostname -I).

3. Create the admin account. Click Sign up and enter a name, an email and a password. The first account becomes the administrator: do this yourself, right away.

4. Turn off the background requests. The chatbot sends hidden requests to make titles and tags, which clutter the trace. Click your name (bottom left), then Admin Panel → Settings → Interface, switch off Title Generation, Tags Generation and Follow Up Generation, and click Save.

5. Pick the model demo-model at the top of a new chat.

6. Send the test messages below, one by one. They hold only fake numbers. Never test with a real person's details, including your own.

Each message gives four lines in the trace:

[trace 48b77b] user wrote (identifiers hidden): Bonjour, je veu changer mon numero de telefone, c'est le «MA_PHONE_1».
[trace 48b77b] fahmek masked: MA_PHONE x1 (policy bank-pilot)
[trace 48b77b] auto-gad fixed: fr: veu → veux, numero → numéro, telefone → téléphone
[trace 48b77b] model receives: Bonjour, je veux changer mon numéro de téléphone, c'est le «MA_PHONE_1».

The masking must match exactly; corrections can differ slightly with LanguageTool's version. With the test backend, the answer repeats your message with the real number back in place: proof that the round trip works, and http://localhost:9200/last_request shows what the backend received, with labels instead of numbers. With a real model, the answers are real ones, and the trace's model receives line is the proof.

What auto-gad corrects, and what it leaves alone#

auto-gad fixes a word only when it is sure; when in doubt, it leaves the word as typed, because the model understands small mistakes anyway.

• Spelling: a misspelled word gets the closest dictionary word, if one is clearly closest.
• Grammar: some rules read the neighbouring words, for example a verb after je or il a. A grammar fix applies only when the rule proposes exactly one single-word replacement. Style and punctuation never change.

Results measured with the installed LanguageTool:

It works sentence by sentence: in a message mixing Darija and French, only the French sentence is corrected. Grammar checks add no measurable delay. If LanguageTool is slow or down, the message goes through uncorrected after 3 seconds; auto-gad never blocks a message.

Managing it on site#

Once installed, fahmek needs about an hour of care a year: renew the license, apply updates, keep the secrets secret. Everything runs from the ~/fahmek folder.

The license, month by month#

fahmek reads license/fahmek.license every minute and moves through these states on its own:

To renew, copy the new file over the old one. Nothing needs restarting:

cp ~/Downloads/*.license license/fahmek.license

The file cannot be edited: changing one character (the date, say) breaks its signature, and fahmek treats it as invalid. An expired license never lets messages through unchecked; it stops them. Putting real values back in answers keeps working.

Changing settings#

Edit .env with a text editor (nano .env; save with Ctrl+O, Enter; quit with Ctrl+X), then apply with $C up -d. The most useful setting is the masking policy:

auto-gad's behaviour lives in auto-gad/config.yaml, with a comment above each setting: Darija words it must never touch, the words it uses to recognise each language, brand names never to correct (never_correct), the grammar rule groups (grammar_categories; empty = spelling only) and the shortest word that may get a letter change (min_word_length, 5). After editing it, restart auto-gad only:

$C restart autogad

Updating#

When the vendor announces a new version. Settings (.env), the license and chat history are kept; run the checks of Step 6 afterwards.

With the package (option A), unpack the new file over the folder, load its parts and restart. A word list you changed yourself (auto-gad/config.yaml) is replaced: copy it aside first and compare.

tar xzf ~/Downloads/fahmek-0.2.0.tar.gz --strip-components=1 -C ~/fahmek
cd ~/fahmek
for f in fahmek-image.tar fahmek-litellm-image.tar fahmek-demo-stub-image.tar fahmek-autogad-image.tar; do docker load -i "$f"; done
$C up -d --remove-orphans

With Git (option B; the vendor sends a new token if the old one expired):

git pull
$C up --build -d --remove-orphans

Logs#

The logs never contain the numbers fahmek detected, or the passwords. The trace shows messages with identifiers hidden, but names, addresses and anything fahmek does not detect appear as typed: switch it off (FAHMEK_TRACE=0, then $C up -d) before real staff use the system. The trace covers only this machine's gateway (port 4000), not a customer's own LiteLLM.

Security rules#

• Keep .env readable only by the admin (chmod 600 .env, done in Step 3).
• Never open ports 3000, 4000, 4001, 8100, 8200 or 9200 to the internet. Keep the machine on the internal network.
• To change the shared secret, put a new FAHMEK_API_KEY in .env and run $C up -d: every part restarts with the new value together.
• Redis keeps the real values in memory only, a few minutes per message, never on disk. Under a strict policy, fahmek refuses to start if Redis is set to save to disk.
• The chatbot keeps its chat history unmasked on this machine's disk. Test only with fake values, and treat the machine as holding personal data once staff use it.

Customer who already runs LiteLLM#

A customer with their own LiteLLM gateway skips the demo chatbot. You start fahmek beside their gateway, then add it from LiteLLM's web UI, with no file to copy and nothing to change on their server.

1. Start fahmek beside the gateway. On a machine the gateway can reach, do Steps 1 to 4 (Docker, fahmek, settings file, license). The masking policy is the FAHMEK_POLICY_ID line of .env: it is chosen on the fahmek side, so no app can ask for a weaker one. With Git (option B), build fahmek's parts once: docker compose -f docker-compose.demo.yml -f docker-compose.autogad.yml build fahmek autogad. Then start fahmek, Redis, auto-gad and LanguageTool, without the chatbot or test backend:

docker compose -f docker-compose.services.yml --profile autogad up -d

fahmek now answers on port 8100 and auto-gad on port 8200 of that machine.

On Kubernetes, replace this step with Customer on Kubernetes.

2. Add the guardrails in the LiteLLM web UI. Open http://<gateway>:4000/ui and sign in as an admin. Use the Submitted Guardrails tab, not Add New Guardrail → Add Provider Guardrail: that form has no field for fahmek's address or key.

1. Teams → Create Team, Team Name fahmek, Create Team. The team is only a label: the guardrails protect every key and request of the gateway.

2. Guardrails → Submitted Guardrails → Add Guardrail. Fill in one line of the table, pick the team fahmek, click Submit for Review. Three times:

   Guardrail Name	Mode	API Base URL	Additional litellm_params
   fahmek-mask	Pre Call	http://<fahmek machine>:8100	{"api_key": "<FAHMEK_API_KEY>", "default_on": true, "unreachable_fallback": "fail_closed"}
   fahmek-restore	Post Call	http://<fahmek machine>:8100	{"api_key": "<FAHMEK_API_KEY>", "default_on": true, "streaming_transform_mode": "incremental_diff"}
   auto-gad (optional)	Pre Call	http://<fahmek machine>:8200	{"api_key": "<FAHMEK_API_KEY>", "default_on": true}

3. Still in Submitted Guardrails, click Approve on each, then Approve again to confirm. They work at once.

Replace <FAHMEK_API_KEY> with the value in .env (grep FAHMEK_API_KEY .env). The UI accepts an address only if the machine is an IP address, localhost, or a name with a dot (fahmek.bank.local works, fahmek does not).

3. Add models and test in the Playground. A model added later needs no extra step: the guardrails are default_on, so they protect every model of the gateway, old and new. Checked in LiteLLM v1.105.1.

1. Models + Endpoints → Add Model. Provider: e.g. Anthropic, or OpenAI-Compatible Endpoints for another compatible server (then also its API Base). LiteLLM Model Name(s): a model from the list (e.g. claude-opus-5-5), or Custom Model Name (Enter below) and the provider's exact name. Public Model Name: the name apps will use. API Key: the provider's key. Click Test Connect, then Add Model.
2. Playground → Chat. Virtual Key Source Current UI Session, Endpoint Type /v1/chat/completions, Select Model: the new model. Leave Guardrails empty: the fahmek guardrails run anyway.
3. Type a message with fake values and press Enter, for example Bonjour, mon RIB est 999001123456789012345691 et ma CIN AB123456. Merci. The answer shows the RIB and CIN as typed: hidden from the model, then put back. The Playground streams, so this needs incremental_diff (LiteLLM v1.96 or newer); otherwise the answer shows «MA_RIB_…» labels (nothing leaked).
4. See what the model received. The Playground shows only the restored answer. In Logs, click the request: a green guardrails evaluated badge lists fahmek-mask, auto-gad and fahmek-restore (fahmek-restore counts once per streamed round). The provider's request logs, if it keeps them, show the labels.

By default LiteLLM's Logs keep neither the message text nor the values. Keep store prompts in spend logs off (Admin Settings → Logging Settings): switched on, LiteLLM stores conversation text in its own database, outside fahmek's protection.

4. What to tell the customer about this mode.

• If fahmek is unreachable, LiteLLM's unreachable_fallback decides, not the fahmek policy: keep fail_closed with bank-pilot or bank-ma-strict. Every fahmek refusal (license, size, rate limit, blocked identifier, images) is a block that LiteLLM always applies.
• Function calling: a request whose function-call history holds an identifier is refused, because LiteLLM cannot take masked arguments back from a web-UI guardrail. Apps that use function calling need the integration kit.
• Streaming: real values are put back only with streaming_transform_mode: incremental_diff, on LiteLLM v1.96 or newer; older releases show the labels (nothing leaks).
• No trace in this mode: check with the Playground and Logs.

Tested on unmodified LiteLLM v1.89.7 to v1.105.1 (integrations/litellm/COMPATIBILITY.md, "web UI" column).

Rehearse on your demo machine first#

docker-compose.litellm-ui.yml starts a second LiteLLM beside the demo, exactly as a customer runs it: the official image, untouched, with the Postgres database its web UI needs, on port 4001 and with no guardrail. Start it (the first time downloads LiteLLM and Postgres):

cd ~/fahmek
$C -f docker-compose.litellm-ui.yml up -d

1. Open http://<machine address>:4001/ui. Sign in with user admin and the LITELLM_MASTER_KEY value from .env as the password. A red banner about "environment-credential login" is expected on a demo.
2. Do step 2 above with these addresses (container names inside Docker): http://fahmek.internal:8000 for both fahmek guardrails and http://autogad.internal:8200 for auto-gad.
3. Do step 3 above: the model demo-model (the one from .env) is already there, and you can add others.

The demo chatbot (port 3000) keeps using the first gateway (port 4000). Remove the second one with $C -f docker-compose.litellm-ui.yml rm -sf litellm-ui litellm-ui-db.

Customer on Kubernetes#

On Kubernetes, nothing in Steps 3 to 7 applies: there is no .env, no $C and no demo chatbot. The Helm chart in the package (deploy/helm/fahmek) installs fahmek, its in-memory Redis and, optionally, auto-gad with LanguageTool. The customer's LiteLLM stays as it is and adds fahmek as guardrails, exactly as in Customer who already runs LiteLLM.

What you need: kubectl and Helm 3 rights on one namespace (Kubernetes 1.25 or newer), a private registry the cluster can pull from, a machine with Docker to push the images, the package file, and the license file.

1. Push the images to the registry. On the machine with Docker, in the unpacked package (Step 2, option A), with your registry's address:

REG=registry.bank.local/fahmek
for f in fahmek-image.tar fahmek-autogad-image.tar; do docker load -i "$f"; done
docker pull redis:7-alpine && docker pull erikvl87/languagetool:6.8
for img in fahmek:0.1.0 fahmek-autogad:0.1.0 redis:7-alpine erikvl87/languagetool:6.8; do
  docker tag "$img" "$REG/$img" && docker push "$REG/$img"
done

Without internet, load Redis and LanguageTool from the files of Installing without internet instead of docker pull.

2. Create the namespace and the two Secrets. The chart never holds a key: it reads these two.

kubectl create namespace fahmek
kubectl -n fahmek create secret generic fahmek-api-key \
  --from-literal=FAHMEK_API_KEY="$(openssl rand -hex 32)"
kubectl -n fahmek create secret generic fahmek-license --from-file=fahmek.license=./fahmek.license

Keep the key: the LiteLLM guardrails need the same value (kubectl -n fahmek get secret fahmek-api-key -o jsonpath='{.data.FAHMEK_API_KEY}' | base64 -d).

3. Write the values file. Save as fahmek-values.yaml; every other setting has a safe default (see values.yaml in the chart):

imageRegistry: registry.bank.local/fahmek
# imagePullSecrets: [{name: regcred}]   # if the registry needs a login
fahmek:
  replicas: 2
  workers: 2          # one per CPU core requested (resources.requests.cpu)
  policyId: bank-pilot
networkPolicy:
  clients:            # only the LiteLLM pods may call fahmek and auto-gad
    - namespaceSelector:
        matchLabels: {kubernetes.io/metadata.name: litellm}
# autogad:
#   enabled: false    # without the spelling corrector (saves about 2 GB of memory)

Replace litellm with the namespace where LiteLLM runs. Network rules take effect only if the cluster's network plugin enforces them (Calico, Cilium and most managed clusters do).

4. Install and check.

helm install fahmek ./deploy/helm/fahmek -n fahmek -f fahmek-values.yaml
kubectl -n fahmek get pods          # all Running and READY 1/1 after a minute or two
helm test fahmek -n fahmek --logs

The test ends with ok, after readyz: ready, license: valid, inspect decision: redact and the auto-gad line. LanguageTool takes a minute or two to load; until then auto-gad is not ready.

5. Add the guardrails in LiteLLM. Do step 2 of Customer who already runs LiteLLM, with the cluster addresses that helm install prints:

(Release name fahmek in namespace fahmek; otherwise http://<release>.<namespace>.svc.cluster.local:8000.) With the integration kit instead, set FAHMEK_URL and AUTOGAD_URL on the LiteLLM deployment to the same addresses. Then add models and test in the Playground as in step 3 there.

Day to day on Kubernetes

The load test from inside a fahmek pod shares that pod's CPU: it shows whether a rate holds, but for capacity figures run it from a separate pod or machine (Going to production).

Rehearse on one machine with kind

kind runs a one-node Kubernetes cluster inside Docker, handy to rehearse before the customer's cluster. Install kind, kubectl and helm from their official sites (pick the build for the machine: amd64 or arm64). In the fahmek folder, with the fahmek images loaded (Step 2; on an ARM machine, built with option B, since the package holds x86-64 images):

kind create cluster --name fahmek
docker pull redis:7-alpine && docker pull erikvl87/languagetool:6.8
ARCH=$(docker version -f '{{.Server.Arch}}')
for img in fahmek:0.1.0 fahmek-autogad:0.1.0 redis:7-alpine erikvl87/languagetool:6.8; do
  docker save --platform "linux/$ARCH" "$img" \
    | docker exec -i fahmek-control-plane ctr --namespace=k8s.io images import --digests --snapshotter=overlayfs -
done

(The loop replaces kind load docker-image, which fails with content digest ... not found on recent Docker. Lines saying mismatched image rootfs and manifest layers can be ignored.) Then do point 2 with ./license/fahmek.license, and install with less CPU:

helm install fahmek ./deploy/helm/fahmek -n fahmek --set fahmek.workers=1 --set fahmek.resources.requests.cpu=250m --set fahmek.resources.limits.cpu=1 --set autogad.languagetool.resources.requests.cpu=250m

Check with point 4, then remove everything with kind delete cluster --name fahmek.

Good to know

• Redis must not save to disk. The chart's Redis runs with --save "" --appendonly no and no volume. Do not replace it with a Redis chart that keeps data on disk (Bitnami's does by default): under bank-pilot, fahmek then refuses to start. To use an existing Redis, set redis.enabled: false and redis.externalUrl.
• Hardening: fahmek, auto-gad and Redis run as non-root users with a read-only file system and no Linux capabilities. Readiness uses a small command inside the container, so the key never appears in the manifests.
• Tested on 2026-10-19 on a one-node kind cluster (Kubernetes 1.31, ARM64): all five pods became ready and helm test passed (readiness, license, a synthetic CIN masked, auto-gad). This first real cluster run found that pods were refused with CreateContainerConfigError (fixed in the chart since). kind does not enforce network rules and has one node, so do the first installation on the customer's test cluster too.

Going to production#

The demo machine is one box: one LiteLLM process, one fahmek, no TLS. A production deployment for many users runs several fahmek instances, a higher rate limit and a load test on the customer's hardware. The full guide, with measurements, is docs/production-sizing.md in the package or repository.

Size from calls per second, not from users.

turns per second = users chatting at peak × messages each sends per minute ÷ 60
fahmek calls/s   = turns per second × calls per turn

A turn costs 2 calls through the web UI (more when streamed: one per round), but one call per message of the conversation with the integration kit. In the sandbox measurements, one fahmek worker (one CPU core) handled about 150 calls per second. Example for 10,000 users chatting at once, one message every 30 seconds (333 turns per second), with 50% headroom:

fahmek is rarely the limit: LiteLLM itself, the model provider's quotas and auto-gad (about 30 LanguageTool cores at that rate; switch it off or run several) need sizing too.

Settings (in .env next to the compose file):

On Kubernetes the same settings are chart values: fahmek.workers, fahmek.inspectRps, fahmek.inspectBurst, fahmek.replicas (or fahmek.autoscaling); the chart's Redis is already in-memory.

Load test. scripts/load_test.py sends fake conversations at fixed rates and prints the turns per second held, the share that succeeded and the latency. It runs with the fahmek image's own Python, so nothing needs installing. Run it against a test instance set up like production, never against production during working hours:

cd ~/fahmek && set -a; . ./.env; set +a
docker run --rm --network host -e FAHMEK_API_KEY \
  -v "$PWD/scripts:/scripts:ro" --entrypoint python fahmek:0.1.0 \
  /scripts/load_test.py --mode ui --url http://localhost:8100 \
  --history 10 --stream-rounds 5 --rates 100,200,400 --duration 60 --max-p99-ms 100

--mode ui copies the web-UI guardrails, --mode kit the integration kit; --history is the customer's typical conversation length. The capacity is the highest rate where ok/s equals the target and ok share is 1.0; plan the peak at two thirds of it. rate_limited means raise FAHMEK_INSPECT_RPS; budget_exceeded or latencies in seconds mean add workers or instances.

Before go-live:

• Peak estimated with the customer's real conversation length.
• Load test passed at 1.5 × that peak, on the production hardware.
• At least two fahmek instances; workers and rate limit set.
• Redis without persistence; /readyz ready on every instance.
• FAHMEK_TRACE=0; LiteLLM prompt storage off.
• bank-pilot (or stricter) and unreachable_fallback: fail_closed.
• License expiry monitored (GET /v1/license, days_left).
• LiteLLM, the provider's quotas and auto-gad sized for the same peak.

Addresses and ports#

From another computer, replace localhost with the machine's address (hostname -I). None of these should be reachable from the internet.

Redis and LanguageTool have no address of their own: only the other parts reach them. Existing company apps connect to port 4000 with the LITELLM_MASTER_KEY, like any OpenAI-compatible service.

Installing without internet#

For a machine that can never reach the internet, carry everything over on a USB disk. The isolated machine needs Docker already installed (your IT team can install it from offline packages).

On a connected machine, save the three outside parts next to the package file:

for img in redis:7-alpine erikvl87/languagetool:6.8 ghcr.io/open-webui/open-webui:v0.7.2; do
  docker pull "$img" && docker save -o "$(echo $img | tr '/:' '__').tar" "$img"
done

Copy the package file, the three .tar files and the license file to the USB disk.

On the isolated machine, put the files in your Downloads folder, do Step 2 option A, then load the outside parts:

for f in ~/Downloads/*.tar; do docker load -i "$f"; done

Continue with Steps 3 and 4, save the $C shortcut of Step 5 and start with $C up -d --remove-orphans, without the download box. Then Steps 6 and 7. A real AI model must be reachable from the isolated network (an internal model server); otherwise keep the test backend.

Troubleshooting#

Most problems come from three things: a terminal in the wrong folder, a part still starting, or a step skipped.

Stop, start, reset#

Redis never saves to disk, so after a restart, labels in answers that were in progress can no longer be filled in. Only chats mid-answer during the restart are affected.

Known limits#

Say these to a customer before the pilot; they decide what to measure.

• Accuracy on real messages is not measured yet. Current figures come from 65 made-up messages. The pilot should measure a reviewed sample of the customer's own messages, per language.
• Identifiers only. Names, addresses and dates of birth are not detected; they reach the model and appear in the trace.
• auto-gad fixes only what is certain. Many mistakes are left as typed, by design.
• The Darija word list (about 100 words in auto-gad/config.yaml) needs review by a native speaker before the pilot.
• Models rewrite labels. fahmek restores them without their « », in bold or with escaped underscores; if a model changes the code itself, the answer shows the label instead of the number. Nothing leaks, but the reply reads badly.
• The demo machine is a pilot setup, not production: no HTTPS between the parts, one shared password, a single Redis, and the chatbot stores its history unmasked on disk. See Going to production.
• LanguageTool's image is community-maintained (erikvl87/languagetool); LanguageTool publishes no official one.
• The first French check after a start takes a few seconds; after that, about 20–50 ms.