MetatroncubeSoftwareSolutions/quorlasocial
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Model-server runs: 64k window, summarise at ~23k tokens

Browse Source 
Matches the laptop Ollama setup (num_ctx 65536 on the model): requests
stay readable within the agent's ~5-minute wait while big tool results
still fit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
metatroncubeswdev 2026-10-26 11:20:30 -04:00
parent 8170133a79
commit d314a7ad7c
1 changed files with 4 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
dashboard/server/jobs.js
View File

@ -152,11 +152,12 @@ function docStamp(slug) {
/** Runs on our own model server (Ollama etc.): every model slot points at the chosen model, and the Anthropic key stays home. */
function runEnv() {
if (!process.env.ANTHROPIC_BASE_URL) return undefined;
// Summarise the conversation at ~16k tokens (half of a 32k window): local models get unreliable on longer requests.
// Summarise at ~23k tokens (35% of the 64k window set on the Ollama model): a laptop reads ~100–150 tokens/s and the
// agent gives up on a request after 5 minutes, while the spare window leaves room for big tool results.
const env = {
...process.env, ANTHROPIC_AUTH_TOKEN: process.env.ANTHROPIC_AUTH_TOKEN || "ollama", CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC: "1",
CLAUDE_CODE_AUTO_COMPACT_WINDOW: process.env.CLAUDE_CODE_AUTO_COMPACT_WINDOW || "32768",
CLAUDE_AUTOCOMPACT_PCT_OVERRIDE: process.env.CLAUDE_AUTOCOMPACT_PCT_OVERRIDE || "50",
CLAUDE_CODE_AUTO_COMPACT_WINDOW: process.env.CLAUDE_CODE_AUTO_COMPACT_WINDOW || "65536",
CLAUDE_AUTOCOMPACT_PCT_OVERRIDE: process.env.CLAUDE_AUTOCOMPACT_PCT_OVERRIDE || "35",
CLAUDE_CODE_MAX_OUTPUT_TOKENS: process.env.CLAUDE_CODE_MAX_OUTPUT_TOKENS || "8000",
};
delete env.ANTHROPIC_API_KEY;