How I sped up local Qwen 27B on a Mac with M4 Max — and what actually happened

I wanted to use a local model to work with code: open a project, give it tens of thousands of tokens of source code, ask questions and continue the conversation without waiting several minutes after each file it reads. The computer is a Mac with an M4 Max, 32 GPU cores and 36 GB of unified memory.

At first the goal seemed straightforward: a 27B-class Q4 model, a long context up to 128k, prefill at about 300 tokens per second and generation at no less than 30 tokens per second. Later I accepted 255 tokens per second as the lower bound for prefill speed. And of course I wanted that speed without a noticeable loss of quality.

Together with an AI assistant, we built a working setup around Qwen3.8-27B and a modified Kapitsa. In one cold test with 144,585 input tokens, it processed the request at 355.48 tokens per second. In another long-context test, code generation reached 48.25 tokens per second. But a Russian-language answer to another request ran at around 22.11 tokens per second. Short additions to a long context could also be processed considerably slower than 255 tokens per second.

These are experimental results for specific requests, not a guaranteed minimum for every context. The main limitation is that we changed the attention computation during prefill to gain speed. All input tokens are retained, but the model's answer need not match its answer with full attention. A test at around 256k context produced a quality error; we could not confirm an overall quality loss of no more than 1–2%.

I will explain how the setup works, what we measured, where we fell short and how to try reproducing it. At the end there is a ready-to-use prompt for an AI assistant to install it and run its own checks.

Where we started and what we achieved

Before the changes, large requests had a prefill speed of roughly 135–200 tokens per second on my machine; for long contexts the reference point was around 135 tokens per second. After tuning, one cold test on 144,585 tokens gave 355.48 tokens per second. Comparing these particular numbers gives approximately 2.6 times faster, or a practical improvement of around 2.5 times relative to my initial observation.

This gives a sense of the scale of improvement in my work, but it is not a strict benchmark of the same request before and after: scenarios and conditions may have differed. We therefore also kept a separate paired measurement using the same long document:

Scroll horizontally if the table does not fit.

Both runs used the same document and speculative decoding, but the generated responses differed in length: 512 and 1,024 tokens. This is one measurement, and generation speed depends on the response content and decoding behaviour; it cannot be taken as a promise of the same improvement on every task. Specific measurements for later versions and other scenarios are presented separately below.

The speedup does not mean we proved that quality was completely unaffected. On six selected LongBench v2 tasks and four questions about a context of around 50k, the answers from the full and accelerated profiles matched. This is a good result for those cases, but the sample is small and approximate prefill changes the computation. The more precise conclusion is that we achieved a substantial speedup and found no differences in these paired checks, but did not establish that quality is preserved on every task.

What exactly I wanted to speed up

First, let us separate the model's work into two stages. Otherwise the numbers are easy to confuse.

Prefill is reading the input. Before starting its answer, the model processes system instructions, conversation history, code, files and tool results. If a request contains 50,000 tokens and the system processes them at an average of 200 tokens per second, reading alone takes around four minutes.

Generation, or decode, is producing the answer after reading the input. For example, 30 tokens per second means a 900-token response takes around 30 seconds after generation starts. Comparing this speed using an answer of one or two tokens is meaningless: such a result is too small and depends on overhead.

A third important measure is cache reuse. When I ask the model about a project a second time, the client usually sends the previous history again. If the beginning matches what has already been processed, the server can restore the saved state and process only the new tail. A speedup from the cache means less repeated work; it does not by itself mean that computing each new token has become faster.

In an interface, all three measures are easy to mix into one. A “tokens per second” figure can be misleading without saying exactly what was counted. We therefore tracked the full input, the matching prefix, tokens actually recomputed, prefill time, generation time and the first emitted token separately.

What makes up the setup

We used Youssofal/Qwen3.8-27B-MTPLX-Bare-Speed as the main model: Qwen3.8-27B weights quantized to Q4. A reproducible model revision is pinned in the accompanying package. On my machine, the three main weight files and tokenizer matched that revision by SHA256.

The server was Kapitsa, a separate native engine for Apple Silicon that uses Metal and the Neural Engine for some computations. It can read compatible weights but is not itself the MTPLX interface. Kapitsa's architecture documentation has more details.

We took a particular Kapitsa commit and modified it slightly: we added approximate prefill, changes to cache handling and monitoring. An ordinary Kapitsa installation or running the same weights in MTPLX will therefore not automatically include all the changes described here.

To speed up generation, we added a separate incoai/Qwen3.8-27B-DFlash2 draft model. It proposes several next tokens, which the main model checks. We used the original BF16 draft weights and enabled loading them in Q4 inside the engine. This is DFlash2, not MTP. The draft helps generation; the measured cold-prefill speedup should not be attributed to it.

We also enabled Q8 for the KV cache: saved data that attention uses in subsequent computations. This is another approximation compared with storing KV at higher precision. Its role is to reduce memory use for long contexts and make the chosen capacity practical on a 36 GB machine.

Scroll horizontally if the table does not fit.

The Neural Engine is indeed used for part of prefill in this setup. The NI_ANE_UNITS=30 setting in our profile specifies a share of the work, expressed by the engine in blocks of 512 channels; it does not claim that the chip has 30 Neural Engine cores. The optimal value for another Mac must be measured again.

Where the fast prefill came from

With full attention, each new position considers the preceding positions available to it. As context grows, the amount of this work grows. We chose an experimental policy that limits the attention area for a substantial part of the input: it retains a global region at the beginning and a recent window.

Our specific profile used:

• the first 8192 positions as the global region
• the last 8192 positions relative to the token under consideration as the recent window
• dense attention for the last 2048 input positions in a cold request
• a dense tail of 128 positions in the selected continuation mode.

Separate parts of a restored tail can be recomputed with dense attention depending on which cache checkpoint is found. During generation, dense attention is applied to the available cache.

The qualification matters. We do not remove input tokens, but during prefill many of their representations are computed using a limited attention area. Later dense decode therefore operates on states that were already computed approximately. It does not turn the earlier processing into full attention. In addition, continuation uses saved states from previous computation epochs: the result can depend on conversation history and differ from a new cold pass over the same complete history.

The speed was not free: we deliberately changed the computation and accepted a risk to quality. For tasks that must strictly account for relationships between all parts of a long document, this profile needs particularly careful testing.

Why the cache sometimes appeared to reset

During real work, an agent does more than append a message. It can read a file, call a tool, rewrite the last message, format reasoning history differently or change system instructions and the tool list. The server compares the actual token prefix. If no suitable saved state exists, part of the history must be recomputed.

We found cases where a large part of the history matched, but the closest available checkpoint was far behind. The model recomputed thousands of already known tokens. From the outside it looked as if the cache had reset: change the subject, and you wait again.

We added several protected checkpoints: at the input boundary, before the user turn, near the response header and at selected points during generation. On restoration, the engine checks the matching prefix, whether the state belongs to the current context, and compatibility of the attention policy. An unfinished state after partial prefill cannot simply be declared a ready cache. After some cancellations the engine returns to an earlier confirmed checkpoint; when no suitable checkpoint exists, it recomputes more or starts again.

One control case looked like this:

Scroll horizontally if the table does not fit.

In another separate test, a change inside a long response produced the following result:

Scroll horizontally if the table does not fit.

In the latter case, the required new tail remained around 3967 tokens in both versions. The gain came from reducing repeated processing, not removing new text. Across both control pairs, all eight checked answers matched before and after. These are individual scenarios, not a guarantee that the cache will never reset again.

The number of checkpoints is finite. For example, a change to an early system instruction or the tool list, a switch to another conversation, or an edit before the available checkpoint can still require substantial recomputation. In our case, an additional archive checkpoint occupied around 189.6 MiB.

What the measurements showed

The main comparison used the engine's final counters, not just the changing figure in the monitor while a request was running. Prefill speed was calculated from tokens actually processed, including necessary replay, rather than the full history already retrieved from cache.

The following table refers to the historical verified E2 build e2dcb749f9b5…, not a separate repeat benchmark of every later change:

Scroll horizontally if the table does not fit.

That is why a headline saying “prefill is now always 355 tokens per second” would look attractive but be false. A large cold request and a short addition to a long cache behave differently. The latter may have different overhead and more work involving the accumulated state. Generation speed also depends on the task and on speculative decoding behaviour.

Later, we separately tested the adaptive packedR4 path for Q8 KV. In one pair of identical Russian tasks with 146,935 input tokens, generation speed increased from 21.858 to 24.366 tokens per second. The checked answers in this series matched the control, but this Russian task still did not reach 30 tokens per second. This result belongs to a separate build and comparison, not the E2 table above.

To put the speed into more familiar minutes: if average speed throughout a request were the same 355.48 tokens per second, processing 131,072 tokens would take around 369 seconds, or 6 minutes 9 seconds. Actual speed can vary with length, memory state and profile. A client with a 300-second timeout could therefore cancel such a request before the answer. The client timeout and the server gateway timeout are separate settings.

We made a separate monitor at /monitor. It shows active and recently completed requests, computed and cached volumes, prefill, generation and time to first token. During a request, speed may be preliminary; final data from completed requests is better for comparison. Request preparation, tokenization and data transfer time can differ from pure prefill computation time.

Did quality suffer?

We compared the variants on a real WordPress project: 10 questions about the source code and 6 small PHP/JavaScript tasks. Inputs and checks were fixed beforehand, and then a replay of 51 requests was run with contexts up to 157,202 tokens. Answers from the version under test were not inserted as later inputs, allowing the fixed requests to be compared.

Across three checked variants, results for the 16 tasks matched:

• Original strict protocol: 9 out of 16
• Separate JSON fact analysis: 10 out of 10
• Separate code checks with a supplemented DOM test: 5 out of 6.

These figures cannot simply be added or substituted for the original 9/16: they use different checking rules. One error also occurred in the full-attention control: a month validator accepted a string with a trailing newline. After an explicit hint in a separate conversation, the corrected version passed 13 out of 13 tests, but that was a result with a hint, not a first-attempt success.

One historical long_config_witness test with 255,747 input tokens failed its quality check. We retained this result after later cache fixes. A small sample of successful tasks does not prove quality loss is within 1–2%, and an allocated window of 262,144 does not prove equal quality throughout it.

If you repeat the experiment, prepare your own reference checks before comparing: facts at the beginning, middle and end of the project, contradiction detection, several tasks with executable tests, and cases involving tools and conversation continuation. Compare the profile with a suitable control and record errors for both. For important decisions, check the model's conclusions against the source materials.

Update of 30 October 2026: testing memory, cache and real questions

After the first measurements, we continued experimenting: we tested fact retrieval in long text, used tasks from public benchmarks and tried another acceleration scheme, SpecPrefill. A fast first request and comfortable project work turned out to be different problems.

All results below were obtained on the same Mac with M4 Max and 36 GB of memory. One measurement is not a guarantee of speed on another Mac or even on every request on this machine. In our tests the main profile used Q4 weights, Q8 KV cache and the approximate prefill described above. We rolled back changes to the working server after unsuccessful experiments.

Speed and fact retrieval in long text

We prepared our own needle-in-a-haystack-style test: inserted 15 random codes, three in each of five regions of a document, and asked the model to return all codes in one answer. The regions were at approximately 1.5%, 25%, 50%, 75% and 98.5% of the text length by characters, not exact token positions. For each label we checked that its specific code matched.

Scroll horizontally if the table does not fit.

Response speed was estimated from the stream between the first and last substantive chunks. These are two specific answers, not a promise of 51 tokens per second for Russian text, code or any 128k request. The 131,636-token measurement required a wait of about six minutes before the answer began. Enough time for tea—but it is better to account for that in client settings.

Note that 15 out of 15 is the result of one request with multiple facts, not a full grid of independent requests at each depth. This shows that the model found the required strings in this material. It does not prove that it reliably understands relationships between documents or writes correct code.

Public RULER and LongBench v2 tests

To test more than retrieval of an individual string, we used NVIDIA RULER. With its official generator we prepared two examples of one type: tracing a chain of variables through four transitions. The generator used our model's tokenizer and seed 42; the API counted around 32.6 thousand input tokens per example. In both examples the model found all five target variables: 10 out of 10, at about 389 tok/s for each request. Scoring followed RULER's rule for this task: the fraction of expected names appearing in the answer.

This is not a full RULER result. We checked two examples in one category, not every task and context length. They cannot support a claim that the model handles long memory with 100% accuracy.

For more realistic questions we took six LongBench v2 examples, one from each category: a document, multiple documents, conversation history, a code repository, structured data and in-context learning. These were the first suitable examples with documents from 40,000 to 180,000 characters, not a random representative sample. Full prompts ranged from around 13,000 to 50,000 tokens; documents were not truncated.

The model encountered a mundane problem: answer format. With the published direct-answer template and a limit of 128 output tokens, it began a detailed explanation in all six cases and exhausted the limit without issuing the final letter. The answer extraction rule formally scored 0 out of 6, but this does not mean the model solved all six incorrectly: no completed answer was produced. We explicitly disabled thinking mode; the normal response text was still too verbose for the limit.

We then repeated the same six tasks with an additional system instruction to give a brief answer immediately. All six answers finished; 2 out of 6 matched the reference—conversation history and repository tasks. This is an adapted test, not an official LongBench v2 result. Six examples are too few for a reliable overall assessment, but they show the difference between fact retrieval and more complex analysis.

Testing the cache in conversation

We separately sent a large project context and asked four consecutive questions. The time is from sending each request to the start of its answer:

Scroll horizontally if the table does not fit.

The cache saved considerable time in this conversation, but did not make the model smarter: speed and accuracy must be evaluated separately. If instructions, tools or early messages change, the common prefix can shrink and part of the input will be recomputed.

We checked such a case too: in a separate context of around 12,000 tokens we changed a code at the beginning of the document. The server reprocessed the input and returned the new value, not the old one. In this specific case zero tokens were reused from cache; that is correct behaviour, although the repeated prefill took around 26 seconds. One check does not guarantee correctness for all clients and all ways of changing history.

SpecPrefill experiment: faster first request, worse continuation

We also tested a separate oMLX profile with SpecPrefill. A small model first evaluates the entire input and selects fragments; the main model then processes only the selected portion. This can speed up the first large request, but discarded fragments do not become a full part of the main model's cache. This is an approximation that may lose important details.

With 20% retained in one test on roughly 100,000 original tokens, we obtained 649 original tok/s to the first token, including selection. This is effective speed relative to the complete original text, not the speed at which the large model physically processed each original token. Answers to four simple control facts matched the reference, but that check does not establish overall quality.

On another set with around 132,000 tokens of context, retaining 30% gave 443 original tok/s and 4 fully correct answers out of 6; our old solution gave 362 tok/s and 3 out of 6. The next two sets on the same synthetic project ended identically for both solutions: 2 out of 4 and 4 out of 4. Engines and profiles differed, so these small samples cannot attribute the difference to selection alone or establish an overall percentage quality loss.

The most interesting result appeared in continuations. In the test oMLX setup with disk cache enabled, the scorer restored 49,152 tokens and repeated selection in about a second. But the main model processed around 15,000 selected tokens again for every new message. In a conversation with an initial context of about 50,000 tokens, time to the second answer was 75.35 s, versus 0.84 s for our old profile, which reused more than 50,000 tokens of the main cache.

We therefore returned to the old solution for daily work: I did not want to wait more than a minute after every follow-up. SpecPrefill was interesting for the first request, but the tested configuration was unsuitable for a long conversation with frequent continuations. A potential compatibility issue with positional handling for this model type was also found in the experimental setup; its effect on quality was not established separately.

We also tried accelerating the old profile while preserving its cache. Reducing the global attention region from 8192 to 6144 positions raised one request from about 372 to 384 tok/s, but the new version got one control question wrong where the original answered correctly. Another GPU operation did not produce a measurable gain on the full request. The small sample does not establish the cause of the different answers, but we also found no reason to enable these changes. 500 tok/s on a full cold input in our old profile has not yet been confirmed.

Comparison with an ordinary MTPLX run

I wanted to understand what the engine changes offered compared with an ordinary run. We therefore compared our build separately with the same Q4 Qwen3.8-27B weights in MTPLX 2.12.2 on the same M4 Max with 36 GB. For MTPLX we used the stable profile, ordinary autoregressive generation without MTP and disabled SSD session cache. KV quantization was disabled in this MTPLX run; our build used Q8 KV, approximate prefill and speculative decoding. Thus the comparison is between two working configurations, not one isolated setting.

On the same short input of 3459 tokens with four questions about the project, both returned correct final answers: 4 out of 4. The final JSON answers also matched, though their wording differed. Results matched on this particular task; that does not prove equal quality on long contexts or other questions.

In a separate first MTPLX run on this input, the first token appeared after 29.72 seconds—about 116 input tokens per second if input length is divided by waiting time. Our build produced its first token after 7.18 seconds, or about 482 input tokens per second calculated in the same way. These are observations, not a precise estimate of superiority: answer limits and configurations differed in the two measurements. Furthermore, these figures are calculated from time to first token, including overhead, not readings from the same prefill counter. A repeat MTPLX run with a 4096-token response limit used the cache, so its time cannot be inserted into a cold-request table.

Long LongBench v2 requests of 13,270 and 19,537 tokens were not run in this MTPLX configuration: the memory guard rejected them before computation. We therefore obtained no comparable MTPLX results on those long tasks. This is a limitation of that run with its memory budget, not a claim that MTPLX cannot support such contexts in principle.

How much does our prefill change quality?

For a more direct comparison, we tested two modes of the same Kapitsa: the main profile with approximate prefill and a control with full attention during prefill. Both used the same Q4 weights and Q8 KV cache. This tests the effect of our attention mask but does not separate the effects of Q4 and Q8 and is not a comparison with a full-precision model.

Scroll horizontally if the table does not fit.

The two modes' answers matched on each of the six tasks. In the 50,000-token check the full response also matched byte for byte. In this one pair, our profile reduced waiting time by about 15%, while calculated speed over the full input increased by about 18%. This is one measurement, not an average over a large set or a promise of the same gain on every request.

Six LongBench questions are a small non-random sample; an extra instruction limited answer length, so the result is not an official overall LongBench v2 score. Matching results here mean only that we observed no difference on the selected tasks. They do not prove approximate prefill never reduces quality or that loss stays within 1–2%. Test the accelerated profile on your own tasks against full attention and prepared reference answers.

How readers can check the results

Before celebrating a number on a graph, test your own workflow:

1. Record the model and quantization, memory sizes, engine settings, prefill mask, KV cache format and generation parameters.
2. For a cold request, save the input token count and time to the first substantive token. Divide the full input by time only with an empty cache; record tokenization and model loading separately.
3. Send several continuations in the same conversation. Record cached_tokens, time to answer and quality; then change an early fragment and check the model does not use the stale value.
4. For fact retrieval, prepare different random values at different places in the text and compare each answer with your own table. Repeat on different documents and lengths.
5. For external checks, use the RULER instructions and LongBench v2 dataset and methodology. State the number of tasks actually run, sample, template and response limit. Do not call a small sample a full benchmark result.

We did not run HumanEval in this series, so we have no result for it. The minimal installation archive does not include the approximately 443 MiB LongBench dataset or complete benchmark results; download and run them separately using the authors' instructions.

How to reproduce the installation

I prepared a minimal anonymized reproduction package. The archive contains a source patch, local launcher, configuration, monitor, model download and measurement scripts, licences and checksums for the package files. Models are downloaded separately. Keys, personal files, conversation history, logs and my project data are not included; the archive does not contain reports with full test results.

Setup requires a separate folder on a Mac, Python 3.10 or newer, Xcode Command Line Tools, internet access and free SSD space. A practical estimate is 30–40 GB for models, the build and temporary files. The original experiment used an M4 Max with 36 GB; window size, memory budget and speed can differ on another configuration.

After extracting, open a terminal in the package folder. The commands below create a separate Python environment, build the required Kapitsa version and download the weights:

python3 -m venv .venv
.venv/bin/python -m pip install 'cmake>=3.24,<5' 'huggingface_hub>=0.34,<2'
git clone --no-checkout --filter=blob:none https://github.com/nace-ai/kapitsa.git upstream
git -C upstream checkout --detach 6bcac82bed6986c1f1b0227c0234aa42e93c895b
git -C upstream apply --check "$PWD/source.patch"
git -C upstream apply "$PWD/source.patch"
.venv/bin/cmake -S upstream -B build -DCMAKE_BUILD_TYPE=Release
.venv/bin/cmake --build build --target kapitsa --parallel 2
cp build/kapitsa runtime/kapitsa
.venv/bin/python download_models.py
.venv/bin/python run.py --dry-run

If the required weights are already downloaded through MTPLX, use the path to your folder instead of downloading a second copy of the target model:

.venv/bin/python download_models.py --model-dir "$HOME/.mtplx/models/Youssofal--Qwen3.8-27B-MTPLX-Bare-Speed"

The script checks checksums for the three main model files and tokenizer and downloads DFlash2 separately. If the dry run reports ready: true, you can start the server. Before starting: the launcher automatically stops a verified idle MTPLX on port 8000 and reports this. On exit it attempts to restore MTPLX, but recovery after a crash is not guaranteed. Save current work and make sure this behaviour suits you.

.venv/bin/python run.py

Keep the terminal open. The monitor appears at http://127.0.0.1:8010/monitor and the local OpenAI-compatible API at http://127.0.0.1:8008/v1. For the model identifier, open http://127.0.0.1:8008/v1/models. Stop it with Ctrl+C in the launch terminal; the monitor button reloads the model and resets its cache.

By default the package profile sets a window of 262,144 tokens total for input and response, a 30 GB memory budget, Q4 for main and draft models, Q8 KV and the approximate attention described above. These are experiment parameters, not a claim that all 262k will fit and yield good quality on any Mac. Check ports before starting: if a particular idle MTPLX is found on port 8000, the launcher may temporarily stop it and attempt to restore it on exit.

For your own cold measurement prepare a UTF-8 file and run:

.venv/bin/python measure.py --file /path/to/context.txt --fresh --max-tokens 512 --show-answer

The script calls the local server and reports actual tokens, cache, prefill, approximate streamed generation speed and file hash. Character count and token count are different. To test cache, repeat the history in the client without --fresh; to assess capability, compare answers with prepared correct results. Do not publish answers containing private project data.

For an additional control, you can stop your server, save a copy of config.json, set approximation.enabled: false and repeat the same requests after restarting. This disables the experimental prefill mask; Q4 weights and Q8 KV remain, so this is not a reference using a fully unquantized model. Full prefill may require more time and resources. Restore the original config afterwards.

By default the server and monitor listen only on localhost. Access from another device requires separate configuration of a trusted home network and a key. The gateway uses unencrypted HTTP; do not expose it to the Internet or publish the key.

If something goes wrong

Downloaded files failed SHA256. Check the path and selected model revision. The package deliberately stops when weights, tokenizer or important configuration files differ. Another build with a similar name may not be compatible with this comparison.

Speed was lower. Inspect final prompt_tokens, cached_tokens and the number of processed tokens. A short new tail with a large cache is not comparable to a cold load of a large file. Also check memory pressure and other processes and repeat the same scenario several times. Hardware and system state affect the result too.

The client cancels a large request. Increase its own stream or response timeout. For example, at an average of around 355 tokens per second, prefill alone for 128 × 1024 tokens exceeds five minutes. Changing the server timeout alone does not fix the client limit.

The answer is empty with a small limit. Reasoning mode may use the available tokens before normal text appears. Increase the response limit or explicitly disable reasoning for a short check if the client supports chat_template_kwargs: {"enable_thinking": false}. The supplied measure.py uses this parameter for the short test.

The monitor did not open. Check whether the local server is ready and port 8010 is free. The direct API uses port 8008. If a port is occupied, first identify its owning application: indiscriminately terminating another process can interrupt work.

The cache is recomputed again. Check not just the last message but also that the beginning of the history, system instructions and tools remain unchanged. Our scheme keeps a limited number of checkpoints and does not promise to avoid recomputation after every edit.

I verified that the patch applies cleanly to the specified commit and the original build compiled on my Mac. The new minimal archive was checked separately after extraction: checksums matched and the dry run completed, as expected reporting models and binary not yet downloaded. I have not yet performed a complete installation from scratch and a long GPU benchmark of this exact minimal archive; below is a prompt asking an assistant to run checks on the reader's machine.

Prompt for setup with an AI assistant

Copy the text below and attach qwen27b-minimal.zip from this article to the chat. The assistant needs access to the Mac terminal, the Internet and permission to change files in a separate setup folder. If the tool sees only chat text, it can only prepare commands.

The current text can also be downloaded as a separate AI prompt file. The prompt is not included in the minimal archive: download it separately or copy it from the page.

You are helping set up local Qwen3.8-27B on my Mac. I have the qwen27b-minimal.zip archive and the article describing the experiment. Perform the setup rather than only offering advice. First read README.md and the archive configuration.

The archive contains a patch, scripts and a local monitor; it contains neither weights nor a ready binary, which must be downloaded and built separately. After extraction, checksums and a dry run without a model were checked; a complete installation from scratch and long test of this exact minimal archive have not yet been confirmed. First check SHA256SUMS.txt, then record your own check results separately. By default the API and monitor are available only on this Mac. After checks, run.py may temporarily stop idle MTPLX on port 8000 and then try to restore it; tell me beforehand and do not stop busy or unverified processes.

The goal is to reproduce the described Kapitsa build and test it on my Mac. The original experiment used an M4 Max with 36 GB RAM. Record my chip, RAM, macOS, free disk space and installed tools. If the computer differs substantially, do not promise the same speed; suggest a suitable window and memory limit and explain the differences. If information is sufficient, make decisions yourself.

Build the engine from the specific Kapitsa commit and source.patch in the archive. Download the pinned Qwen3.8-27B-MTPLX-Bare-Speed Q4 and incoai/Qwen3.8-27B-DFlash2 using download_models.py; if compatible weights already exist on disk, use --model-dir and check their hashes first. Weights are downloaded separately; do not substitute another model or patch version without explanation. The package allows using my existing weights without overwriting them.

Run the dry check, start the server and connect a local OpenAI-compatible client. Before stopping or restarting any running server, tell me in advance; do not terminate other people's or busy processes. If a port is occupied, find out by what and stop if the required ports cannot be used safely. Do not delete existing applications' data, projects or settings. Do not print or transmit API keys or my file or log contents to third parties. For measurements using my project files, process them locally and do not modify them.

Check a short request and the monitor, then perform cold and repeated measurements at suitable context sizes. For each, report actual input tokens, tokens from cache, tokens recomputed, prefill time and generation time. Do not assess generation speed from an answer of 1–2 tokens. Compare identical inputs and parameters; distinguish native tok/s from an approximate window value. Test multiple turns, tool calls and cancellation. If the client changes the start of the history, explain the unavoidable recomputation.

Treat the article's profile as experimental: Q4 weights, Q8 KV cache, approximate prefill and DFlash2. All tokens are retained, but the restricted attention mask changes computation; dense decode does not make this prefill exact. The previous experiment includes a long test with an error and has no proven 1–2% quality-loss bound. Do not present 255/30 tok/s as guaranteed. Compare quality against a suitable control profile on my test tasks, recording errors and results separately; do not call matching results on a couple of tasks a general guarantee.

Test retrieval of multiple random facts at different depths and several context lengths; state how depths were measured and how many separate requests were made. If desired, test RULER and LongBench v2 following their authors' instructions: record the sample, template, response limit, scoring method and results for each task. Do not present a partial or adapted check as a complete official result. Do not execute generated code without suitable isolation and resource limits.

For cache, separately test a cold request, continuation, addition of new data and a change to an early part of context. Record time to first token and matching cached volume; check that the model does not return the old value after the change. If experimenting with SpecPrefill, measure selection time and main-model cache reuse separately. Do not call selected fragments a complete KV cache.

By default leave the API and monitor on localhost. Enable home-network access only if I ask: configure the correct private subnet and a new key, do not print it in logs or expose the service to the Internet. Test access and protection. Finally report the address, actual model, measured speeds, quality-test results, differences from the article and all remaining limitations. If something failed, show exactly where and what can be reproduced.

Sources and materials

• Kapitsa source and engine description, experiment base commit 6bcac82bed6986c1f1b0227c0234aa42e93c895b.
• Qwen3.8-27B-MTPLX-Bare-Speed, revision b59d7002368575f08a58ba0d26686b53a9c162d6.
• Qwen3.8-27B-DFlash2, revision 015e795645c74b1a0eeef3b570031fb62e769bc5.
• MTPLX documentation—if the weights are already installed through the app.
• NVIDIA RULER—generator and methodology for long-context tasks.
• LongBench v2 and dataset—questions about long documents and codebases.
• The minimal archive includes SHA256SUMS.txt for its own files. Original prompts, benchmark results, my project data and model weights are not included.

The patch and supporting files are distributed with the licence and notices in the package. Check model licences separately on their authors' pages.