Seyraby IncrementPlus Chat with Seyra
Seyra against a small language model
The same 269 conversation turns, run turn by turn through Seyra and through Qwen 3.5 0.8B, the smallest language model on the test machine. Quality was graded blind; energy was measured from the chip's power sensor.
Quality against energy
Blind-graded quality per reply (1–5) against the energy each reply used. Up and to the left is better. The energy axis is logarithmic: each gridline is ten times the last.
Good answers per watt-hour
Replies graded 4 or 5 out of 5, per watt-hour of energy used. Logarithmic scale. Seyra's bar shows its range: from the meter's upper bound to the CPU-time estimate.
How often each system got it right
Three views of the same runs, as a share of turns or pairs.
Blind score by test set
Average blind grade (1–5) in each set of conversations. The number of graded turns in each set is in brackets; small sets swing more.
Energy per reply
Joules above idle, logarithmic scale.
Time per reply
Median, logarithmic scale.
All the numbers
How this was measured
- Same conversations for both. 229 multi-turn conversation turns from seven test sets (including five written blind, before fixes) plus 40 factual and reasoning questions. Each system saw the conversation so far. The model got a short instruction to be Seyra: warm, brief, honest that it's an AI.
- Blind grading. 59 turns sampled across every set. For each, the two replies were shown as A and B in random order, and graded 1–5 for being right, natural and on-topic, before the answer key was opened.
- Automatic checks. Each turn has an expected-content pattern, plus checks for echoing the person's words and for repeated replies.
- Energy. The SoC package power sensor (CPU and integrated graphics), sampled ten times a second, minus an idle baseline. Whole-laptop battery draw agreed: 147 J per reply for the model. Seyra's work stayed below the sensor's noise even over thousands of replies. So its figure is its CPU time per reply (2.1 ms) times the per-core power seen under load, with the measured noise ceiling (0.007 J) as an upper bound.
What to keep in mind
- This isn't a comparison with large models. Qwen 3.5 0.8B is a small open model that runs locally. Hosted models such as Claude are far larger and more capable, and weren't tested here.
- The grader knew one system's style. Labels were hidden, but the grader helped build Seyra and could recognise its style. The automatic checks were also written around Seyra's wording. The gap is large either way, but treat the exact numbers with that in mind.
- Live data is not counted. Seyra looks up weather and facts on the web (about 10 of the 269 turns need it), and the energy those servers use isn't measured. The model has no tools.