Active Parameters and Emergent Abilities: Why a Small MoE Model Can Outperform a Large One

A model's active parameters determine the amount of computation for a particular token, but they do not describe the neural network's full capacity. An MoE model can store significantly more parameters than it uses during generation, while the quality of its response also depends on the architecture, training, expert routing, and additional computation.

A model with a smaller active size can sometimes outperform a larger system on specific tests. Comparing only billions of parameters is not enough.

What Are Active and Inactive Parameters

In a dense language model, every token passes through almost the entire network. If a model contains 70 billion parameters, this figure simultaneously describes its capacity and the approximate scale of computation per token. In a sparse model, or Mixture of Experts, the relationship is different: the overall network includes many experts, and a router selects only some of them.

Active parameters participate in processing a particular token. Inactive parameters do not perform computations for the selected route at that moment, although they remain part of the model's weights and occupy storage space if the system does not use special offloading or distributed placement. Active size primarily affects computational load, while total size determines the potential capacity of the expert set.

How an MoE Model Is Structured: The Qwen Example

The developers describe Qwen3.8-Flash-Next as a multimodal MoE model with a main network containing 125 billion parameters, additional N-gram Embedding parameters, and approximately 6 billion parameters activated per token. A user-written account cites an additional 55 billion and a total size of 180 billion, whereas the published repository description specifies 51 billion N-gram Embedding parameters. These figures must not be mixed: for a technical comparison, the model version and its model card must be checked separately.

In this architecture, N-gram Embedding expands the representation of local context and, according to the developers' description, can be placed in the computer's RAM with advance preloading. The architecture also combines Gated DeltaNet with an attention mechanism that selects important fragments of a long context. These solutions show why overall efficiency cannot be reduced to simply dividing the total number of parameters by the active number.

The router selects experts depending on the current token and the context accumulated so far. Different combinations of parameters will be useful for code, business correspondence, mathematical reasoning, and multimodal requests, so two requests of the same size may pass through different parts of the network.

Why Total Size Is Not Equal to the Load per Token

When an MoE model is run, the hardware must provide access to the weights of all experts or to their distributed copies, even if only part of the network is activated at each step. Memory is needed not only for active matrices: it also includes inactive weights, auxiliary structures, buffers, the KV cache, and memory used by the inference engine.

The computational load may indeed be closer to that of a model with 6 billion active parameters, but storage requirements do not become the same as those of a dense model with 6 billion. This is a key distinction for local deployment. A fast SSD and tiered placement reduce pressure on RAM, but memory bandwidth and transfer latency still affect speed.

The author of the described experiment links running Qwen on a home computer to the Strata platform, which, in his assessment, uses available memory, including disk space, as tiered storage. The requirements he cites are approximately 16–24 GB of VRAM and 64–128 GB of RAM, while the final configuration depends on quantization, the processor, the storage subsystem, and the selected generation speed. This is the author's experience, not a universal Qwen specification for every deployment method.

Where the Requirement for 70 Billion Active Parameters Came From

The figure of 70 billion active parameters is not a generally accepted technical threshold for personal artificial intelligence. The author previously used it as a working assumption, associating that scale with the emergence of the capabilities he needed, but later called the assumption an unsubstantiated “creation myth.”

Such a threshold has no universal meaning without specifying the task. The needs of autonomous programming, complex fact-checking, multi-user memory, and long-term planning differ, while a model for classification, searching an internal database, or preparing drafts may work with a smaller active size. The question should not be phrased as “Does the model have 70 billion active parameters?” but rather as “How does it solve my tasks on an independent set of examples?”

The claim about a desktop model with 4 billion active parameters cannot be considered confirmed by the cited materials. The available description of Meta Muse Spark Open lists 816 billion total and 42 billion active parameters, and its deployment is associated with two Apple Mac Studio computers using M5 Ultra, not with a standard gaming PC. This example is still important, but it demonstrates something else: a large sparse model can run outside a conventional data center with a suitable hardware configuration.

Emergent Abilities and Alternatives to Scaling

An emergent ability is behavior that becomes noticeably apparent in a model of a certain scale and is poorly predicted by simply extending the curve for smaller models. However, the research literature does not fully agree on which jumps truly reflect a new internal skill and which arise from the metric, task format, or insufficient test examples.

A review of research on emergent abilities points to several sources of nonlinear improvement. These include model growth and an increase in the training compute budget, instruction tuning, synthetic data, chain-of-thought approaches, reinforcement learning, search during generation, and architectural changes. A single metric may show a sharp jump in accuracy, while a more detailed metric reveals gradual improvement in intermediate answers.

Additional computation during generation is especially important for reasoning models. A system can construct several possible solutions, check them, break a task into stages, or use an external tool without increasing the number of parameters in the base network. This does not make a small model equal to a large one in every mode, but it changes the practical boundary of its capabilities.

Comparing a Model with 6 Billion and a Model with 17 Billion Active Parameters

In the comparison presented by the author, a modern model with 6 billion active parameters outperformed an older model with 17 billion active parameters in 8 out of 14 tests. In the other six tests, it fell behind by no more than 2,6 points. The result applies to specific models, versions, settings, and evaluation methods, so it cannot be turned into a rule that all smaller models are superior.

The comparison shows that active size is not equal to quality. The newer model may have had a more efficient architecture, better data, a different instruction-tuning procedure, better tokenization, or more accurate routing. The older model with 17 billion active parameters may have had greater computational capacity per token but used it less effectively.

The Role of Experts, Routing, and Knowledge Integration

The large total size of an MoE model makes it possible to form large experts, each of which can specialize in its own set of patterns. The active router selects one or more such blocks, and the model combines their results before moving to the next layer. When configured well, this provides a diversity of knowledge without fully computing the entire network for every token.

Activating several experts simultaneously is useful for tasks at the intersection of different fields. A request to develop a financial web application requires simultaneously considering programming code, interface requirements, business logic, and security constraints. One expert may process syntax, another the domain context, and a third the verification of the sequence of actions, but the outcome depends on how well the router distributed the request.

The advantage does not arise automatically. A poorly trained router overloads some experts, ignores others, or selects blocks that are unsuitable for the current task. The architectural number of experts and the total number of parameters must be checked together with routing stability, training quality, and results on the required classes of requests.

How to Evaluate a Model Instead of Focusing on a Single Metric

For local AI, it is useful to create your own task set and divide it into several groups. These include ordinary dialogues, work with long documents, programming, structured output, error detection, multimodal analysis, and agent scenarios. Each group should be evaluated using separate criteria, since an average score hides failures on critical operations.

1. Define the task. Describe the input data, the expected response format, and the acceptable level of error.
2. Compare under identical conditions. Use one temperature, the same context, an equal token limit, and a comparable set of tools.
3. Separate the model from the surrounding system. Evaluate the base response, RAG, rechecking, decomposition, and the agent loop separately.
4. Check reproducibility. Repeat each scenario several times and save not only the best results but also the unsuccessful ones.
5. Measure local cost. Take into account equipment purchases, energy consumption, maintenance, speed, and the specialist's time spent on configuration.

The author estimates the cost of assembling a computer for local deployment in Ukraine at approximately 1500–2500 dollars, associating the lower bound with acceptable speed and the upper bound with more comfortable operation. This amount applies to his scenario, evaluation period, and specific set of components, so it cannot be used as a current universal price.

Local deployment provides control over access to the model and reduces dependence on a cloud provider's decision to shut down a service or change its pricing. In return, the user incurs equipment costs and must configure the engine, updates, quantization, data storage, and additional infrastructure independently—features that are already built into a cloud product.

Conclusion: active parameter count matters, but it is not everything

A model with 6 billion active parameters may be more practical than a model with 17 billion if it is better trained, routes experts more efficiently, and receives more appropriate compute during generation. MoE expands overall capacity without necessarily increasing compute proportionally for each token, but it also raises memory and infrastructure requirements.

The requirement of 70 billion active parameters should be considered a hypothesis, not a universal threshold for personal AI. A reliable choice is determined not by the ranking of total parameter counts, but by a repeatable test on your own tasks, where quality, speed, memory, cost, and error frequency can be assessed separately.

Frequently asked questions

Are active parameters the same as the model's actual size?

No. Active parameters describe the portion of the computation used for a particular token, while the total size includes all experts and additional components that must be stored or loaded.

Why does a smaller model outperform a larger one?

The reasons may include higher-quality data, a new architecture, successful fine-tuning, efficient routing, quantization, and the specifics of the test. The number of parameters alone does not guarantee a better answer.

Is Qwen3.8-Flash-Next suitable for an ordinary home computer?

That depends on the weight version, quantization, engine, and available memory. The scenario described by the author requires a powerful configuration, not just any office PC; its requirements should not be considered an official universal recommendation.

Does a local model replace leading cloud-based systems?

According to the author, a local model can handle common tasks involving text, long context, and some programming, but it does not replace cloud-based systems in all complex expert and agentic scenarios. The result should be checked on your own data.