Concurrency and connections

wasmCloud gives you two families of controls that decide how much work a component does at once and how much of the network it may hold open while doing it:

• Instance limits set how many calls a single warm instance (a component the host keeps ready between calls) serves concurrently, and how many warm instances the host keeps. They live on the component, in the Workload (or WorkloadDeployment) custom resource.
• Connection quotas set how many outbound and inbound connections a workload may hold. They live on the host, set through Helm values or host flags, and apply to every workload the host runs.

Raising instance concurrency raises how many connections the workload needs, so they are worth tuning as a pair. This page explains how they fit together and how to size them.

For the full field reference, see component runtime fields and workload security.

Scoping

Three settings decide a component's total in-flight work, each at a different scope:

Scope	Field	What it sets
Instance	maxConcurrency	calls one warm instance serves at the same time
Component	poolSize	warm instances the host keeps for the component
Deployment	replicas	copies of the workload across the host group

• maxConcurrency and poolSize are set on the component entry.
• replicas is set on the WorkloadDeployment.
• A single component's warm capacity is poolSize × maxConcurrency in-flight calls. Multiply by replicas for the deployment's total. Each of those calls can open its own outbound connections, so the connections a workload needs grow with the same product.

That is the interaction to keep in mind: instance concurrency sets the demand, connection quotas set the ceiling.

Instance limits (per component)

These five fields sit directly on a component entry, under spec.template.spec.components[*] in a WorkloadDeployment or spec.components[*] in a Workload:

Field	Default when unset	What it does
poolSize	none kept (every call gets a fresh instance)	how many warm instances the host keeps between calls
maxConcurrency	1	how many calls one warm instance serves at the same time
maxInvocations	unlimited reuse	retires a warm instance after it admits this many calls, then replaces it
reclaimWindowSeconds	never reclaimed	lets an idle pool shrink: each window, instances beyond what peak load needed are drained and retired (since 2.9.0)
reclaimMinInstances	0	floor on reclaim; sweeps never shrink the pool below this count (since 2.9.0)

yaml

spec:
  template:
    spec:
      components:
        - name: api
          image: ghcr.io/example/api:1.0.0
          poolSize: 10
          maxConcurrency: 8
          maxInvocations: 1000
          reclaimWindowSeconds: 60

• maxConcurrency above 1 lets one instance overlap calls while it waits on I/O, which is what makes pooling pay off for network-bound work. It is only safe for a guest (the component's own code) that yields while it waits rather than blocking the thread.

• maxInvocations bounds how long a warm instance lives, which is useful when a component accumulates state you would rather reset periodically. Both are only meaningful alongside poolSize, since an unpooled component starts fresh on every call.

• reclaimWindowSeconds (since 2.9.0) lets the pool shrink back after a spike: every window the host retires idle instances beyond what the window's peak load needed, never interrupting a call in flight. reclaimMinInstances sets a floor. By default a pool holds its high-water mark until the workload stops. Like the limits above, reclaim is only meaningful alongside poolSize.

The component runtime fields reference covers the pooling semantics in full, including how linked components share a store and what a guest trap does to calls in flight.

Connection quotas (per workload)

Every workload gets one connection quota across three surfaces. You set these on the host, either as Helm values in a host group's networking block (runtime.hostGroups[].networking; rendered by the chart since 2.9.0, so on earlier charts use extraArgs) or as host flags, and they apply to each workload the host runs:

Helm value	Host flag	Default	Bounds
maxOutboundHttpConnectionsPerWorkload	--max-outbound-http-connections-per-workload	128	pooled outbound HTTP and gRPC connections a workload may hold (idle keep-alive connections count)
maxOutboundSocketConnectionsPerWorkload	--max-outbound-socket-connections-per-workload	256	raw wasi:sockets outbound connections a workload may hold
maxInboundSocketConnectionsPerWorkload	--max-inbound-socket-connections-per-workload	256	inbound published-port connections a workload serves at once
maxConnections	--max-connections	derived from the process file-descriptor limit	host-wide ceiling across every workload
maxHttpIngressConnections	--max-http-ingress-connections	a quarter of the descriptor limit, floor 256	connections the host's own HTTP listener holds open, across every workload (since 2.9.0)

The three per-workload surfaces behave differently when a workload reaches its limit:

• Outbound HTTP waits. A request over the quota waits for a slot, up to --http-connection-wait (default 5s), then fails. This is the surface most sensitive to instance concurrency, because the outbound pool is what warm, overlapping calls draw from.
• Raw sockets and inbound connections are refused immediately. They do not wait, which avoids a guest deadlocking against its own quota.
• The host's ingress ceiling sheds. Connections past maxHttpIngressConnections are accepted and immediately closed, so the host stays responsive rather than exhausting descriptors (since 2.9.0).

The three per-workload quotas roll up into the host-wide maxConnections ceiling; maxHttpIngressConnections bounds the host's own listener separately, derived from the same descriptor budget. Since 2.9.0 the host raises its own soft file-descriptor limit toward the hard limit at startup, and an unlimited descriptor limit derives a large budget instead of a silent 512, so hosts running under LimitNOFILE=infinity permit far more concurrent connections after upgrading. The workload security page covers these quotas alongside the raw-socket egress policy they ship with.

Sizing them together

Take the component above: poolSize: 10 and maxConcurrency: 8 is up to 80 calls in flight on one replica. If each call makes one outbound HTTP request, the workload wants up to 80 outbound connections at peak, comfortably under the default 128. Raise maxConcurrency to 16, or have each call fan out to several backends, and the demand crosses the default. Past that point, outbound requests wait on --http-connection-wait and then fail, which shows up as latency spikes and timeouts under load rather than an obvious error.

Two separate things are at work here, and only one of them is a value you set:

• Keep-alive pool sizing is automatic. For reuse, the host keeps some idle connections warm to each destination, and it sizes that from the component's declared poolSize × maxConcurrency, so raising concurrency keeps proportionally more connections ready without a second setting to keep in sync. Idle connections above real demand close on their own after a timeout. You do not tune this.
• The connection ceiling is the value you set. maxOutboundHttpConnectionsPerWorkload (default 128) caps how many outbound HTTP connections the workload may hold at once, across every destination. This is the limit a request waits behind, and it is what you raise when peak demand outgrows the default.

So the tuning workflow is:

1. Set poolSize and maxConcurrency for the throughput you want.
2. Estimate peak outbound connections as poolSize × maxConcurrency × (outbound requests per call).
3. If that estimate approaches maxOutboundHttpConnectionsPerWorkload (default 128), raise the quota, and confirm the host-wide maxConnections still leaves room for every other workload.

note

These quotas are per host process. A workload scaled to several replicas gets the quota once per replica, not divided across them, so scaling out multiplies the workload's total connection budget the same way it multiplies throughput.

Messaging admission (per component)

As of wasmCloud 2.8.0, messaging-triggered components have their own in-flight limit, applied before any instance work begins (trigger services are exempt). The max_in_flight config key on the component (default: the host's per-component ceiling) caps concurrent deliveries across the component's replicas on a host. A delivery over the limit waits up to admission_wait (default 30s) for a slot, then is dropped with a warning.

This limit is separate from maxConcurrency: maxConcurrency bounds calls on one warm instance, while max_in_flight bounds total messaging deliveries executing for the component. Since 2.9.0, async (0.3.0) deliveries to a pooled component run on its warm instances under the same poolSize and maxConcurrency rules as HTTP, so for messaging workloads, size max_in_flight first, then the instance limits beneath it. Sync (0.2.0) deliveries still run in a store per message regardless of poolSize. See Messaging admission control for the config keys and host flags.

Replicas

replicas is the third scaling axis: copies of the whole workload spread across the host group (the pool of hosts the operator manages). It scales throughput past what one host's instance pool can serve, and it is the axis a HorizontalPodAutoscaler or KEDA drives through the /scale subresource. See Autoscaling for how to scale replicas automatically, and why the host group is a precondition for it.

Takeaways

• A component's warm capacity is poolSize × maxConcurrency calls in flight; replicas multiplies that across hosts. With reclaimWindowSeconds set, an idle pool shrinks back after a spike.
• Instance concurrency sets connection demand; the per-workload quota sets the ceiling. Size them as a pair.
• The keep-alive pool is sized automatically from declared concurrency, so you tune concurrency and the quota, not the pool directly.
• Outbound HTTP over quota waits up to --http-connection-wait (default 5s) then fails; raw sockets and inbound connections are refused immediately.
• The connection quota applies once per replica, so scaling out multiplies the total connection budget.
• Messaging deliveries have their own admission limit (max_in_flight, as of 2.8.0), applied before instance limits come into play.