Skip to content

Request Latency and Observed Throughput

Summarize actual latency samples and completed-request rates, or add explicit serial stages without inventing P99 or maximum capacity.

Use this result well

Inputs that matter
Preprocessing time (ms), Core operation time (ms), Postprocessing time (ms), Queue and dispatch time (ms), and 5 more
Output to expect
Entered serial path time, Observed sample P99, Completed requests per second
  • Check the units and required inputs before comparing results.
  • Keep the assumptions with a copied result so you can reproduce the calculation later.
Was this tool helpful?

Related Tools

Tools you might need next

Audio Buffer Latency Calculator: Buffer Contribution

Estimate input and output buffer duration from frames and sample rate. Understand what this excludes before comparing real audio latency.

Music Technology
Network Latency Calculator

Estimate network latency from fiber distance and processing delay. Free network latency calculator for network, latency, and more.

Telecommunications
Communication Latency

Estimate robot communication latency over wireless links. Free online robotics calculator with instant, accurate results.

Robotics

Reference & details

How it works

Non-overlapping service-path timing

Total path time = preprocessing + core operation + postprocessing + queue/dispatch + transport/other A sum of supplied stage times. Do not add overlapping wall times, CPU times and wall times, or stage percentiles as if they were one end-to-end percentile. No P99, maximum QPS, concurrency or SLA is predicted. Warmup, batching, queuing, synchronization and transport must match the measurement scope.

Measured latency sample summary

Sort N measurements; percentile p is sorted[ceil(p×N)−1]. Mean = sum/N. Values are milliseconds Nearest-rank percentiles of an observed sample. Other software may interpolate percentiles; this method deliberately returns an observed value. A small sample P99 can be the maximum and is not a precise population tail estimate. This does not repair coordinated omission, remove warmup or classify failures. Record those choices explicitly.

Observed completed-request rate

Completed rate = completed requests / seconds; successful rate = (completed − failed)/seconds; failure share = failed/completed Observed counts over one interval. This is not maximum capacity, offered request rate or a percentile. Unfinished requests are not completions; record timeouts, cancellations and pending work under a consistent policy. A high completed rate with failures or a growing queue does not establish adequate capacity.

Updated: September 2026

Example Scenarios

Add compatible non-overlapping request or inference stage times. Queue and transport time remain explicit, and no tail percentile is inferred.

Summarize a supplied sample of complete request or inference latencies with explicitly defined nearest-rank percentiles.

Divide completed requests by an observed elapsed interval and show failed completions separately from successful ones.

Common Mistakes to Avoid

Applying non-overlapping service-path timing outside its stated assumptions

Do not add overlapping wall times, CPU times and wall times, or stage percentiles as if they were one end-to-end percentile. No P99, maximum QPS, concurrency or SLA is predicted. Warmup, batching, queuing, synchronization and transport must match the measurement scope.

Applying measured latency sample summary outside its stated assumptions

Other software may interpolate percentiles; this method deliberately returns an observed value. A small sample P99 can be the maximum and is not a precise population tail estimate. This does not repair coordinated omission, remove warmup or classify failures. Record those choices explicitly.

Applying observed completed-request rate outside its stated assumptions

This is not maximum capacity, offered request rate or a percentile. Unfinished requests are not completions; record timeouts, cancellations and pending work under a consistent policy. A high completed rate with failures or a growing queue does not establish adequate capacity.

FAQ

No. Measure a latency distribution; a mean does not determine its tail.

Not generally. Actual concurrency, arrival rate, queueing and batch behavior require load testing.

No. Paste actual recorded durations.

No. Evaluate representative traffic, duration, outcome coverage and uncertainty.

Yes. Rates are zero; the failure percentage is then undefined.

It supplies one observed rate. Capacity planning needs realistic load, latency, errors and resource evidence.

About Request Latency and Observed Throughput

Summarize actual latency samples and completed-request rates, or add explicit serial stages without inventing P99 or maximum capacity. Choose the mode that matches your measurements or study design, enter the stated units and keep the method and limits with the result.