Paper trading that behaves like a real execution venue

"Would this order have filled on the live venue?" Ask the team that, then check the simulator against the venue's actual order lifecycle. A green P&L figure is not evidence of realistic execution. Paper trading is useful for unattended software only when it reproduces the mechanics that can change an order's price, size, status or exposure.

Run the checks below against the broker or exchange you intend to use. Use the same symbols, session times, order types and API path where possible. Record mismatches as test results, not as footnotes to a strategy report. A simulator can still be valuable if it differs from the venue, but the difference needs to be explicit and bounded.

Define what "like the venue" means

Start with the venue's contract, not the simulator's feature list. Choose one instrument and write down its tick size, lot or contract size, minimum quantity, trading hours, price bands, order types and account permissions. Include the rules that apply outside the main session. A market order accepted during regular hours may be rejected, queued or handled differently in an extended session.

Next, describe the order lifecycle your software expects. For a limit order, that usually means submitted, acknowledged, resting, partially filled, filled, cancelled or rejected. Capture the exact status strings and timestamps returned by the API. Check whether a cancel request can race with a fill, whether a replace keeps the original queue priority, and whether a late fill can arrive after the client believes the order is cancelled.

Write down the target behaviour before testing. For example, a buy limit should not fill merely because the last traded price touched the limit.

It should fill only when the simulation's trade or queue model says enough opposing volume has reached it. If the simulator has no queue model, record that limitation and avoid treating its fill rate as a forecast.

• Pin the API version, account type, symbol mapping and session calendar.
• Save raw requests, responses, timestamps, market data and order events.
• Separate strategy decisions from simulator behaviour in your logs.
• Set a fixed test window and seed any randomised fill model if the tool supports it.

Keep the first test narrow. One liquid instrument and two order types are enough to expose many assumptions. Expand only after the lifecycle is understood.

Compare the execution mechanics

Test market, limit, stop and cancel-replace behaviour separately. For each order, compare the submitted quantity and price with the acknowledgement, subsequent events and final position. A single final fill message can hide partial executions, duplicate events or an incorrect average price. Your client should reconstruct state from events and tolerate repeated delivery without placing a second order.

Measure latency at several points: client send, server acknowledgement, first fill event and final fill event. Use a monotonic clock locally and keep the venue timestamp separately. A simulator that returns every acknowledgement in 2 ms may be convenient for functional tests, but it will not exercise a strategy that depends on a 100 ms timeout or an order that sits in a queue.

The figures below are typical market ranges, not guarantees or audited results for a particular provider. They vary with venue, session, connectivity, instrument and market conditions. Use them to select test cases, then measure your own live API path where permitted.

Do not feed a simulator only clean, continuous prices. Replay gaps, crossed or stale quotes, abrupt spread widening and periods with no trade updates. Check whether a market order consumes available depth or receives the mid-price. If the model fills the full quantity at the top of book regardless of size, it is testing intent, not execution cost.

"A fill model that never says no is a strategy optimiser wearing a simulator's name tag."

Exercise data, errors and account state

Execution realism depends on the data stream as much as the order endpoint. Compare the simulator's bid and ask with the target feed at the same timestamps. Check whether updates are snapshots or incremental changes, whether sequence gaps are detectable, and how the client recovers after reconnecting. If your strategy acts on a stale quote, the order simulation should expose that condition rather than quietly supplying a fresh price.

Test normal and abnormal responses. Send a quantity below the minimum, a price outside the permitted range, an unsupported time-in-force and an order while the market is closed.

Trigger rate limits with a controlled burst. The expected result is not always rejection: some venues accept an order and later cancel it, or return a warning alongside an identifier. Your software must distinguish these cases.

Reconcile account state after every test run. Compare open orders, executions, cash, buying power, realised P&L and positions with the events your process observed. Then disconnect the client, allow orders to change, reconnect and request a fresh snapshot. Check that the client can converge on the broker's state without replaying an old intent as a new order.

Include these failure cases in the test record:

• Delayed or duplicated order events, including events arriving out of order.
• A timeout after submission where the server may have accepted the order.
• A partial fill followed by a cancel, replace or risk-limit rejection.
• A dropped data connection during an open position.
• Market closure, trading halt, symbol change or a daily session boundary.

For a timeout, query by client order ID before resubmitting. Retrying a POST blindly can create duplicate exposure. Use idempotency keys where available, but do not assume they remove the need for reconciliation.

Run a one-week practitioner checklist

Use a small, repeatable plan this week. Keep the strategy simple enough that a discrepancy can be traced to the simulator, data or order-handling code. Store the test inputs and outputs in version control or an equivalent change-controlled record.

1. Monday, set the baseline. Select one target venue and instrument. Record its trading rules, API version, supported order types, quantity limits and session calendar. Save a sample of market data and raw API responses.
2. Tuesday, test order lifecycles. Submit marketable and non-marketable limits, then test partial fills, cancel-replace and expiry. Verify every state transition and calculate average fill price from executions, not from the final order status alone.
3. Wednesday, inject rough conditions. Replay fast moves, spread widening, stale quotes, delayed messages and reconnects. Measure event gaps and end-to-end latency. Confirm that a timeout leads to a state query, not an automatic duplicate order.
4. Thursday, check account and risk rules. Test minimum size, maximum size, buying power, position limits and rate limits. Reconcile the resulting orders, executions, cash and positions against the simulator's account view.
5. Friday, compare and decide. Review the logs, list every material mismatch and classify it as acceptable, fixable or blocking. Set a threshold for each blocking measure, such as maximum stale-data age or unaccounted order count, then rerun the relevant tests after any code change.

A short scorecard makes the result actionable. Track fill price difference in ticks or basis points, rejection rate by reason, acknowledgement and event latency percentiles, duplicate or missing events, and reconciliation breaks. Report counts as well as rates. Ten successful orders do not establish a reliable rejection rate.

Set an acceptance gate for unattended runs

There is no single fidelity score that makes a simulator safe for every strategy. A market-making agent and a daily rebalance script are sensitive to different errors.

Set acceptance limits from the strategy's actual exposure to spread, latency, queue position and order state. If a strategy loses its edge when a limit fills one tick less often, a simulator that assumes touch fills cannot validate that edge.

Separate functional acceptance from performance claims. Functional acceptance means the client handles acknowledgements, rejects, reconnects and reconciliation correctly. Performance acceptance means the simulated costs and fills are close enough to the target venue for the question being tested. Passing the first does not establish the second.

Keep a known-differences register alongside your test suite. It might say that queue position is approximated, market impact is omitted, or fees use a static schedule. These are not automatic reasons to reject a simulator. They are reasons to avoid claims that rely on the missing mechanics. A platform such as Agentic Trading Index's paper-trading service can be assessed with the same venue-specific checks as any other environment.

Before a live unattended run, require the client to stop or enter a defined safe state when market data goes stale, account state cannot be reconciled, or an order remains in an unknown state. Keep the thresholds in configuration, log the trigger and make recovery explicit. A paper environment can test whether those controls fire; it cannot prove that every live venue response has been modelled.

Most paper accounts still make execution look cleaner than it is.

#broker API testing #market data latency #paper trading simulation #execution simulation #order fill modelling #slippage modelling #trading agent testing #simulated order execution