Paper trading that behaves like a live venue

Two operators, two calls, one misleading simulator

At 09:31, Operator A sends a marketable buy for 500 shares. Operator B, seeing the same signal, posts a limit order at the best bid and waits.

In a basic paper account, both orders may fill instantly at the displayed price. That result hides the decision each system actually made: A paid to cross the spread, while B joined a queue and might not trade at all.

This is the central failure in much paper trading. It tests whether an API accepts a request, then treats acceptance as execution.

An unattended agent needs more than a green response. It needs a credible sequence of acknowledgements, fills, cancels, rejects and market data updates, including the possibility that an order remains live while the strategy has moved on.

A paper venue cannot reproduce every hidden order or routing decision. It can, however, preserve the mechanics that change strategy behaviour and make its limitations measurable.

The goal is not a convincing screen. It is a test that can reveal when an agent depends on fills, timing or costs that are unlikely to exist live.

Acceptance is not a fill

Many sandboxes return an order ID as soon as they receive a syntactically valid request. The simulator may then fill the order at the latest quote, without checking whether that price was still available, how much displayed size remained, or how many orders were already ahead. A limit order can appear safe because it never pays the spread, while its simulated fills quietly assume the most favourable outcome.

Separate the order lifecycle into events. A request can be submitted, acknowledged, routed, partially filled, cancelled or rejected.

Each event needs a timestamp and a state transition. An acknowledgement means the venue accepted the instruction, not that the order has reached the book or traded. Preserve that distinction in the API your agent uses, even if a simplified simulator cannot model every intermediate step.

Use the same asynchronous contract

Keep the strategy-facing interface consistent across paper and live environments. Deliver acknowledgements and executions through the same event path, rather than returning a fill inside the order-placement call.

Include order IDs, cumulative filled quantity, remaining quantity, average fill price, fees and rejection codes. A cancel request should not erase an order immediately: a fill can arrive while the cancel is in flight.

This catches common unattended-agent bugs. For example, a process restart may cause it to resubmit an order because it did not persist the acknowledgement. A late fill can leave the position larger than the strategy expects. A duplicate cancel or retry can behave differently from a clean first request. Build the simulator to expose these states, and make event ordering reproducible with recorded inputs or a fixed random seed.

Where paper trading usually diverges

Queue position and partial fills

For a passive limit order, the best bid or offer is not a promise of a fill. The order joins a queue, and available volume may trade against orders ahead of it first. A useful simulator estimates queue position from displayed depth and subsequent trade flow. It should reduce that estimate when trades execute at the price, and handle cancellations cautiously because public data generally cannot show exactly which resting orders disappeared ahead of yours.

Even a rough queue model beats an unconditional fill. Use conservative assumptions where data is incomplete: for example, count only a fraction of observed cancellations as reducing the queue ahead. Test several queue assumptions and report how much strategy performance depends on them. If the strategy only works when every cancellation improves its position, the result is not solid.

Do not turn a partial execution into a full fill merely because the agent asked for a round number. The simulator should return cumulative quantity and leave the remainder working, or cancel it according to the time-in-force instruction. Position limits, stop logic and follow-up orders must respond to the filled quantity, not the requested quantity.

Latency, costs and venue rules

A simulator that uses the same market-data snapshot for signal generation and order matching removes the delay that occurs between those steps. Add configurable latency between quote receipt, decision, submission, acknowledgement and fill. Replay recorded market events where possible, and ensure an order cannot execute against a quote that arrived after its simulated arrival time.

Fees, rebates, tick sizes, lot sizes and time-in-force rules also change outcomes. Model them per instrument and account configuration.

A fill at the touch is not free: a marketable order generally pays the spread, while a passive fill can carry adverse selection if the price moves against it immediately afterwards. Report gross and net results separately, with slippage measured against a stated reference such as decision-time mid-price or order-arrival mid-price.

These are broad typical market ranges, not guarantees. Actual values depend on instrument, venue, broker, order size, volatility and network route.

Build around observable mechanics

Start with what the venue exposes, not with a claim that the simulator reproduces a complete order book. If you have level-one quotes and trades, you can model spread crossing, trade-throughs and a conservative queue estimate. If you have depth updates, you can improve queue estimates, but hidden liquidity, internalisation and routing can still make a paper fill differ from a live one.

Keep the execution model separate from strategy logic. It should consume timestamped market events and order events, then emit fills, rejects and state changes through the ordinary adapter.

Record the inputs needed to replay an incident: instrument, quote and trade sequence, order parameters, latency assumptions, configuration version and random seed. Replaying a failure should produce the same event trace.

A practical build sequence is:

1. Match the contract. Use the live order schema, state transitions, time-in-force options and error handling. Mark unsupported venue behaviour explicitly instead of silently accepting it.
2. Add deterministic replay. Feed historical market events in event-time order, apply configured latency and prevent fills against prices that were no longer available on arrival.
3. Model uncertainty. Add queue estimates, partial fills, variable latency and conservative costs. Run optimistic, central and adverse assumptions rather than presenting one estimate as fact.
4. Compare with live observations. Use small, controlled live or broker-provided execution samples to estimate the gap, without treating a short sample as proof that the model is accurate in every regime.

For developers who want a place to exercise agent order flows, Agentic Trading Index offers agentic paper trading from $9.99 a month. The same engineering test still applies: inspect which mechanics the environment models, then keep a record of what it simplifies.

Measure the gap before trusting the result

Compare simulated and observed execution by instrument, order type, time of day and volatility regime. A single average slippage number can hide a simulator that works for market orders but fills passive orders too generously. Track distributions and tails, not just averages, because a small number of bad executions can dominate an unattended strategy's risk.

Useful measures include:

• Fill ratio and time to first fill, split by marketable and passive orders.
• Slippage against decision-time and arrival-time reference prices, with fees included separately.
• Partial-fill frequency, average completion time and quantity left working after a cancel request.
• Reject rate by reason, stale-quote rate, duplicate submissions and order-state mismatches.
• Results under adverse queue, latency and spread assumptions, compared with the strategy's expected edge.

Set tolerances in relation to that edge. If a strategy expects 4 bps per trade, a 2 bps error in modelled costs is material.

If simulated passive fills exceed observed fills by 20 percentage points in a relevant sample, treat simulated performance as overstated until the cause is understood. These are diagnostic examples, not universal acceptance thresholds.

Paper results are useful for finding software faults, checking event handling and comparing controlled changes. They are not evidence that live execution will match a model. The advice here does not apply to a signal-only system where a separate execution desk owns every order; in that case, test the hand-off, signal timing and decision record rather than simulating fills the agent never controls.

#order execution testing #partial fills #queue position modelling #paper trading simulator #trading API testing #venue execution simulation #order book replay #simulated slippage