Strategy description metrics: measure what plain language hides

One operator buys a sharp dip in a liquid share at the close of a five-minute bar. Another skips the same setup because the spread widened before the order reached the market. Both say they follow the same plain-language strategy: buy a sharp dip and sell on a recovery. Their calls differ because the description leaves timing, price and execution rules unstated.

For software that places orders without supervision, those omissions become code paths. A useful strategy description must be measurable enough that two independent implementations make the same decision from the same data. It also needs metrics that show where intended behaviour diverges from actual fills, costs and risk.

Turn the idea into testable decisions

Plain language is a good starting point, not a trading specification. "Sharp dip" could mean a one-bar return below a threshold, a move away from a rolling mean, or a price change adjusted for volatility. "Recovery" could mean a touch of the mean, a bar close above it, or a fixed profit target. Each choice changes trade count, holding time and exposure.

Write the strategy as a chain of decisions with explicit inputs and timestamps. For a simple mean-reversion example, the specification might say: use completed five-minute bars; calculate the mean and standard deviation from the previous 20 completed bars; enter long when the close is more than two standard deviations below the mean; exit at the mean or after 15 minutes. That is still incomplete, but each condition can be checked.

Record at least these fields for every signal:

• Data basis: instrument identifier, venue or feed, bar construction, timezone and treatment of missing or corrected data.
• Decision time: bar close time, time the agent received the data, and maximum acceptable signal age.
• Eligibility: trading hours, spread ceiling, volatility limits, position cap and whether an existing order blocks another entry.
• Exit and failure rules: target, time stop, price stop, cancellation condition and behaviour after a disconnect.

Keep separate versions of the strategy logic and configuration. A change from a 20-bar to a 30-bar lookback is a different test, even if the prose stays the same. Without versioned parameters, performance comparisons mix changes in the idea with changes in implementation.

Measure the order path, not just the signal

Signal accuracy is not the same as tradable performance. A signal may predict a rebound but arrive after most of the move. A market order may capture the move while paying the spread and impact. A resting limit may reduce price paid but miss the fill, then leave the agent exposed to a later signal. Measure each stage against a named reference price and a defined clock.

For each order, preserve the signal timestamp, decision timestamp, submit time, acknowledgement time, every fill time and price, cancellation or rejection reason, and final position. Calculate implementation shortfall from the arrival midpoint: for a buy, fill price minus arrival midpoint; for a sell, arrival midpoint minus fill price. Report it in basis points and currency. Include commissions and exchange fees separately, so a low price impact does not conceal a high all-in cost.

The figures below are indicative typical ranges for liquid US large-cap shares during regular hours, for modest order sizes. They are measurement context, not performance targets or guarantees. Venue, volatility, order size and market conditions can move results well outside them.

These ranges need instrument-specific baselines. Compare orders within similar time-of-day, spread and volatility buckets.

A strategy trading the opening minutes should not be judged against midday fills. Track both filled and unfilled orders: excluding missed limit orders makes execution look better while hiding the opportunity cost of not participating.

Separate prediction, execution and portfolio results

Use a measurement stack that can attribute a disappointing result to a cause. Start with the signal: count eligible setups, entries, exits and rule violations.

Then measure execution: rejection rate, fill ratio, time to first fill, partial-fill frequency, slippage, fees and cancellation rate. Finally measure portfolio outcomes: net return, turnover, maximum drawdown, time in market, gross and net exposure, and concentration by instrument.

Report both per-trade and aggregate figures. Mean slippage can hide a small number of expensive outliers, so include the median and 90th percentile. Break out costs by side, order type and market regime. For returns, show gross and net results, and state whether the calculation includes unfilled signals. Keep a count and denominator beside every rate. "Three per cent rejected" means little without the number of orders and the rejection categories.

Useful operational checks include:

• Signal-to-submit delay, split into data receipt, computation and order construction.
• Percentage of orders submitted with a stale signal, defined by a fixed age threshold.
• Difference between intended position and broker-confirmed position after each fill or cancellation.
• Exposure and loss limits breached, including how long each breach remained open.

For a software agent, these are not merely reporting fields. They are controls. A stale signal can be discarded; an unexpected position can block new entries and trigger reconciliation.

An agent trading platform evaluation should check whether order events and timestamps are available with enough detail to calculate these measures. The FX-Tech agent trading resource can sit alongside that platform review, but the measurement definitions still need to match the chosen broker and instrument.

Build a repeatable measurement loop

Before enabling unattended orders, make each run reproducible. Save the input data, configuration version, decision log and broker responses. Recompute the same signals from the saved data and check that the agent produces the same intended orders. Then compare simulated and observed order paths, rather than comparing only headline returns.

A practical sequence is:

1. Freeze the rule set. Record indicator definitions, thresholds, bar timing, eligible hours, sizing, exits and risk caps. Give the configuration a version identifier.
2. Define metric formulas. Specify benchmark prices, clock boundaries, fee treatment, denominators and how cancelled, rejected and unfilled orders count.
3. Run historical and paper tests. Use the same decision logic, while recording which parts of the simulated execution model differ from the live venue.
4. Reconcile every order event. Match client order IDs to acknowledgements, fills, cancels and resulting positions. Flag missing or out-of-order events rather than silently dropping them.
5. Review cohorts and tails. Compare costs by time, spread, volatility and order type. Inspect high-cost outliers and the 90th percentile, not only the average.
6. Set operational limits. Define thresholds for stale signals, unexpected exposure, rejection bursts and cost overruns. Specify whether the agent pauses, cancels orders or reduces risk when a limit trips.

A plain-language description remains useful for communication, but it cannot settle every implementation choice. This measurement approach does not apply unchanged to strategies whose edge depends on human interpretation of news, order-book context or discretionary judgement, because those inputs cannot be reduced to a stable rule set without changing the strategy itself.

#signal definition #execution costs #automated trading #strategy measurement #trading strategy metrics #order fill analysis #strategy evaluation #realised exposure