Assessing the benefits of independent TCA in verifying FX algo performance

By Pete Eggleston

When embarking upon a process to implement a TCA framework an institution is faced with the strategic decision of whether to build such a framework in-house, purchase a solution from an independent provider, or rely on TCA supplied by the brokers that they execute with. This article explores the advantages and disadvantages of these options, and distills them into the following themes:

1. Ability to compare and contrast performance
   using consistently computed metrics
2. Trade data availability, especially enriched algo
   fill data
3. Independence
4. Peer Analysis
5. Costs

Ability to compare and contrast performance using
consistently computed metrics

Accessing execution performance metrics, that are computed in a consistent way to allow fair comparative analysis, is more challenging that one may think. It is tricky to do this in a listed market such as equities, where there is publicly available national best bid and offer data, and a consistent format for the reporting of many data fields, but in an OTC market such as FX it is significantly more challenging. Broadly, there are 2 major areas that create issues:

i. Market data
ii. Consistent methodologies and definitions

Within an OTC market, without an official market tape, or centrally cleared price, there is no universal truth when it comes to determining a mid price. It can only be estimated by aggregating from multiple sources of liquidity provision. Thus, there is always the potential for differences across liquidity providers depending on both the sources they use to aggregate to estimate the market mid, and also the methodology they use with regards to data cleansing and aggregation. To provide some indication of the scale and complexity of the market data aggregation issue, at BestX we consume billions of price updates a day from over 120 liquidity providers. This is complex enough for spot FX prices, but as soon as you turn your attention to forward points, the problems multiply.

For example, how do you interpolate between the published forward dates, which tend to be standard tenors (e.g. 1m, 3m, 6m etc) from most sources. You obviously also need to ensure you are taking into account holiday calendars and day count conventions globally, in addition to allowing for the ‘turn’ effect. Clearly, this is a significant undertaking and therefore an in-house build project would need to account for this, but it also exemplifies how difficult it is to compare output from different TCA reports supplied by liquidity providers. There is no universal truth for spot FX (as shown in Figure 1 below which compares prices from 2 of the largest sources for the most liquid pair, EURUSD) but for forward FX it is even harder to estimate. Using a single independent TCA provider, that utilises an extensive and representative market data set, at least allows a user to compare and contrast performance using consistent estimates of bid, mid and offer for any given time stamp. In addition to the consistency issue, there are also considerations over market coverage. It is time consuming and expensive to gather all the data to cover all currency pairs, e,g, NDFs, restricted EM pairs etc.

For example, for a fund that has executed a USDKRW trade, in order to do an in-house transaction cost analysis, the fund needs to know whether this trade is non-deliverable/ deliverable, onshore or offshore and has to have all the relevant market data in the database, which is a complex and expensive undertaking. However, an independent TCA specialist should have economies of scale when it comes to the use of market data and hence should be in a position to provide cheaper indirect access to a broader market data set. Consistent market data, covering the full market, is only part of the issue though. You then need to decide what you are going to do with it in terms of constructing benchmarks. As with price estimation, there is plenty of scope here for the introduction of inconsistencies. For example, if you are replying on TCA reports supplied by your liquidity providers, and want to compare performance of particular benchmarks, then it is essential that the methodologies used for the benchmark construction are exactly the same. For example, generally the Arrival Price benchmark is thought to be pretty straightforward, simply comparing performance price to the price observed at the market arrival time stamp.

Putting the issues described above to one side, even if all your liquidity providers were to be using exactly the same market data set, which is pretty unlikely, what arrival price is actually used? Some liquidity providers use mid, which tends to be a market convention, but some use top of book bid or offer, which would render comparison of performance to be meaningless. Using an independent TCA provider, or indeed building it yourself, at least provides assurance that you are truly comparing apples to apples as the benchmarks are constructed consistently.

Trade Data Availability

Data issues are not only related to market data. High quality TCA is also dependent on accurate, and enriched, transaction data. This is generally available via broker TCA for their own transactions, but unfortunately a fully enriched data set is not always available via an EMS (e.g. algo fills that are tagged with venue, execution type etc). If the requirement is for a more basic level TCA (e.g. if it is for a client that largely executes via voice or via a custodian) then such additional data enrichment may not be a priority but for any clients trading electronically, and especially algos, then it will be required to gain full value from the analysis. It is possible to access such enriched data via an independent provider, although due diligence should be carried out during the selection process to ensure that the provider has full direct connectivity to all of the required brokers to access this data and isn’t just relying on a subset of the data typically provided by an EMS.

Independence

Another theme to explore when making a decision on a TCA solution relates to independence. Clearly, there are benefits from having an independent TCA provider perform the analysis for you, using unbiased data sets and metrics. Indeed, using TCA provided by liquidity providers is sometimes referred to as ‘marking your own homework’, which you may be comfortable with for certain use cases. However, if you intend to use output from your TCA in, for example, PRIPPs reporting (costs and charges disclosures) then independently computed metrics may be more appropriate. Indeed, the PRIPPs legislation is clear that for FX, it is not permitted to use pricing from a single counterparty or platform, as per below. The ‘marking your own homework’ theme also extends to the in-house build option. Asset owners have become increasingly aware of the value of rigorous best execution and TCA processes, and it is now commonplace that when owners put RFPs out to tender they ask managers to clarify whether a) they have a TCA process in place and b) if they do, is this TCA solution independent. An asset owner would instinctively be more comfortable knowing that the manager is measuring and reporting its costs using an independent provider.

Peer Analysis

Although an in-house build does provide the benefit of retaining control over the development of the platform, there are additional disadvantages in addition to the previously mentioned complexity and costs related to market data sourcing and aggregation. There is increasing demand for high quality peer analysis in FX TCA, a theme that has evolved in the Equity market for many years. To access a statistically significant data set for rigorous peer analysis is a major challenge, and cannot be delivered by an in-house solution as asset managers are obviously extremely unlikely to share their transaction data directly with a competitor. Equally, a broker-delivered TCA will only ever be based on transactions executed by that broker so will only ever represent a fraction of any one asset manager’s flow. It is also likely to be biased depending on that broker’s particular strengths in given markets or currencies. It is therefore only possible to access meaningful peer analysis via an independent TCA provider, although care should be taken during the selection process to ensure the universe is significant. Figure 2 graphically illustrates how contributing to an anonymised community data pool (labelled as Peer) increases the sample size, and therefore the validity of the analysis.

Costs

Lastly, but clearly not least, cost is also a very important factor. Broker supplied TCA is generally free and provided as a service as part of the execution package, although as discussed, limited to only those trades that the manager executes with that broker, and generally only available for algos. The costs associated with an in-house build should not be underestimated, both the development costs but also the ongoing costs, including market data licence fees and data storage costs. It is also worth noting that the development of an in-house solution is not a one-off investment. Any platform will need to be maintained (e.g. data feeds change in format and content, new brokers may require additional connectivity building, user requirements for reports or other functionality change etc) but also the strategic nature of TCA and best execution is still evolving as regulatory requirements increase and market structure, especially in OTC markets, continues to develop. Deploying an independent solution, especially one that is cloud-based, provides advantages here as the product should be kept updated, often at minimal ongoing cost, to reflect the changing requirements and market developments.

Conclusion

In Table 1 we summarise the various pros and cons of the different options discussed in this article. The ultimate decision will boil down to a trade off between the requirements, the value to be extracted and the associated cost of the solution. An independent provider will need to be engaged if the requirement set includes the following:
• regulatory adherence and the need for independent reporting and oversight,
• the ability to compare performance of products and liquidity providers on a consistent basis,
• peer analysis
The cost of any such independent solution can be an issue, although this can be mitigated to some extent by ensuring that maximum value is extracted from the product. If TCA is simply used to ‘tick a box’, then some may feel the cost of an independent solution is not value for money. However, if the TCA framework is firmly embedded in the institution’s best execution process, allowing more informed decisions to be made around all aspects of the execution process (e.g. product/ broker selection, trade timing, trade sizing etc), then any decent independent solution should pay for itself very quickly in terms of execution cost savings made.