High Frequency Trading
Systems Designs
An "Idiots Guide" to the Design and
Implementation of High Frequency Trading
Systems using standard CPUs, "Custom chips"
and networking components
Created by
Power Broadcasting
Computational Research Department
Adelaide / Wellington / Vancouver
This is a thought experiment (Gedankenexperiment) research white paper.
This research should not be construed as formal financial advice.
Version : 03 September 2012
How can one understand a complex
system? Design one.




Most complex computing technology is especially hard
to understand unless you have designed something
similar to it.
Computer technology is essentially logic gates fed by
a clock signal, generally with an operating system.
Complicated systems are often less complex than one
might imagine.
The "original system designers" hate complexity too.
Complexity++ == Reliability--
What do most honest HF trading
systems do?






Carry out programmed trades very quickly.
The programmed trades must be reliable and traceable,
and must be backed up.
HF trading systems must be interoperable with existing
trading systems.
A lot of sitting and waiting. Yes, this is in the specifications.
Trading systems must be made of materials and systems
“common enough” to be modular, reparable and
replaceable.
The systems mostly should not be interesting.
What do dishonest HF trading
systems do?



Run trades (by using existing intercept technology to
intercept "trades in progress" and take
countermeasures to obtain a small transaction profit).
Use programmed strategies to manipulate prices to
obtain a slightly larger transaction profit.
Work with other similar systems for general market
manipulation.
Malfeasance did not come 1st
The dishonest “price and market manipulation” HF trading systems
features were not in the original specifications for the technology.
These dishonest features evolved out of a general regulatory
vacuum and an unconcerned public in Europe and the Americas and
significant parts of Asia.
The honest, passive transactional stuff is in the original design
specification for HF trading systems. From this one must work one's
way to the original design base systems specificaitons.
What is the fastest way to complete
a transaction?







Simplify a "transaction" into its most elementary operations. Do
this by using a device independent language like Systems
Query Language (SQL).
A 3 to 5 binary record structure can imitate the “inputs and
outputs” to a SQL trading database.
~150000 logic gates can do a “trade transaction” with not too
much complexity. This is less logic than a 68000 CPU.
SQL to VHDL (Hardware Description Language) translation is
not easy!
Translate like this SQL → C++ → (VHDL or Verilog)
The minimalist system will not be capable of doing anything
else but completing a transaction.
PS: VHDL ~= Verilog
Poof! Completed.
A chip that does simple trades...




It will have to be a FPGA or a similar device.
You still have to make it communicate with the
outside world using existing standards – so no
proprietary interfaces.
The FPGA will have to live on a removable
system board with another CPU, GPU etc...
No cryogenic cooling is assumed here, good
designs don't need this botheration (but it does
wonders for speed).
What does an FPGA board look like?
On the system board...





… the FPGA now lives. There must be oceans of RAM
memory – and spare CPUs & GPUs (General Purpose
CPUs) on the board to be super fast.
So order 1000 system boards! There will be burnout.
RISC > CISC CPUs : The system board must have a
controlling CPU that is dead simple (yet fast) so you can
catch your design flaws.
Now the interfacing software is needed. You need a Real
Time Operating System (RTOS)!
Application-specific integrated circuits (ASICs) can also do
what FPGAs do, so there will always be more than one
way of making your system board...
The system board CPU & GPU...


… has to be RISC (Reduced Instruction Set
Computing) aka SPARC, ARM architecture, etc
to be fast enough to do this job ...
… has to be multicore, hyperthreading if
possible.
Post 2010 GPUs (Graphics Processing Units) will
work here too, but are not ideal for this kind of
application. Experiment here if you want.
The system board OS ...



RTOSes worth using : VxWorks, Nano-RK,
QNX, FreeRTOS, etc ... as Cyclic Executives
are just too rigid to be useful.
All the system board OS has to do is run a real
time database + applicable support applications
and be programmable in real time.
There is a cesspit of complexity here, but
somehow the systems actually used are
actually very simple.
The server, it holds the system
board ... and is not much different.




The servers containing real time trading systems can
have more software on them to make them more
usable.
Massive parallelism design rules : for reasons of
maximizing reliability and bandwidth flexibility.
Interfacing the real time trading system with the
"market standard" systems that are slower poses
hundreds of buffering and coherency problems.
Deep pockets (aka money) can solve these difficult
and obstinate problems.
Interfaces





Your real time trading system has to be accessible on
your trading floor (and at home), so it must have a
user friendly interface!
A real time trading system that is accessible across
the ocean can help.
So London and Amsterdam must be able to trade in
New York and Chicago etc ... Sydney and Tokyo.
So, lots of computing power is needed for lots of
communications link encryption.
To keep within real time, you need a fiber optic path
along all points of the transaction chain.
Transaction redundancy





As a rule, each market transaction must be backed up
in at least 10 places to be domestically legal and 20
places to be internationally legal.
Redundancy : It's mostly just temporary buffering ...
deletable from 3 days to 3 years.
So, buy lots of Hard Disks (and Redundant Arrays of
them aka RAIDs).
On a slow day, a firm might use 20% of a RAID but a
fast day may yield 335% RAID consumption.
Yes, there will be LOG FILES!
Malfeasance 101




OK, the HF trading system is up. Now what?
Do everything in New York City or London. Not much
regulation here. Anything goes ... and honestly gained
normal trading efficiencies made the system pay for
itself in 2 weeks.
This system has probably cost 20 millions SDRs,
enough to run Radio Canada International for a year –
but can it "earn" 120 millions? Maybe.
So, lets get programming …
Oops! Design constraints...




Cross market malfeasance takes work.
HF trading systems designed for stocks will not
work in the FOREX market. Several specialized
trading platforms need to be built -- and these
platforms must talk to each other.
This will take time, and some protocol reworking
is needed. Ding, 60 millions SDRs.
Coordinated HF trading systems may have
payback horizons of a week vs a month.
Malfeasance 101 : Tax avoidance




A trading system that engages in trades that are
designed in such a way so as to avoid taxes
can be a real money saver for the firm.
Being a party of both sides of the transaction
can really help in tax avoidance.
The real time transactions have to be complex
enough so that the Inland Revenue can't
comprehend it and thus can't tax it.
So, use the system in the 'black pools' of the
unregulated derivative markets.
Malfeasance 101
"Price manipulation"



Price Manipulation using HF trading systems is an art and
a science. Possibly it is the main reason for ~20% of all HF
trades.
Little is actually known about the full extent of price
manipulation, but the results can be clearly seen.
HF Trading systems have to be on 'both sides of the
transaction' (where possible) else mainly do 'man in the
middle' trading attacks.
This amoral practice can be harmless but … megalomania is
rampant in multi-generational CEOs who have never
actually worked for a living. So grand scale price
manipulation schemes will come down autocratically and
often.
Malfeasance 101 : Tech support




The IT Department that makes all your HF trading possible
has to be paid off. This is a Cost of Doing Business write
off (in the USA anyway).
Oddly, most (in the know) IT staff that create and oversee
these HF trading systems are probably massively
underpaid.
IT staffs may be underpaid by an order of 125x to 2048x,
so there is plenty of theft going around.
Any finance firm CEO paying their IT departments less
than 20% (of all profits extracted from HF trading systems)
are begging for 25 years in prison or execution in China.
HF trading strategies, externalities



The cost of having any market sector (like FOREX)
saturated with HF trading systems engaging in
programmed malfeasance is not obvious to anyone
but 'day traders' – who can actually see the outcomes.
You can't beat a fast [and clever] machine that has
more knowledge than you do. This is a law of nature.
If HF trading systems are possibly causing you to lose
money in the markets -- then HF trading systems are
directly causing you to lose money in the markets ...
HF Trading, externalities – fixes
Yes, it is possible to fix the issue of malfeasance in HF
Trading systems – but this may take decades for any
government to do anything. Governments are sadly in
on the corruption in the finance system, as nepotism
and favoritism (with the private sector) as all but
obliterated functioning government.
However, for an individual there are quick fixes.
Liquidate your holdings cautiously and leave the market.
Most people opt for cash -- and later on Gold or Silver.
Australia, Canada, New Zealand – are not exempt. This
is not a problem exclusive to the US or UK.
HF trading, more externalities




HF trading systems alone will not cause the collapse of a
nation or region.
Societies (and nations) do truly collapse long before their
economies finally collapse.
Failure and collapse has a thousand parents, and HF
trading is just one of them.
Complex systems that have become corrupted and
unstable implode with clockwork regularity, as a general
rule of nature. Finance systems are no exception.
HF Trading, Historical Perspective
HF trading is one of the dozens of significant (but not actual)
causes of the North American 'Great Depression'. Stock
tickers in the 1920s ran at about 5 characters per second,
and slowed down the 1929 crash by at least a fortnight.
There have been some flash recessions in the developing
world since the 1980s. High speed transaction finance has
contributed to at least two (and possibly five) flash
recessions since 1975.
Small investors “in the modern era” almost never directly
cause flash recessions. These recessions are more the
product of large institutions. The small investor has lost
ground with respect to the large investor each decade after
1970 -- with respect to causing “money flight events”.
Why small investors will not cause the
next global finance meltdown...





Mainly, it boils down to actual market knowledge and
access to information.
Small investors are swamped with seas of useless
information – that does little to help them know what is
actually going on.
Small investors actually work, and available time is not
as great. Institutional investors have time.
Large investors (banks, trading houses etc...) have
clear government backing, and deep pockets. This
allows for the possibilty of infinite malfeasance.
The “power disparity” is as true in North America as it
is in Europe, Asia and Australasia.
End
If you liked this research document, please feel
free to make a donation. This research may be
suitable for a book or documentary later on.
Please contact the author at : dist23 at juno dot com