PeachPy: A Python Framework for Developing
High-Performance Assembly Kernels
Marat Dukhan
School of Computational Science and Engineering
College of Computing
Georgia Institute of Technology
Presentation on PyHPC 2013
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
1 / 43
Outline
1
Motivation
2
PeachPy Foundations
3
PeachPy Basics
4
Assembly Programming Automation
5
Metaprogramming
6
Instruction Streams
7
Experimental Validation
8
Conclusion
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
2 / 43
The Problem
dot_product:
XORPD xmm0, xmm0
XORPD xmm1, xmm1
XORPD xmm2, xmm2
SUB r8, 6
JAE .restore
align 16
.batch:
MOVUPD xmm3, [rcx]
MOVUPD xmm7, [rdx]
MULPD xmm3, xmm7
ADDPD xmm0, xmm3
MOVUPD xmm4, [rcx +
MOVUPD xmm7, [rdx +
MULPD xmm4, xmm7
ADDPD xmm1, xmm4
MOVUPD xmm5, [rcx +
MOVUPD xmm7, [rdx +
MULPD xmm5, xmm7
ADDPD xmm2, xmm5
ADD rcx, 48
ADD rdx, 48
SUB r8, 6
JAE .batch
.restore:
ADD r8, 6
JZ .reduce
.remainder:
MOVSD xmm3, [rcx]
MOVSD xmm4, [rdx]
MULSD xmm3, xmm4
ADDSD xmm0, xmm3
ADD rcx, 8
ADD rdx, 8
SUB r8, 1
JNZ .remainder
.reduce:
ADDPD xmm0, xmm1
ADDPD xmm0, xmm2
HADDPD xmm0, xmm0
RET
Marat Dukhan (Georgia Tech)
PeachPy
16]
16]
32]
32]
PyHPC 2013
3 / 43
The Problem
dot_product:
XORPD xmm0, xmm0
XORPD xmm1, xmm1
XORPD xmm2, xmm2
SUB r8, 6
JAE .restore
align 16
.batch:
MOVUPD xmm3, [rcx]
MOVUPD xmm7, [rdx]
MULPD xmm3, xmm7
ADDPD xmm0, xmm3
MOVUPD xmm4, [rcx +
MOVUPD xmm7, [rdx +
MULPD xmm4, xmm7
ADDPD xmm1, xmm4
MOVUPD xmm5, [rcx +
MOVUPD xmm7, [rdx +
MULPD xmm5, xmm7
ADDPD xmm2, xmm5
ADD rcx, 48
ADD rdx, 48
SUB r8, 6
JAE .batch
.restore:
ADD r8, 6
JZ .reduce
.remainder:
MOVSD xmm3, [rcx]
MOVSD xmm4, [rdx]
MULSD xmm3, xmm4
ADDSD xmm0, xmm3
ADD rcx, 8
ADD rdx, 8
SUB r8, 1
JNZ .remainder
.reduce:
ADDPD xmm0, xmm1
ADDPD xmm0, xmm2
HADDPD xmm0, xmm0
RET
Marat Dukhan (Georgia Tech)
dot_product:
XORPS xmm0, xmm0
XORPS xmm1, xmm1
XORPS xmm2, xmm2
SUB r8, 12
JAE .restore
align 16
.batch:
MOVUPS xmm3, [rcx]
MOVUPS xmm7, [rdx]
MULPS xmm3, xmm7
ADDPS xmm0, xmm3
MOVUPS xmm4, [rcx +
MOVUPS xmm7, [rdx +
MULPS xmm4, xmm7
ADDPS xmm1, xmm4
MOVUPS xmm5, [rcx +
MOVUPS xmm7, [rdx +
MULPS xmm5, xmm7
ADDPS xmm2, xmm5
ADD rcx, 48
ADD rdx, 48
SUB r8, 12
JAE .batch
.restore:
ADD r8, 12
JZ .reduce
.remainder:
MOVSS xmm3, [rcx]
MOVSS xmm4, [rdx]
MULSS xmm3, xmm4
ADDSS xmm0, xmm3
ADD rcx, 4
ADD rdx, 4
SUB r8, 1
JNZ .remainder
.reduce:
ADDPS xmm0, xmm1
ADDPS xmm0, xmm2
HADDPS xmm0, xmm0
HADDPS xmm0, xmm0
RET
16]
16]
32]
32]
PeachPy
16]
16]
32]
32]
PyHPC 2013
4 / 43
The Problem
dot_product:
XORPD xmm0, xmm0
XORPD xmm1, xmm1
XORPD xmm2, xmm2
SUB r8, 6
JAE .restore
align 16
.batch:
MOVUPD xmm3, [rcx]
MOVUPD xmm7, [rdx]
MULPD xmm3, xmm7
ADDPD xmm0, xmm3
MOVUPD xmm4, [rcx +
MOVUPD xmm7, [rdx +
MULPD xmm4, xmm7
ADDPD xmm1, xmm4
MOVUPD xmm5, [rcx +
MOVUPD xmm7, [rdx +
MULPD xmm5, xmm7
ADDPD xmm2, xmm5
ADD rcx, 48
ADD rdx, 48
SUB r8, 6
JAE .batch
.restore:
ADD r8, 6
JZ .reduce
.remainder:
MOVSD xmm3, [rcx]
MOVSD xmm4, [rdx]
MULSD xmm3, xmm4
ADDSD xmm0, xmm3
ADD rcx, 8
ADD rdx, 8
SUB r8, 1
JNZ .remainder
.reduce:
ADDPD xmm0, xmm1
ADDPD xmm0, xmm2
HADDPD xmm0, xmm0
RET
16]
16]
32]
32]
dot_product:
XORPS xmm0, xmm0
XORPS xmm1, xmm1
XORPS xmm2, xmm2
SUB r8, 12
JAE .restore
align 16
.batch:
MOVUPS xmm3, [rcx]
MOVUPS xmm7, [rdx]
MULPS xmm3, xmm7
ADDPS xmm0, xmm3
MOVUPS xmm4, [rcx +
MOVUPS xmm7, [rdx +
MULPS xmm4, xmm7
ADDPS xmm1, xmm4
MOVUPS xmm5, [rcx +
MOVUPS xmm7, [rdx +
MULPS xmm5, xmm7
ADDPS xmm2, xmm5
ADD rcx, 48
ADD rdx, 48
SUB r8, 12
JAE .batch
.restore:
ADD r8, 12
JZ .reduce
.remainder:
MOVSS xmm3, [rcx]
MOVSS xmm4, [rdx]
MULSS xmm3, xmm4
ADDSS xmm0, xmm3
ADD rcx, 4
ADD rdx, 4
SUB r8, 1
JNZ .remainder
.reduce:
ADDPS xmm0, xmm1
ADDPS xmm0, xmm2
HADDPS xmm0, xmm0
HADDPS xmm0, xmm0
RET
Marat Dukhan (Georgia Tech)
16]
16]
32]
32]
dot_product:
XORPD xmm0, xmm0
XORPD xmm1, xmm1
XORPD xmm2, xmm2
SUB rdx, 6
JAE .restore
align 16
.batch:
MOVUPD xmm3, [rdi]
MOVUPD xmm7, [rsi]
MULPD xmm3, xmm7
ADDPD xmm0, xmm3
MOVUPD xmm4, [rdi +
MOVUPD xmm7, [rsi +
MULPD xmm4, xmm7
ADDPD xmm1, xmm4
MOVUPD xmm5, [rdi +
MOVUPD xmm7, [rsi +
MULPD xmm5, xmm7
ADDPD xmm2, xmm5
ADD rdi, 48
ADD rsi, 48
SUB rdx, 6
JAE .batch
.restore:
ADD rdx, 6
JZ .reduce
.remainder:
MOVSD xmm3, [rdi]
MOVSD xmm4, [rsi]
MULSD xmm3, xmm4
ADDSD xmm0, xmm3
ADD rdi, 8
ADD rsi, 8
SUB rdx, 1
JNZ .remainder
.reduce:
ADDPD xmm0, xmm1
ADDPD xmm0, xmm2
HADDPD xmm0, xmm0
RET
PeachPy
16]
16]
32]
32]
dot_product:
XORPS xmm0, xmm0
XORPS xmm1, xmm1
XORPS xmm2, xmm2
SUB rdx, 12
JAE .restore
align 16
.batch:
MOVUPS xmm3, [rdi]
MOVUPS xmm7, [rsi]
MULPS xmm3, xmm7
ADDPS xmm0, xmm3
MOVUPS xmm4, [rdi +
MOVUPS xmm7, [rsi +
MULPS xmm4, xmm7
ADDPS xmm1, xmm4
MOVUPS xmm5, [rdi +
MOVUPS xmm7, [rsi +
MULPS xmm5, xmm7
ADDPS xmm2, xmm5
ADD rdi, 48
ADD rsi, 48
SUB rdx, 12
JAE .batch
.restore:
ADD rdx, 12
JZ .reduce
.remainder:
MOVSS xmm3, [rdi]
MOVSS xmm4, [rsi]
MULSS xmm3, xmm4
ADDSS xmm0, xmm3
ADD rdi, 4
ADD rsi, 4
SUB rdx, 1
JNZ .remainder
.reduce:
ADDPS xmm0, xmm1
ADDPS xmm0, xmm2
HADDPS xmm0, xmm0
HADDPS xmm0, xmm0
RET
PyHPC 2013
16]
16]
32]
32]
5 / 43
The Problem
dot_product:
XORPD xmm0, xmm0
XORPD xmm1, xmm1
XORPD xmm2, xmm2
SUB r8, 6
JAE .restore
align 16
.batch:
MOVUPD xmm3, [rcx]
MOVUPD xmm7, [rdx]
MULPD xmm3, xmm7
ADDPD xmm0, xmm3
MOVUPD xmm4, [rcx +
MOVUPD xmm7, [rdx +
MULPD xmm4, xmm7
ADDPD xmm1, xmm4
MOVUPD xmm5, [rcx +
MOVUPD xmm7, [rdx +
MULPD xmm5, xmm7
ADDPD xmm2, xmm5
ADD rcx, 48
ADD rdx, 48
SUB r8, 6
JAE .batch
.restore:
ADD r8, 6
JZ .reduce
.remainder:
MOVSD xmm3, [rcx]
MOVSD xmm4, [rdx]
MULSD xmm3, xmm4
ADDSD xmm0, xmm3
ADD rcx, 8
ADD rdx, 8
SUB r8, 1
JNZ .remainder
.reduce:
ADDPD xmm0, xmm1
ADDPD xmm0, xmm2
HADDPD xmm0, xmm0
RET
16]
16]
32]
32]
dot_product:
VXORPD ymm0, ymm0
VXORPD ymm1, ymm1
VXORPD ymm2, ymm2
SUB r8, 12
JAE .restore
align 16
.batch:
VMOVUPD ymm3, [rcx]
VMOVUPD ymm7, [rdx]
VMULPD ymm3, ymm7
VADDPD ymm0, ymm3
VMOVUPD ymm4, [rcx + 32]
VMOVUPD ymm7, [rdx + 32]
VMULPD ymm4, ymm7
VADDPD ymm1, ymm4
VMOVUPD ymm5, [rcx + 64]
VMOVUPD ymm7, [rdx + 64]
VMULPD ymm5, ymm7
VADDPD ymm2, ymm5
ADD rcx, 96
ADD rdx, 96
SUB r8, 12
JAE .batch
.restore:
ADD r8, 12
JZ .reduce
.remainder:
VMOVSD xmm3, [rcx]
VMOVSD xmm4, [rdx]
VMULSD xmm3, xmm4
VADDSD ymm0, ymm3
ADD rcx, 8
ADD rdx, 8
SUB r8, 1
JNZ .remainder
.reduce:
VADDPD ymm0, ymm1
VADDPD ymm0, ymm2
VEXTRACTF128 xmm1, ymm0, 1
VADDPD xmm0, xmm1
VHADDPD xmm0, xmm0
RET
Marat Dukhan (Georgia Tech)
dot_product:
XORPS xmm0, xmm0
XORPS xmm1, xmm1
XORPS xmm2, xmm2
SUB r8, 12
JAE .restore
align 16
.batch:
MOVUPS xmm3, [rcx]
MOVUPS xmm7, [rdx]
MULPS xmm3, xmm7
ADDPS xmm0, xmm3
MOVUPS xmm4, [rcx + 16]
MOVUPS xmm7, [rdx + 16]
MULPS xmm4, xmm7
ADDPS xmm1, xmm4
MOVUPS xmm5, [rcx + 32]
MOVUPS xmm7, [rdx + 32]
MULPS xmm5, xmm7
ADDPS xmm2, xmm5
ADD rcx, 48
ADD rdx, 48
SUB r8, 12
JAE .batch
.restore:
ADD r8, 12
JZ .reduce
.remainder:
MOVSS xmm3, [rcx]
MOVSS xmm4, [rdx]
MULSS xmm3, xmm4
ADDSS xmm0, xmm3
ADD rcx, 4
ADD rdx, 4
SUB r8, 1
JNZ .remainder
.reduce:
ADDPS xmm0, xmm1
ADDPS xmm0, xmm2
HADDPS xmm0, xmm0
HADDPS xmm0, xmm0
RET
dot_product:
XORPD xmm0, xmm0
XORPD xmm1, xmm1
XORPD xmm2, xmm2
SUB rdx, 6
JAE .restore
align 16
.batch:
MOVUPD xmm3, [rdi]
MOVUPD xmm7, [rsi]
MULPD xmm3, xmm7
ADDPD xmm0, xmm3
MOVUPD xmm4, [rdi +
MOVUPD xmm7, [rsi +
MULPD xmm4, xmm7
ADDPD xmm1, xmm4
MOVUPD xmm5, [rdi +
MOVUPD xmm7, [rsi +
MULPD xmm5, xmm7
ADDPD xmm2, xmm5
ADD rdi, 48
ADD rsi, 48
SUB rdx, 6
JAE .batch
.restore:
ADD rdx, 6
JZ .reduce
.remainder:
MOVSD xmm3, [rdi]
MOVSD xmm4, [rsi]
MULSD xmm3, xmm4
ADDSD xmm0, xmm3
ADD rdi, 8
ADD rsi, 8
SUB rdx, 1
JNZ .remainder
.reduce:
ADDPD xmm0, xmm1
ADDPD xmm0, xmm2
HADDPD xmm0, xmm0
RET
dot_product:
VXORPS ymm0, ymm0
VXORPS ymm1, ymm1
VXORPS ymm2, ymm2
SUB r8, 24
JAE .restore
align 16
.batch:
VMOVUPS ymm3, [rcx]
VMOVUPS ymm7, [rdx]
VMULPS ymm3, ymm7
VADDPS ymm0, ymm3
VMOVUPS ymm4, [rcx + 32]
VMOVUPS ymm7, [rdx + 32]
VMULPS ymm4, ymm7
VADDPS ymm1, ymm4
VMOVUPS ymm5, [rcx + 64]
VMOVUPS ymm7, [rdx + 64]
VMULPS ymm5, ymm7
VADDPS ymm2, ymm5
ADD rcx, 96
ADD rdx, 96
SUB r8, 24
JAE .batch
.restore:
ADD r8, 24
JZ .reduce
.remainder:
VMOVSS xmm3, [rcx]
VMOVSS xmm4, [rdx]
VMULSS xmm3, xmm4
VADDSS ymm0, ymm3
ADD rcx, 4
ADD rdx, 4
SUB r8, 1
JNZ .remainder
.reduce:
VADDPS ymm0, ymm1
VADDPS ymm0, ymm2
VEXTRACTF128 xmm1, ymm0, 1
VADDPS xmm0, xmm1
VHADDPS xmm0, xmm0
VHADDPS xmm0, xmm0
RET
dot_product:
VXORPD ymm0, ymm0
VXORPD ymm1, ymm1
VXORPD ymm2, ymm2
SUB rdx, 12
JAE .restore
align 16
.batch:
VMOVUPD ymm3, [rdi]
VMOVUPD ymm7, [rsi]
VMULPD ymm3, ymm7
VADDPD ymm0, ymm3
VMOVUPD ymm4, [rdi + 32]
VMOVUPD ymm7, [rsi + 32]
VMULPD ymm4, ymm7
VADDPD ymm1, ymm4
VMOVUPD ymm5, [rdi + 64]
VMOVUPD ymm7, [rsi + 64]
VMULPD ymm5, ymm7
VADDPD ymm2, ymm5
ADD rdi, 96
ADD rsi, 96
SUB rdx, 12
JAE .batch
.restore:
ADD rdx, 12
JZ .reduce
.remainder:
VMOVSD xmm3, [rdi]
VMOVSD xmm4, [rsi]
VMULSD xmm3, xmm4
VADDSD ymm0, ymm3
ADD rdi, 8
ADD rsi, 8
SUB rdx, 1
JNZ .remainder
.reduce:
VADDPD ymm0, ymm1
VADDPD ymm0, ymm2
VEXTRACTF128 xmm1, ymm0, 1
VADDPD xmm0, xmm1
VHADDPD xmm0, xmm0
RET
PeachPy
16]
16]
32]
32]
dot_product:
XORPS xmm0, xmm0
XORPS xmm1, xmm1
XORPS xmm2, xmm2
SUB rdx, 12
JAE .restore
align 16
.batch:
MOVUPS xmm3, [rdi]
MOVUPS xmm7, [rsi]
MULPS xmm3, xmm7
ADDPS xmm0, xmm3
MOVUPS xmm4, [rdi + 16]
MOVUPS xmm7, [rsi + 16]
MULPS xmm4, xmm7
ADDPS xmm1, xmm4
MOVUPS xmm5, [rdi + 32]
MOVUPS xmm7, [rsi + 32]
MULPS xmm5, xmm7
ADDPS xmm2, xmm5
ADD rdi, 48
ADD rsi, 48
SUB rdx, 12
JAE .batch
.restore:
ADD rdx, 12
JZ .reduce
.remainder:
MOVSS xmm3, [rdi]
MOVSS xmm4, [rsi]
MULSS xmm3, xmm4
ADDSS xmm0, xmm3
ADD rdi, 4
ADD rsi, 4
SUB rdx, 1
JNZ .remainder
.reduce:
ADDPS xmm0, xmm1
ADDPS xmm0, xmm2
HADDPS xmm0, xmm0
HADDPS xmm0, xmm0
RET
dot_product:
VXORPS ymm0, ymm0
VXORPS ymm1, ymm1
VXORPS ymm2, ymm2
SUB rdx, 24
JAE .restore
align 16
.batch:
VMOVUPS ymm3, [rdi]
VMOVUPS ymm7, [rsi]
VMULPS ymm3, ymm7
VADDPS ymm0, ymm3
VMOVUPS ymm4, [rdi + 32]
VMOVUPS ymm7, [rsi + 32]
VMULPS ymm4, ymm7
VADDPS ymm1, ymm4
VMOVUPS ymm5, [rdi + 64]
VMOVUPS ymm7, [rsi + 64]
VMULPS ymm5, ymm7
VADDPS ymm2, ymm5
ADD rdi, 96
ADD rsi, 96
SUB rdx, 24
JAE .batch
.restore:
ADD rdx, 24
JZ .reduce
.remainder:
VMOVSS xmm3, [rdi]
VMOVSS xmm4, [rsi]
VMULSS xmm3, xmm4
VADDSS ymm0, ymm3
ADD rdi, 4
ADD rsi, 4
SUB rdx, 1
JNZ .remainder
.reduce:
VADDPS ymm0, ymm1
VADDPS ymm0, ymm2
VEXTRACTF128 xmm1, ymm0, 1
VADDPS xmm0, xmm1
VHADDPS xmm0, xmm0
VHADDPS xmm0, xmm0
RET
PyHPC 2013
6 / 43
The Research Problem
This reasearch is about the problem of generating multiple similar assembly
kernels:
Kernels which perform similar operations
I
E.g. vector addition/subtraction
Kernels which do same operation on dierent data types
I
E.g. single- and double-precision dot product
Kernels which target dierent microarchitectures or ISA
I
E.g. dot product for AVX, FMA4, FMA3
Kernels which use dierent ABIs
I
E.g. x86-64 on Windows and Linux
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
7 / 43
Outline
1
Motivation
2
PeachPy Foundations
3
PeachPy Basics
4
Assembly Programming Automation
5
Metaprogramming
6
Instruction Streams
7
Experimental Validation
8
Conclusion
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
8 / 43
Assembly Compilation Process
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
9 / 43
Assembly Compilation Process
Lets replace macro processor with something
More exible
More standardized
More popular
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
10 / 43
Assembly Compilation Process
Lets replace macro processor with something
More exible
More standardized
More popular
Python!
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
11 / 43
Introducing PeachPy
PeachPy is. . .
. . . an automation and metaprogramming tool for assembly
programming
. . . an Assembly-like DSL: PeachPy user is exposed to the same
low-level details as assembly programmer
. . . a Python framework: any PeachPy code is a valid Python
code
PeachPy is not. . .
. . . a compiler: PeachPy does not oer high-level programming
abstractions
. . . an assembler: PeachPy does not generate machine code
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
12 / 43
PeachPy Philosophy
PeachPy is for writing high-performance codes
I
I
I
No support for invoke, OOP, and other "high-level assembly"
No kernel-mode instructions
No system instructions
PeachPy is for writing assembly codes
I
Not a replacement for high-level compiler
All optimizations possible to do in assembly should be possible to do
in PeachPy
Everything that can be automated in assembly programming should be
automated
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
13 / 43
Outline
1
Motivation
2
PeachPy Foundations
3
PeachPy Basics
4
Assembly Programming Automation
5
Metaprogramming
6
Instruction Streams
7
Experimental Validation
8
Conclusion
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
14 / 43
Minimal PeachPy Function
from peachpy.x64 import *
abi = peachpy.c.ABI('x64-sysv')
assembler = Assembler(abi)
x_argument = peachpy.c.Parameter("x",
peachpy.c.Type("uint32_t"))
arguments = (x_argument,)
function_name = "f"
microachitecture = "SandyBridge"
with Function(assembler, function_name,
arguments, microarchitecture):
RETURN()
print assembler
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
15 / 43
Modules
PeachPy functionality is concentrated in three Python modules:
peachpy.c for C compatilibity classes (C types and ABIs)
peachpy.x64 for x86-64 assembly classes
peachpy.arm for ARM assembly classes
Assembly modules are intended to be imported into program workspace:
# This will make the syntax of PeachPy codes
# very similar to native assembly
from peachpy.x64 import *
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
16 / 43
Assembler and Function
Assembler
I
I
I
Container for functions
Contains only functions with specied ABI
Normally may be saved as assembly le to disk
Function
I
I
Created using with syntax: with Function(...):
Creates an active instruction stream
Microarchitecture
I
I
String parameter for Function constructor
Restricts the set of supported instructions
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
17 / 43
Instructions
All instructions are named in uppercase
All instructions are python objects
When an instruction is called (as Python object), it is generated and
added to the active PeachPy function (as assembly instruction)
PeachPy veries the correctness of instruction operands
Most computational x86-64 and many ARM instructions are supported
Traditional Assembly
PeachPy
.loop:
ADDPD xmm0, [rsi]
ADD rsi, 16
SUB rcx, 2
JAE .loop
LABEL( "loop" )
ADDPD( xmm0, [rsi] )
ADD( rsi, 16 )
SUB( rcx, 2 )
JAE( "loop" )
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
18 / 43
Registers
PeachPy maps dierent types of architectural registers on Python classes
x86 register classes:
I
I
I
I
I
I
I
I
GeneralPurposeRegister (base class)
GeneralPurposeRegister8
GeneralPurposeRegister16
GeneralPurposeRegister32
GeneralPurposeRegister64
MMXRegister
SSERegister
AVXRegister
ARM register classes:
I
I
I
I
GeneralPurposeRegister
SRegister
DRegister
QRegister
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
19 / 43
Registers
Architectural registers are represented in PeachPy as Python objects
All register names are in lowercase
Traditional x86-64 Assembly
PeachPy
MOVZX rax, al
PADD mm0, mm1
ADDPS xmm0, xmm1
VMULPD ymm0, ymm1, ymm2
MOVSX( rax, al )
PADD( mm0, mm1 )
ADDPS( xmm0, xmm1 )
VMULPD( ymm0, ymm1, ymm2 )
Traditional ARM Assembly
PeachPy
ADD r0, r0, r1
VLD1.32 {d0[]}, [r2]
VFMA.F32 q2, q1, q1
ADD( r0, r0, r1 )
VLD1.F32( (d0[:],), [r2] )
VFMA.F32( q2, q1, q1 )
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
20 / 43
Outline
1
Motivation
2
PeachPy Foundations
3
PeachPy Basics
4
Assembly Programming Automation
5
Metaprogramming
6
Instruction Streams
7
Experimental Validation
8
Conclusion
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
21 / 43
Register Allocation
Traditional x86-64 Assembly
VMOVAPD ymm0, [rsi]
VMOVAPD ymm1, ymm0
VFMADD132PD ymm1, ymm13, ymm12
VFMADD231PD ymm0, ymm1, ymm14
VFMADD231PD ymm0, ymm1, ymm15
PeachPy
ymm_x = AVXRegister()
VMOVAPD( ymm_x, [xPointer]
ymm_t = AVXRegister()
VMOVAPD( ymm_t, ymm_x )
VFMADD132PD( ymm_t, ymm_t,
VFMADD231PD( ymm_x, ymm_t,
VFMADD231PD( ymm_x, ymm_t,
Marat Dukhan (Georgia Tech)
)
ymm_log2e, ymm_magic_bias)
ymm_minus_ln2_hi, ymm_x )
ymm_minus_ln2_lo, ymm_x )
PeachPy
PyHPC 2013
22 / 43
In-place Memory Constant Declarations
Traditional x86-64 Assembly
Right here:
section .rdata rdata
c0 dq 3.141592, 3.141592
In a galaxy far far away:
section .text code
MULPD xmm0, [c0]
PeachPy
MULPD( xmm_x, Constant.float64x2(3.141592) )
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
23 / 43
Hexadecimal Floating-Point Constants
Hexadecimal oating-point constants provide an accurate and portable way
to specify oating-point constants and without rounding errors
Required in C99 standard
Supported by gcc, clang, icc, xlc, and NASM
But not supported by GNU Assembler
PeachPy lets programmers use hexadecimal oating-point constants on all
supported platforms
C99
const double ln2 = 0x1.71547652B82FEp+0;
ARM Assembly (GNU)
x86-64 Assembly (NASM)
ln2: .quad 0x3FF71547652B82FE
ln2 dq 0x1.71547652B82FEp+0
PeachPy (x86-64 and ARM)
ln2 = Constant.float64('0x1.71547652B82FEp+0')
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
24 / 43
Calling conventions
The Problem
Consider Assembly implementations for the C function
uint64_t add(uint64_t x, uint64_t y) {
return x + y;
}
Assembly for Microsoft x86-64 calling convention
add:
LEA rax, [rcx + rdx * 1]
RET
Assembly for System V x86-64 calling convention
add:
LEA rax, [rdi + rsi * 1]
RET
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
25 / 43
Calling conventions
PeachPy Approach
PeachPy code
from peachpy.x64 import *
asm = Assembler(peachpy.c.ABI("x64-ms")) # or "x64-sysv"
x_arg = peachpy.c.Parameter("x",
peachpy.c.Type("uint64_t"))
y_arg = peachpy.c.Parameter("y",
peachpy.c.Type("uint64_t"))
with Function(asm, "add", (x_arg, y_arg), "Bobcat"):
x = GeneralPurposeRegister64()
LOAD.PARAMETER( x, x_arg ) # Does the magic!
y = GeneralPurposeRegister64()
LOAD.PARAMETER( y, y_arg ) # Does the magic!
LEA( rax, [x + y * 1] )
RETURN()
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
26 / 43
ISA-based runtime dispatching
PeachPy known the instruction set of each instruction
PeachPy also collects ISA information about each function
This helps to do ne-grained runtime dispatching
I
More ecient vs recompiling the function for each ISA with high-level
compiler
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
27 / 43
Outline
1
Motivation
2
PeachPy Foundations
3
PeachPy Basics
4
Assembly Programming Automation
5
Metaprogramming
6
Instruction Streams
7
Experimental Validation
8
Conclusion
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
28 / 43
Parametrized Unroll
with Function(asm, "dot_product", args, "SandyBridge"):
xPointer, yPointer, zPointer, length = LOAD.PARAMETERS()
reg_size = 32
reg_elements = 8
unroll_regs = 8
acc = [AVXRegister() for _ in range(unroll_regs)]
temp = [AVXRegister() for _ in range(unroll_regs)]
...
LABEL( "process_batch" )
for i in range(unroll_regs):
VMOVAPS( temp[i], [xPointer + i * reg_size] )
VMULPS( temp[i], [yPointer + i * reg_size] )
VADDPS( acc[i], temp[i] )
ADD( xPointer, reg_size * unroll_regs )
ADD( yPointer, reg_size * unroll_regs )
SUB( length, reg_elements * unroll_regs )
JAE( "process_batch" )
...
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
29 / 43
Parametrization by Element Type
reg_size = 32
reg_elements = reg_size / element_size
unroll_regs = 8
SIMD_LOAD = {4: VMOVAPS, 8: VMOVAPD}[element_size]
SIMD_MUL = {4: VMULPS, 8: VMULPD}[element_size]
SIMD_ADD = {4: VADDPS, 8: VADDPD}[element_size]
...
LABEL( "process_batch" )
for i in range(unroll_regs):
SIMD_LOAD( temp[i], [xPointer + i * reg_size] )
SIMD_MUL( temp[i], [yPointer + i * reg_size] )
SIMD_ADD( acc[i], temp[i] )
ADD( xPointer, reg_size * unroll_regs )
ADD( yPointer, reg_size * unroll_regs )
SUB( length, reg_elements * unroll_regs )
JAE( "process_batch" )
...
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
30 / 43
Supporting Multiple Instruction Sets
if Target.has_fma():
VMLAPS = VFMADDPS if Target.has_fma4() else VFMADD231PS
else:
def VMLAPS(x, a, b, c):
t = AVXRegister()
VMULPS( t, a, b )
VADDPS( x, t, c )
...
LABEL( "processBatch" )
for i in range(unroll_regs):
VMOVAPS( temp[i], [xPointer + i * reg_size] )
VMLAPS( acc[i], temp[i], [yPointer + i * reg_size], acc[i] )
ADD( xPointer, reg_size * unroll_regs )
ADD( yPointer, reg_size * unroll_regs )
SUB( length, reg_elements * unroll_regs )
JAE( "processBatch" )
...
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
31 / 43
Outline
1
Motivation
2
PeachPy Foundations
3
PeachPy Basics
4
Assembly Programming Automation
5
Metaprogramming
6
Instruction Streams
7
Experimental Validation
8
Conclusion
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
32 / 43
Introduction
Usually we want assembly instructions to appear in the same order as we
write them.
To increase IPC it is useful to interleave two parts of code using
dierent types of instructions. However, it might be convenient to
write the code of those parts separately.
I
I
ARM Cortex-A9 can decode one SIMD instruction and one scalar
instruction per cycle. By interleaving SIMD and scalar processing we
can achieve higher performance.
On x86 we may use scalar instructions to detect special cases while
SIMD units are busy doing calculations.
For software pipelining we may want the skew the sequences of similar
instructions relative to each other.
I
But we don't want to skew our code
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
33 / 43
Instruction Stream Objects
The Python with statement can be used to redirect generated instructions
to an InstructionStream object.
scalar_stream = InstructionStream()
with scalar_stream:
x = GeneralPurposeRegister64()
MOV( x, [xPointer] )
CMP.JA( x, threshold, "above_threshold" )
vector_stream = InstructionStream()
with vector_stream:
...
Instructions from instruction stream can then be re-issued to current
instruction stream:
while scalar_stream or vector_stream:
scalar_stream.issue()
vector_stream.issue()
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
34 / 43
Software Pipelining
Instruction streams are useful for implementing software pipelining
instruction_columns = [InstructionStream(),
InstructionStream(), InstructionStream()]
for i in range(unroll_regs):
with instruction_columns[0]:
VMOVDQU( ymm_x[i], [xPointer + i * reg_size] )
with instruction_columns[1]:
VPADDD( ymm_x[i], ymm_y )
with instruction_columns[2]:
VMOVDQU( [zPointer + i * reg_size], ymm_x[i] )
with instruction_columns[0]:
ADD( xPointer, reg_size * unroll_regs )
with instruction_columns[2]:
ADD( zPointer, reg_size * unroll_regs )
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
35 / 43
Outline
1
Motivation
2
PeachPy Foundations
3
PeachPy Basics
4
Assembly Programming Automation
5
Metaprogramming
6
Instruction Streams
7
Experimental Validation
8
Conclusion
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
36 / 43
Is It Worth The Eort?
Albeit PeachPy simplies develiping assembly kernels, PeachPy is still
assembly, with all its drawbacks.
For many HPC scientists C code with compiler intrinsics is a viable
alternative to writing assembly:
C code with intrinsics is more portable with assembly
Many of the problems targeted by PeachPy become irrelevant (e.g.
calling convention)
Compiler could take into account more processors details than humans
We did a simple experiment to check if PeachPy (and assebly in general)
can deliver better performance than optimizing compilers.
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
37 / 43
Experimental Setup
For the experiment we used branchless versions of vector logarithm and
exponential functions from Yeppp! library. These are high-performance
implementation originally developed and tuned using Peach-Py.
We converted the assembly instructions one-to-one to C++ intrinsics and
compiled with modern C++ compilers. The C++ code is a nearly ideal
input for a compiler:
Code is already vectorized with intrinsics.
Each function processes 40 elements and has only one branch.
The only parts left to the compilers are register allocation and
instruction scheduling.
Initial instruction schedulling is close to optimal.
A lot of room for improving instruction scheduling: the original version
contains 581 instructions for log function and 400 instructions for exp.
The produced codes are benchmarked on Intel Core i7-4770K processor
with the recent Haswell microarchitecture.
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
38 / 43
Benchmarking Results
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
39 / 43
Outline
1
Motivation
2
PeachPy Foundations
3
PeachPy Basics
4
Assembly Programming Automation
5
Metaprogramming
6
Instruction Streams
7
Experimental Validation
8
Conclusion
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
40 / 43
Plans and Goals
The current priorities for PeachPy are
Support for PowerPC (including Blue Gene/Q) and Xeon Phi
architectures
Distribute PeachPy via Python Package Index
Enable generation of machine code directly from PeachPy
Provide additional features for x86-64 and ARM architectures (e.g.
table lookups)
ARM64 and x86-32 ports
In the long term we hope that PeachPy will replace conventional assembly
in HPC workow.
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
41 / 43
Public Availability
PeachPy repository is hosted on bitbucket.org/MDukhan/peachpy
The primary user of PeachPy is Yeppp! library (www.yeppp.info).
I
The codegen directory in Yeppp! source tree contains a large number
of Peach-Py codes.
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
42 / 43
Funding
This research was supported in part by
National Science Foundation
(NSF) under NSF CAREER
award number 0953100.
A grant from the Defense
Advanced Research Projects
Agency (DARPA) Computer
Science Study Group program.
Declaimer
Any opinions, conclusions or recommendations expressed in this presentation are
those of the authors and not necessarily reect those of NSF or DARPA.
Marat Dukhan (Georgia Tech)
PeachPy
PyHPC 2013
43 / 43