CS 332
Programming Language Concepts
Lecture 12 – Alternative Language
Examples
(API Concurrency vs. Built-in
Language Concurrency)
March 28, 2016
 Sam Siewert
Interesting Google Trends
https://www.google.com/trends/ - Search Trends
Normalized to all Searches, Auto-scaled between 0-100
 Sam Siewert
As of 4/6/2015
2
Major Concepts
Concurrent Processing
– Processes, Tasks, Threads
1. For Multi-programming to Overlap I/O Blocking or
Leverage Multi-core with Shared Memory – POSIX
Threads
2. For Multi-core and Multiple CPU Systems for Services
and Parallel Applications (Shared Memory) - OpenMP
3. For HPC Clusters to Scale Processing – MPI, OpenMPI
For Tasking with Shared Memory (Threads) #1 and #2,
Alternative is Built-in Language Concurrency
1. Ada95 Tasking, Using Linux GNAT, “sudo apt-get install gnat”
2. Java Threads, DrJava or Eclipse on Linux for Java
3. C++ 2011 – FAQ, New C++11 Design Goals, C++11 Features
 Sam Siewert
3
Java Example
public class simpleThreads {
private static int gsum=0;
private static class incrementThread implements Runnable {
public void run() {
for(int i=0; i < 100; i++) {
gsum=gsum+1;
System.out.format("incrementThread: gsum=%d\n", gsum); } }
}
private static class decrementThread implements Runnable {
public void run() {
for(int i=0; i < 100; i++) {
gsum=gsum-1;
System.out.format("decrementThread: gsum=%d\n", gsum); } } }
public static void main(String args[]) throws InterruptedException {
System.out.println("Starting increment and decrement threads");
Thread inct = new Thread(new incrementThread());
Thread dect = new Thread(new decrementThread());
inct.start();
dect.start();
System.out.println("Main will loop until it can join both threads");
// Option 1 - just wait indefinitely
inct.join();
dect.join();
System.out.println("TEST COMPLETE\n");}}
 Sam Siewert
4
Java Output
Fairly Deterministic for
10 iterations each
For 100, More Variation
(Chance for Context
Switching)
Are the Outputs Even in
the Right Order? (Check
with Timestamps)
Are the Outputs Valid?
 Sam Siewert
Welcome to DrJava.
> run simpleThreads
Starting increment and decrement threads
Main will loop incrementThread: gsum=1
incrementThread: gsum=1
incrementThread: gsum=2
incrementThread: gsum=3
incrementThread: gsum=4
incrementThread: gsum=5
incrementThread: gsum=6
incrementThread: gsum=7
incrementThread: gsum=8
incrementThread: gsum=9
decrementThread: gsum=0
decrementThread: gsum=8
decrementThread: gsum=7
decrementThread: gsum=6
decrementThread: gsum=5
decrementThread: gsum=4
decrementThread: gsum=3
decrementThread: gsum=2
decrementThread: gsum=1
decrementThread: gsum=0
TEST COMPLETE
>
5
C Pthreads Example
int gsum=0;
void *incThread(void *threadp){
int i; threadParams_t *threadParams = (threadParams_t *)threadp;
for(i=0; i<COUNT; i++) gsum++;}
void *decThread(void *threadp){
int i; threadParams_t *threadParams = (threadParams_t *)threadp;
for(i=0; i<COUNT; i++) gsum--;
}
int main (int argc, char *argv[])
{
int rc;
int i=0;
threadParams[i].threadIdx=i;
pthread_create(&threads[i],(void *)0,incThread, (void *)&(threadParams[i]) );
i++;
threadParams[i].threadIdx=i;
pthread_create(&threads[i], (void *)0,decThread, (void *)&(threadParams[i]));
for(i=0; i<2; i++)
pthread_join(threads[i], NULL);
printf("TEST COMPLETE\n");
}
 Sam Siewert
6
C Output
Which Goes First
Appears to Go to
Completion
Could we Count on a
Specific Sequence
We can Count on 0 at
End?
 Sam Siewert
$ ./pthread
Decrement thread
Decrement thread
Decrement thread
Decrement thread
Decrement thread
Decrement thread
Decrement thread
Decrement thread
Decrement thread
Decrement thread
Increment thread
Increment thread
Increment thread
Increment thread
Increment thread
Increment thread
Increment thread
Increment thread
Increment thread
Increment thread
TEST COMPLETE
idx=1,
idx=1,
idx=1,
idx=1,
idx=1,
idx=1,
idx=1,
idx=1,
idx=1,
idx=1,
idx=0,
idx=0,
idx=0,
idx=0,
idx=0,
idx=0,
idx=0,
idx=0,
idx=0,
idx=0,
gsum=-1
gsum=-2
gsum=-3
gsum=-4
gsum=-5
gsum=-6
gsum=-7
gsum=-8
gsum=-9
gsum=-10
gsum=-9
gsum=-8
gsum=-7
gsum=-6
gsum=-5
gsum=-4
gsum=-3
gsum=-2
gsum=-1
gsum=0
7
C Output on Multi-Core
Multi-Core has More
Switching
Less of a Sequence
Shows This Really is
Non-Deterministic
Result is Still 0
 Sam Siewert
[ssiewert@everest]$ ./pthread
Increment thread idx=0, gsum=1
Increment thread idx=0, gsum=1
Increment thread idx=0, gsum=2
Increment thread idx=0, gsum=3
Increment thread idx=0, gsum=4
Increment thread idx=0, gsum=5
Decrement thread idx=1, gsum=0
Decrement thread idx=1, gsum=5
Decrement thread idx=1, gsum=4
Decrement thread idx=1, gsum=3
Decrement thread idx=1, gsum=2
Decrement thread idx=1, gsum=1
Decrement thread idx=1, gsum=0
Increment thread idx=0, gsum=6
Increment thread idx=0, gsum=0
Increment thread idx=0, gsum=1
Increment thread idx=0, gsum=2
Decrement thread idx=1, gsum=-1
Decrement thread idx=1, gsum=1
Decrement thread idx=1, gsum=0
TEST COMPLETE
8
More on Concurrency
Covered in Operating Systems – Fair threading on one
Multi-core Platform
Real-Time Operating Systems
– AMP – Asymmetric Multi-core Processing (Unbalanced)
– SMP – Symmetric Multi-core Processing (Balanced)
Parallel and High Performance Computing
– NUMA vs. UMA (Uniform, Non-Uniform Memory Access)
– Message Passing vs Shared Memory
CUDA – Thread Grids for Data Transformation
MapReduce - Hadoop
 Sam Siewert
9
Computer Architecture Trends
Multi-Core, Many-Core
–
E.g. Intel Xeon Phi MICA, not MICA wearable
Vector Processing Instructions
–
–
–
ARM NEON
Intel SSE
POWER Architecture ALTIVEC
Vector Co-Processing
–
–
GP-GPU (E.g. NVIDIA Tegra X1, Tesla/Kepler/Fermi)
AMD ATI StreamProc
HP – “The Machine”, Memristor, Optical, Collapsed Memory Hierarchy
IBM – Power8 Architecture
ARM – SoCs [Altera Cyclone V SoC, Qualcomm, NVIDIA, Apple, Texas
Instruments, Marvel, …]
General Commodity Software-Defined Clusters [Google, Facebook, Hadoop,
Seamicro (Now Defunct)]
Clear Trend toward Concurrency, Threading, Message Passing, MapReduce
 Sam Siewert
10
Discussion
More Latest PL Revisions Include Concurrency as a
Built-in Language Feature – Good? Bad? Don’t-Care?
Should Concurrency Be an Operating System Feature?
Should it be a PL Run-time Feature?
Should it Be Provided by a Library?
Why is Concurrency a Contemporary Issue in Computer
Engineering, Software Engineering and PL Design?
 Sam Siewert
11