advertisement

A Multi-Level Parallel Implementation of a Program for Finding Frequent Patterns in a Large Sparse Graph Steve Reinhardt, Interactive Supercomputing [email protected] George Karypis, Dept. of Computer Science, University of Minnesota Outline Problem definition Prior work Problem and Approach Results Issues and Conclusions Graph Datasets Flexible and powerful representation Evidence extraction and link discovery (EELD) Social Networks/Web graphs Chemical compounds Protein structures Biological Pathways Object recognition and retrieval Multi-relational datasets Finding Patterns in Graphs M. Many Kuramochi and G. Karypis. Finding frequent patterns in a large sparse graph. Dimensions In SIAM International Conference on Data Mining (SDM-04), 2004. http://citeseer.ist.psu.edu/article/kuramochi04finding.html MIS calculation for frequency Structure of the graph dataset connected subgraphs induced subgraphs exact approximate upper bound Algorithm vertical (depth-first) horizontal (breadth-first) Nature of the algorithm Finds all patterns that satisfy the minimum support requirement Complete Finds some of the patterns single-graph setting Type of patterns graph transaction setting one large graph many small graphs Incomplete Nature of the pattern’s occurrence The pattern occurs exactly in the input graph Exact algorithms There is a sufficiently similar embedding of the pattern in the graph Inexact algorithms Single Graph Setting Find all frequent subgraphs from a single sparse graph. Choice of frequency definition Input Graph Size 6 Frequency = 1 Size 7 Frequency = 6 vSIGRAM: Vertical Solution Candidate generation by extension Add one more edge to a current embedding. Solve MIS on embeddings in the same equivalence class. No downward-closure-based pruning Two important components Frequency-based pruning of extensions Treefication based on canonical labeling vSIGRAM: Connection Table Frequency-based pruning. Trying every possible extension is expensive and inefficient. A particular extension might have been tested before. Categorize extensions into equivalent classes (in terms of isomorphism), and record if each class is frequent or not. If a class becomes infrequent, never try it in later exploration. Parallelization Two clear sources of parallelism in the algorithm The code is typical C code structs, pointers, frequent mallocs/frees of small areas, etc. nothing like the “Fortran”-like (dense linear algebra) examples shown for many parallel programming methods Parallel structures need to accommodate dynamic parallelism Amount of parallelism from each source not known in advance Dynamic specification of parallel work Dynamic allocation of processors to work Chose OpenMP taskq/task constructs Proposed extensions to OpenMP standard Support parallel work being defined in multiple places in a program, but be placed on a single conceptual queue and executed accordingly ~20 lines of code changes in ~15,000 line program Electric Fence was very useful in finding coding errors Algorithmic Parallelism vSiGraM (G, MIS_type, f) 1. F ← 2. F1 ← all frequent size-1 subgraphs in G 3. for each F1 in F1 do 4. M(F1) ← all embeddings of F1 5. for each F1 in F1 do 6. F ← F vSiGraM-Extend(F1, G, f) return F // high-level parallelism vSiGraM-Extend(Fk, G , f) 1. F ← 2. for each embedding m in M(Fk) do // low-level parallelism 3. Ck+1 ← C k+1 {all (k+1)-subgraphs of G containing m} 4. for each Ck+1 in Ck+1 do 5. if Fk is not the generating parent of Ck+1 then 6. continue 7. compute Ck+1.freq from M(Ck+1) 8. if Ck+1.freq < f then 9. continue 10. F ← F vSiGraM-Extend(Ck+1, G, f) 11.return F Simple Taskq/Task Example main() { int val; #pragma intel omp taskq val = fib(12345); } fib(int n) { int partret[2]; if (n>2) #pragma intel omp task for(i=n-2; i<n; i++) { partret[n-2-i] = fib(i); } return (partret[0] + partret[1]); } else { return 1; } } High-Level Parallelism with taskq/task // At the bottom of expand_subgraph, after all child // subgraphs have been identified, start them all. #pragma intel omp taskq for (ii=0; ii<sg_set_size(child); ii++) { #pragma intel omp task captureprivate(ii) { SubGraph *csg = sg_set_at(child,ii); expand_subgraph(csg, csg->ct, lg, ls, o); } // end-task } Low-Level Parallelism with taskq/task #pragma omp parallel shared(nt, priv_es) { #pragma omp master { nt = omp_get_num_threads(); //#threads in par priv_es = (ExtensionSet **)kmp_calloc(nt, sizeof(ExtensionSet *)); } #pragma omp barrier #pragma intel omp taskq { for (i = 0; i < sg_vmap_size(sg); i++) { #pragma intel omp task captureprivate(i) { int th = omp_get_thread_num(); if (priv_es[th] == NULL) { priv_es[th] = exset_init(128); } expand_map(sg, ct, ams, i, priv_es[th], lg); } } } } // end parallel section; next loop is serial reduction for (i=0; i < nt; i++) { if (priv_es[i] != NULL) { exset_merge(priv_es[i],es); } } kmp_free(priv_es); } Implementation due to Grant Haab and colleagues from Intel OpenMP library group Experimental Results SGI Altix™ 32 Itanium2™ sockets (64 cores), 1.6GHz 64 GBytes (though not memory limited) Linux No special dplace/cpuset configuration Minimum frequencies chosen to illuminate scaling behavior, not provide maximum performance Dataset 1 - Chemical Graph Frequency Type of Parallelism Number of processors 1 2 4 8 16 30 60 Time in seconds (speed-up) 17.01 (2.03) 14.76 (2.40) 13.89 (2.58) 14.00 (2.56) 13.97 (2.57) Low 32.51 (0.98) 31.52 (1.01) 37.95 (0.83) 42.18 (0.74) 49.56 (0.63) Both 17.52 (1.96) 14.88 (2.37) 15.80 (2.21) 29.85 (1.08) 44.37 (0.70) 48.86 (1.97) 27.12 (3.71) 16.82 (6.39) 15.05 (7.29) 14.52 (7.61) Low 94.36 (1.00) 92.18 (1.02) 112.17 (0.83) 133.40 (0.70) 116.31 (0.80) Both 48.38 (1.99) 27.27 (3.69) 61.52 (1.55) 315.94 (0.29) 281.83 (0.33) 142.02 (2.00) 62.73 (4.64) 34.44 (8.76) 19.40 (16.56) 15.06 (22.27) Low 283.19 (1.00) 293.6 (0.96) 400.55 (0.70) 262.82 (1.07) 197.27 (1.44) Both 140.47 (2.03) 81.18 (3.55) 242.09 (1.17) 513.39 (0.55) 581.04 (0.48) High 500 High dtp 100 High 50 31.94 93.96 282.15 15.80 (21.03) Dataset 2 – aviation Graph Frequency Type of Parallelism Number of processors 1 2 4 8 16 30 60 21.74 (22.30) 18.85 (27.29) Time in seconds (speed-up) High 358.27 54.92 (7.19) 1750 171.04 (2.13) Low air1 1500 High 1250 High 1000 High 771.82 112.30 (7.20) 39.40 (22.89) 33.99 (27.30) 1503.49 209.08 (7.37) 67.54 (24.31) 56.56 (29.58) 3909.95 490.38 (8.06) 155.33 (26.13) 158.14 (25.65) Performance of High-level Parallelism When sufficient quantity of work (i.e., frequency threshold is low enough) Good speed-ups to 16P Reasonable speed-ups to 30P Little or no benefit above 30P No insight into performance plateau Poor Performance of Low-level Parallelism Several possible effects ruled out Granularity of data allocation Barrier before master-only reduction Source: highly variable times for register_extension ~100X slower in parallel than serial, … but different instances from execution to execution Apparently due to highly variable run-times for malloc Not understood Issues and Conclusions OpenMP taskq/task were straightforward to use in this program and implemented the desired model Performance was good to a medium range of processor counts (best 26X on 30P) Difficult to gain insight into lack of performance High-level parallelism 30P and above Low-level parallelism Backup Datasets Dataset Connected Components Vertices Edges Vertex Labels Edge Labels Aviation 2,703 101,185 98,482 6,173 51 Citation 16,999 29,014 42,064 50 12 2,633 12,752 11,542 23 1 VLSI Aviation Dataset Generally, vSIGRAM is 2-5 times faster than hSIGRAM (with exact and upper bound MIS) Largest pattern contained 13 edges. Citation Dataset But, hSIGRAM can be more efficient especially with upper bound MIS (ub). Largest pattern contained 16 edges. VLSI Dataset Exact MIS never finished. Longest pattern contained 5 edges (constraint). Comparison with SUBDUE SUBDUE Dataset Freq. Credit DTP VLSI Size 341 395 387 6 5 5 4,957 4,807 1,950 2 2 6 773 773 244 1 1 1 vSIGRAM (approximate MIS) Runtime [sec] Freq. Largest Size Patterns Runtime [sec] 200 9 11,696 4 20 13 613,884 461 500 7 190 20 10 21 112,535 311 200 5 137 3 25 5 1,452 18 517 1,525 16 Similar results with SEuS Summary With approximate and exact MIS, vSIGRAM is 2-5 times faster than hSIGRAM. With upper bound MIS, however, hSIGRAM can prune a larger number of infrequent patterns. The downward closure property plays the role. For some datasets, using exact MIS for frequency counting is just intractable. Compared to SUBDUE, SIGRAM finds more and longer patterns in shorter amount of runtime. Thank You! Slightly longer version of this paper is also available as a technical report. SIGRAM executables will be available for download soon from http://www.cs.umn.edu/~karypis/pafi/ Complete Frequent Subgraph Mining— Existing Work So Far Input: A set of graphs (transactions) + support threshold Goal: Find all frequently occurring subgraphs in the input dataset. AGM (Inokuchi et al., 2000), vertex-based, may not be connected. FSG (Kuramochi et. al., 2001), edge-based, only connected subgraphs AcGM (Inokuchi et al., 2002), gSpan (Yan & Han, 2002), FFSM (Huan et al., 2003), etc. follow FSG’s problem definition. Frequency of each subgraph The number of supporting transactions. Does not matter how many embeddings are in each transaction. What is the reasonable frequency definition? Two reasonable choices: The frequency is determined by the total number of embeddings. Not downward closed. Too many patterns. Artificially high frequency of certain patterns. The frequency is determined by the number of edge-disjoint embeddings (Vanetik et al, ICDM 2002). Downward closed. Since each occurrence utilizes different sets of edges, occurrence frequencies are bounded. Solved by finding the maximum independent set (MIS) of the embedding overlap graph. Embedding Overlap and MIS E2 E1 Edge-disjoint embeddings { E1, E2, E3 } { E1, E2, E4 } Create an overlap graph and solve MIS E3 E4 Vertex Edge Embedding Overlap OK. Definition is Fine, but … MIS-based frequency seems reasonable. Next question: How to develop mining algorithms for the single graph setting. How to Handle Single Graph Setting? Issue 1: Frequency counting Exact MIS is often intractable. Issue 2: Choice of search scheme Horizontal (breadth-first) Vertical (depth-first) Issue 1: MIS-Based Frequency We considered approximate (greedy) and upper bound MIS too. Approximate MIS may underestimate the frequency. Upper bound MIS may overestimate the frequency. MIS is NP-complete and not be approximated. Practically simple greedy scheme works pretty well. Halldórsson and Radhakrishnan. Greed is good, 1997. Issue 2: Search Scheme Frequent subgraph mining Exploration in the lattice of subgraphs Horizontal Level-wise Candidate generation and pruning Joining Downward closure property Frequency counting Vertical Traverse the lattice as if it were a tree. hSIGRAM: Horizontal Method Natural extension of FSG to the single graph setting. Candidate generation and pruning. Downward closure property Tighter pruning than vertical method Two-phase frequency counting All embeddings by subgraph isomorphism Anchor edge list intersection, instead of TID list intersection. Localize subgraph isomorphism MIS for the embeddings Approximate and upper bound MIS give subset and superset respectively. TID List Recap Lattice of Subgraphs T1 size k + 1 size k T2 TID( TID( TID( ) = { T1, T2, T3 } ) = { T1, T2, T3 } ) = { T1, T3 } TID( ) TID( ) ∩ TID( = { T1, T3 } T3 ) ∩ TID( ) Anchor Edges Lattice of Subgraphs size k + 1 size k Each subgraph must appear close enough together. Keep one edge for each. Complete embeddings require too much memory. Localize subgraph isomorphism. Treefication Lattice of Subgraphs Treefied Lattice size k + 1 size k size k - 1 : a node in the search space (i.e., a subgraph) Based on subgraph/supergraph relation Avoid visiting the same node in the lattice more than once.