ES-MPICH2: A Message Passing Interface with Enhanced Security
advertisement
ES-MPICH2: A Message Passing Interface with Enhanced Security Authors: Xiaojun Ruan, Member, Qing Yang, Mohammed I. Alghamdi,Shu Yin, Student Member, and Xiao Qin This is a postprint of an article that originally appeared in Performance Computing and Communications Conference (IPCCC), 2010 IEEE 29th International in December 2010. http://dx.doi.org/10.1109/PCCC.2010.5682312 Xiaojun Ruan; Qing Yang; Alghamdi, M.I.; Shu Yin; Xiao Qin, "ES-MPICH2: A Message Passing Interface with Enhanced Security," Performance Computing and Communications Conference (IPCCC), 2010 IEEE 29th International , vol.9, no.3, pp.161-168 2010 http://dx.doi.org/10.1109/PCCC.2010.5682312 Made available through Montana State University’s ScholarWorks scholarworks.montana.edu 1 ES -MP ICH2 :A Me ssagePass ingIn te r facew i th EnhancedSecu r i ty X iao junRuan ,Membe r ,IEEE ,Q ingY ang ,Membe r ,IEEE , MohammedI .A lghamd i , Membe r ,IEEE , ShuY in ,S tuden t Membe r ,IEEE ,andX iao Q in ,Sen io r Membe r ,IEEE ✦ Abs t rac t—Aninc reas ingnumbe ro fcommod i tyc lus te rsa reconne c ted toeacho the rb ypub l icne two rks , wh ichha vebecomeapo ten t ia lth rea t tosecu r i tysens i t ive pa ra l le lapp l ica t ionsrunn ingonthec lu s te r s .T o add ressth issecu r i tyissue , wede ve lopedan MP I(Message Pa s s ing In te r face )imp lemen ta t iontop rese rvecon f iden t ia l i to f me s sage scom mun ica tedamongnodeso fc lus te rs inanunsecu redne two rk . Wefo cu s on MP Ira the rthano the rp ro toco ls ,because MP Iisoneo fthe mo s t popu la rcommun ica t ion p ro toco lsfo r pa ra l le lcompu t ing onc lu s te r s . Ou r MP Iimp lemen ta t ion-ca l led ES -MP ICH2- was bu i l t ba sed on MP ICH2de ve lopedb ytheA rgonneNa t iona lLabo ra to ry .L ike MP ICH2 , ES -MP ICH2a imsa tsuppo r t inga la rgeva r ie tyo fcompu ta t ionandcom mun ica t ionp la t fo rmsl ikecommod i tyc lus te rsandh igh -speedne two rk s . Wein teg ra tedenc ryp t ionanddec ryp t iona lgo r i thmsin tothe MP ICH2 l ib ra ry w i ththes tanda rd MP Iin te r faceand ;thus ,da tacon f iden t ia l i to f MP Iapp l ica t ionscanberead i lyp rese rved w i thou taneedtochange thesou rcecodeso fthe MP Iapp l ica t ions . MP I -app l ica t ionp rog ramme r s canfu l lycon f igu r an ycon f iden t ia l i t se rv icesin MP ICH I2 , be cau se a secu red con f igua t ionf i l in ES -MP ICH2 o f fe rsthe p rog ramme r s f lx ib i l i tyinchoos ingan yc ryp tog raph icschemesandke ysseam le s s l y inco rpo ra ted inES -MP ICH2 . WeusedtheSand ia M ic roBen chma rkand In te l MP IBenchma rksu i testoe va lua teandcompa rethepe r fo rman ce o f ES -MP ICH2 w i ththe o r ig ina l MP ICH2 ve rs ion . Ou re xpe r imen t s showtha to ve rheadincu r red b ythe con f iden t ia l i t se rv i ce sin ES MP ICH2is ma rg ina lfo rsma l l messages .Thesecu r i tyo ve rheadinES MP ICH2becomes mo rep ronouncedw i thla rge r messages . Ou rre su l t s a lsoshowtha tsecu r i tyo ve rheadcanbes ign i f ican t l redu cedin ES MP ICH2 b yh igh -pe r fo rmancec lus te rs . The e xecu tab le b ina r ie s and sou rcecodeo fthe ES -MP ICH2imp lemen ta t iona ref ree lya va i lab lea t h t tp : / /www .eng .aubu rn .edu /∼xq in /so f twa re /es -mp ich2 / . 1 INTRODUCT ION 1 .1 Mo t iva t ion Du etoth ef a s td ev e lopm en to fth eIn t e rn e t ,anin c r ea s ingnumb e ro fun iv e r s i t i e sandcompan i e sa r econn e c t ing th e i rc lu s t e rcompu t ingsy s t em sto pub l i cn e two rk sto X . J . Ru ani sw i thth eD ep a r tm en to fC ompu t e rS c i en c e , W e s tC h e s t e r Un iv e r s i tyo fP enn sy lv an i a ,PA19383 , USA .E -m a i l :x ru an@w cu p a . e du . Q .Y angi sw i thth eD ep a r tm en to fC ompu t e rS c i en c e , M on t an aS t a t e Un iv e r s i ty , MT59717 , USA .E -m a i l :q ing .y ang@ c s .m on t an a . edu . M . I .A lgh amd ii sw i thth e D ep a r tm en to fC ompu t e rS c i en c e ,A l -B a h a Un iv e r s i ty ,A l -B ah aC i ty ,K ingd omo fS aud iA r a b i a .E -m a i l :m i a lmu s h i l ah@ bu . edu . s a . S .Y ini sw i thth eD ep a r tm en to fC ompu t e rS c i en c eand S o f tw a r e En g in e e r ing , Au bu rn Un iv e r s i ty , Au bu rn , AL 36849 -5347 , USA .E -m a i l : s zy0004@ au bu rn . edu . X .Q ini sw i thth eD ep a r tm en to fC ompu t e rS c i en c eand S o f tw a r e En g in e e r ing , Au bu rn Un iv e r s i ty , Au bu rn , AL 36849 -5347 , USA .E -m a i l : x q in@ au bu rn . edu . URL :h t tp : / /www . eng . au bu rn . edu /∼x q in . M anu s c r ip tr e c e iv edJun e XX ,201X ;r ev i s edJ anu a ry XX ,201X . p rov id eh igha c c e s s ib i l i ty . Tho s ec lu s t e r sconn e c t ingto th eIn t e rn e tcanb ea c c e s s edbyanyon ef romanywh e r e . Fo rexamp l e ,compu t ingnod e sina d i s t r ibu t edc lu s t e r sy s t em p ropo s edbySun M i c ro sy s t em sa r eg eog raph i ca l lyd ep loy edinva r iou scompu t ings i t e s .In fo rma t ion p ro c e s s edin a d i s t r ibu t edc lu s t e ri ssha r ed among a g roupo fd i s t r ibu t edta sk so ru s e r sbyth ev i r tu eo fm e s sag e pa s s ing p ro to co l s( e .g .m e s sag e pa s s ingin t e r fa c e - MP I )runn ingona pub l i cn e two rkl ik eth eIn t e rn e t . Th i sop en sapo s s ib i l i tyfo rs e cu r i tyl eakag e si fth e r ei s noin fo rma t iona s su ran c ep ro t e c t iononc la s s ifi edo rcon fid en t ia lda tat ran sm i t t edtoandf romc lu s t e rcompu t ing nod e s . P r e s e rv ing da taconfid en t ia l i tyina m e s sag epa s s ing env i ronm en t ov e ran un t ru s t ed n e two rki sc r i t i ca lfo r aw id esp e c t rum o fs e cu r i ty -awa r e MP I app l i ca t ion s , b e cau s e unau tho r iz ed a c c e s stoth es e cu r i ty s en s i t iv e m e s sag e s by un t ru s t ed p ro c e s s e scanl eadtos e r iou s s e cu r i ty b r ea ch e s . H en c e ,i ti simp e ra t iv eto p ro t e c t confid en t ia l i tyo fm e s sag e sex chang edamongag roup o ft ru s t edp ro c e s s e s . I ti sa non t r iv ia landcha l l eng ing p rob l emto o f f e r confid en t ia l i tys e rv i c e sfo rla rg e s ca l ed i s t r ibu t edc lu s t e r sb e com e s ,b e cau s eth e r ei sanop ena c c e s s ib l ena tu r e o fth eop enn e two rk s .T oadd r e s sth i si s su e ,w eenhan c ed th es e cu r i ty o fth e MP I(M e s sag e Pa s s ingIn t e r fa c e ) p ro to co l byen c ryp t ingand d e c ryp t ing m e s sag e ss en t andr e c e iv edamongcompu t ingnod e s . Inth i ss tudy w efo cu son MP Ira th e rthano th e rp ro to co l s ,b e cau s e MP Ii son eo fth e mo s tpopu la rcommu n i ca t ionp ro to co l sfo rc lu s t e rcompu t ingenv i ronm en t s . Num e rou ss c i en t ifi candcomm e r c ia lapp l i ca t ion srun n ingonc lu s t e r sw e r ed ev e lop edu s ingth e MP Ip ro to co l . Amonga va r i e tyo f MP Iimp l em en ta t ion s ,w ep i ck ed MP ICH2 d ev e lop edbyth eA rgonn e Na t iona l Labo ra to ry .Th ed e s igngoa lo f MP ICH2-a w id e lyu s ed MP I imp l em en ta t ion-i stocomb in e po r tab i l i ty w i th h igh p e r fo rman c e[15 ] . W ein t eg ra t eden c ryp t iona lgo r i thm s in toth e MP ICH2l ib ra ry Thu s , da taconfid en t ia l i ty o f MP I app l i ca t ion scan b er ead i ly p r e s e rv ed w i thou ta n e edtochang eth esou r c ecod e so fth e MP Iapp l i ca t ion s . Da tacommun i ca t ion so faconv en t iona l MP Ip rog ram can b es e cu r ed w i thou tconv e r t ingth ep rog ramin to th eco r r e spond ings e cu r ev e r s ion ,s in c ew ep rov id ea 2 •W ehav eu s edES -MP ICH2top e r fo rmad e ta i l edca s e s e cu r i tyenh an c ed MP I l ib ra ry w i thth es tanda rd MP I s tudyu s ingth eSand ia M i c roB en chma rk sandth eIn t e l in t e r f a c e . MP Ib en chma rk s . W efo cu s onrun t im ep e r fo rman c e ov e rh eadin t rodu c edbyth ec ryp tog raph i ca lgo r i thm s . 1 .2 Poss ib leApp roaches Th e r ea r eth r e epo s s ib l eapp roa ch e stoimp rov ings e cu r i tyo f MP Iapp l i ca t ion s .In fi r s tapp roa ch ,app l i ca t ion p rog r amm e r sc anaddsou r c ecod etoadd r e s sth ei s su e o fm e s s ag econfid en t ia l i ty . Fo rexamp l e ,th ep rog ram m e r s m ayr e ly on ex t e rna ll ib ra r i e s( e .g . , SEAL[27 ] and N exu s[ 1 1 ] )toimp l em en ts e cu r eapp l i ca t ion s .Su ch anapp l i c a t ion l ev e ls e cu r i tyapp roa chno ton ly mak e s th e MP Iapp l i c a t ion se r ro r -p ron e ,bu ta l sor edu c e sth e po r t ab i l i ty and fl ex ib i l i ty o fth e MP I app l i ca t ion s .In th es e condapp ro a ch ,th e MP Iin t e r fa c ecanb eex t end ed inth ew ayth a tn ews e cu r i ty -awa r e AP I sa r ed e s ign ed andimp l em en t ed( s e e ,fo rexamp l e , MP IS e cI/O[23 ] ) . Th i s MP I in t e r f a c e l ev e lso lu t ionenab l e sp rog ramm e r s to w r i t es e cu r e MP Iapp l i ca t ion sw i th m in ima lchang e s toth ein t e r f a c e .A l thoughth es e condapp roa chi sb e t t e rth anth efi r s ton e ,th i s MP I in t e r fa c e l ev e lso lu t ion typ i c a l lyr equ i r e sex t racod etod ea lw i thda taconfid en t i a l i ty .Th eth i rdapp roa ch-achann e l l ev e lso lu t ion-i s p ropo s edinth i ss tudytoadd r e s sth ed rawba ck so fth e abov etwoapp ro a ch e s . Ou rchann e l l ev e lso lu t iona im s a tp rov id ing m e s sag econfid en t ia l i tyinacommun i ca t ion ch ann e lth a timp l em en t sth e Chann e lIn t e r fa c e 3in MP ICH 2( s e eF ig .1 ) . 1 .3 Con t r ibu t ions In wh a tfo l low s ,w esumma r iz eth efou r ma jo rcon t r ibu t ion so fth i ss tudy . •W eimp l em en t edas tanda rd MP Im e chan i smca l l ed ES -MP ICH 2too f f e rda taconfid en t ia l i tyfo rs e cu r en e t wo rkcommun i c a t ion sin m e s sag epa s s ingenv i ronm en t s . Ou rp ropo s eds e cu r i tyt e chn iqu ein co rpo ra t edinth e MP ICH 2l ib r a ryc anb ev e ryu s e fu lfo rp ro t e c t ingda ta t ran sm i t t edinop enn e two rk sl ik eth eIn t e rn e t . •Th eES -MP ICH2 m e chan i sma l low sapp l i ca t ionp ro g ramm e r stoe a s i ly w r i t es e cu r e MP Iapp l i ca t ion sw i th ou tadd i t ion a lcod efo r da ta confid en t ia l i ty p ro t e c t ion . W es e ekach ann e l l ev e lso lu t ionin wh i chen c ryp t ion andd e c ryp t ionfun c t ion sa r ein c lud edin toth e MP ICH2 l ib r a ry . Ou rES -MP ICH2 ma in ta in sas tanda rd MP Iin t e r f a c etoex ch ang em e s sag e s wh i l ep r e s e rv ingda taconfi d en t i a l i ty . •Th eimp l em en t edES -MP ICH2f ram ewo rkp rov id e s as e cu r ed configu ra t ion fi l etha t enab l e s app l i ca t ion p rog r amm e r stos e l e c t iv e lychoo s e anyc ryp tog raph i c a lgo r i thmandsymm e t r i c -k eyinES -MP ICH2 .Th i sf ea tu r em ak e si t po s s ib l efo rp rog ramm e r stoea s i lyand fu l lycon t ro lth es e cu r i tys e rv i c e sin co rpo ra t edinth e MP ICH I 2l ib r a ry .T od emon s t ra t eth i sf ea tu r e ,w eim p l em en t edth e AESand T r ip l e DESa lgo r i thm sin ES MP ICH 2 .W ea l soshowinth i spap e rhowtoaddo th e r c ryp tog r aph i ca lgo r i thm sintoth e ES -MP ICH2f ram e wo rk . 1 .4 Roadmap Th e pap e ri so rgan iz eda sfo l low s :S e c t ion2 d emon s t ra t e sth evu ln e rab i l i t i e so fex i s t ing MP Iimp l em en ta t ion sbyd e s c r ib ingas e cu r i tyth r ea t mod e lfo rc lu s t e r s conn e c t edbypub l i cn e two rk s .S e c t ion3no ton ly p ro v id e sar ea sonfo rfo cu s ingonth econfid en t ia l i tyi s su e o f MP ICH2ra th e rthano th e r MP Iimp l em en ta t ion s ,bu t a l so g iv e s an ov e rv i ew o fth e MP ICH2imp l em en ta t ion .S e c t ion4p r e s en t sth e mo t iva t iono fth i s wo rkby show ing whys e cu r ed MP Ii sanimpo r tan ti s su eand a l soou t l in e sth ed e s igno f ES -MP ICH2-th em e s sag e pa s s ingin t e r fa c ew i thenhan c eds e cu r i ty .S e c t ion5d e s c r ib e sth eco r r e spond ingimp l em en ta t iond e ta i l so fES MP ICH2 .S e c t ion6d i s cu s s e ssom eexp e r im en ta lr e su l t s andcompa r e sth ep e r fo rman c eo fES -MP ICH2 w i ththa t o f MP ICH2 .S e c t ion7p r e s en t sp r ev iou sr e s ea r chr e la t ed toou rp ro j e c t .F ina l ly ,S e c t ion8s ta t e sth econ c lu s ion s andfu tu r e wo rko fth i ss tudy . 2 THREAT MODEL Ag eog raph i ca l ly d i s t r ibu t edc lu s t e rsy s t emi s on ein wh i chcompu t ingcompon en t sa tlo ca lc lu s t e rcompu t ingp la t fo rm scommun i ca t eandcoo rd ina t eth e i ra c t ion s by pa s s ing m e s sag e sth rough pub l i cn e two rk sl ik eth e In t e rn e t .T oimp rov eth es e cu r i tyo fc lu s t e r sconn e c t ed toth epub l i cn e two rk s ,on e maybu i ldap r iva t en e two rk toconn e c t an a r ray o flo ca lc lu s t e r stofo rm ala rg e s ca l ec lu s t e r . Bu i ld ing a p r iva t en e two rk , how ev e r ,i s no taco s t e f f e c t iv e waytos e cu r ed i s t r ibu t edc lu s t e r s . Th eIn t e rn e t-av e ryla rg ed i s t r ibu t edsy s t em-canb e u s edtosuppo r tla rg e s ca l ec lu s t e rcompu t ing .B e inga pub l i cn e two rk ,th eIn t e rn e tb e com e sapo t en t ia lth r ea t tod i s t r ibu t edc lu s t e rcompu t ingenv i ronm en t s . W efi r s td e s c r ib eth econfid en t ia l i tya sp e c to fs e cu r i ty inc lu s t e r sfo l low edbyth r e esp e c ifi ca t ta ckin s tan c e s .W e b e l i ev en ewa t ta ck sa r el ik e lytoem e rg e ,bu tth econfi d en t ia l i tya sp e c tw i l lr ema inun chang ed .Confid en t ia l i ty a t ta ck sa t t emp t stoexpo s em e s sag e sb e ingt ran sm i t t ed among a s e to f co l labo ra t ing p ro c e s s e sin a c lu s t e r . Fo r examp l e ,i fa t ta ck e r s ga in n e two rk adm in i s t ra to r p r iv i l eg e ,th eycanin t e r c ep tm e s sag e sandexpo r tth e m e s sag e stoa da taba s efi l efo rfu r th e rana ly s i s . Ev en w i thou tl eg i t ima t ep r iv i l eg e ,ana t ta ck e rs t i l lcansn i f f andin t e r c ep ta l l m e s sag e sinac lu s t e r onth e pub l i c n e two rk . Su ch a t ta ck sr e su l tinth ein fo rma t ionl eak ag eo f m e s sag e s pa s s ed among compu t ing nod e sin g eog raph i ca l ly d i s t r ibu t edc lu s t e r s .C ryp tog raphyand a c c e s scon t ro la r ew id e lyapp l i edtocompu t e rsy s t em s tosa f egua rdaga in s tconfid en t ia l i tya t ta ck s . W eid en t i fyth efo l low ingth r e econfid en t ia l i tya t ta ck s on MP Ip rog ram srunn ingond i s t r ibu t edc lu s t e r s : 3 •Sn i ffin g M e s s ag eT r a ffi c: M e s sag et ra ffi co fan MP I p rog r amc anb esn i f f ed .Fo rexamp l e , wh en MPCH2i s d ep loy edinac lu s t e rconn e c t edbya G igab i tE th e rn e t n e two rk ,a t t a ck e r scansn i f fp la in t ex tm e s sag e st ran sm i t t edth roughth eTCPso ck e t .M e s sag esn i ffingcanr ev ea l s e cu r i ty s en s i t iv eda ta ,m e tada ta ,andin fo rma t ion . •Sn o o p in gon M e s s ag e Bu f f e r:In an MP Ip rog ram , bu f f e r sa r e emp loy edtos end andr e c e iv e m e s sag e s . R eg a rd l e s so fsp e c ifi c MP Iimp l em en ta t ion s , m e s sag e bu f f e r sa r ec r e a t edb e fo r eth es endandr e c e iv ep r im i t iv e s a r einvok ed .A t t a ck e r s who snoopin toth e m e s sag e bu f f e r sin m emo ry can a c c e s s da ta andin fo rma t ion w i thou tb e ingg iv ensp e c ifi ca c c e s sp r iv i l eg e s . •M e s s a g eT r a ffi cP r ofi l ing: M e s sag et ra ffi cp rofi l ing a t t a ck ss e ektou s em e s sag etyp e ,t im e s tamp s ,m e s sag e s i z e ,ando th e rm e tada tatoana lyz em e s sag eex chang e p a t t e rn sandtyp e so fp ro to co l sb e ingu s edin m e s sag e t ran sm i s s ion s . Fo r examp l e , an a t ta ck e r can mon i to r th en e two rkconn e c t ion o f ac lu s t e rrunn ing an MP I p rog r am .I fa m e s sag eha sb e enr egu la r lyt ran sm i t t ed , th ea t t a ck e rc ansp e cu la t eth eimpo r tan c eo fth em e s sag e andin t e r c ep tth econ t en to fth em e s sag e . Confid en t i a l i ty s e rv i c e s can e f f e c t iv e ly coun t e rth e a fo r em en t ion edth r ea t sin MP Iapp l i ca t ion srunn ingon c lu s t e r sconn e c t edbyapub l i cn e two rk .Inth i sr e s ea r ch , w een cod e m e s s ag e su s ingth e Advan c ed En c ryp t ion S tand a rd(AES )andth eT r ip l eDa taEn c ryp t ionS tanda rd (T r ip l e -DESo r3DES ) .I ti s wo r thno t ingtha t3DESi sno t con s id e r edfo rany mod e rnapp l i ca t ion s ,w einv e s t iga t e 3DESinth i ss tudyb e cau s eo ftwor ea son s .F i r s t ,3DES i ss t i l lu s edin manyl ega cyapp l i ca t ionsy s t em sinth e indu s t ry .S e cond ,3DESi sgoodba s e l in eso lu t ion u s ed fo rth epu rpo s eo fcompa r i son .Inth eca s etha ta t ta ck e r s in t e r c ep tm e s s ag e sinan MP Ip rog ram ,th eya r eunab l e tot r an s fo rmth ec iph e r t ex tin toth eo r ig ina lp la in t ex tdu e toth el a cko fd a taen c iph e rm en tk ey s . 3 MP ICH2 OVERV IEW MP ICH- on eo fth e mo s t popu la r MP Iimp l em en ta t ion s- w e r ed ev e lop eda tth eA rgonn e Na t iona l Lab o ra to ry[ 1 5 ] . Th e ea r ly MP ICH v e r s ionsuppo r t sth e MP I 1s t and a rd . MP ICH2- asu c c e s so ro f MP ICHno ton ly p rov id e ssuppo r tfo rth e MP I -1s tanda rd ,bu t a l sof a c i l i t a t e sth en ew MP I -2s tanda rd , wh i chsp e c ifi e s fun c t ion a l i t i e sl ik eon e s id edcommun i ca t ion , dynam i c p ro c e s sm an ag em en t ,and MP II/O[13 ] .Compa r ed w i th th eimp l em en t a t iono f MP ICH , MP ICH2 wa scomp l e t e ly r ed e s ign edandd ev e lop edtoa ch i ev eh ighp e r fo rman c e , m ax imumfl ex ib i l i ty ,andgoodpo r tab i l i ty . F ig .1show sth eh i e ra r ch i ca ls t ru c tu r eo fth e MP ICH2 imp l em en t a t ion , wh e r eth e r ea r efou rd i s t in c tlay e r so f in t e r f a c e sto m ak eth e MP ICH2 d e s ign po r tab l e and fl ex ib l e . Th efou rlay e r s ,f romtopto bo t tom ,a r eth e m e s s ag ep a s s ingin t e r fa c e2(MP I -2 ) ,th eab s t ra c td ev i c e in t e r f a c e(AD I 3 ) ,th echann e lin t e r fa c e(CH3 ) ,andth e low l ev e lin t e r f a c e . AD I3-th eth i rdg en e ra t iono fth e ab s t r a c td ev i c ein t e r fa c e-inth eh i e ra r ch i ca ls t ru c tu r e U s e rA p p l i c a t i o n MPID−Layer C h a n n e lIm p l em e n t a t i o no fAD I− 3 [ . . .] Device−Layer M e s s ag eP a s s ing In t e r f a c e2 MP IIm p l em e n t a t i o n MPI−Layer t h e r n e t s I n f i n i− E h a r e d / IP )m B a n d (TCP em o r y C omm u n i c a t i o nS y s t em Ab s t r a c tD ev i c e In t e r f a c e3 Ch ann e l In t e r f a c e3 [ . . .] Low−L ev e l In t e r f a c e s F ig .1 :H i e ra r ch i ca lS t ru c tu r eo f MP ICH2[14 ] ( s e eF ig .1 )a l low s MP ICH2tob eea s i lypo r t edf romon e p la t fo rmtoano th e r .S in c ei ti snon t r iv ia ltoimp l em en t AD I3a safu l l f ea tu r edab s t ra c td ev i c ein t e r fa c ew i th manyfun c t ion s ,th e CH3lay e rs imp lyimp l em en t sa doz enfun c t ion sin AD I3[19 ] . A sshowninF ig .1 ,th eTCPso ck e tChann e l ,th esha r ed m emo rya c c e s s(SHMEM )chann e l ,andth er emo t ed i r e c t m emo rya c c e s s(RDMA )chann e la r ea l limp l em en t ed inth elay e ro f CH3tofa c i l i ta t eth e ea s eo f po r t ing MP ICH2on va r iou sp la t fo rm s . No t etha tea chon eo f th ea fo r em en t ion edchann e l simp l em en t sth e CH3in t e r fa c efo raco r r e spond ingcommun i ca t iona r ch i t e c tu r e l ik eTCPso ck e t s ,SHMEM ,andRDMA . Un l ik ean AD I3 d ev i c e ,achann e li sea sytoimp l em en ts in c eon eon ly ha stoimp l em en ta doz enfun c t ion sr e l evan tfo rw i th th echann e lin t e r fa c e . T oadd r e s sth ei s su e so fm e s sag esnoop inginth em e s sag epa s s ingenv i ronm en t sonc lu s t e r s ,w es e ektoim p l em en tas tanda rd MP Im e chan i sm w i thconfid en t ia l i ty s e rv i c e stocoun t e rsnoop ingth r ea t sin MP Ip rog ram s runn ingonac lu s t e rconn e c t edanun s e cu r edn e two rk . Mo r esp e c ifi ca l ly ,w ea imtoimp l em en tc ryp tog raph i c a lgo r i thm sinth eTCPso ck e tchann e linth e CH3lay e r o f MP ICH2( s e eF ig .2ands e c t ion5fo rd e ta i l so fhow tocon s t ru c tac ryp to sy s t eminth echann e llay e r ) . 4 THE DES IGN OF ES -MP ICH2 4 .1 Scopeo fES -MP ICH2 Confid en t ia l i ty ,in t eg r i ty ,ava i lab i l i ty ,andau th en t i ca t ion a r efou rimpo r tan ts e cu r i tyi s su e sto b eadd r e s s edin c lu s t e r s conn e c t ed by an un s e cu r ed pub l i cn e two rk . Ra th e rthanadd r e s s inga l lth es e cu r i tya sp e c t s ,w epay pa r t i cu la ra t t en t iontoconfid en t ia l i tys e rv i c e sfo rm e s sag e spa s s edamongcompu t ingnod e sinanun s e cu r ed c lu s t e r . A l though p r e s e rv ingconfid en t ia l i tyi s ou rp r ima ry con c e rn , anin t eg r i tych e ck ings e rv i c ecan b er ead i ly in co rpo ra t edin toou rs e cu r i tyf ram ewo rkbyapp ly inga pub l i c -k eyc ryp tog raphys ch em e .Inan MP If ram ewo rk equ ipp ed w i thth epub l i c -k eys ch em e ,s end ingnod e scan en cod em e s sag e su s ingth e i rp r iva t ek ey s .Inth em e s sag e r e c e iv ing p ro c edu r e , any nod e s can u s e pub l i ck ey s co r r e spond ingtoth ep r iva t ek ey sto d e cod em e s sag e s . I f on ea l t e r sth e m e s sag e s ,th ec iph e r t ex t canno tb e 4 d e c iph e r edco r r e c t lyu s ingpub l i ck ey sco r r e spond ingto th ep r iv a t ek ey s .Thu s ,th er e c e iv ingnod e scanp e r fo rm m e s s ag ein t eg r i tych e ck w i thou tth es e cu r eex chang eo f s e c r e tk ey s .P l e a s er e f e rtoS e c t ion5 .6fo rd e ta i l so fhow toaddin t eg r i tych e ck ings e rv i c e sinou r MP If ram ewo rk c a l l edES -MP ICH 2 . 4 .2 Des ignIssues Th ego a lo fth ed ev e lopm en to fth eES -MP ICH2 m e ch an i smi stoen ab l eapp l i ca t ion p rog ramm e r stoea s i ly imp l em en ts e cu r eenhan c ed MP Iapp l i ca t ion sw i thou t add i t ion a lcod efo rda ta confid en t ia l i typ ro t e c t ion .W i th ES -MP ICH 2inp l a c e ,s e cu r e MP Iapp l i ca t ionp rog ram m e r sa r e ab l eto fl ex ib ly choo s e ac ryp tog raph i ca l go r i thm ,k eys i z e , and da ta b lo cks iz efo rea ch MP I app l i c a t ionth a tn e ed sda taconfid en t ia l i typ ro t e c t ion . ES -MP ICH 2o f f e r sm e s sag econfid en t ia l i tyinan MP I p rog r amm ing env i ronm en t byin co rpo ra t ing MP ICH2 w i then c ryp t ionandd e c ryp t iona lgo r i thm s .Inth ep ro c e s so fd e s ign ing ES -MP ICH2 ,w ein t eg ra t edth e AES and3DESa lgo r i thm sin toth e MP ICH2l ib ra ry . Th e ES -MP ICH2imp l em en ta t ion ha sth efo l low ing fou rd e s igngo a l s : •M e s s ag e Confiden t ia l i ty : ES -MP ICH2a im stop r e s e rv e m e s s ag e confid en t ia l i tyf rom unau tho r iz ed a c c e s s e sbyun t ru s t edp ro c e s s e s . W el ev e rag eth e AESto p ro t e c tth econfid en t ia l i tyo fm e s sag e s ,b e cau s e AESi s anen c ryp t ions t anda rdadop t edbyth eU .S .gov e rnm en t . Fo r comp a r i son pu rpo s e , w ea l so con s id e r 3DESin th ed e s igno f ES -MP ICH2 . AES w i th128 -b i tk ey scan p rov id ead equ a t ep ro t e c t ionfo rc la s s ifi ed m e s sag e sup toth e SECRETl ev e l . Th eimp l em en ta t ion o f AESin p rodu c t sin t end edto p ro t e c tna t iona ls e cu r i tysy s t em s and/o rin fo rm a t ion mu s tb er ev i ew edandc e r t ifi edby NSAp r io rtoth e i ra cqu i s i t ionandu s e[1 ] .Inth i ss tudy , w ein t eg r a t edd a t aconfid en t ia l i tys e rv i c e sw i th MP ICH2 byimp l em en t ingth ec ryp tog raph i ca lgo r i thm sinaCH3 ch ann e l . •Comp l e t eT r an spa ren cy :P r e s e rv ing m e s sag econfi d en t i a l i tyin MP ICH2i sen t i r e lyt ran spa r en ttoapp l i ca t ion p rog r amm e r s .Su chconfid en t ia l i tyt ran spa r en cyi s f e a s ib l eandth er ea soni stwo fo ld .F i r s t ,th een c ryp t ion and d e c ryp t ion p ro c e s s e scanb ebu i l tinth e MP ICH2 l ib r a ry a tth echann e lt ran sm i s s ionlay e r .S e cond ,w e ma in t a inth es am ein t e r fa c ea sth e AP I so fth e MP ICH2 imp l em en t a t ion .Th e r e fo r e ,i ti sno tn e c e s sa ryto mod i fy MP Ip rog r am stoadap tES -MP ICH2 . •Comp a t ib i l i tyandPo r tab i l i ty :Id ea l ly ,ES -MP ICH2 n e ed stob ee a s i lypo r t edf romon ep la t fo rmtoano th e r w i th no add i t iontoth e app l i ca t ionsou r c ecod e . ES MP ICH 2i s an ex t en s ion o f MP ICH2 and ;thu s , ES MP ICH 2shou ld hav eth esam el ev e lo f po r tab i l i tya s MP ICH 2 . How ev e r ,i ti scha l l eng ingto a ch i ev eh igh po r t ab i l i tyin ES -MP ICH2 ,b e cau s ew e hav etoimp l e m en tac ryp tog r aph i csub sy s t eminea chchann e linth e CH 3l ay e rin MP ICH2 . •E x t en s ib i l i ty :ES -MP ICH2 mu s ta l lowapp l i ca t ion p rog r amm e r stos e l e c t iv e lychoo s eanyc iph e rt e chn iqu e s Sende r Ne two rk Rece ive r AD I3 AD I3 CH3 CH3 P la in tex t T ransm i t Message TCPSocke t P la in tex t TCPSocke t F ig .2 :M e s sag epa s s ingimp l em en ta t ions t ru c tu r ein MP ICH2 . and k ey sin co rpo ra t edin MP ICH2 . Th i sd e s ign goa l mak e si tpo s s ib l efo rp rog ramm e r stofl ex ib lys e l e c tany c ryp tog raph i ca lgo r i thmimp l em en t edin ES -MP ICH I2 . A l though w eimp l em en t ed AESand3DESinth echann e l lay e ro f MP ICH2 , w e w i l lshowinth en ex ts e c t ion howtoaddo th e rc ryp tog raph i ca lgo r i thm s( e .g . ,E l l ip t i c Cu rv eC ryp tog raphy ,[2 ] )toth e ES -MP ICH2env i ron m en t . 5 IMPLEMENTAT ION DETA ILS Du r ing th eimp l em en ta t ion o f ES -MP ICH2 , w e ad d r e s s edth efo l low ing fiv ed ev e lopm en t qu e s t ion s :(1 ) Amongth e mu l t ip l elay e r sinth eh i e ra r ch i ca ls t ru c tu r e o f MP ICH2 ,in wh i chlay e rshou ld w eimp l em en tc ryp tog raph i ca lgo r i thm s ?(2 ) Wh i chc ryp to sy s t emshou ld w echoo s etoimp l em en t ?(3 ) Howtoimp l em en ts e cu r e k ey manag em en t ?(4 ) Howtou s eth eimp l em en t edES MP ICH2 ?(5 ) Howtoaddin t eg r i tych e ck ings e rv i c e sto ES -MP ICH2 ? 5 .1 C iphe rsintheChanne lLaye r F ig . 2 ou t l in e sth e m e s sag e pa s s ingimp l em en ta t ion s t ru c tu r einth eo r ig ina lv e r s iono f MP ICH2 .Insu cha h i e ra r ch i ca ls t ru c tu r eo f MP ICH2 ,m e s sag e sa r epa s s ed f romas end ing p ro c e s stoar e c e iv ingp ro c e s sth rough th eab s t ra c td ev i c ein t e r fa c e(AD I3 ) ,th echann e lin t e r fa c e (CH3 ) ,andth eTCPso ck e tchann e l .C ryp tog raph i csub sy s t em s mayb eimp l em en t edinon eo fth eth r e elay e r s ( i . e . , AD I3 ,CH3 ,o rth eTCPso ck e tchann e l ) .T oa ch i ev e th ed e s igngoa lo fcomp l e t et ran spa r en cy ,w echo s eto imp l em en tc ryp tog raph i ca lgo r i thm sinth eTCPso ck e t chann e l .Compa r ed w i th AD I3andCH3 ,th eTCPso ck e t chann e li sth elow e s tlay e ro fth e MP ICH2 h i e ra r chy . Imp l em en t ing c ryp to sy s t em sinth elow e s tlay e r can p r e s e rv e m e s sag e confid en t ia l i tyin any conv en t iona l MP Ip rog ram w i thou t add ing ex t ra cod eto p ro t e c t m e s sag e s .F ig .3 d ep i c t sth eimp l em en ta t ions t ru c tu r e o f ES -MP ICH2 , wh e r eac ryp to sy s t emi simp l em en t ed inth e TCPso ck e tlay e r .Thu s ,m e s sag e sa r een c ryp t ed andd e c ryp t edinth eTCPso ck e tchann e lra th e rthanth e AD I3and CH3lay e r s . F ig .4show stha tth een c ryp t ionandd e c ryp t ionfun c t ion sin ES -MP ICH2in t e ra c tw i thth e TCPso ck e tto 5 Ne two rk Sende r Mas te r Node Rece ive r AD I3 AD I3 CH3 CH3 Send DEK Compu t ing Node1 Symme t r ic Keyfo r ES -MP ICH2 Compu t ing Node2 Enc ryp t Message P la in tex t TCPSocke t Enc ryp ted T ex t T ransm i t Message Enc ryp ted T ex t Dec ryp t Message P la in tex t TCPSocke t F ig .3 :M e s s ag epa s s ingimp l em en ta t ions t ru c tu r ein ES -MP ICH 2w i then c ryp t ionandd e c ryp t ionp ro c e s s e s . Ac ryp to sy s t emi simp l em en t edinth eTCPso ck e tlay e r toa ch i ev eth ed e s igngoa lo fcomp l e t et ran spa r en cy . MP I MP I_Send MP I_ ISend . . . . . En c r yp t ion MP I_Recv MP I_Reduce Dec ryp t ion So c ke t W r i t ing Socke t Read ing Socke t Node1 ’s Pub l ic Key Node2 ’s Pub l ic Key Node3 ’s Pub l ic Key Node4 ’s Pub l ic Key Node5 ’s Pub l ic Key Compu t ing Node3 Compu t ing Node4 Compu t ing Node5 F ig .5 :K ey manag em en tinES -MP ICH2 .Pub l i ck ey c ryp tog raphyemp loy edinES -MP ICH2r e l i e son in t e r chang ek ey s( i . e . ,pub l i candp r iva t ek ey s )to ex chang eda taen c iph e rm en tk ey s(DEK )inas e cu r e way . C .W efo cu sonb lo ckc iph e r sinth eimp l em en ta t iono f ES -MP ICH2 ,b e cau s eab lo ckc iph e rt ran s fo rm safix ed l eng thb lo cko fp la in t ex tin toab lo cko fc iph e r t ex to fth e sam el eng th .I fth eca s e wh e r ec iph e r t ex tandp la in t ex t a r ed i f f e r en tinl eng th , MP I app l i ca t ion s hav eto b e awa r eo fsu chad i f f e r en c eino rd e rtoco r r e c t lyd e cod e c iph e r s .K e ep ingin m indtha ts e cu r e lypa s s ing m e s sag e s shou ldb et ran spa r en tto MP Iapp l i ca t ionp rog ramm e r s , w eadvo ca t eth eu s eo fb lo ckc iph e r sra th e rthannon b lo ck c iph e r stha tfo r c ep rog ramm e r stob eawa r eo fth e l eng th so fp la in t ex tandc iph e r t ex t . F ig .4 :Th ein t e r f a c eb e tw e enth een c ryp t ion/d e c ryp t ion 5 .3 Key Managemen t p ro c e s s e sandth eTCPso ck e t .ES -MP ICH2 ma in ta in s Th egoa lo fk ey manag em en ti sto dynam i ca l lye s tab th es am e AP Ia stha to f MP ICH2 . l i sh s e cu r e m e s sag e -pa s s ing chann e l s by d i s t r ibu t ing c ryp tog raph i ck ey s .ES -MP ICH2 ma in ta in stwotyp e so f k ey s-da taen c iph e rm en tk ey s(a .k .a . ,s e s s ionk ey s )and p rov id em e s s ag econfid en t ia l i ty p ro t e c t ioninth e TCP in t e r chang ek ey s .Inth e MP Iin i t ia l iza t ionpha s eo fea ch so ck e tl ay e r .B e fo r ea m e s sag ei sd e l iv e r edth roughth e MP Iapp l i ca t ion ,ada taen c iph e rm en tk eyi sc r ea t edand TCPso ck e tch ann e l , da tacon ta in edinth em e s sag ei s sha r edamonga l lth ecommun i ca t ingp ro c e s s e s . en c ryp t edbyac e r ta inc ryp tog raph i ca lgo r i thml ik e AES F ig .5 p r e s en t sth ek ey manag em en tin f ra s t ru c tu r e and3DES .Uponth ea r r iva lo fanen c ryp t ed m e s sag eina in ES -MP ICH2 . Pub l i ck eyc ryp tog raphyemp loy edin r e c e iv ingnod e ,th enod einvok e sth eco r r e spond ingd e - ES -MP ICH2r e l i e sonin t e r chang ek ey s( i . e . ,pub l i cand c ryp t ionfun c t iontod e c ryp tth em e s sag e .F ig .4d emon - p r iva t ek ey s )tos e cu r e lyex chang es e s s ion k ey sinan s t r a t e sth a tES -MP ICH2 ma in ta in sth esam eapp l i ca t ion un s e cu r ed n e two rk . Mo r esp e c ifi ca l ly , wh en a ma s t e r p rog r amm ingin t e r fa c eo r AP Ia stha to f MP ICH2 by nod ea t t emp t stosha r en ews e s s ionk ey sw i tho th e rs lav e imp l em en t ingth een c ryp t ionandd e c ryp t iona lgo r i thm s nod e s ,th e ma s t e rnod eu s e sth es lav enod e s ’pub l i ck ey s inth e TCPso ck e tl ev e l . Th econfid en t ia l i tys e rv i c e so f toen c ryp tth es e s s ionk ey s .Th es lav enod e s mak eu s e ES -MP ICH 2w e r eimp l em en t edinth e MP ICH2l ib ra r i e s , o fth e i rp r iva t ek ey sto d e c iph e rm e s sag e scon ta in ing th e r eby b e ingto ta l lyt ran spa r en tto MP I app l i ca t ion th es e s s ionk ey s .Th en ,th e ma s t e rnod eands lav enod e s p rog r amm e r s . cans e cu r e lycommun i ca t eu s ingth e MP If ram ewo rk . In t e r chang ek ey sa l socanb es to r edonphy s i ca ld ev i c e s l i k e sm a r t c a r d a n d R OM [ 7 ] [ 9 ] [ 2 1 ] . 5 .2 B lockC iphe rs W eh av e noin t en t ion o fr e inv en t ing ac ryp tog raph i c l ib r a ry ,b e c au s ei ti sv e ryco s t lyto gua ran t e etha tth e s e cu r i tyo fyou rownimp l em en ta t ioni sh igh e rthantha t o fex i s t ingt e s t edandaud i t eds e cu r i tyl ib ra r i e s .Inth e ES -MP ICH 2f r am ewo rk ,w eadop t edth eimp l em en ta t ion o fth e AESand3DESc ryp tog raph i ca lgo r i thm so f f e r ed byth e Po l a rSSLl ib ra ryin MP ICH2 v e r s ion1 .0 .7 . Po l a rSSLi sanop en sou r c ec ryp tog raph i cl ib ra ry w r i t t enin 5 .4 Socke tP rog ramm ing Inso ck e tp rog ramm ing ,th e r ei sabu f f e rcon ta in ingda ta s en tandr e c e iv edth roughth eTCPso ck e tchann e l .F ig .6 d emon s t ra t e sth een c ryp t ionandd e c ryp t ionp ro c e s sin ES -MP ICH2 . ES -MP ICH2en c ryp tth ep la in t ex tinth e bu f f e ro fth es end ing nod eth en d e c ryp ti t onth er e c e iv ing nod e .B e cau s eth ep la in t ex tandc iph e r t ex ta r e 6 m e s sag e su s ing p r iva t ek ey s wh e r ea sr e c e iv ing nod e s u s eth eco r r e spond ing pub l i ck ey sto d e c iph e rth ec i ph e r t ex t .B e fo r eanod ed e l iv e r sanen c ryp t ed m e s sag e inES -MP ICH2 ,th enod een c ryp t sth em e s sag eu s ingth e Enc ryp t Dec ryp t p r iva t ek eyo fth es end ingnod e .T och e ckth ein t eg r i ty o fth ec iph e rm e s sag e ,ar e c e iv ingnod es imp lyn e ed sto Socke tBu f fe r So c ke tBu f fe r d e cod eth ec iph e rm e s sag ebyapp ly ingth epub l i ck ey o fth es end ingnod e . Ha sh Fun c t ion s : Wh eni tcom e stola rg em e s sag e s , W r i teSocke t ReadSo c ke t ha shfun c t ion sa r ew id e lyu s edinin t eg r i tych e ck ingand d ig i ta ls igna tu r e s . MD5 ,SHA -1 ,andSHA -2canb eim So c ke t p l em en t edtoch e ckth ein t eg r i tyo fen c ryp t ed m e s sag e s inES -MP ICH2 .Th eha shi sac ryp tog raph i cch e ck sumo r F ig .6 :ES -MP ICH2So ck e tD e ta i l s m e s sag ein t eg r i tycod etha ts end ingandr e c e iv ingnod e s mu s tcompu t etov e r i fy m e s sag e s .Sp e c ifi ca l ly ,as end ing n o d e u s e s a h a s h f u n c t i o n t o c om p u t e a c h e c k s um f o ra id en t i c a linl eng thinb lo ckc iph e ra lgo r i thm s ,th es iz e s l a r g e m e s s a g e . T h e c h e c k s um i s s h o r t e r t h a n t h e o r i g i na l o fth ebu f f e r sinbo thth es end ingandr e c e iv ingnod e s m e s s a g e . T h e n , t h e s e n d i n g n o d e s i g n s t h e h a s h v a l u e r em a inun ch ang eda f t e rth een c ryp t ionandd e c ryp t ion w i thasha r edk ey .Indo ingso ,th ein t eg r i tyo fth ela rg e p ro c e s s e s . m e s sag ecanb ech e ck edinane ffi c i en t way . P eop l eu s eha shandth ens ignth eha sh w i thth ek ey , 5 .5 Usage th es iz eo fha shb e ing mu chsho r t e rthanth em e s sag e . Th es e cu r i tyf e a tu r e so fES -MP ICH2canb econfigu r ed w i thou t mod i fy ingany MP Iapp l i ca t ionsou r c ecod e .T o ION s e cu r e lyp a s sm e s sag e su s ingES -MPCH2 ,th efo l low ing 6 EXPERIMENTALEVALUAT oeva lua t eth ef ea tu r e sandp e r fo rman c eo fES -MP ICH2 , configu r a t ion s mu s tb es e tb e fo r e MP Iin i t ia l iza t ion . T eimp l em en t ed ES -MP ICH2and d ep loy edi tontwo F i r s t ,as e cu r i tyop t ionshou ldb eenab l edo rd i sab l ed . w lu s t e r sw i th d i f f e r en tconfigu ra t ion s . Th efi r s tc lu s t e r S e cond , on eh a stos e l e c tasp e c ifi cc ryp tog raph i ca l - c h a s s i x n o d e s o f 2 . 2 GH z I n t e l C e l e r o n p r o c e s s o r s w i t h 2 go r i thmimp l em en t edin ES -MP ICH2 . Th i rd ,ex chang e k ey s mu s tb es e cu r e lys to r edinaconfigu ra t ionfi l eo ra GB m emo ry .Th es e condc lu s t e rcon ta in st ennod e s .Th e s t e rnod eha sa3 .0 GHzIn t e lP en t ium Co r e2 Duo phy s i c a ld ev i c einea chnod e( s e eS e c t ion5 .3fo rd e ta i l s ma ro c e s so rw i th1 GB m emo ry , wh e r ea sth en in es lav e onth ek ey m an ag em en ti s su e . )Th en ,u s e r scanrunth e i r p e shav e333 MHzIn t e lP en t iumI Ip ro c e s so r sw i th MP Ip rog r am sinth esam e waya sth eyshou ldrunth e nod emo ry . Th es ix nod e sinth efi r s tc lu s t e ra r e p rog r am sin MP ICH2 .Thu s ,i fan MP Ip rog ramcanb e 64 MB m e c t edbya1 Gbp sE th e rn e tLAN .Th e10nod e sin ex e cu t edin MP ICH2 ,on ecana l sorunth e MP Ip rog ram conn es e condc lu s t e ra r econn e c t edbya100 Mbp sE th e rn e t inES -MP ICH 2w i thou t mod i fy ingth esou r c ecod eo fth e th LAN . Appa r en t ly ,th eov e ra l lp e r fo rman c eo fth efi r s t p rog r am . c lu s t e ri sh igh e rthantha to fth es e condc lu s t e r . W e u s e a f a s t c l u s t e r ( i . e . , t h e fi r s t o n e ) a n d a s l ow( i . e . , 5 .6 Inco rpo ra t ingIn teg r i tySe rv icesinES -MP ICH2 t h e s e c o n d o n e ) c l u s t e r t o c o n d u c t e x p e r im e n t s , b e c a u s e Inadd i t iontoconfid en t ia l i tys e rv i c e s ,in t eg r i tych e ck ing o n e o f t h e g o a l s i s t o i l l u s t r a t e t h e im p a c t o f c om p u t i n g s e rv i c e sc an b es eam l e s s lyin co rpo ra t edin toth e ES c i tyo fc lu s t e r sonth ep e r fo rman c eo fES -MP ICH2 . MP ICH 2f r am ewo rk .In wha tfo l low s ,w eadd r e s sth e capa Socke tBu f fe r So c ke tBu f fe r imp l em en t a t ioni s su eo fhowtoin t eg ra t ein t eg r i tych e ck ings e rv i c e sinES -MP ICH2 . Sp r e ad ingf e a tu re o fb lo cken c ryp t iona lgo r i thm s : B lo cken c ryp t iona lgo r i thm s hav easp r ead ingf ea tu r e wh i ch m e an si f ev en 1 b i ti schang edinc iph e r t ex t , th ed e c ryp t edt ex tw i l lb ecomp l e t e lyd i f f e r en tf romth e o r ig in a lp l a in t ex t .A l t e r ed m e s sag e scau s ingfa ta le r ro r s c anno tb ein t e rp r e t ed .A l though u s ingth esp r ead ing f ea tu r ei sno tar e l iab l eso lu t ion ,th esp r ead ingf ea tu r e do e sp rov id eanin t eg r i tych e ck ing m e thod .S in c ebo th AES and 3DES a r eb lo ck en c ryp t ion a lgo r i thm s , ES MP ICH 2m ayr e lyonth esp r ead ingf ea tu r etop e r fo rm in t eg r i tych e ck ing . Pub l i ck ey :Anin t eg r i tys e rv i c eta i l edfo rsma l lm e s sag e sc an b e add edin to ES -MP ICH2 u s ing a pub l i c k eyen c ryp t ions ch em e ,in wh i chs end ingnod e sen cod e TABLE1 :Th e Configu ra t iono f A6 -Nod e C lu s t e ro fIn t e lC e l e ronP ro c e s so r s CPU M emo ry OS K e rn e lv e r s ion N e two rk Nod e ×6 In t e lC e l e ron4 5 02 . 2GHz 2GB Ubun tu9 . 0 4J aun tyJ a ck a lop e 2 . 6 . 2 8 1 5 -g en e r i c 1 0 0 0Mbp s 6 .1 A6 -nodeC lus te ro fIn te lCe le ronP rocesso rs 6 .1 .1 Expe r imen ta lT es tbed L e tu sfi r s teva lua t eth ep e r fo rman c eo fbo th MP ICH2 and ES -MP ICH2 on a6 -nod ec lu s t e r .T ab l e 1r epo r t s th econfigu ra t ion o fth efi r s tc lu s t e rw i ths ixid en t i ca l 7 3DES -ba s ed ES -MP ICH2 . W e ob s e rv ef romth i s figu r e tha t wh enth em e s sag es iz ei ssma l l( e .g . ,1 KB ) ,th e p e r fo rman c eo f ES -MP ICH2i sv e ry c lo s etotha to f MP ICH2 .Fo rexamp l e ,th een c ryp t ion modu l e sin ES MP ICH2on ly mod e s t lyin c r ea s eth eex e cu t iont im eby l e s sthan2% .Th e s er e su l t sind i ca t etha tES -MP ICH2can p r e s e rv econfid en t ia l i tyo fsma l lm e s sag e sw i thn eg l ig i b l eov e rh ead . Sand ia M ic roBenchma rkI te r_ t 6 10 O r ig ina l MP I AES 3DES 5 10 4 T ime(ms ) 10 3 10 2 10 1 10 0 10 0 2 4 6 8 10 n 12 14 16 18 20 2 By tes F ig .7 :S and ia M i c roB en chma rki t e rt im e compu t ing nod e so fIn t e lC e l e ron p ro c e s so r s . Th eop e ra t ingsy s t em u s edinth es ix nod e si s Ubun tu 9 .04 Jaun tyJ a ck a lop e . Th ecompu t ing nod e sa r econn e c t ed bya1 Gbp sn e two rk .A l lth es lav enod e ssha r ead i sk onth e m a s t e r nod eth roughth en e two rk fi l esy s t em (NFS )[ 2 6 ] .Th e MP Il ib ra ryu s edinth e6 -nod ec lu s t e ri s MP ICH 2v e r s ion1 .0 .7 .W erunth eSand ia M i c roB en ch ma rk sandth eIn t e l MP IB en chma rk stoeva lua t eand comp a r eth ep e r fo rman c eo f MP ICH2andES -MP ICH2 . Wh en w et e s t ES -MP ICH2inea chexp e r im en t ,w es e t th ec ryp tog r aph i cs e rv i c eto AESand3DES ,r e sp e c t iv e ly . Th el eng th o fd a taen c iph e rm en tk ey sg en e ra t edand d i s t r ibu t edinES -MP ICH2i s192 -b i t . TABLE2 :P e r fo rman c eM e t r i c su s edinth eSand ia M i c roB en chma rkSu i t e(SMB ) M e t r i c i t e rt wo rk t Exp lana t ion to ta lamoun to ft im efo rth elooptocomp l e t e fo rea chi t e ra t iono fth epo s t -wo rk -wa i tloop th eamoun to f wo rkp e r fo rm ed ov e rh eadt th el eng tho ft im etha tap ro c e s so ri seng ag edinth e t ran sm i s s iono rr e c ep t iono fea ch m e s sag e ba s et m e s sag et ran s f e rt im eca l cu la t ionth r e sho ld 6 .1 .2 SMB :Sand ia M ic roBenchma rk Th eS and i a N a t iona l Labo ra to ry d ev e lop edth e San d i a M i c ro B en chma rk Su i t e(a .k .a . , SMB )to eva lua t e andt e s th igh -p e r fo rman c en e two rkin t e r fa c e sandp ro to co l s .T ab l e2 d e s c r ib edth efou rp e r fo rman c e m e t r i c su s edinth eSMB b en chma rksu i t e . Th e s em e t r i c s in c lud eto t a l ex e cu t iont im e( i . e . ,i t e rt ) , CPU ex e cu t iont im efo ri t e ra t ion s( i . e . , wo rkt ) , m e s sag e pa s s ing ov e rh e ad( i . e , ov e rh eadt ) , and m e s sag et ran s f e r t im ec a l cu l a t ionth r e sho ld( i . e ,th r e sho ldo rba s et ) .Th e d e t a i l edin fo rm a t ion onth e s e m e t r i c s can b efound a th t tp ://www . c s . sand ia .gov/ smb .P l ea s eno t etha tth e m e s s ag ep a s s ingov e rh eadcanb ed e r iv edbysub t ra c t ing th e CPUex e cu t iont im ef romth eto ta lex e cu t iont im e . Ea chb en chm a rkha s1000i t e ra t ion s . F ig . 7show sth eto ta l ex e cu t iont im eo fth e SMB b en chm a rkrunn ingonth eo r ig ina l MP Iimp l em en ta t ion ( i . e . , MP ICH 2 )a sw e l la s AES -ba s ed ES -M IPCH2and F ig .8 (a ) ,F ig .8 (b ) ,F ig .8 ( c ) ,andF ig .8 (d )showth e to ta lex e cu t iont im e ,CPUt im e ,ov e rh ead ,andth r e sho ld o f MP ICH2 and ES -MP ICH2 wh enth e m e s sag es iz e i ss e tto2 KB ,16 KB ,128 KB ,512 KB ,and1024 KB , r e sp e c t iv e ly . Th er e su l t sp lo t t edinF ig .8 (a )showtha t AES -ba s edES -MP ICH2and MP ICH2hav es im i la rp e r fo rman c einth eca s eo fsma l lm e s sag e s .Fo rexamp l e , th e AES modu l einES -MP ICH2in c r ea s e sth eex e cu t ion t im e so fi t e rtandwo rk tby5 .9%and1 .2% ,r e sp e c t iv e ly . How ev e r , wh en3DESi semp loy edinES -MP ICH2 , th es e cu r i tyov e rh eado fES -MP ICH2b e com e sno t i c eab l e ev enfo rsma l lm e s sag e s .Fo rexamp l e ,l e tu scon s id e ra ca s e wh e r eth em e s sag es iz ei s2KB . Compa r ew i thth e 3DES modu l e ,th e AES modu l ecanr edu c eth eex e cu t ion t im e so fi t e rtandba s etbyapp rox ima t e ly56%and 45% ,r e sp e c t iv e ly .F ig s .8 (b ) -8 (d )i l lu s t ra t etha t bo th AESand3DESinES -MP ICH2in t rodu c e mu chov e rh ead tha t mak e s ES -MP ICH2p e r fo rm s wo r s ethan MP ICH2 -th eo r ig ina l MP Iimp l em en ta t ion .S e cu r i ty ov e rh ead inES -MP ICH2b e com e s mo r ep ronoun c ed w i thin c r ea s ing m e s sag es iz e .S in c e AES ha sb e t t e rp e r fo rman c e than3DES , AES -ba s edES -MP ICH2i ssup e r io rto3DES ba s ed ES -MP ICH2 . W er e comm endth efo l low ingtwo app roa ch e stolow e r ing ov e rh ead cau s ed by en c ryp t ionandd e c ryp t ion modu l e sinES -MP ICH2 .F i r s t ,on e canr edu c eth es e cu r i ty ov e rh eadin ES -MP ICH2 by enhan c ingth ep e r fo rman c eo fb lo ckc iph e ra lgo r i thm s . S e cond , mu l t i co r ep ro c e s so r scanboo s te ffi c i en cyo fth e en c ryp t ionandd e c ryp t ion modu l e s ,th e r ebyb en efi t ing th ep e r fo rman c eo fES -MP ICH2 . TABLE3 :In t e l MP IB en chma rk s B en chm a rk s P ingPong P ingP ing S end r e cv Ex ch ang e B c a s t A l lg a th e r A l lg a th e rv S c a t t e r S c a t t e rv G a th e r G a th e rv A l l to a l l A l l to a l lv R edu c e R edu c eS c a t t e r A l l r edu c e W indow C l a s s ifi c a t ion S ing l eT r an s f e r S ing l eT r an s f e r P a r a l l e lT r an s f e r P a r a l l e lT r an s f e r Co l l e c t iv e Co l l e c t iv e Co l l e c t iv e Co l l e c t iv e Co l l e c t iv e Co l l e c t iv e Co l l e c t iv e Co l l e c t iv e Co l l e c t iv e Co l l e c t iv e Co l l e c t iv e Co l l e c t iv e O th e r S em an t i c s MP I 1 MP I 1 MP I 1 MP I 1 MP I 1 MP I 1 MP I 1 MP I 1 MP I 1 MP I 1 MP I 1 MP I 1 MP I 1 MP I 1 MP I 1 MP I 1 MP I 2 8 Sand ia M ic roBenchma rk MsgS ize=1024KB 1800 1600 Sand ia M i c roBen chma r kM sgS i ze=16KB 6000 O r ig ina l MP I AES 3DES O r ig ina l MP I AES 3DES 5000 1400 4000 T ime(ms ) T ime(ms ) 1200 1000 800 600 3000 2000 400 1000 200 0 i te r_ t wo rk_ t ove rhead_ t 0 ba se_ t i te r_ t (a )M e s sag eS iz ei s2KB ba se_ t Sand ia M i c roBen chma r kM sgS i ze=1024KB 5 x10 3 .5 O r ig ina l MP I AES 3DES 4 o ve rhead_ t (b )M e s s ag eS i z ei s1 6KB Sand ia M ic roBenchma rk MsgS ize=128KB 4 x10 4 .5 wo r k_ t O r ig ina l MP I AES 3DES 3 3 .5 2 .5 T ime(m s ) T ime(m s ) 3 2 .5 2 2 1 .5 1 .5 1 1 0 .5 0 .5 0 i te r_ t wo rk_ t ove rhead_ t ba se_ t ( c )M e s sag eS iz ei s128KB 0 i te r_ t wo r k_ t o ve rhead_ t ba se_ t (d )M e s s ag eS i z ei s1 0 2 4KB F ig .8 :S and i aM i c roB en chma rk ,I t e ra t ionT im e ,W o rkT im e , Ov e rh eadT im e ,andBa s eT im eon D i f f e r en tM e s sag e S iz ef rom6 -nod eC lu s t e ro fIn t e lC e l e ron 6 .1 .3 IMB :In te l MP IBenchma rks Th eIn t e l MP Ib en chma rksu i t eo rIMB wa sd ev e lop ed fo rt e s t ingandeva lua t ingimp l em en ta t ion so fbo th MP I 1[ 8 ]and MP I 2[12 ]s tanda rd s .IMBcon ta in sapp rox i m a t e ly1 0 , 0 0 0l in e so fcod eto m ea su r eth ep e r fo rman c e o fimpo r t an t MP Ifun c t ion s[5 ] [25 ] . W e hav e eva lu a t edth ep e r fo rm an c eo f ES -MP ICH2andth eo r ig ina l MP ICH 2 byrunn ingth eb en chma rk s onth e6 -nod e c lu s t e r .T ab l e 3l i s t sa l lth eIn t e lb en chma rk su s edto m ea su r eth ep e r fo rman c eo fES -MP I2and MP ICH2 .Th e b en chm a rk sinIMB -MP I1can b eca t ego r iz edinth r e e g roup s :s ing l et r an s f e r , pa ra l l e lt ran s f e r ,andco l l e c t iv e b en chm a rk s .S ing l et ran s f e rb en chma rk sa r efo cu s ingon as ing l em e s s ag et ran s f e r r edb e tw e entwocommun i ca t ingp ro c e s s e s . Un l ik es ing l et ran s f e rb en chma rk s ,pa ra l l e lt r an s f e rb en chma rk sa ima tt e s t ingpa t t e rn sanda c t iv i t i e sinag roupo fcommun i ca t ingp ro c e s s e sw i thcon cu r r en ta c t ion s .Co l l e c t iv eb en chma rk sa r eimp l em en t ed tot e s th igh e rl ev e lco l l e c t iv efun c t ion s , wh i chinvo lv e p ro c e s so r sw i th inad efin edcommun i ca to rg roup .P l ea s e r e f e rto h t tp :// so f twa r e . in t e l . com/ en -u s/a r t i c l e s/ in t e l mp i -b en chm a rk sfo r mo r ein fo rma t ioncon c e rn ingIMB . F ig s .9 ( a )and9 (b )showth ep e r fo rman c eo fP ingPong andP ingP ing-twos ing l et ran s f e rb en chma rk sinIMB . S in c es ing l et r an s f e rb en chma rk sa r eu s edtot e s tapa i r o ftwoa c t iv ep ro c e s s e s ,w erunP ingPongandP ingP ing ontwonod e so fth e6 -nod ec lu s t e r .Th eto ta lex e cu t ion t im e so fP ingPongandP ingP inggoup wh enth em e s sag es iz ein c r ea s e sb e cau s ela rg e rm e s sag e sg iv er i s eto h igh e ren c ryp t ionandd e c ryp t ionov e rh ead s .Compa r ed w i th MP ICH2 ,th eex e cu t iont im e so f AES -ba s edand 3DES -ba s edES -MP ICH2a r e mo r es en s i t iv eto m e s sag e s iz e . Now w eana lyz eth ep e r fo rman c eo fS end r e cvand Ex chang e-two pa ra l l e lt ran s f e rb en chma rk sinIMBrunn ing on ES -MP ICH2and MP ICH2 onth e6 -nod e c lu s t e r .S end r e cv , in wh i chth e ma inpu rpo s ei stot e s tth e MP I S end r e cvfun c t ion ,con s i s t so fp ro c e s s e sfo rm inga p e r iod i ccommun i ca t ioncha in .S im i la r ly ,Ex chang ei sa b en chma rkfo cu s ingonth eeva lua t iono fth e MP IIS end , MP I W a i ta l l ,and MP I R e cvfun c t ion s . Un l ik eth ea fo r e m en t ion eds ing l et ran s f e rb en chma rk s ,m e s sag epa s s ing op e ra t ion sinth e s etwo pa ra l l e lb en chma rk sa r ep e r fo rm edinpa ra l l e l . F ig .9 ( c )p lo t sth ep e r fo rman c er e su l t so fth eS endR e cv b en chma rk onth ec lu s t e r , wh e r e ea ch nod er e c e iv e s da taf romi t sl e f tn e ighbo randth ens end s da tatoi t s r igh tn e ighbo r .Th eto ta lex e cu t iont im eo fth eS endR e cv b en chma rk do e s no t no t i c eab lychang e wh en w eva ry th enumb e ro fcompu t ing nod e sinth ec lu s t e r . W ea t - 9 IMBP ingPong 6 10 5 10 O r ig ina l MP I AES 3DES 5 10 4 10 T ime(ms ) T ime(ms ) O r ig ina l MP I AES 3DES 4 10 3 10 3 10 2 10 2 1 10 10 1 10 0 10 0 IMBP ingP ing 6 10 0 5 10 n 15 10 0 20 5 MsgS ize2 By tes (a )P ingPong 5 10 O r ig ina l MP I AES 3DES 5 10 4 20 O r ig ina l MP I AES 3DES 4 10 T ime(m s ) T ime(m s ) 15 IMBE x change 6 10 10 3 10 3 10 2 10 2 1 10 10 1 10 0 10 0 n (b )P ingP ing IMBSend recv 6 10 10 M sgS i ze2 B y te s 0 5 10 MsgS ize2n By tes 15 20 ( c )S endR e cv 10 0 5 10 M sgS i ze2n B y te s 15 20 (d )Ex ch ang e F ig .9 :In t e l MP IB en chma rk s ,P ingPongandP ingP inga r eS ing l eT ran s f e rB en chma rk s ,S endR e cvandEx chang e a r ePa ra l l e lB en chma rk son6 -nod eC lu s t e ro fIn t e lC e l e ron av e rag eex e cu t iont im e s .F ig s .10 (a ) -10 (b )showth ep e r fo rman c eo fth efi r s tg roup o fn in eco l l e c t iv eb en ch ma rk s . W e ob s e rv ef romth e s e figu r e stha tth eto ta l ex e cu t iont im eo fea chco l l e c t iv eb en chma rkcon t inua l ly in c r ea s e s w i thin c r ea s ing m e s sag es iz e . MP ICH2 ha s b e t t e rp e r fo rman c ethan AES -ba s edand3DES -ba s edES MP ICH2a c ro s sa l lth eco l l e c t iv eb en chma rk s ,b e cau s e th econfid en t ia l i tyi sp r e s e rv eda tth eco s to fm e s sag e pa s s ingp e r fo rman c e .F ig s .11 (a ) -11 (b )p lo tth eex e cu t ion t im e so fth es e cond g roup o fth r e eb en chma rk s . Th e p e r f o rm a n c e r e s u l t s o f t h e s e c o n d b e n c hm a r k g r o u p F ig .9 (d )d ep i c t sth eto ta lex e cu t iont im eo fth e Ex r econ s i s t en tw i ththo s eo fth efi r s tb en chma rkg roup ch ang eb en chm a rk . Compa r ingF ig .9 (d )w i thF ig .9 ( c ) , a epo r t edinF ig s .10 (a ) -10 (b ) . w er e a l i z eth a tr ega rd l e s so fth e MP Iimp l em en ta t ion s , r th eex e cu t iont im eo fth eEx chang eb en chma rki slong e r th anth a to fth eS endR e cv b en chma rk und e rth econ d i t ion o fs am e m e s sag es iz e . Th i si s ma in ly b e cau s e F ig .12show sth er e su l t so fth eW indowb en chma rk , in Ex ch ang ee a ch nod et ran s f e r da tato bo thl e f tand wh i cha im stot e s t MP I -2fun c t ion sl ik e MP IW in c r ea t e , r igh tn e ighbou r sinth e commun i ca t ion cha in . Thu s , MP I W in f en c e ,and MP IW in f r e e .Inth i sb en chma rk , commun i c a t iont im ein Ex chang ei sla rg e rthantha t a w indows iz e m e s sag ei st ran s f e r r edto ea ch nod e , in S endR e cv .A s ar e su l t ,th eto ta l ex e cu t iont im eo f wh i chintu rnc r ea t e sa w indowu s ingasp e c ifi eds iz e . Ex ch ang ei sapp rox ima t e lytwot im e sh igh e rthantha t F ig .12ind i ca t e stha tth eex e cu t iont im eo fth eb en ch o fS end r e cv wh en m e s sag es iz ei sla rg e . ma rki sno ts en s i t iv eto m e s sag es iz e .Th er e su l t scon t r ibu t eth i st r endtoth efa c to rtha tm e s sag epa s s ingin mu l t ip l e nod e sa r eca r r i edou tin pa ra l l e lra th e rthan s e r i a l ly .Thu s ,in c r ea s ingth enumb e ro fnod e sdo e sno t a f f e c tS endR e cv ’ sto ta lex e cu t iont im e . W i thr e sp e c tto pa r a l l e lt r an s f e r s ,th ep e r fo rman c eo f AES -ba s edand 3DES -b a s ed MP ICH2i sc lo s etotha to fth eo r ig ina l v e r s ion o f M IPCH2 wh en m e s sag es iz ei sr e la t iv e ly sm a l l . Wh eni tcom e stola rg em e s sag e s , AES -ba s edES MP ICH 2h a sb e t t e r pa ra l l e lt ran s f e rp e r fo rman c ethan 3DES -b a s ed MP ICH2 . L e tu sv a ry m e s sag es iz e andeva lua t eth ep e r fo r - fi rmtha t AES -ba s edES -MP ICH2imp rov e sth es e cu r i ty man c eo fco l l e c t iv eb en chma rk s .W erunth eb en chma rk o fth e W indowb en chma rkon MP ICH2 w i th ma rg ina l 10t im e s one a ch MP Iimp l em en ta t ionandr epo r tth e ov e rh ead . 1 0 IMBCo l lec t ivep rocess#=6 1400 7000 1200 6000 1000 800 400 2000 200 1000 0 0 B ca s t A l lga the rA l lga the r vS ca t te r Sca t te rv Ga the r Ga the rv A l l toa l l A l l toa l lv MsgS ize=1KB 10000 4000 3000 (a )M e s sag eS iz ei s1KB O r ig ina l MP I AES 3DES 14000 12000 5000 600 IMBCo l lec t ivep rocess#=6 16000 O r ig ina l MP I AES 3DES 8000 T ime(m s ) T ime(m s ) 1600 IMBCo l lec t ivep rocess#=6 9000 O r ig ina l MP I AES 3DES T ime(m s ) 1800 8000 6000 4000 2000 0 Bcas t A l lga the rA l lga the rvSca t te r Sca t te rv Ga the r Ga the rv A l l toa l l A l l toa l lv MsgS ize=8KB (b )M e s s ag eS i z ei s8KB Bcas t A l lga the rA l lga the rvSca t te r Sca t te rv Ga the r Ga the rv A l l toa l l A l l toa l l v MsgS ize=16KB ( c )M e s s ag eS i z ei s1 6KB F ig .1 0 :In t e l MP IB en chma rk s , Co l l e c t iv eB en chma rk s ,G roup A ,ag roupo fb en chma rk sf rom Co l l e c t iv eon 6 -nod eC lu s t e ro fIn t e lC e l e ron IMBCo l lec t ivep rocess#=6 1400 1200 O r ig ina l MP I AES 3DES 1000 8000 T ime(ms ) T ime(ms ) T ime(ms ) 4000 600 O r ig ina l MP I AES 3DES 10000 5000 800 IMBCo l lec t ivep rocess#=6 12000 IMBCo l lec t ivep rocess#=6 6000 O r ig ina l MP I AES 3DES 3000 6000 2000 4000 1000 2000 400 200 0 Redu ce Reduce_Sca t te r MsgS ize=1KB 0 A l l reduce (a )M e s sag eS iz ei s1KB Reduce Reduce_Sca t te r MsgS ize=8KB 0 A l l reduce (b )M e s s ag eS i z ei s8KB Reduce Reduce_Sca t te r MsgS ize=16KB A l l redu ce ( c )M e s s ag eS i z ei s1 6KB F ig .1 1 :In t e l MP IB en chma rk s , Co l l e c t iv eB en chma rk s ,G roupB ,ag roupo fb en chma rk sf rom Co l l e c t iv eon 6 -nod eC lu s t e ro fIn t e lC e l e ron on ly64 MB m emo ry , wh e r ea sth e ma s t e r nod e ha s1 GB m emo ry .A l lth et ennod e sa r econn e c t edbya100 Mbp sE th e rn e tn e two rk .L ik eth efi r s tc lu s t e r ,a l lnod e s inth e10 -nod ec lu s t e rsha r ed i skspa c eonth e ma s t e r nod eth roughth en e two rkfi l esy s t em(NFS ) . IMB W indowp rocess#=6 1200 1000 T ime(m s ) 800 600 TABLE4 :Th e Configu ra t iono f A10 -Nod eC lu s t e r o fIn t e lP en t iumI IP ro c e s so r s 400 O r ig ina l MP I AES 3DES 200 0 0 2 4 6 8 10 12 W indow 2 n By tes CPU 14 16 18 20 F ig .1 2 :In t e lM i c roB en chma rk W indow6nod e s M emo ry OS K e rn e l N e two rk Ad ap t e r M a s t e r ×1 S l av e s×9 P en t ium Co r e P en t iumI I 2 Duo3 . 0 0GH z 3 3 3MHz 1GB 6 4MB F edo r a Co r e F edo r a Co r e r e l e a s e4 r e l e a s e4 2. 6. 12−1. 1456 2. 6. 17−1. 2142 FC 4 smp FC 4 1 0 0Mbp s 1 0 0Mbp s 6 .2 A10 -nodeC lus te ro fIn te lPen t iumI IP rocesso rs 6 .2 .1 E xpe r imen ta lT es tbed Now w e ev a lu a t eth ep e r fo rman c eo f MP ICH2 and ES -MP ICH 2 on a10 -nod ec lu s t e ro fIn t e lP en t iumI I p ro c e s so r s . Th ec lu s t e rconfigu ra t ioni ssumma r iz edin T ab l e4 .Th eop e r a t ingsy s t emrunn ingonth i sc lu s t e ri s F edo r a Co r er e l e a s e4(S t en tz ) .A l thoughth ep ro c e s so r s o fth en in es l av enod e sa r e333 MHzIn t e lP en t iumI I , th em a s t e rnod econ ta in sa3 .0 GHzIn t e lP en t ium Co r e 2 Duop ro c e s so r , wh i chi sa lmo s tt ent im e sfa s t e rthan th ep ro c e s so r sinth es lav enod e s .Ea chs lav enod eha s 6 .2 .2 SMB :Sand ia M ic roBenchma rk F ig s . 13 (a ) -13 ( c )r ev ea lth eto ta l ex e cu t iont im e , CPU t im e , ov e rh ead , andth r e sho ld o f MP ICH2 and ES MP ICH2 wh enth e m e s sag es iz ei ss e tto 1 KB , 16 KB ,and32 KB ,r e sp e c t iv e ly .Th er e su l t sshowtha tth e p e r fo rman c eo f AES -ba s edand3DES -ba s edES -MP ICH2 i sno t i c eab ly wo r s ethantha to f MP ICH2 ,b e cau s eth een c ryp t ionandd e c ryp t ion modu l e sinES -MP ICH2sp end s ign ifi can tamoun to ft im einen c ryp t ingandd e c ryp t ing 1 1 Sand ia M i c roBenchma rk MsgS ize=1KB 6000 Sand ia M ic roBenchma rk MsgS ize=16KB 4 x10 7 O r ig ina l MP I AES 3DES 6 5000 Sand ia M ic roBenchma rk MsgS ize=32KB 4 x10 14 O r ig ina l MP I AES 3DES O r ig ina l MP I AES 3DES 12 5 10 4 8 3000 T ime(m s ) T ime(m s ) T ime(m s ) 4000 3 6 2000 1000 0 i te r_ t wo r k_ t ove rhead_ t base_ t (a )M e s sag eS iz ei s1KB 2 4 1 2 0 i te r_ t wo rk_ t ove rhead_ t base_ t 0 (b )M e s s ag eS i z ei s1 6KB i te r_ t wo rk_ t ove rhead_ t ba se_ t ( c )M e s s ag eS i z ei s3 2KB F ig .13 :Sand ia M i c roB en chma rk sonth e10 -nod ec lu s t e ro fIn t e lP en t iumI I m e s s ag e si s su edf romth eb en chma rk s .Th i st r endi st ru e ev en wh en m e s s ag e sa r esma l l( e .g . ,s e eF ig .13 (a ) wh e r e m e s s ag es i z ei s1 KB ) . Comp a r ingF ig .13 (a )andF ig .8 (a ) ,w ed rawth efo l low ingth r e eob s e rva t ion s .F i r s t ,th efi r s t6 -nod ec lu s t e r i ss ign ifi c an t lyf a s t e rthanth es e cond10 -nod ec lu s t e r . A l thoughth e1 0 -nod ec lu s t e rha s mo r ecompu t ingnod e s th anth es e condon e ,th e nod e so fth e10 -nod ec lu s t e r h av elow e rcompu t ingcapa c i tythantho s einth e6 -nod e c lu s t e r .Th i si sb e cau s eth eha rdwa r econfigu ra t iono fth e 1 0 -nod ec lu s t e ri s wo r s ethantha to fth e6 -nod ec lu s t e r . In o th e r wo rd s ,th ep ro c e s so r sinth e10 -nod ec lu s t e r a r eP en t iumI I CPU s ;th e6 -nod ec lu s t e rr e l i e sonIn t e l C e l e ron4 5 0torunth eb en chma rk s .S e cond ,compa r ed w i thth e1 0 -nod es lowc lu s t e r ,th e6 -nod efa s tc lu s t e r a l low sth ei t e rtandth e wo rktb en chma rk stosp end sm a l l e rt im ep e r iod sd ea l ing w i thth es e cu r i ty modu l e s in MP ICH 2 . Fo r examp l e , onth e6 -nod efa s tc lu s t e r , i t e rtand wo rktsp endapp rox ima t e ly7. 1%and3. 2% o fth eb en chm a rk s ’to ta lex e cu t iont im e sinth e AES ba s eds e cu r i ty modu l e s(S e eF ig .8 (a ) ) . Onth e10 -nod e s lowc lu s t e r ,th e AES modu l e sa c coun tfo rabou t35. 5% and39. 3%o fth e s etwo b en chma rk s ’to ta l ex e cu t ion t im e s (S e eF ig .1 3 (a ) ) . Th i rd ,th ep e r fo rman c eo f AES b a s ed MP ICH 2onth efi r s tc lu s t e ri sv e ryc lo s etotha t o f MP ICH 2 wh en m e s sag es iz ei ssma l l e rthan2 KB . Th eabov eob s e rva t ion sind i ca t etha tg iv en m e s sag e in t en s iv e MP Iapp l i ca t ion s , on ecanimp rov eth ep ro c e s so rcompu t ingcapa c i tyo fac lu s t e rtosub s tan t ia l ly r edu c eth epo r t iono ft im e(ou to fth eapp l i ca t ion s ’to ta l ex e cu t iont im e )sp en tinp ro c e s s ings e cu r i ty modu l e sin ES -MP ICH 2 . 6 .2 .3 IMB :In te l MP IBenchma rks F ig s .1 4 ( a ) -F ig .1 4 (d )d ep i c tth ep e r fo rman c eo fth eP ing Pong ,P ingP ing ,S endR e cv ,andEx chang eb en chma rk sin IMB .Th eto t a lex e cu t iont im e so fth efou rIMBb en ch ma rk sin c r e a s e sw i thin c r ea s ing m e s sag es iz e .Compa r ed w i th MP ICH 2 ,th eex e cu t iont im eo fES -MP ICH2i s mo r e s en s i t iv eto m e s s ag es iz e . Mo r eimpo r tan t ly ,F ig s .14 (a ) F ig .1 4 (d )d emon s t ra t etha t wh en ES -MP ICH2i sd e p loy edonas lowc lu s t e r ,ES -MP ICH2p r e s e rv e sm e s sag e confid en t i a l i ty bysub s tan t ia l ly d eg rad ingth ep e r fo r - man c eo fth eo r ig ina l MP ICH2 .Bycompa r ingth eIn t e l b en chma rkp e r fo rman c eonbo thth e6 -nod ec lu s t e r( s e e F ig s .9 (a ) -F ig .9 (d )and10 -nod ec lu s t e r s( s e eF ig s .14 (a ) F ig .14 (d ) ) ,w e ob s e rv etha tth ep e r fo rman c e gap b e tw e en MP ICH2and ES -MP ICH2onth efa s tc lu s t e ri s mu chsma l l e rthanth ep e r fo rman c e gap onth es low c lu s t e r .Animp l i ca t iono fth i sob s e rva t ioni stha ts e cu r i ty ov e rh eadinES -MP ICH2canb es ign ifi can t lyr edu c edby d ep loy ingES -MP ICH2inah igh endc lu s t e r . 7 RELATED WORK Me s sage Pa s s ingIn te r fa ce . Th e M e s sag e Pa s s ingIn t e r fa c es tanda rd(MP I )i sa m e s sag e pa s s ingl ib ra ry s tanda rdu s edfo rth ed ev e lopm en to fm e s sag e -pa s s ing pa ra l l e lp rog ram s[15 ] . Th egoa lo f MP Ii stofa c i l i ta t e ane ffi c i en t ,po r tab l e ,andfl ex ib l es tanda rdfo rpa ra l l e l p rog ram su s ing m e s sag epa s s ing . MP ICH2-d ev e lop ed byth eA rgonn e Na t iona lLabo ra to ry-i son eo fth e mo s t popu la rand w id e ly d ep loy ed MP Iimp l em en ta t ion sin c lu s t e rcompu t ingenv i ronm en t s . MP ICH2p rov id e san imp l em en ta t iono fth e MP Is tanda rd wh i l esuppo r t ing ala rg eva r i e tyo fcompu ta t ionandcommun i ca t ionp la t fo rm sl ik ecommod i tyc lu s t e r s ,h igh -p e r fo rman c ecom pu t ingsy s t em s ,andh igh sp e edn e two rk s[14 ] . A sea r lya s1997 ,B r igh tw e l le ta l .f romth e Sand ia Na t iona l Labo ra to ryin s igh t fu l ly po in t ed ou t ba r r i e r s toc r ea t ingas e cu r e MP If ram ewo rk[3 ] . Th e ba r r i e r s in c lud econ t ro land da tainadd i t iontoc ryp tog raph i c i s su e s .Inas e cu r e MP I ,bo thcon t ro landda ta m e s sag e s mu s tb ep ro t e c t edf romunau tho r iz eda c c e s so fa t ta ck e r s and ma l i c iou su s e r s .A l thoughth e r ei sa w id erang e o fimp l em en ta t ion so fth e MP I and MP I -2s tanda rd s ( e .g . , MP ICHand MP ICH2a r etwof r e e ly-ava i lab l eim p l em en ta t ion sf romth eA rgonn e Na t iona lLabo ra to ry ) , th e r ei s ala ck o fs e cu r e MP If ram ewo rk sd ev e lop ed fo rla rg e s ca l ec lu s t e r sd i s t r ibu t eda c ro s sw id ea r ean e t wo rk s . Da ta confiden t ia l i tyin MP I I /O . P rabhaka re ta l . d e s ign ed andimp l em en t ed a s e cu r ein t e r fa c e ca l l ed MP IS e cI/Ofo rth e MP I I/Of ram ewo rk[23 ] . MP IS e c I/Op r e s e rv e sth eadvan tag e so fbo thpa ra l l e lI/Oand da taconfid en t ia l i ty w i thou ts ign ifi can t lyimpa c t ingp e r fo rman c eo f MP I app l i ca t ion s .I ti sfl ex ib l efo r MP I 1 2 IMBP ingPong 7 10 IMBP ingP ing 8 10 O r ig ina l MP I AES 3DES 6 10 7 10 O r ig ina l MP I AES 3DES 6 10 5 10 5 T ime(ms ) T ime(ms ) 10 4 10 3 10 4 10 3 10 2 10 2 10 1 10 1 10 0 10 0 0 2 4 6 8 10 12 n 14 16 18 20 10 0 22 5 MsgS ize2 By tes (a )P ingPong 7 10 O r ig ina l MP I AES 3DES 7 10 6 10 20 5 10 O r ig ina l MP I AES 3DES 6 5 T ime(m s ) 10 T ime(m s ) 15 IMBE x changep ro ce s s#=10 8 10 10 4 10 4 10 3 10 2 3 10 1 10 10 2 10 1 10 0 10 0 n (b )P ingP ing IMBSendRecvp rocess#=10 8 10 10 M sgS i ze2 B y te s 0 5 10 MsgS ize2n By tes 15 20 ( c )S endR e cv 10 0 5 10 M sgS i ze2n B y te s 15 20 (d )Ex ch ang e F ig .1 4 :In t e l MP IB en chma rk s ,P ingPongandP ingP inga r eS ing l eT ran s f e rB en chma rk s ,S endR e cvandEx chang e a r ePa ra l l e lB en chma rk son10 -nod eC lu s t e ro fP en t iumI I p rog r amm e r sto manua l lys e ten c ryp t ionru l e sin MP IS e c I/O .D a t ac anb een c ryp t edand w r i t t enon to d i sk sin MP IS e cI/O ,th enen c ryp t edda tacanb er eadf romth e d i sk sb e fo r eb e ingd e c ryp t ed .Th e r ea r etwoin t e r e s t ing f e a tu r e so f MP IS e cI/O .F i r s t , MP IS e cI/Op rog ramm e r s n e edtoc a r e fu l lys e tupen c ryp t ionandd e c ryp t ionru l e s inth e i r MP Ip rog ram s .O th e rw i s e ,som eda ta mayb ee i th e rs to r edond i sk sw i thou ten c ryp t iono rr ead w i thou t d e c ryp t ionanda sar e su l t ,th e MP Ip rog ram sa r eunab l e tofun c t ionp rop e r lyun t i lth eru l e sa r es e tinaco r r e c t way .S e cond , MP IS e ci sno tcomp l e t e lycompa t ib l ew i th non s e cu r e MP Il ib ra r i e s .Ino th e r wo rd s ,p r e s e rv ingda ta confid en t i a l i tyin MP IS e cI/Oi sno tt ran spa r en tto MP I app l i c a t ionp rog r amm e r s . On eha sto mod i fyth esou r c e cod eo fconv en t iona l MP Ip rog ram stoimp rov es e cu r i ty o fth e MP Ip rog r am s . Apa r tf romupda t ingth esou r c e cod eo fth e MP Ip rog ram sb e fo r e MP IS e cI/Ocanb e u s ed p rop e r ly ,d i sk r e s id en t da ta mu s tb e ma rk eda s en c ryp t edo run en c ryp t ed . B lo ckc iph e r s .Th e Da ta En c ryp t ionS tanda rd(DES ) p rov id e sar e l a t iv e lys imp l em e thodo fen c ryp t ion .3DES en c ryp t sd a t ath r e et im e sin s t eado fon eu s ingth e DES s tand a rd[ 4 ] . 3DESi sa b lo ck andsymm e t r i cc iph e r cho s enbyth eU .S . Na t iona lBu r eauo fS tanda rd sa san o ffi c i a lF ed e r a lIn fo rma t ionP ro c e s s ingS tanda rdin1976 . 3DESin c r ea s e sth ek eys iz eo f DESto p ro t e c taga in s t b ru t efo r c ea t ta ck sw i thou tr e ly ing onany n ew b lo ck c iph e ra lgo r i thm . Aha rdwa r eimp l em en ta t iono f3DES i ss ign ifi can t lyfa s t e rthanth eb e s tso f twa r eimp l em en ta t ion so f3DES[10 ][16 ] . Aso f twa r eimp l em en ta t iono f 3DES wa sin t eg ra t edinES -MP ICH2 . Aha rdwa r e3DES cansub s tan t ia l lyimp rov ep e r fo rman c eo f3DES -ba s ed ES -MP ICH2 . In Nov emb e r2001 ,th esymm e t r i cb lo ckc iph e rR i jnda e l wa ss tanda rd iz ed byth e Na t iona lIn s t i tu t eo f S tanda rd sandT e chno logya sth e Advan c edEn c ryp t ion S tanda rd(AES )[6 ] . AES-th esu c c e s so ro fth e Da taEn c ryp t ionS tanda rd(DES )-ha sb e en w id e lyemp loy edto p r ev en tconfid en t ia lda taf romb e ingd i s c lo s edbyunau tho r iz eda t ta ck e r s .AEScanb eu s edinh igh -p e r fo rman c e s e rv e r sa sw e l la ssma l l and mob i l econ sum e rp rod u c t s . AESi sth ep r e f e r r edc ryp tog raph i ca lgo r i thmto b eimp l em en t edinES -MP ICH2 , wh i ch wa sbu i l tba s ed onsymm e t r i cb lo ckc iph e r s .A l though AESin t rodu c e s ov e rh ead du etoadd i t iona ls e cu r i ty op e ra t ion sin ES MP ICH2 ,th eov e rh eadcau s edby AESin ES -MP ICH2 canb es ign ifi can t lyr edu c edby AESha rdwa r ea r ch i t e c tu r e s( s e e[20 ]fo rd e ta i l so fah igh lyr egu la rands ca lab l e AESha rdwa r ea r ch i t e c tu r e ) . Se cu r i tyenhan cemen tinc lu s te r s .Th e r ea r es ev e ra l 1 3 r e s e a r ch wo rk sfo cu s ing ons e cu r i ty enhan c em en tin commod i ty c lu s t e r s . Fo r examp l e ,L e e and K im d e v e lop edas e cu r i tyf ram ewo rkinth eInfin iBanda r ch i t e c tu r e( IBA )[ 1 8 ] . Fo rconfid en t ia l i ty and au th en t i ca t ion ,L e e and K im p ropo s edth e pa r t i t ion l ev e l and QP l ev e ls e c r e tk ey manag em en ts ch em e s . Th es e cu r i tyinIBAi simp rov ed w i th m ino r mod ifi ca t ion sto th eIBAsp e c ifi c a t ion . Ram su r runandSoy jaudahcon s t ru c t ed a h igh ly ava i lab l et ran spa r en tL inux c lu s t e r s e cu r i ty mod e l , wh i cho f f e r san ewapp roa chtoenhan c ingc lu s t e rs e cu r i ty[24 ] . Ko en ige t .a l .imp l em en t eda too lth a t mon i to r sp ro c e s s e sa c ro s scompu t ingnod e sin ac lu s t e r[ 1 7 ] . Th etoo ld e l iv e r sr ea l t im ea l e r t s wh en th e r ea r eimm ed ia t eth r ea t s .S im i la r ly , Pou rzand ie ta l . inv e s t ig a t edth es e cu r i tyi s su e so fd e t e c t ingth r ea t sand h az a rd sind i s t r ibu t edc lu s t e r s[22 ] .Th ea fo r em en t ion ed s e cu r i tyso lu t ion sd ev e lop edfo rc lu s t e r sa r einad equa t e tod i r e c t lysuppo r ts e cu r i ty s en s i t iv e MP Ip rog ram s ,b e c au s eth eex i s t ings e cu r i tyso lu t ion sg en e ra l lyr equ i r eap p l i c a t iond ev e lop e r stoimp l em en ts e cu r i tyfun c t iona l i ty inth e i r MP Ip rog ram s . 8 CONCLUS IONS AND FUTURE WORK T oadd r e s sth ei s su eo fp rov id ingconfid en t ia l i tys e rv i c e s fo rl a rg e s c a l ec lu s t e r sconn e c t edbyanop enun s e cu r ed n e two rk ,w ea ima timp rov ingth es e cu r i tyo fth em e s s ag ep a s s ingin t e r fa c e(MP I )p ro to co lbyen c ryp t ingand d e c ryp t ing m e s s ag e scommun i ca t edamongcompu t ing nod e s .Inth i ss tudy ,w eimp l em en t edth e ES -MP ICH2 f ram ewo rk , wh i chi sba s edon MP ICH2 .ES -MP ICH2i s as e cu r e ,comp a t ib l e ,and po r tab l eimp l em en ta t ion o f th em e s s ag ep a s s ingin t e r fa c es tanda rd .Compa r ed w i th th eo r ig in a lv e r s iono f MP ICH2 ,ES -MP ICH2p r e s e rv e s m e s s ag econfid en t ia l i tyin MP Iapp l i ca t ion sbyin t eg ra t ingen c ryp t iont e chn iqu e sl ik e AESand3DESin toth e MP ICH 2l ib r a ry . In l igh to f ES -MP ICH2 ,p rog ramm e r s can ea s i ly w r i t es e cu r e MP Iapp l i ca t ion sw i thou tadd i t iona lsou r c e cod efo rd a t a confid en t ia l i ty p ro t e c t ioninop en pub l i c n e two rk s . Th es e cu r i tyf ea tu r eo f ES -MP ICH2i s en t i r e lyt r an sp a r en tto MP Ip rog ramm e r sb e cau s een c ryp t ionand d e c ryp t ionfun c t ion sa r eimp l em en t eda tth e ch ann e l l ev e linth e MP ICH2l ib ra ry . MP I -app l i ca t ion p rog r amm e r sc anfu l lyconfigu r eanyconfid en t ia l i tys e r v i c e sin MP ICH I 2 ,b e cau s eas e cu r edconfigu ra t ionfi l ein ES -MP ICH 2o f f e r sth ep rog ramm e r sfl ex ib i l i tyinchoo s ing any c ryp tog raph i cs ch em e s and k ey ss eam l e s s ly in co rpo r a t edinES -MP ICH2 .B e s id e sth eimp l em en ta t ion o f AESand3DESin ES -MP ICH2 ,o th e rc ryp tog raph i c a lgo r i thm sc anb er ead i lyin t eg ra t edinth eES -MP ICH2 f ram ewo rk .W eu s edth eSand ia M i c roB en chma rk sand th eIn t e l MP Ib en chma rk stoeva lua t eandana lyz eth e p e r fo rm an c eo f MP ICH2 . Confid en t i a l i tys e rv i c e sin ES -MP ICH2 doin t rodu c e add i t ion a lov e rh eadb e cau s eo fs e cu r i tyop e ra t ion s .In th ec a s eo fsm a l lm e s sag e s ,th eov e rh eadin cu r r edby th es e cu r i tys e rv i c e si s ma rg ina l .Th es e cu r i tyov e rh ead cau s ed by AESand3DES b e com e s mo r ep ronoun c ed inES -MP ICH2 w i thla rg e rm e s sag e s( e .g . ,th em e s sag e s iz ei sla rg e rthan256 KB ) . Ou rexp e r im en ta lr e su l t s showtha tth es e cu r i tyov e rh eadinES -MP ICH2canb e s ign ifi can t lyr edu c edbyh igh -p e r fo rman c ec lu s t e r s .Fo r examp l e ,th eov e rh eadadd edby AESinES -MP ICH2i s r edu c edby mo r ethanha l f wh enth e6 -nod ec lu s t e ro f In t e lC e l e roni su s edin s t eado fth e10 -nod ec lu s t e ro f In t e lP en t iumI I .Inadd i t ionto h igh endc lu s t e r s ,th e fo l low ingtwoso lu t ion scan b eapp l i edtofu r th e rr e du c edov e rh eadcau s edbyconfid en t ia l i tys e rv i c e sinES MP ICH2 .F i r s t , AES/3DES ha rdwa r eimp l em en ta t ion s canlow e rs e cu r i ty ov e rh eadin ES -MP ICH2 .S e cond , s e cu r i tyco -p ro c e s so r scanh id eth eov e rh eadbya l low ing th een c ryp t ionandd e c ryp t ionp ro c e s s e stob eex e cu t ed inpa ra l l e lw i thth em e s sag epa s s ingp ro c e s s e s . W ea r ecu r r en t lyinv e s t iga t ingva r i e sm ean so fr edu c ings e cu r i tyov e rh eadinES -MP ICH2 .Fo rexamp l e ,w e p lantos tudyi f mu l t i co r ep ro c e s so r scansub s tan t ia l ly low e rth e ov e rh ead o fconfid en t ia l i tys e rv i c e sin ES MP ICH2 . Ano th e rin t e r e s t ing d i r e c t ionfo rfu tu r e wo rki sto con s id e rs ev e ra ls t rongande ffi c i en tc ryp tog raph i ca lgo r i thm sl ik eth eE l l ip t i cCu rv eC ryp tog raphy(ECC )inES MP ICH2 .S in c eECCi sane ffi c i en tandfa s tc ryp tog raph i c so lu t ion ,bo thth ep e r fo rman c eandth es e cu r i tyo f ES MP ICH2 a r el ik e lyto b eimp rov ed byin co rpo ra t ing ECC . Ath i rd p rom i s ing d i r e c t ionfo rfu r th e r wo rki sto in t eg ra t een c ryp t ionandd e c ryp t iona lgo r i thm sino th e r commun i ca t ionchann e l sl ik eSHMEMandInfin iBandin MP ICH2b e cau s eanin c r ea s ingnumb e ro fcommod i ty c lu s t e r sa r ebu i l tu s ings tanda lon eandadvan c ed n e t wo rk ssu cha sInfin ibandand My r in e t . Th e cu r r en tv e r s ion o f ES -MP ICH2i sfo cu s ed on s e cu r ingth et ran sm i s s ioncon t ro lp ro to co l(TCP )conn e c t ion sonth eIn t e rn e t ,b e cau s ew eadd r e s s edth eda tacon fid en t ia l i tyi s su e song eog raph i ca l ly d i s t r ibu t edc lu s t e r compu t ingsy s t em s .Inadd i t iontoth e MP Il ib ra ry ,o th e r pa ra l l e lp rog ramm ingl ib ra r i e sw i l lb einv e s t iga t ed .Can d ida t el ib ra r i e sin c lud eth esha r ed m emo rya c c e s sl ib ra ry (SHMEM )andth er emo t ed i r e c tm emo rya c c e s sl ib ra ry (RDMA ) .W ep lantop rov id econfid en t ia l i tys e rv i c e sin th eSHMEMandRDMAl ib ra r i e s . La s t bu t no tl ea s t , w e w i l l quan t i ta t iv e ly eva lua t e th ep e r fo rman c eo fin t eg r i tych e ck ings e rv i c e stha ta r e in co rpo ra t edin toth eES -MP ICH2f ram ewo rk .Th egoa l o fd ev e lop ingth ein t eg r i tych e ck ings e rv i c e si stor edu c e ov e rh eado fth es e rv i c e sinES -MP ICH2 . AVA ILAB IL ITY Th e ex e cu tab l e b ina r i e s and sou r c e cod e o f ES -MP ICH2 a r e f r e e ly ava i lab l e , a long w i th do cum en ta t ion and b en chma rk sfo r exp e r im en ta t ion , a t h t tp ://www . eng .aubu rn . edu/∼xq in/ so f twa r e/ e s mp i ch2/ . 1 4 ACKNOWLEDGMENT Th e wo rkr epo r t edinth i s pap e r wa ssuppo r t ed by th e US N a t ion a lS c i en c e Founda t ion und e r G ran t s CCF 0 8 4 5 2 5 7 (CAREER ) , CNS -0757778 (CSR ) , CCF 0 74 2 1 8 7(CPA ) , CNS -0917137(CSR ) , CNS -0831502(Cy b e rT ru s t ) , CNS 0 855251(CR I ) , OC I -0753305(C I -TEAM ) , DUE 0 8 3 7 3 4 1(CCL I ) ,and DUE -0830831(SFS ) ,a sw e l l a s Aubu rn Un iv e r s i ty und e r as ta r tup g ran t and a g i f t(Numb e r2 0 05 -04 -070 )f romth eIn t e l Co rpo ra t ion . Moh amm ed A lgh amd i ’ sr e s ea r ch wa s suppo r t ed by th eK ing Abdu l az iz C i tyfo rS c i en c eand T e chno logy (KACST )and AL -Baha Un iv e r s i ty . REFERENCES [1 ] Na t iona lS e cu r i ty Ag en cy . Na t iona l po l i cy onth eu s eo fth e advan c eden c ryp t ions tanda rd(a e s )top ro t e c tna t ion a ls e cu r i ty sy s t em sandna t iona ls e cu r i tyin fo rma t ioncn s spo l i cyno .1 5f a c t sh e e tno .1 ,Jun e2003 . [2 ] Ian F .B lak e ,G .S e rou s s i , and N .P . Sma r t . E l l ip t i ccu r v e sin c ryp t og r aphy . Camb r idg e Un iv e r s i tyP r e s s ,N ew Y o rk , NY , USA , 1999 . [3 ] RonB r igh tw e l l , Dav idS .G r e enb e rg ,B r ianJ . Ma t t ,and G eo rg eI . Dav ida . Ba r r i e r stoc r ea t ingas e cu r e mp i ,1997 . [4 ] D .Copp e r sm i th ,D .B .John son ,S .M . Ma tya s ,T .J .W a t son , DonB . John son ,andS t eph en M . Ma tya s .T r ip l ed e sc iph e rb lo ckch a in ing w i thou tpu tf e edba ck ma sk ing ,1996 . [5 ] In t e lCo rpo ra t ion .In t e l mp ib en chma rk su s e rgu id eand m e thod o logyd e s c r ip t ion ,2008 . [6 ] Joan Da em enand V in c en tR i jm en . Th ed e s igno fr i jnd a e l ,2 0 0 2 . [7 ] Do ro thyE .D enn ing .S e cu r ep e r sona lcompu t inginanin s e cu r e n e two rk .C ommun . ACM,22 (8 ) :476–482 ,1979 . [8 ] Ja ckJ . Donga r ra ,S t ev e W .O t to , Ma r cSn i r ,and Dav id W a lk e r . Anin t rodu c t iontoth e mp is tanda rd .T e chn i ca lr epo r t ,Knoxv i l l e , TN , USA ,1995 . [9 ] W . Eh r sam ,S . Ma tya s ,C . M ey e r ,and W . Tu chman . Ac ryp tog raph i ck ey manag em en ts ch em efo rimp l em en t ingth ed a t a en c ryp t ions tanda rd .IBMSy s t em sJ ou rn a l ,17 (2 ) :106– 1 2 5 ,1 9 7 8 . [10 ] AdamJ .E lb i r t , W .Y ip ,B . Ch e twynd ,and C . Paa r . Anfpg a ba s edp e r fo rman c eeva lua t iono fth ea e sb lo ckc iph e rc and id a t e a lgo r i thm fina l i s t s . IEEE T r an s .V e ry L a r g eS c a l eIn t e g r .S y s t . , 9 (4 ) :545–557 ,2001 . [11 ]Ian Fo s t e r ,N i cho la sT . Ka ron i s , Ca r lK e s s e lman ,G r eg Ko en ig , andS t ev enTu e ck e . As e cu r ecommun i ca t ion sin f ra s t ru c tu r efo r .6THIEEE h igh -p e r fo rman c ed i s t r ibu t edcompu t ing .InINPROC SYMP . ON H IGHPERFORMANCE D ISTR IBUTEDCOMPUT ING, pag e s125–136 .IEEE Compu t e rSo c i e tyP r e s s ,1996 . [12 ] A lG e i s t ,W i l l iam G ropp ,S t ev en Hu s s -L ed e rman , And r ewLum s da in e ,Ew ingL .Lu sk ,W i l l iamSaph i r ,T onySk j e l lum ,and M a r c Sn i r . Mp i -2 : Ex t end ingth em e s sag e -pa s s ingin t e r fa c e . InEu r o P a r’96 :P r o c e ed ing so fth eS e c ondIn t e rn a t i on a l Eu r o -P a rC on f e r en c e ,pag e s128–135 ,London , UK ,19 9 6 .Sp r ing e r onP a r a l l e lP r o c e s s ing V e r lag . [13 ]R .G rabn e r ,F .M i e tk e ,and W .R ehm .Imp l em en t ingan mp i ch 2 chann e ld ev i c eov e rvap ioninfin iband .InP a r a l l e land D i s t r i bu t e d ag e s P r o c e s s ingSymp o s ium ,2004 .P r o c e ed ing s .18 thIn t e rn a t i on a l, p 184– , Ap r i l2004 . [14 ]R en G rabn e r ,F rank M i e tk e ,and W o l fgang R ehm .Imp l em en t ing an mp i ch -2chann e ld ev i c eov e rvap ioninfin iband .P a r a l l e lan d D i s t r i bu t edP r o c e s s ingSymp o s ium ,In t e rn a t i on a l,9 :184a ,2 0 0 4 . [15 ] W i l l iam G ropp , Ew ing Lu sk , Na than Do s s ,and An thonySk j e l lum . Ah igh -p e r fo rman c e , po r tab l eimp l em en ta t iono fth e mp i a r a l l e lC ompu t .,2 2 ( 6 ) : 7 8 9 – m e s sag epa s s ingin t e r fa c es tanda rd . P 828 ,1996 . [16 ]P . Hama la in en ,M . Hann ika in en ,T . Hama la in en ,andJ .S a a r in en . Configu rab l eha rdwa r eimp l em en ta t iono ft r ip l e -d e sen c ryp t ion a lgo r i thmfo rw i r e l e s slo ca la r ea n e two rk . InICASSP’ 0 1 :P r o c e ed ing so fth eA c ou s t i c s ,Sp e e ch ,andS ign a lP r o c e s s ing ,2 0 0 .onIEEE In t e rn a t i on a lC on f e r en c e ,pag e s1221–1224 , W a sh ing ton , DC , USA , 2001 .IEEE Compu t e rSo c i e ty . [ 1 7 ]G r ego ry A . Ko en ig ,X in M eng , Ad amJ .L e e , M i cha e lT r ea s t e r , N ad i rK iy an c l a r , and W i l l i am Yu r c ik . C lu s t e rs e cu r i ty w i th nv i s ion c c :P ro c e s s mon i to r ingbyl ev e r ag ingem e rg en tp rop e r t i e s . lu s t e rC om pu t in gan dG r i d(CCG r i d ,2 005 . InInIEEEC [ 1 8 ] M anh e eL e eandEunJungK im . Acomp r eh en s iv ef ram ewo rkfo r enh an c ings e cu r i tyininfin ib anda r ch i t e c tu r e .IEEET r an s .P a r a l l e l D i s t r i b .S y s t .,1 8 ( 1 0 ) : 1 3 9 3 – 1 4 0 6 ,2 0 0 7 . [ 1 9 ]J .L iu ,W .J i ang ,P . Wy cko f f ,D .K .P and a ,D .A sh ton ,D .Bun t ina s , W .G ropp ,andB .T oon en .D e s ignandimp l em en t a t iono f mp i ch2 ov e rinfin ib and w i thrdm asuppo r t . InP a r a l l e land D i s t r i bu t ed P r o c e s s in gS ym p o s ium ,2 0 0 4 .P r o c e e d in g s .1 8 t hIn t e rn a t i on a l, pag e s 1 6 – , Ap r i l2 0 0 4 . [ 2 0 ]S t e f an M ang a rd , M an f r ed A ign e r ,and S and r a Dom in iku s . A h igh lyr egu l a rands c a l ab l ea e sh a rdw a r ea r ch i t e c tu r e .IEEET r an s . C om pu t .,5 2 ( 4 ) : 4 8 3 – 4 9 1 ,2 0 0 3 . [ 2 1 ]S .M a ty a sandC .M ey e r .G en e r a t ion ,d i s t r ibu t ion ,andin s ta l la t ion y s t em sJ ou rn a l ,1 7 ( 2 ) : 126–137 ,1978 . o fc ryp tog r aph i ck ey s .IBMS [ 2 2 ] M .Pou r z and i ,D . Go rdon ,W .Yu r c ik ,and G .A . Ko en ig .C lu s t e r s ands e cu r i ty :d i s t r ibu t eds e cu r i tyfo rd i s t r ibu t edsy s t em s .InC lu s t e rC om pu t in gan dt h eG r i d ,2 0 0 5 . CCG r i d2 0 0 5 .IEEEIn t e rn a t i on a l S ym p o s iumon,vo ag e s9 6 – 1 0 4V o l .1 ,M ay2005 . lum e1 ,p [ 2 3 ]R amy aP r abh ak a r , Ch r i s t in aP a t r i ck , and M ahmu t Kand em i r . Mp i s e ci/o :P rov id ing d a t aconfid en t i a l i tyin mp i i/o . C lu s t e r C om pu t in gan dt h eG r i d ,IEEEIn t e rn a t i on a lS ym p o s iumon,0 :388– 3 9 5 ,2 0 0 9 . [ 2 4 ]V .R am su r runand K .M .S .Soy j aud ah . Ah igh lyava i lab l et ran s p a r en tl inuxc lu s t e rs e cu r i ty mod e l .InP e r f o rm an c e ,C ompu t ingand C ommun i c a t i on sC on f e r en c e ,2 0 0 8 .IPCCC2 0 0 8 .IEEEIn t e rn a t i on a l, p ag e s6 9 – 7 6 ,D e c .2 0 0 8 . [ 2 5 ] Subh a sh S a in i , Rob e r tC io t t i ,B r i an T .N . Gunn ey , Thoma sE . Sp e l c e ,A l i c e Kon ig e s , Don Do s s a ,P an ag io t i s Adam id i s , Ro l f R ab en s e i fn e r , Sun i lR .T iyy agu r a , and M a t th i a s Mu e l l e r . P e r fo rm an c e ev a lu a t ion o f sup e r compu t e r su s ing hp c c andimb .C om pu t .S y s t .S c i . ,7 4 ( 6 ) : 9 6 5 – 9 8 2 ,2 008 . b en chm a rk s .J [ 2 6 ]S . Sh ep l e r ,B .C a l l agh an ,D . Rob in son ,R . Thu r low ,C .B eam e , M .E i s l e r ,and D . Nov e ck . N e two rk fi l esy s t em(n f s )v e r s ion4 p ro to co l .2 0 0 3 . [ 2 7 ] Dun c anS .W ong ,H e c to r HoFu en t e s ,and Agn e sH . Chan . Th e p e r fo rm an c em e a su r em en to fc ryp tog r aph i cp r im i t iv e sonpa lm d ev i c e s . InIn P r o c e e d in g so ft h e1 7 t h Annu a lC ompu t e rS e cu r i ty A p p l i c a t i on sC on f e r en c e(ACSAC,p ag e s9 2 – 1 0 1 ,2 0 01 .
