Inference of Organizational Tensions
Using Email Social Network Analysis
April 26, 2010
Ben Collingsworth
Department of Computer Sciences, Florida Institute of Technology
email:bcolling@fit.edu
Proposal
Utilize social network analysis on network derived from
email activity to detect social tension within
organizations.
Email is a good data source for analysis because:
• Reflects social activity not technology.
• Self-organized system where whole is bigger than sum of
parts.
• Activity reflects anticipation of events; as a result, detection of
activity could provide early warning mechanism.
• Emergent characteristics are robust, resistant to small
changes in system [1].
Motivation
Unexpected failure of many financial organizations
indicates more diagnostic tools are needed:
• Citigroup
• Bear Stearns
• General Motors
• Fannie Mae
• AIG
Social Network Definition
Social networks are complex networks that exhibit the
following properties [2]:
• Heavy–tailed degree distribution.
• High level of clustering.
• High level of homophily.
Case Study - Enron
Enron email dataset made public by Federal Energy
Regulatory Commission (FERC) during investigation of
2002 collapse:
• Available for download at http://www-2.cs.cmu.edu/~enron
• Highly pruned -148 employees (less than one percent of
Enron personnel at peak employment).
• Full set of email folders available for each participant.
• Positions of 113 of participants are posted.
Network Views
“Partial” Network - Consist of nodes and edges of
participants whose email folders are known.
• 148 nodes
• 783 edges
“Full” Network – Includes partial network plus additional
participants discerned from known email folders.
•
•
14647 nodes
162,160 edges
Partial Network Visualization
Full Network Visualization
Social Network Property Comparison
Network
Nodes
Edges
Avg Path
Len
Clustering
Coeff
Degree Prob
Coeff
Enron Full
14647
181195
3.5
0.334
1.36
Film Actors
Collaboration [2]
449913
25516482
3.5
0.79
2.3
Astrophysics
Collaboration [3]
16706
121251
4.8
0.726
1.94
Pruned
Astrophysics
Collaboration
2083
2580
7.4
0.327
1.33
Random [4]
59912
10.1
0.00005
N/A
Experimental Results
Social network analysis will show:
1. Social network activity correlates to events occurring
at Enron.
2. Network activity tends to precede actual events.
3.Correlation is stronger within homopholous groups.
Enron Monthly Event Score
Month
Event
Score
July 2000
Enron announces that its Broadband unit (EBS) has joined forces with Blockbuster to supply video-on-demand.
4
Aug 2000
Azurix (water trading division) collapses after investment of $3 billion, Rebecca Mark, CEO of Azurix, resigns.
4
Sept 2000
Stock price levels o at all-time high of $90 per share.
2
Oct 2000
Enron lawyers alert senior company executives to potentially criminal trading strategies in use in the California electricity market.
2
Nov 2000
Raptor partnerships expanded, Enron reports pretax earning for last two quarters of $650 million, 80 percent of profits derived from unsecured Raptors.
Jeff Skilling sells $10 million in Enron Stock.
3
Dec 2000
Enron announces that President and COO Jeff Skilling will take over as chief executive in February. Kenneth Lay will remain as chairman.
6
Jan 2001
Severe shortages of electricity in California lead to rolling blackouts and accusations that Enron manipulated prices. Enron's West Coast power desk has
its most profitable month ever: 254 million dollars in gross profits.
4
Feb 2001
Jeff Skilling made CEO, Kenneth Lay retains chairman post.
5
Mar 2001
After repeated clashes with Jeff Skilling over questionable accounting practices, Cliff Baxter, former head of trading unit and briefly CEO of Enron North
America, resigns. Raptor deals restructured so Enron can record $36.6 million credit reserve loss instead of $504 million. Enron and Blockbuster
announce the cancellation of their video-on-demand deal, Enron reports $111 million in profit from Blockbuster deal even though it never closed. Raptor
deals begin to collapse, Enron takes $500 million charge against earnings to reflect Raptor short falls. Enron treasurer Jeff McMahon approaches Skilling
with grave concerns about Raptor funds. Fortune magazine prints major article about Enron being overpriced. Enron transfers large portions of EES
business into wholesale to hide EES losses. The stock has steadily declined since Skilling became CEO.
8
Apr 2001
Kenneth Lay meets with Vice President Dick Cheney and head of the National Energy Policy Development Group. Lay presents recommendations in eight
policy areas of the Federal Energy Regulatory Commission. Seven of the eight policies are adopted, including the recommendation that price caps on
wholesale energy sales be avoided. This policy allows Enron to continue manipulation the California electricity market. Under pressure to maintain the
illusion of Enron profitability, Skilling makes famous "asshole" remark during stock analyst meeting. Enron top management and external analysts were
stunned when Skilling publically called speculator Richard Grubman an asshole in response to a fair question. It was highly unusual for the CEO of a
Fortune 500 Company to behave this way. People thought this was a sign that Skilling was falling apart from the pressure of maintaining the illusion of
Enron profitability.
7
May 2001
India stops making payments for electricity generated by the huge Enron Dabhol plant after Enron has invested almost $1 billion in the plant. Enron shuts
the plant down in May. Lou Pai, Chairman and Chief Executive Officer of Enron Energy Services resigns, selling 338,897 shares of Enron stock, and
exercising stock options that resulted in the sale of 572,818 shares to the open market. The sales resulted in an estimated $200 million. The Dabhol
crises was a "nightmare" for Enron. Kenneth Lay personally chaired a task force to deal with the problem.
8
Enron Monthly Event Score
Month
Event
Score
Jun 2001
Skilling goes to California to deliver a speech at San Francisco's Common Wealth Club. The venue is surrounded by protesters. Security personal warn
Skilling they may not be able to protect him. Skilling is later hit in the face with pie. The event shows that public awareness of Enron corruption is
growing.
4
July 2001
Enron officially acknowledges failure of Broadband division, reporting \$102 million operating loss, merges broadband with wholesale. Chief
executive of Enron Broadband Services Ken Rice sells 386,000 Enron shares. The exposure of the Broadband division as a failure was a big setback.
Skilling had repeatedly hyped Broadband as "The Big Enchilada" that was going to take Enron to the next level of success and profitability.
6
Aug 2001
At board meeting Chief Risk Officer Rick Buy outlines disaster scenario if Enron’s stock starts to fall - All SPEs (special purpose entities created to
isolate financial risk) will crash. Skilling resigns Aug 14 after only six months as CEO, cites "personal reasons". Kenneth Lay re-takes CEO position Aug
15. Sharron Watkins sends famous anonymous letter outlining concerns about potential disaster at Enron to Kenneth Lay. Kenneth Lay emails
employees, stating "one of my highest priorities is to restore investor confidence in Enron and regain stock value." Skilling's resignation was a
complete shock and surprise. Many employees felt betrayed. They feared Skilling was leaving because he knew Enron was falling apart.
8
Sept 2001
Jeff Skilling sells 500,000 Enron shares worth more than $15 million. The sale of this huge amount of stock supports the fears that Skilling knew Enron
was about to collapse when he resigned.
4
Oct 2001
Arthur Andersen hires law firm to prepare a defense for the company. Arthur Andersen begins shredding documents. Enron announces a third
quarter loss of $618 million. To correct an accounting error on the Raptor partnerships devised by CFO Andrew Fastow, Enron's assets (shareholder
equity) are reduced by $1.01 billion. The Enron 401(k) retirement plan is frozen for administrative changes. Enron announces that the U.S. Securities
and Exchange Commission (SEC) has begun an inquiry into Enron's accounting practices with its partnerships. CFO Andrew Fastow is forced to leave
Enron. Kenneth Lay seeks help from Washington, contacting Chairman of the Federal Reserve Alan Greenspan, Secretary of the Treasury Paul H.
O'Neill, and Secretary of Commerce Donald L. Evans. Washington maintains hands-off policy regarding Enron crisis. Kenneth Lay meets with Dynegy
chairman Charles Watson about possible buyout of Enron. October 2001 was the beginning of the rapid collapse of Enron.
9
Nov 2001
Enron announces it overstated profits by $586 million over five years. The SEC subpoenas Arthur Andersen officials. Dynegy announces it will acquire
Enron for $9 billion. Enron announces the payment of a $690 million note is nearly due as a result of the descent of its credit rating. Dynegy
acquisition negotiations collapse. Enron stock price drops to less than $1 a share. The events of Nov 2001 lead to panic among employees. Their jobs
were in jeopardy and, additionally, many had their life savings frozen in Enron stock while the stock price dropped to almost zero.
10
Dec 2001
Dec 2 Enron files for bankruptcy. Dec 3 Enron lays off 21,000 in Houston. Enron is no longer operating as a viable and functioning corporation. To
employees, to worst case scenario has unfolded.
10
Jan 2001
Enron persists as a corporate entity only to facilitate the government investigation and any possible recovery of investor funds. Email and other
corporate communication is very limited as remaining employees fear government monitoring and potential legal backlash.
2
Email Correlated to Event Score
Pearson Correlation Coefficient
Perfectly Negative
-1
High Negative
> -1 < -0.5
Medium Negative
>= -0.5 < -0.3
Low Negative
>= -0.3 < 0
Absence of a Relationship 0
Low Positive
> 0 <= 0.3
Medium Positive
> 0.3 <= 0.5
High Positive
> 0.5 < 1
Perfectly Positive
1
Email Correlated to Time-Shifted Event Score
Pearson Correlation Coefficient
Absence of a Relationship 0
Low Positive
> 0 <= 0.3
Medium Positive
> 0.3 <= 0.5
High Positive
> 0.5 < 1
Perfectly Positive
1
Focus on Homophily
Activity between alike connected to alike:
Important to analysis because during life changing events people
communicate with well known peers and friends to make sense of
what is going on [5].
Homophilous Group Identification
• Employee Position
• Cliques and Clans
• Clustering Coefficient
Position Messages Correlated to Time-Shifted Event Score
Number of Cliques Correlated to Time-Shifted Event Score
Number of 2-Clans Correlated to Time-Shifted Event Score
Clustering Coefficient Correlated to Time-Shifted Event Score
Correlation to Other Organizational Indicators
• Stock Price - Difference between each month's average stock price
and the average stock price over the entire analysis period.
• News Stories - Number of headlines containing the word “Enron”
per month.
Scores calculated for each month and normalized to value between
zero and ten.
Monthly Stock Price Score Calculation
• Average stock price over analysis period = USD $51.2
• Monthly score = ABS(Avg Stock Price for Month - Avg stock price over analysis period)
• Scores normalized to values between 0 and 10.
Example:
• Average stock price for month of September 2001 = 26.9
• Stock Score = ABS(26.9 - 51.2) = 24.3
• Normalization Divisor = 5.09 from largest stock score of 50.9 in Nov 2001
• Normalized Stock Score = 24.3/5.09 = 4.8
Number of Cliques Correlated to Time-Shifted Stock Score
Monthly News Score Calculation
LexisNexis Academic used to search for the number of headlines
containing the word Enron for each month:
Number of Cliques Correlated to Time-Shifted News Score
Temporal Analysis
Goal – Optimize time parameters for :
• Time shift of event score
• Dataset size
Time Shift Optimization
Clustering Coefficient Correlated to Event Score
Data Size Optimization
Clustering Coefficient Correlated to Event Score
Conclusions
• Successfully demonstrated that shifts in emergent
network properties coincided with real-world events.
• Enron is interesting case study because email logs
span from period of success and prosperity to total
collapse.
• Social network analysis can be used to complement
existing audit and assessment tools.
CompleNet 2009 Response
Future Work
• Further testing on other organizational instances will
test hypothesis.
• Automation and refinement of mechanisms used in
analysis.
• Evolution into “real time” monitoring tool.
• Application as cultural assessment tool.
Bibliography
1. Reka Albert, Hawoong Jeong, and Albert-Laszlo Barabasi. Error and attack
tolerance of complex networks. Nature, 406(6794):378-382, July 2000.
2. M. E. J. Newman. The structure and function of complex networks. SIAM
Review, 45:167-256, 2993.
3. M. E. J. Newman. The structure of scientific collaboration networks. Proc Natl
Acad Sci U S A, 98(2):404-409, January 2001.
4. Holger Ebel, Lutz-Ingo Mielsch, and Stefan Bornholdt. Scale-free topologyof email networks. Phys. Rev. E, 66(3):035103:(1-4), Sep 2002.
5. Patricia M. Sias. Organizing Relationships. Sage Publications, 2009.
6. Ben Collingsworth and Ronaldo Menezes. Identification of social tension in
organizational networks. In Complex Networks, pages 209-223. Springer Berlin /
Heidelberg, May 2009.
7. Ben Collingsworth, Ronaldo Menezes, and Paulo Martins. Assessing
organizational stability via network analysis. In 2009 IEEE Symposium on
Computational Intelligence for Financial Engineering Proceedings, pages 43-50.
IEEE, April 2009.
Analysis Tools
Combination of existing and custom tools:
• Social network analysis – Pajek, UCINET
• Visualization – BioLayout Express 3D, Cytoscape
• Custom – implemented using MySQL and J2EE
Pearson Correlation Coefficient
• Indicates the strength and direction of a linear relationship
between two random variables.
• Used when the variables to be correlated are continuous
(measured on at least an interval scale).
• Assumes the relationship between the two variables
approximates a linear one.
y
y
y
X
Positive Correlation
X
No Correlation
X
Negative Correlation