Profile
Pic
Mike Dodge
Timeline
About
Photos 614
Friends 926
Update Info
About
What’s on your mind?
Client Platform Engineer at Facebook HQ
Mike Dodge
Automation at Hacker School
Cant wait to give my talk @MacSysAdmin
Lives in San Francisco, California
From San Jose, California
Followed by 641 people
More
Self Service / Carrot
So now what?
im done
Im Awesome,
100% of your machines are
managed ...
Right?
Wrong!
What would you do, if you weren't afraid?...
Bash / Casper
Python / Casper
+ Custom Code
Ruby /Munki / Chef
code.google.com/p/munki/
Why Chef ?
Infrastructure as Code
Transparency
Deterministic
Idempotent
Client side computation
+ very Scaleable
Extremely Flexible
What else did we learn?
Tools
Coding Practices
▪
Source Control: Git, Mercurial, SVN
▪
Abstractions
▪
Code linters / Standards(PEP8)
▪
Central code base.
▪
Code Review: Phabricator
▪
Shards ( For slowed roll outs).
▪
Data Tools : Splunk, Sentry
▪
Right tool for the job.
▪
Monitoring / Alerts
▪
Reuse and Share
▪
IRC
Open Source
Autoinit
Launch Deamon Init
Code Sync
github.com/facebook/IT-CPE
Abstractions
Logical grouping
Step up everything once
Conceptually Simple
Easily extendable
*_tools.py
Ensures Consistently api_tools.py
account_tools.py
casper_tools.py
fs_tools.py
shell_tools.py
sys_tools.py
user_tools.py
Pyexec
Sets up your path
Catchs all exceptions
Command Line Execution
Step up everything once
getsentry.com
Shards
Breaks up your fleet into
chunks, so that you can roll
out slowly
Shards
💩
0
1
2
3
13
4
Open Source
*_tools.py
pyexec
github.com/facebook/IT-CPE
Profile
Pic
Ajay Chand
Timeline
About
Photos 614
Update Info
About
What’s on your mind?
Client Platform Engineer at Facebook HQ
Ajay Chand
Studied MSIS at Santa Clara University
CPE Team 2013
Lives in Newark, California
From Newark, California
Followed by 498 people
Friends 926
More
Log Congregation
▪
Search Analyze Data
▪
BI
Logger
▪
Tool to write to syslog
▪
Script: logger -t cpe “testing”
Sample Logger Query
▪
host=* process=myapp |
rex field=_raw “Rebind=(?<status>\S+) | stats dc(host) by status
▪
Output report on sums of each condition
source="/Library/Managed Installs/Logs/install.log"
| rex field=_raw "(Install of (?<app>[^:]+)|: (?<status>[^$]+))"
| stats count by app
|sort -count
|head 10
https://code.google.com/p/munki/
System Info File
▪
File with relevant attributes
▪
Updated daily with LaunchD or mgmt tool
▪
Indexed in Splunk daily
System Info Examples
▪
Finding Regional Systems
▪
Finding Departmental Systems
▪
Permutations of Attributes (AND OR NOT)
▪
Multiplatform
System Info Examples
▪
Top 10 apps installed by host count
▪
▪
Photoshop by Office Campus
▪
▪
sourcetype=munki | rex "Install of (?P<app_name>[^:]+)" | stats count by
app_name | sort -count | head 10
sourcetype=munki | rex "Install of (?P<app_name>[^:]+)" | search
app_name="Adobe Photoshop*" | join host [search sourcetype=cpe_info] |
stats count by office_campus
Photoshop by Office Campus on 10.9.4 and laptop
▪
sourcetype=munki | rex "Install of (?P<app_name>[^:]+)" | search
app_name="Adobe Photoshop*" | join host [search sourcetype=cpe_info
is_desktop="False" os_ver="10.9.4"] | stats count by office_campus
Proactive Support
▪
Hardware Support - Kernel Panics
▪
Example KP Query: sourcetype=syslog *DumpPanic*
▪
Software Support - Outlook
Outlook Crashes
Take Aways
Data can drive impact
Impact can be measured!
Stop scripting, Start CODING
Reuse and share
What would you do,
If you weren't AFRAID
Q&A?
Group: fb.com/groups/macbrained/
Group: fb.com/groups/TheITThinkTank/
IRC: freenode.irc.net #ITThinkTank
GitHub: github.com/facebook/IT-CPE
Web: facebook.com/IT
(c) 2009 Facebook, Inc. or its licensors. "Facebook" is a registered trademark of Facebook, Inc.. All rights reserved. 1.0