Improving Application
Security After
An Incident
Cory Scott
Matasano Security
Where Do Application
Security Programs Come
From?
Unlikely.
Maybe.
Most likely.
An incident, you say?
• Could be a near miss
• Or an unfortunate impact
• That’s fine, we’ll pull out our trusty dusty
(network) response plan...
FAIL
Traditional Network
Incident Response
•
•
•
•
•
•
Root cause is one or more of the following:
credentials, access control, patch, or
configuration.
There’s an app for that.
And a process template.
And an audit guideline.
Whew... All Done!
Usually one neck to choke.
Application Anarchy!
•
•
•
•
•
Could be one of many root causes.
Could be the fault of the developer, the framework
author, third-party plug-ins, application operations,
poor requirement definition, client-side security, etc
etc.
There’s probably an app for some of that. But
you’re going to need some process for it too...
Quick - how do you audit a secure coding
practice?
How many necks can you choke?
Queue Foreshadowing
Music Here
Oh, the people you’ll meet!
• Internal Auditors (grr!)
• External Auditors (eep!)
• Executives (*cringe*)
• Development Managers (hey, you!)
• Network Security People (...)
• Application Security Salesmen in your
C[X]Os office (WTF!)
The Opportunity &
The Problem
•
•
•
•
Taking the root cause
to the bank
You can prove that the Quick Fix is not the
fix.
You’ve just got some funding for an appsec
program. Congratulations! OR
You may be getting funding... IF you can
show that you’re going to do something
meaningful with it. OR
You may have to go back into the trenches
until the next one.
AppSec Stallout!
•
•
•
•
•
•
•
•
Management priority shift.
Fatigue, fear, and loathing.
Bought the $PRODUCT, the problem is solved.
Right? Right?
Got the Pentest, all clean! Right?
X days without a workplace incident, all good!
Analysis Paralysis
Auditor Pile-On
The LCD of Compliance
READY... FIRE...
AIM!
Assessment Strategies to Prevent Stallout
Identify High-Risk
Applications
• Emphasis on high-risk
• Enforce the two-sentence rule to
identify loss potential
• Existing inventories are usually
insufficient
• Don’t fight against intuition
• Get it over with
Scoping is Critical
• Get this wrong and you’ve just wasted
thousands of dollars.
Scoping is Collaborative
• Get everyone to the table, including:
• Application Owner
• Development Guy
• Information Security Guy
• The Tester
• Ambiguity at the beginning is okay, but
not at the end. Respect the fact that this
make some people uncomfortable.
Best of both worlds
•Embrace Design Reviews in addition to
implementation-oriented assessments
•HOWEVER:
•Questionnaires are to Design Reviews
what Web Vulnerability Scanners are to
Penetration Testing
Flexible & Standardized at
the same time?!
• Define a short-list of vulnerabilities and
weaknesses.
• Choices are good!
•
•
•
•
Design review
Tools
Code review
Manual Penetration Testing
• Standardize approach and deliverable
for each choice.
Pick your battles and
weapon of choice
• The first few engagements are the most
important.
• Insert a QA checkpoint and a postassessment feedback process.
• Pick “friendly” application teams to start.
• Bring in external teams at the beginning
to crib off of their approach and delivery.
Management Strategies to Prevent Stallout
Get funding for remediation
upfront
• Strike while the iron is hot. (and the
wallet is open)
• Rule of thumb: remediation cost equals
assessment cost.
• Consider a two-level approach for each
app: a pre-approved “not-to-exceed”
amount and a separate budget request
for larger initiatives.
• You’ll make friends!
Assign Specialists
• Understand the business unit
• Maintain a watchlist of applications
• Scope and schedule assessments
• Assist in Incident Response
Process Change
• SDL improvements
• Small steps with pilot groups
• Leverage specialists
• Vendor management
• Give them a risk assessment that
they can self-operate to start
• Encourage reusable assessments
Detection & Response
• You worked so hard to get situational
awareness, don’t lose it!
• First on your wish-list: logging and audit
trails that you didn’t have pre-incident
that would have helped you respond
faster and with less legwork.
• Specialists can help in preparation and
response.
Metrics
•
KRIs
•
Vulnerabilities still open for each application
•
•
•
Applications within open vulnerabilities that have suffered a successful
attack within the last year
Applications with open vulnerabilities with no clear path towards
remediation or where the risk has been accepted by the business unit
KPIs