Scalable Configuration Management For Secure Web Services Infrastructure Prepared For DIMACS Workshop on Security of Web Services and E-Commerce, May 5-6, 2005 Sanjai Narain Senior Research Scientist Telcordia Technologies narain@research.telcordia.com (732) 699 2806 Outline Focus on architectural aspect of web-services security. Components can be robust, but architecture into which they are integrated can be fragile and vulnerable. How do we answer questions such as “is there a single point of failure?”, “is there sufficient defense-in-depth?” Show an approach based on model-finding Show how to scale this approach to realistic size and complexity DIMACS (Narain) – 2 Deploying Web Services Security Infrastructure Component Configuration Is Central Operation Defense-in-depth via DMZ Gateway Router F W Bulk encryption via fault-tolerant network of IPSec Tunnels Gateway Router WAN Gateway Router Gateway Router XML Gateway Cluster F W Application Servers Credit card encryption Authentication Authorization F W XML Gateway Cluster F W Application Servers F W XML Gateway Cluster F W Application Servers DIMACS (Narain) – 3 Yet, there is no theory of configuration System requirements on security, functionality, fault-tolerance Configuration Synthesis Requirement Strengthening Component Adds & Deletes Configuration Error Diagnosis Configuration Sequencing Configuration Error Fixing Requirement Verification Operations on requirements Components DIMACS (Narain) – 4 Quotes Although setup (of the trusted computing base) is much simpler than code, it is still complicated, it is usually done by less skilled people, and while code is written once, setup is different for every installation. So we should expect that it’s usually wrong, and many studies confirm this expectation. – Butler Lampson, Computer Security In the Real World. Proceedings of Annual Computer Security Applications Conference, 2000. – http://research.microsoft.com/lampson/64-SecurityInRealWorld/Acrobat.pdf 65% of attacks exploit configuration errors. – British Telecom/Gartner Group. http://www.btglobalservices.com/business/global/en/products/docs/28154_219475secur_bro_sin gle.pdf ...operator error is the largest cause of failures...and largest contributor to time to repair ... in two of the three (surveyed) ISPs.......configuration errors are the largest category of operator errors. – David Oppenheimer, Archana Ganapathi, David A. Patterson. Why Internet Services Fail and What Can Be Done About These? Proceedings of 4th Usenix Symposium on Internet Technologies and Systems (USITS ‘03), 2003. – http://roc.cs.berkeley.edu/papers/usits03.pdf Consider this: ….the complexity [of computer systems] is growing beyond human ability to manage it….the overlapping connections, dependencies, and interacting applications call for administrative decision-making and responses faster than any human can deliver. Pinpointing root causes of failures becomes more difficult. –Paul Horn, Senior VP, IBM Research. Autonomic Computing: IBM’s Perspective on the State of Information Technology. – http://www.research.ibm.com/autonomic/manifesto/autonomic_computing.pdf DIMACS (Narain) – 5 New Concept: Requirement Solver System Requirements in FOL System Components Alloy (MIT) FOL Boolean logic compiler Requirement Solver With policy-based networking, this work has to be done by system designer. SAT Solvers (Very fast) Configurations Transformations FOL formula model System components, e.g., hosts, servers, routers, firewalls DIMACS (Narain) – 6 Verification: Where R is a requirement, to show that S R is valid show S R is unsatisfiable Component adds/deletes: Solve S for new set of components Configuration synthesis: • Find system configuration C: S is satisfiable Configuration Sequencing: SAT for planning Also Quantified Boolean Formulas S = System Requirement Formalizing Configuration Management Problems Configuration error diagnosis: • Where C is current system configuration, is SC satisfiable? Requirement Strengthening: Solve S NewReq Configuration error fixing: Find new configuration C: S is satisfiable and cost of migration to C is acceptable DIMACS (Narain) – 7 Fully Configured Fault-Tolerant VPN IPSec Tunnel Hub Router GRE Tunnel XML GW Cluster XML GW Cluster Spoke Router WAN Router Spoke Router Hub Router Full mesh of IPSec tunnels does not scale DIMACS (Narain) – 8 Network Components Interface •Physical Interface Subnet • Internal Interface • Internal Subnet • External Interface • External Subnet •hubExternalInterface •spokeExternalInterface Protocols • ike • esp • gre Permissions • permit • deny RIP Routing Domain ipPacket OSPF Routing Domain Component Attributes IPSec Tunnel interface – Spoke Router – GRE Tunnel – ipsecTunnel – Access Server (router subtype) – firewallPolicy – – – – – Hub Router localPhysical: externalInterface remotePhysical:externalInterface routing:routingDomain firewallPolicy – WAN Router Legacy Router local: externalInterface, remote: externalInterface, protocolToSecure: protocol greTunnel – chassis: router network: subnet routing: routingDomain prot: protocol action: permission protectedInterface: physicalInterface ipPacket – – – source:interface, destination:interface, prot:protocol DIMACS (Narain) – 9 Fault-Tolerant VPN Requirements GRERequirements RouterInterfaceRequirements 1. 2. 3. 4. Each spoke router has internal and external interfaces Each access server has internal and external interfaces Each hub router has only external interfaces Each WAN router has only external interfaces SubnettingRequirements 5. 6. 7. 8. 9. A router does not have more than one interface on a subnet All internal interfaces are on internal subnets All external interfaces are on external subnets Every hub and spoke router is connected to a WAN router No two non-WAN routers share a subnet There is a GRE tunnel between each hub and spoke router RIP is enabled on all GRE interfaces 12. 13. SecureGRERequirements For every GRE tunnel there is an IPSec tunnel between associated physical interfaces that secures all GRE traffic 14. FirewallPolicyRequirements 15. Each hub and spoke external interface permits esp and ike packets RoutingRequirements 10. 11. RIP is enabled on all internal interfaces OSPF is enabled on all external interfaces Human administrators reason with these in different ways to synthesize initial network, then reconfigure it as operating conditions change. Can we automate this reasoning? DIMACS (Narain) – 10 Current VPN Configuration Process hostname AI-RTR New Cisco IOS configuration needs to be implemented at all VPN peer routers! For 4 node VPN that is more than 240 command lines ! hostname SN2-RTR ! interface Tunnel2 crypto isakmp policy 1 interface Tunnel2 cryptoipisakmp policy 1 authentication pre-share address 36.36.36.2 255.255.255.0 ip address 34.34.34.2 255.255.255.0 hostnameauthentication PN1BS-RTR pre-share crypto isakmp key SN1BS-RTR_key_with_AI-RTR address 128.128.128.2 tunnel source 148.148.148.2 tunnel source 158.158.158.2 hostname SN1BS-RTR ! crypto isakmp key PN1BS-RTR_key_with_SN2-RTR address 148.148.148.2 crypto isakmp key PN1BS-RTR_key_with_AI-RTR address 148.148.148.2 tunnel destinationtunnel 138.138.138.2 ! 138.138.138.2 cryptopolicy isakmp1destination key AI-RTR_key_with_SN2-RTR address 158.158.158.2 ip classless crypto isakmp map vpn-map-Ethernet0/0 crypto isakmp key SN2-RTR_key_with_AI-RTR address 138.138.138.2 crypto isakmpcrypto policy 1 authentication pre-share cryptocrypto isakmpmap key SN1BS-RTR_key_with_SN2-RTR address 128.128.128.2 ! vpn-map-Ethernet0/0 ! pre-share authentication ! crypto isakmp SN1BS-RTR_key_with_PN1BS-RTR address 128.128.128.2 ! interface Tunnel2 ! key ! interface Ethernet0/0 crypto isakmp key PN1BS-RTR_key_with_SN1BS-RTR address 148.148.148.2 crypto ipsec transform-set IPSecProposal esp-des esp-sha-hmac crypto isakmp key A1-RTR_key_with_PN1BS-RTR address 158.158.158.2 crypto ipsec transform-set IPSecProposal esp-des esp-sha-hmac ip address 32.32.32.2 255.255.255.0 interface Tunnel2 address 128.128.128.2 255.255.255.0 crypto isakmpipkey AI-RTR_key_with_SN1BS-RTR address 158.158.158.2 Ethernet0/0 isakmp key SN2-RTR_key_with_PN1BS-RTR address 138.138.138.2 ! interface ! tunnel sourcecrypto 148.148.148.2 ip address 34.34.34.1 crypto 255.255.255.0 map vpn-map-Ethernet0/0 crypto isakmp key address 138.138.138.2 ! SN2-RTR_key_with_SN1BS-RTR cryptoipmap vpn-map-Ethernet0/0 33 ipsec-isakmp address 158.158.158.2 255.255.255.033 ipsec-isakmp tunnel destination 138.138.138.2 crypto map vpn-map-Ethernet0/0 tunnel! source 138.138.138.2 ! crypto ipsec transform-set IPSecProposal esp-des esp-sha-hmac set peer 148.148.148.2 crypto map vpn-map-Ethernet0/0 crypto map vpn-map-Ethernet0/0 set peer 128.128.128.2 tunnelcrypto destination interface 128.128.128.2 Ethernet0/1 ipsec transform-set IPSecProposal esp-sha-hmac ! set transform-setesp-des IPSecProposal ! set transform-set IPSecProposal crypto! map vpn-map-Ethernet0/0 ip address 50.50.50.1 255.255.255.0 ! address crypto map vpn-map-Ethernet0/0 33 ipsec-isakmp match 142 interface Ethernet0/0 ! ! match address34 142 crypto map vpn-map-Ethernet0/0 33 ipsec-isakmp interface Ethernet0/1 set peer crypto 128.128.128.2 map vpn-map-Ethernet0/0 ipsec-isakmp ip address 148.148.148.2 255.255.255.0 interface router rip set Ethernet0/0 peer 148.148.148.2 crypto map vpn-map-Ethernet0/0 set transform-set IPSecProposal set peer 158.158.158.2 crypto map vpn-map-Ethernet0/0 ip address 80.80.80.1 255.255.255.0 34 ipsec-isakmp ip address 138.138.138.2 version 255.255.255.0 2 set transform-set IPSecProposal set peer 148.148.148.2 match address 142 set transform-set IPSecProposal ! crypto match map vpn-map-Ethernet0/0 network 50.0.0.0! address 142 crypto map vpn-map-Ethernet0/0 34 ipsec-isakmp match address 143 set transform-set IPSecProposal interface Ethernet0/1 ! network 31.0.0.0router crypto map vpn-map-Ethernet0/0 34 ipsec-isakmp rip match address35 set peer crypto 158.158.158.2 map vpn-map-Ethernet0/0 ipsec-isakmp ip address 192.110.175.1 255.255.255.0 143 interface network 34.0.0.0 set Ethernet0/1 peer 158.158.158.2 set transform-set IPSecProposal set peer 128.128.128.2 version 2 crypto map vpn-map-Ethernet0/0 35 ipsec-isakmp ! ip address 60.60.60.1 network 255.255.255.0 35.0.0.0 set transform-set IPSecProposal match address 143 set transform-set IPSecProposal router rip network 80.0.0.0 set peer35138.138.138.2 ! ! match address 143 vpn-map-Ethernet0/0 ipsec-isakmp match address 144 version 2 crypto map routercrypto rip map vpn-map-Ethernet0/0 ip classless network 35.0.0.0 set transform-set IPSecProposal 35 ipsec-isakmp set peer !138.138.138.2 network 192.110.175.0 version ip route 0.0.0.0 0.0.0.0 128.128.128.1 set2peer 138.138.138.2 match address 144 network 33.0.0.0 set transform-set IPSecProposal interface Tunnel0 network 31.0.0.0 network no ip IPSecProposal http server set 60.0.0.0 transform-set ! match address 144 32.32.32.1 ip address 255.255.255.0 network 33.0.0.0 network 36.0.0.0 network 32.0.0.0 ! 144 match address ! tunnel source 138.138.138.2 interface Tunnel0 network 32.0.0.0 ! network 142 permit gre host 128.128.128.2 host 148.148.148.2 ! 34.0.0.0 access-list interface tunnel Tunnel0 destinationip148.148.148.2 ! address 35.35.35.2 255.255.255.0 network 36.0.0.0 access-list 143 permit gre host 128.128.128.2 host 158.158.158.2 interface Tunnel0 ip classless 31.31.31.2 255.255.255.0 crypto map vpn-map-Ethernet0/0 ip classless ip address source host 158.158.158.2 ! access-list255.255.255.0 144 permit gre host tunnel 128.128.128.2 138.138.138.2 ip address 31.31.31.1 tunnel source ! 148.148.148.1 ip148.148.148.2 route 0.0.0.0 158.158.158.1 ip route 0.0.0.0 0.0.0.0 tunnel0.0.0.0 destination 128.128.128.2 ip classless tunnel source! 128.128.128.2 tunnel destination 128.128.128.2 interface Tunnel1 no ip http server no ip http server ip routetunnel 0.0.0.0 0.0.0.0 end 138.138.138.1 crypto map vpn-map-Ethernet0/0 destinationcrypto 148.148.148.2 map vpn-map-Ethernet0/0 ip address 36.36.36.1 255.255.255.0 ! no ip http server ! source crypto map vpn-map-Ethernet0/0 ! ! permit tunnel 138.138.138.2 access-list 142 gre host 148.148.148.2 host 128.128.128.2 ! ! 142 permit gre host 158.158.158.2 host 128.128.128.2 interface interface Tunnel1 destination 158.158.158.2 access-list 143 permit tunnel greaccess-list host 148.148.148.2 hostTunnel1 158.158.158.2 access-list 142 Tunnel1 permit gre host 138.138.138.2 host 148.148.148.2 interface ip address 33.33.33.1 255.255.255.0 crypto map vpn-map-Ethernet0/0 ip address 33.33.33.2 access-list 144 permit greaccess-list host 148.148.148.2 host 138.138.138.2 143 permit gre host255.255.255.0 158.158.158.2 host 148.148.148.2 access-list 143 permit gre host 138.138.138.2 host 158.158.158.2 ip address 35.35.35.1 255.255.255.0 tunnel source 148.148.148.2 ! ! tunnel sourcegre 158.158.158.2 access-list 144 permit host 158.158.158.2 host 138.138.138.2 access-list 144 permit gre host 138.138.138.2 host 128.128.128.2 tunnel source 128.128.128.2 tunnel destination 158.158.158.2 end tunnel destination 148.148.148.2 ! tunnel destinationcrypto 158.158.158.2 ! map vpn-map-Ethernet0/0 crypto map vpn-map-Ethernet0/0 end crypto map vpn-map-Ethernet0/0 end DIMACS (Narain) – 11 Requirements In Alloy pred RouterInterfaceRequirements () { (all x:spokeRouter | some y:internalInterface | y.chassis = x) && (all x:spokeRouter | some y:spokeExternalInterface | y.chassis = x) && (all x:accessServer | some y:internalInterface | y.chassis = x) && (all x:accessServer | some y:externalInterface | y.chassis = x) && (all x:hubRouter | some y:hubExternalInterface | y.chassis = x)&& (all x:wanRouter | some y:externalInterface | y.chassis = x) } pred SecureGRERequirements () {all g:greTunnel | some p:ipsecTunnel | p.protocolToSecure=gre && ((p.local = g.localPhysical && p.remote = g.remotePhysical) or (p.local = g.localPhysical && p.remote = g.remotePhysical))} DIMACS (Narain) – 12 Sample Output From Requirement Solver routing : samples/router/routingDomain = {externalInterface_0 -> ospfDomain_0, externalInterface_1 -> ospfDomain_0, externalInterface_2 -> ospfDomain_0, externalInterface_3 -> ospfDomain_0, externalInterface_4 -> ospfDomain_0, hubExternalInterface_0 -> ospfDomain_0, hubExternalInterface_1 -> ospfDomain_0, internalInterface_0 -> ripDomain_0, internalInterface_1 -> ripDomain_0, internalInterface_2 -> ripDomain_0, spokeExternalInterface_0 -> ospfDomain_0, spokeExternalInterface_1 -> ospfDomain_0} chassis : samples/router/router = {externalInterface_0 -> accessServer_0, externalInterface_1 -> wanRouter_0, externalInterface_2 -> wanRouter_0, externalInterface_3 -> wanRouter_0, externalInterface_4 -> wanRouter_0, hubExternalInterface_0 -> hubRouter_0, hubExternalInterface_1 -> hubRouter_1, internalInterface_0 -> spokeRouter_0, internalInterface_1 -> accessServer_0, internalInterface_2 -> spokeRouter_1, spokeExternalInterface_0 -> spokeRouter_1, spokeExternalInterface_1 -> spokeRouter_0} network : samples/router/subnet = {externalInterface_0 -> externalSubnet_0, externalInterface_1 -> externalSubnet_0, externalInterface_2 -> externalSubnet_1, externalInterface_3 -> externalSubnet_2, externalInterface_4 -> externalSubnet_3, hubExternalInterface_0 -> externalSubnet_2, hubExternalInterface_1 -> externalSubnet_3, internalInterface_0 -> internalSubnet_0, internalInterface_1 -> internalSubnet_0, internalInterface_2 -> internalSubnet_1, spokeExternalInterface_0 -> externalSubnet_0, spokeExternalInterface_1 -> externalSubnet_1} DIMACS (Narain) – 13 RouterInterfaceRequirements 1. 2. 3. 4. SubnettingRequirements 5. 6. 7. 8. 9. Configuration Synthesis: Physical Connectivity and Routing Each spoke router has internal and external interfaces Each access server has internal and external interfaces Each hub router has only external interfaces Each WAN router has only external interfaces RIP Domain A router does not have more than one interface on a subnet All internal interfaces are on internal subnets All external interfaces are on external subnets Every hub and spoke router is connected to a WAN router No two non-WAN routers share a subnet OSPF Domain Spoke Router RoutingRequirements 10. 11. Hub Router WAN Router RIP is enabled on all internal interfaces OSPF is enabled on all external interfaces To synthesize network, satisfy R1-R11 for – – – – – – – – – 1 hub router, 1 WAN router, 1 spoke router, 1 internal subnet, 2 external subnets 1 internal interface, 4 external interfaces, RIP domain, 1 OSPF domain Requirement Solver generates solution. Note that Hub and Spoke routers are not directly connected, due to Requirement 9 DIMACS (Narain) – 14 RouterInterfaceRequirements 1. 2. 3. 4. SubnettingRequirements 5. 6. 7. 8. 9. 11. GRE Tunnel RIP Domain Hub Router A router does not have more than one interface on a subnet All internal interfaces are on internal subnets All external interfaces are on external subnets Every hub and spoke router is connected to a WAN router No two non-WAN routers share a subnet RoutingRequirements 10. Strengthening Requirement: Adding Overlay Network Each spoke router has internal and external interfaces Each access server has internal and external interfaces Each hub router has only external interfaces Each WAN router has only external interfaces OSPF Domain Spoke Router WAN Router RIP is enabled on all internal interfaces OSPF is enabled on all external interfaces GRERequirements 12. 13. There is a GRE tunnel between each hub and spoke router RIP is enabled on all GRE interfaces To synthesize network, satisfy R1-R13 for previous list of components & 1 GRE tunnel NOTE: GRE tunnel set up and RIP domain extended to include GRE DIMACS (Narain) – 15 interfaces automatically! RouterInterfaceRequirements 1. 2. 3. 4. Strengthening Requirement: Adding Security For Overlay Network Each spoke router has internal and external interfaces Each access server has internal and external interfaces Each hub router has only external interfaces Each WAN router has only external interfaces Hub Router SubnettingRequirements 5. 6. 7. 8. 9. A router does not have more than one interface on a subnet All internal interfaces are on internal subnets All external interfaces are on external subnets Every hub and spoke router is connected to a WAN router No two non-WAN routers share a subnet RoutingRequirements 10. 11. IPSec Tunnel OSPF Domain Spoke Router WAN Router RIP is enabled on all internal interfaces OSPF is enabled on all external interfaces GRERequirements 12. 13. There is a GRE tunnel between each hub and spoke router RIP is enabled on all GRE interfaces SecureGRERequirements 14. For every GRE tunnel there is an IPSec tunnel between associated physical interfaces that secures all GRE traffic To synthesize network, satisfy R1-R14 for previous list of components & 1 IPSec tunnel DIMACS (Narain) – 16 NOTE: IPSec tunnel securing GRE tunnel set up automatically Component Addition: Adding New Spoke Router Hub Router Spoke Router WAN Router Spoke Router To add another spoke router satisfy requirements R1-R15 for previous components and one additional spoke router and related components Note: New subnets, GRE and IPSec tunnels set up, and routing domains extended automatically DIMACS (Narain) – 17 Component Addition: Adding New Hub Router Hub Router OSPF Domain Spoke Router WAN Router Spoke Router Access Server Hub Router To add another hub router satisfy requirements R1-R15 for previous components and one additional hub router (and related com[ponents) New subnets, GRE and IPSec tunnels set up, and routing domains extended automatically DIMACS (Narain) – 18 Verification: Adding Firewall Requirements & Discovering Design Flaw Hub Router OSPF Domain Spoke Router WAN Router Spoke Router Hub Router Symptom: Cannot ping from one internal interface to another Define Bad = ip packet is blocked Check if R1-R16 & Bad is satisfiable Answer: WAN router firewalls block ike/ipsec traffic Action: Create new policy that allows WAN router firewalls to pass esp/ike packets DIMACS (Narain) – 19 Scalability Approaches Can we write specifications in such a way that they are “efficient”? Two heuristics: Small number of quantifiers in formulas – Scope splitting – Even with these, Alloy crashes for 10 sites (200 object instances) New approach is required DIMACS (Narain) – 20 Divide, Conquer, Verify Heuristic: Instead of creating VPN for all sites all at once, create it incrementally by adding sites one at a time Goal: Solve R for component set C If C is large, Alloy will take a long time or crash Split C into C1,..,Ck and solve R for C1,..,Ck generating solutions M1,..,Mk. Then take union of M1,..,Mk = M Verify that M is a solution for R. Using Alloy for verification will restore inefficiency However, one can use Prolog DIMACS (Narain) – 21 Any FOL formula can be expressed in full Prolog. pred greTunnelEveryHubSpoke () {all x:hubExternalInterface, y:spokeExternalInterface | some g:greTunnel | (g.localPhysical=x && g.remotePhysical=y) or (g.localPhysical=y && g.remotePhysical=x)} ----------------------------------------------------------------------------------------------------------------------------------greTunnelEveryHubSpoke if not counterExampleGreTunnelEveryHubSpoke. counterExampleGreTunnelEveryHubSpoke if type(X,hub),type(Y,spoke), not existsGRE(X,Y). existsGRE(X,Y) if localPhysical(GT, P1),remotePhysical(GT,P2), chassis(P1, X), chassis(P2, Y) existsGRE(X,Y) if localPhysical(GT, P1),remotePhysical(GT,P2), chassis(P2, X), chassis(P1, Y) ----------------------------------------------------------------------------------------------------------------------------------- Represent model as a collection of Prolog ground facts. Now, evaluate Prolog requirement. This is a database integrity checking problem DIMACS (Narain) – 22 Summary & Future Directions Problem: Focus on architectural aspects of security Configuration plays central role in web services infrastructure synthesis & management We need a theory of configuration to solve following fundamental problems: 1. 2. 3. 4. 5. 6. 7. 8. Specification languages Configuration synthesis Incremental configuration (requirement strengthening, component addition) Configuration error diagnosis Configuration error troubleshooting Verification Configuration sequencing Distributed configuration Proposed formalization of 1-6 via Alloy and SAT solvers Proposed scalability approach by complementary use of Prolog Future directions: – – Scalable algorithms to solve above problems. Combine FOL reasoning systems, Linear Programming systems, relational databases, graph algorithms into single programming model DIMACS (Narain) – 23 Thank You DIMACS (Narain) – 24 There is no theory of configuration. There is a deep logic that governs infrastructure. Security and routing interfere Move from coding to configuration Why is diagnosis hard: work in isolation but not together Policies across components, and across layers……! GUIs don’t even have expressive power of Boolean logic Self-healing architecture VG Emphasize Language (syntax/semantics). Semantics: services at each layer Integrate fault-tolerant protocols Scalability LP DIMACS (Narain) – 25