Senior Systems Administrator

advertisement
Daniel Kingshott
4117 171st street se, Bothell, WA, 98012. dan@thekingshotts.com, 425-623-4359
LinkedIn Profile
Experienced Technologist and leader with a successful track record of proven delivery in
ecommerce and multi-datacenter environments. Improves system reliability and performance
through advanced deployment of emerging technologies and integration with existing
infrastructure in a heterogeneous environment. Enhances service management and operations
functions through use of ITSM processes, talent motivation and effective communication.
Tangible business outcomes include:

Infrastructure and Application Management: Improvement of datacenter and space
efficiency through the development of ‘fit for purpose’ hardware utilizing traditional
servers as well as Sled and Blade based systems. Increased capacity and throughput
achieved by the effective use of existing systems and multi-tenant architecture. In depth
data gathering and analysis utilizing tools like ExtraHop, JMX, Munin and Splunk leading
to application performance and response time gains enhancing the customer experience
and conversion.

Application Reliability and Innovation: Provides detailed feedback loop and
collaboration with development teams, at Expedia this was used to great effect with the
Voyager development team. Re-factored configuration file management for Tomcat
application leading to faster and more reliable deployments. Re-formatted logs to
increase Splunk field parsing, added additional fields for operational insight, this
enabled Daniel to develop Splunk dashboards showing a detailed, business centric view
of the environment as a whole, leading to reduced mean time to resolution.

Application Performance and Capacity Management: Performed detailed analysis of
bottlenecks in the environment to determine that 35% of all DNS lookups were failing or
delayed due to misbalanced load, poorly configured appliances and multi-hop
resolution, this was compounded by failed IPv6 queries also determined that a bug in
Cloudera Hadoop was causing 11k invalid DNS queries per second leading to reduced
capacity. Once resolved 99 percentile response times were reduced from 5 seconds to
less than a second.
Proven Competencies that Drive Outstanding Performance








Infrastructure and Application Support
High Availability and Scalability
Leadership and Team Development
Performance Engineering
Strategic use of emerging technology
Business Continuity and Disaster Recovery
ECommerce infrastructure and applications
Datacenter Management and Consolidation








IT Service Management
Technology Management and Strategy
Capacity Planning and Management
Performance Engineering
Virtualization and Cloud Computing
Information Security and Compliance
Portfolio and Project Management
Advanced application and infrastructure
Architecture
A Career of Successful Delivery and Leadership:
BestBuy.com
June 2015 – present
Cloud Engineering Lead
 Architectural oversight of OpenStack deployment.
 Defined and deployed high performance storage aggregate for SQL/noSQL use
cases.
 Improved team moral through application of FIRO-B techniques.
 Increased throughput by streamlining CI/CD processes and checkpoints
 Responsible for product roadmap and architecture
 Managed migration of multiple tenants from Essex and Havanna based clouds to
a Kilo based cloud.
 Defined and perfected in place Openstack upgrades without the need for
workload migration, upgraded existing cloud from Icehouse to Kilo.
HP, Inc.
August 2014 – June 2015
Chief OpenStack Architect – Helion Professional services
 Advanced architecture for large customers, including workload placement,
neutron planning and scale out multi-petabyte swift and distributed control
plane services.
 Best practice consultation to large Telco’s and Retail customers.
 Creation of service deliverables and reference architectures.
 Troubleshooting failed deployments for key customers.
 Global point of contact and bridge for professional services with Product
Management, Evangelist Teams, Engineering and Security.
 Recorded for Audio and Video Podcasts
 Author of advanced blog posts on OpenStack technology for HP.
Expedia, Inc.
October 2008 – April 2014
Director, Platform Engineering
 Responsibility for the team of 8 engineers undertaking Infrastructure
Engineering and Architecture activities for Global Infrastructure Services.
 Lead team through a successful Mirantis Openstack Deployment of 2 regions
each with 2 availability zones utilizing CEPH, ISCSI, GlusterFS, SCALR and Cisco
UCS, EMC VNX and HP hardware.
 Undertook several key initiatives including Fit for purpose hardware SKU’s,
Always Available DR Document storage and native Active Directory Integration of
Linux Hosts utilizing SSSD, this lead to a reduction in operations ticket ingress of
30%.
 Developed technical team members from Windows only engineers to include
Linux and Open Source applications.

Re-Negotiated support contracts with major vendors to include onsite support,
Priority level guidelines for incidents and SLA’s as well as support and service
processes.
Principal System Engineer, Platform Engineering
 Provided technical oversight to the Platform engineering team and acted as a
bridge between leadership and 8 matrixed engineers.
 Re-designed GSLB infrastructure from in-efficient DNS based system to a hybrid
OSPF / EIGRP Route-Health-Injection based system utilizing existing Citrix
Netscaler technology.
 Designed and Delivered Gemfire Grid Computing solution utilizing 3584 CPU
Cores and 56TB of RAM across 112 nodes with redundant front and back 10GBit
network interfaces on the Cisco UCS platform to provide 2N+1 redundancy.
 Provided infrastructure and application deployment consultancy to internal
customers.
 Responsible for defining architecture standards across Global Infrastructure
Services
Operations Architect / Senior Applications Engineer.
 Working in the Customer Service and Financial Application spaces with Tomcat
Applications, Oracle DB and ERP Suite, WS02 Middleware, Windows / Linux and
Splunk.
 4th Line escalation point for Application Engineering teams.
 Implemented custom RPM packages and YUM Repo for system preparation
activities including, Tomcat7, Java 6 and Operational deployment scripts,
enabling engineers to take bare metal to running applications in under 30
seconds.
 Deployed Graylog2 centralized syslog solution with MongoDB and Elastic Search
backend for Cisco UCM Voice systems allowing trended alerts based on call
disconnects.
 Undertook significant remediation of WS02 Middleware environment including
MySQL optimization and circular replication, Application tuning and system
patching.
Unix Engineering Manager
 Managed team of 7 Unix Engineers providing support and engineering to
Application Engineering teams.
 Reduced Average ticket resolution times from several days to 2 hours.
 Deployed Beyond Trust power broker identity services to integrate Linux servers
into existing Active Directory environment.
 Remediated External DNS solution from multiple internal masters used for AXFR
transfers to UltraDNS to de-centralized solution allowing internal teams to
manage their own DNS.
 Managed and oversaw Puppet deployment project allowing for centralized
configuration management of over 6000 linux servers.
Global Operations Manager

Managed team of 20 Systems Administrators (Linux, Windows, Network,
Database and SAN) working 1st shift in the tier 3 operations center.
 Ran all major incidents including ‘Code Red’ events.
 Direct input to all change advisory board activities.
 Drove process improvements through more precise support documentation.
Senior Systems Administrator, Hotels.com
 Working within the hotels.com brand of Expedia, responsible for all hotels.com
Application (Tomcat 6), OS infrastructure and Citrix Netscaler Administration.
 Reduced application instability by undertaking a full root cause analysis, utilizing
Java Thread Dumps, Java Heap Dumps and Wireshark captures to understand
the anatomy of an incident.
 Streamlined Application deployment process from several hours across 160+
tomcat instances to less than 7 minutes.
 Produced all documentation and architecture diagrams reducing technical and
operational debt.
Globrix.com
October 2007 – October 2008
Senior Systems Administrator
 Blank canvas startup requiring full architecture and implementation as first IT
employee.
 Implemented Active-Passive Corosync managed MySQL Cluster with multiple
read only slaves.
 LVS Load Balancers with custom built code to take action upon health check
failure.
 Build out of DR site with GRE VPN connectivity.
 Ruby on Rails application server deployment.
 Cisco Catalyst network implementation
 Dell ISCSI SAN providing shared Image storage to media cluster
 As the sole systems administrator / systems engineer in the company I also had
to provide end user support and telephony
Betfair.com
October 2006 – October 2007
Senior Systems Administrator
 Built RHEL 4 PXE system using kickstart templates and PXE entry points.
 Deployed and Supported JBOSS Java Application servers.
 Deployed Oracle Coherence for session management.
 Designed and built ‘internet simulator’ which introduced random latency and
packet loss to provide the QA team with a local environment which matched the
real world infrastructure which was spread out between the UK, Malta and
Australia.
Rackspace
May 2006 – October 2006
Lead Linux Engineer (Intensive)
 Sole lead Linux engineer in the intensive enterprise support team with 2 reports
(systems administrators)

Primary escalation for major clients including MetaCafe, Renault and the
Ministry of Defence.
 Deployed RedHat cluster suite controlled Active / Passive MySQL clusters with
both direct and SAN attached storage.
 Advanced Apache configuration for performance, security and PCI purposes.
 Provided JBOSS deployment, support and tuning services to major customers.
 Wrote monthly FAQ section for Linux Format Magazine
 Undertook Architectural consultations for enterprise customers.
 Identified command and control channels involved in DDOS attacks and shut
them down.
Gala Coral Group
June 2003 – May 2006
Senior Systems Administrator
 First Linux systems administrator in the company employed to migrate
www.coral.co.uk and www.eurobet.com from Oracle Application server to
LAMP.
 Migrated away from Cisco Local Director Load balancers to Linux based LVS load
balancers and then later onto F5 LTM.
 Support and deployment of Oracle 8.1.7 and 9.2.4 on Linux and Windows
Platforms.
 Migrated all corporate email away from Windows SMTP / POP3 and onto Linux
using Postfix / Dovecot with AmavisD and Spam Assassin.
 Support and Migration of Java 5 / Resin Based Casino system from Windows to
Linux .
 Defended against several DDOS attacks ranging from standard Syn Flood,
scripted attacks as well as reflective attacks.
Research in Motion
September 2002 – June 2003
Blackberry Operations Administrator
 Operation, monitoring and support of the Blackberry EMEA / APAC platform.
 Deployment and support of Mirapoint IMAP servers for the ‘prosumer’ platform.
 Performed scheduled maintenance on Solaris, Windows and Linux systems.
 First line support of Oracle Databases and Network infrastructure.
 Supported MS Exchange and BES servers.
BT Syntegra
September 2000 – September 2002
Solaris Systems Administrator
 Solaris 2.6 and 7 Systems administration on SPARC 2E based Periphonics IVR.
 Deployed NIS domain for centralized account management.
 Provisioning of E1 spans on Nortel DMS 100 platform.
 Defined call routing and architecture for CDN (Control Delivery Number)
platform within BT Customer service.
 Deployed SNMP monitoring using Netsaint.
 Developed CGI based system to allow business users to search tokenized call
records.
RGH Associates
September 1999 – September 2000
Systems Administrator
 Remote and onsite support of NT4 based infrastructure across multiple
continents.
 Deployment and support of MS Exchange 5.5 with x-500 and site connectors
 Setup and deployed Norton Ghost imaging system.
 2nd Line Support of Sendmail servers on linux
 Implemented ICMP based wan monitoring system for multinational clients.





Professional training and certifications:
Redhat 4 Certified Engineer (RHCE) – RedHat (expired)
Microsoft Certified Systems Engineer (MCSE) – Microsoft (expired)
Cisco Certified Network Associate (CCNA) - Cisco systems (expired)
Leading Technical Professionals – Blessing White
Maximizing your leadership potential – Center for Creative Leadership
Download