Systems Engineer - Infrastructure

Details

Demonware creates and provides the online services behind hugely popular video game franchises such as Call of Duty and Skylanders. We do matchmaking, microtransactions, and data analytics for almost half a billion users. Demonware has offices in Dublin, Ireland; Vancouver, Canada; and Shanghai, China, and is a wholly-owned subsidiary of Activision Blizzard, Inc.

Position Summary

The Infrastructure department is responsible for the design, build and operation of our network of data centres and all infrastructure including network, compute and storage therein. We also have a significant infrastructure footprint in AWS and provide tooling for accounts, IAM, VPC and VPN access into AWS. We are responsible for systems provisioning in our data centres, hardware inventory, DNS and all IP connectivity to our data centres including connectivity to the Internet and AWS. Our goal is to provide reliable data centre and cloud infrastructure to Demonware’s Title, Product and Platform teams. Some of our current projects include data centre-wide virtualization, data centre standardisation, commodity storage solutions and development of best practices for running infrastructure in AWS.

As our hosting platform is evolving we are looking for experienced Systems Engineers to join our Systems Engineering team. You will help improve efficiency in our existing hardware fleet which spans multiple data centres and is in excess of 4,000 physical servers. You will also help us to deploy Infrastructure in the public cloud in a secure, scalable and repeatable way. You will have worked at scale with key abstraction and scheduling technologies such as virtualization, Linux containers, Docker, Openstack, Mesos/Marathon and/or Kubernetes and have practical hands-on experience working with compute hardware in data centres. You will also have practical experience with public cloud providers which will be beneficial as we launch more production services in AWS. In order to provision infrastructure in AWS in a secure and repeatable way we are moving towards infrastructure as code using frameworks such as Terraform. This is a great career opportunity for a highly motivated individual who wants to utilize and extend his or her broad set of skills.

Responsibilities
  • Ownership of all compute systems (IBM/Lenovo) and storage solutions (NFS/Ceph etc) in our data centres along with hardware inventory, provisioning and DNS
  • Be a key player in providing compute and storage infrastructure to other teams in Demonware, both in our physical data centres and AWS
  • Deliver on Demonware’s hosting strategy to virtualise the workloads in our data centres and provide the underlying infrastructure for deployments into AWS. We are also making plans for our next generation hosting platform, which will host scheduled containers in our data centres and AWS via a CI/CD pipeline
  • Working with other teams in Demonware, assist with load testing and qualification of public cloud infrastructure for supporting Demonware services
  • Create and manage tools for AWS user access management and account/VPC creation, VPC peering and cost optimisation
  • Be, or be willing to become, a subject matter expert on network, compute and storage services in the public cloud
  • Act as an escalation point for systems and hardware problems
  • Interact and maintain relationships with data centre remote hands, third party vendors & contractors as required
  • Participate in out of hours on-call as required
Requirements
  • 5+ years of practical infrastructure experience working with network, compute and storage hardware
  • Firm understanding of compute hardware and architecture including rackmounts and blades and provisioning same at scale
  • 5+ years of hands-on Linux administration
  • Firm grasp of core operating system concepts (processes, threading, memory management, file system types, ACLs)
  • Extensive understanding of networking/distributed computing environment concepts
  • Experience in hardware qualification & performance benchmarking
  • In-depth practical knowledge of open source hardware and operating system abstraction technologies: virtualization (preferably KVM) and/or Linux containers (e.g. LXC, Docker)
  • Proven experience with architecting and provisioning services in AWS
  • OpenLDAP experience, architecting and maintaining
  • Strong scripting skills: Unix shell programming and ideally at least one of Python or Perl, to assist with identification and automation of repetitive tasks
  • Strong working knowledge of TCP/IP networking as well as common application protocols
  • Familiarity with common system services, e.g. Cobbler, PXE, DNS/BIND, DHCP/BOOTP, NFS, SNMP, IPMI, RAID
  • Excellent analytical / troubleshooting skills
  • Strong written & verbal communication skills
  • Ability to work and deliver consistently under pressure
  • Ability & willingness to take part in second level on-call rotation
Desired
  • Experience working in 24x7x365 online environments
  • Experience deploying or operating an OpenNebula cloud environment
  • Experience working with container cluster management technologies such as Mesos and Kubernetes
  • Experience describing and operating infrastructure as code using tools such as Terraform and Ansible
  • Experience enabling product teams with Continuous Integration and Continuous Delivery pipelines
  • Experience with Unix like Operating Systems apart from Linux
What Demonware offers:
  • An excellent salary, with annual performance reviews and discretionary bonuses
  • 5 weeks of vacation, standard
  • Pension matching and contributions
  • Top-tier medical, dental, and life insurance coverage
  • Flexible working practices, based on the belief that the quality of your work is not dependent on the number of hours sitting behind a desk
  • A culture that emphasizes continual learning and improvement, with opportunities for career progression, travel, and a commitment to open source
Demonware by the numbers:
  • 469+ million gamers
  • 3.2+ million concurrent online gamers
  • 100+ games
  • 300,000 requests per second at peak
  • Average query response time of <.02 second
  • 630,000+ metrics a minute
  • 132 billion+ API calls per month
Please note: We do not require agency assistance at this time.

  • Dublin

  • Vancouver

  • Shanghai