Eugene de Beste

Senior HPC Technologist

“The right man in the wrong place can make all the difference in the world.”

- G-man, Half-Life 2.

Current Role

Senior HPC Technologist based in the Centre for High Performance Computing’s (CHPC) Advanced Computer Engineering (ACE) Lab with a primary focus on HPC education, cloud computing systems and distributed storage. The CHPC operates under the National Integrated Cyber Infrastructure System (NICIS), which in turn falls under the umbrella of the Council for Scientific and Industrial Research (CSIR) of South Africa.

Responsibilities

• • Cloud computing research and development, primarily focussing on OpenStack. Manage the OpenStack Research Cloud initiative, which aims to provide a test-bed platform for investigating OpenStack services and developing solutions on-top of OpenStack for the CHPC and CSIR.
• • Distributed storage research and development, primarily focussing on CEPH. Manage and optimise the ACE CEPH cluster and investigate it’s ability to serve as a tier 1 filesystem to compete with traditional High Performance Computing (HPC) filesystems.
• • Contributing to the international Student Cluster Competition (SCC) initiative. The SCC aims to educate young South African tertiary education students in Linux and HPC. Duties include planning the event, educational video content creation, preparing tutorial tasks for students, providing students with cloud computing resources to train on, innovating on assignments and grading and preparing the online education management system.
• • Infrastructure procurement, management and maintenance, including maintenance of legacy OpenNebula cloud deployment.
• • Software system innovations, which include implementing monitoring and alerting for infrastructure, new hardware and inventory management systems, improvements to knowledge-base and general software stack innovations.
• • Supporting and training staff with software and infrastructure in the broader CSIR organisation. Duties include, but are not limited to, introducing staff to new concepts in software, assisting with programming and debugging software, training staff on cloud computing paradigms and software containers and more.
• • Support the South African Development Community (SADC) through the HPC Ecosystems project (donation HPC equipment and training). Duties include liaison with Universities across Africa in order to deliver HPC donation equipment, educating recipients with HPC and Linux administration concepts and skills, contributing to OpenHPC training materials and implementing monitoring and reporting services.
• • Network configuration and debugging in the ACE Lab data center in order to facilitate new equipment being installed and to standardise network configurations to match upstream CSIR projects.
• • Automation engineering and bringing infrastructure in the ACE Lab into an infrastructure-as-code approach as to optimise efficiency and advise the broader CSIR on new and better paradigms for hardware and software management.
• • Training of graduate staff. Duties include introducing graduate training staff to various concepts such as cloud computing, networking and HPC and providing hands-on scenarios for them to develop skills.

Technology Experience

Non-exhaustive

Special Projects

NB: Details of projects listed here are reduced in work experience section below to remove redundancies.

• • CHPC = Centre for High Performance Computing, South Africa
• • UCT = University of Cape Town
• • SANBI = South African National Bioinformatics Institute
• • HISP = Health Information Systems Program
• • NICD = National Institute for Communicable Diseases
• • ICS = Internet Communication Services
• • NICD = National Institute for Communicable Diseases
• • HPC = High Performance Computing
• • UWC = University of the Western Cape

Student Cluster Competition

CHPC

• (2020) Due to the coronavirus pandemic, the CHPC needed to move their student cluster competition online.
• Duties included:
• • The generation of many high-quality videos for the educational part of the competition, with topics such as memory, network interconnects, NUMA, CPU vectorisation, software containers, BASH tips and tricks and more.
• • Preparing content on the learning management system (Canvas).
• • Assisting students with tutorials.
• • Tutoring students in “office-hours” sessions.

Ilifu project

UCT/SANBI

• (2018 - 2020) The ilifu project is currently being used in a production setting.
• The ilifu project is a data-intensive research cloud aimed at academic research work with specifics in astronomy and bioinformatics. It is a joint venture between multiple universities, as well as the Department of Science and Technology in South Africa. This project involved the planning, prototyping and deployment of an OpenStack based private cloud with a multi-petabyte CEPH cluster as its storage.
• Duties included the following:
• • Planning of private cloud deployment with a team from UCT. This included the necessary hardware, network configuration and OpenStack services.
• • Development and implementation a prototype OpenStack instance on testing hardware. Experimentation of OpenStack services were conducted on this prototype implementation.
• • Research and implementation of CEPH distributed storage system with erasure coding to maximize available end-user storage space.
• • Development and implementation of production OpenStack private cloud utilising the aforementioned CEPH storage backend using the Kolla and Kolla-ansible deployment method.
• • Exploring and implementing OpenStack Manila to allow the use of the CEPH backend as a tier 1 filesystem for end-users of the cloud.
• • Consulting services and generalised second-tier technology support support for CEPH implementation.
• • Project details here: http://www.ilifu.ac.za/.

HISP OpenStack Deployment

SANBI

• (2019 - 2020) HISP and SANBI entered into an agreement for assistance with deploying an OpenStack cluster.
• Duties included:
• • Introducing software containers and training HISP users on the topic.
• • Planning the hardware and network layout for HISP OpenStack by discussing needs and expectations for the cloud.
• • Planning the deployment of the OpenStack cloud and training the HISP IT staff on OpenStack deployment and usage.
• • Running multiple deployment scenarios with the HISP IT team and training them with hands-on deployment of OpenStack in a simulated lab.

NICD HPC Deployment

SANBI

• (2019 - 2020) The NICD requested assistance from SANBI to deploy an HPC software stack.
• Duties included:
• • Evaluating and understanding the NICD hardware configuration.
• • Deploying the HPC stack with SLURM and Singularity containers.
• • Deploying the Galaxy (bioinformatics) platform on top of the HPC software stack.
• • Training the IT staff of the NICD in order to maintain and improve the deployed software stack and provide new tools to users via Singularity containers.

UWC HPC Deployment

SANBI

• (2019 - 2020) UWC eResearch contacted SANBI to request help with deploying an HPC cluster.
• The goal of this project was to allow UWC to be able to offer common resources to researchers and students across departments in a unified research-first offering.
• Duties included:
• • Working with the Internet Communication Services (ICS) team to discuss and plan a hardware and software stack as well as the networking configuration for the cluster. This involved the re-use of older existing computer hardware that was donated to the project.
• • Deploying bare-metal servers using MAAS (highly available) to provision and manage the hardware.
• • Deploying the cluster software stack (SLURM based) using Ansible.
• • Providing support for the cluster software and hardware stacks.

Professional Work Experience

Job Roles

2020-ongoing (Current) Senior High Performance Computing Technologist — Advanced Computer Engineering Lab, Centre for High Performance Computing, Council for Scientific and Industrial Research

• Refer to the Responsibilities section in Current Role above.

---

2018-2020 External Consultant — ILIFU Project, University of Cape Town

• • Planning, development, implementation, consulting and support of a private cloud deployment for use primarily by astronomy and bioinformatics researchers.

2017-2020 Systems Developer — South African National Bioinformatics Institute, University of the Western Cape

• • Trained multiple teams of students to enter the Centre for High Performance Computing Student Cluster Competition, one of which proceeded to compete internationally at the International Supercomputing Conference’s Student Cluster Challenge in 2017. (Source.)
• • Migration of in-house developed VM management software to OpenStack Cloud. This involved inventorying the fleet of bare-metal hosts and VMs, deploying OpenStack using Kolla-ansible and migrating VMs to the new platform.
• • Provided training and assistance on the implementation of an OpenStack based cloud to the Health Information Systems Program (HISP) organisation.
• • Optimisation of CEPH storage performance by correcting inefficient legacy configuration, upgrading to latest releases and re-balancing data distribution of CEPH cluster.
• • Provided CEPH as a filesystem to High Performance Computing (HPC) users. This allowed for the management of one single filesystem that could be used for both end-users as well as administrators.
• • Implemented the software stack for an HPC cluster and Galaxy server at the National Institute for Communicable Diseases (NICD) in Johannesburg.
• • Assisted the University of the Western Cape (UWC) Internet Communication Services (ICS) team with deployment of an HPC cluster for UWC eResearch HPC offering.
• • Containerised (Docker) many system and administrative services in order to optimise the efficiency of IT staff.
• • Containerised (Singularity) many bioinformatics scientific user applications, creating a substantially simpler way for SANBI to package and deploy software applications with complicated and/or conflicting software dependencies for their HPC cluster.
• • Migrated from legacy system of manual infrastructure deployment with automated infrastructure deployments utilising MAAS and PXE.
• • Implemented infrastructure-as-code approach with Ansible and Terraform for IT management and up-skilled development staff to utilise these tools to reduce their development and testing time.
• • Implemented pfSense firewall to migrate legacy system which involved reworking SANBI networking configuration.
• • Deployed HPC clusters in cloud and on bare-metal (with focus on the SLURM scheduler.).
• • Wrote many Ansible roles and playbooks for automated deployment of various internal systems including HPC clusters, administrative services and more.
• • Migrated legacy authentication LDAP system to FreeIPA.
• • Implemented monitoring and alerting automation with Prometheus, Grafana and alertmanager in order to preemptively catch and solve infrastructure issues.
• • Managed IT infrastructure for academic researchers, including provisioning of virtual and physical resources for computational activities.
• • Provided IT consulting and advanced support to end-users, academic staff and development staff.
• • Procurement for various projects, ranging from user laptops to high performance servers and switching equipment.
• • Established change management control for IT staff and knowledge repositories for both end-users and IT staff.
• • Network debugging and tracing (VLANs, Openvswitch, port mappings).
• • Optimised and improved security for IT staff.
• • Implementation of a helpdesk ticketing system for IT staff to track user issues and generate reports.

2015-2016 Systems Administrator Intern — VOSS Solutions

• • Upgrading production service software from source.
• • VMWare management - Network tracing and VLAN tagging configuration.
• • Debugging issues related to removable storage passthrough to virtual machines on VMWare ESXI.
• • AD, DNS and WSUS management.
• • User management on Cisco ASA devices.
• • User management on Linux devices.
• • Gathering quotes and creating purchase orders.
• • Desktop support for employees (Operating system installs, software troubleshooting, etc.).

2013-2014 Computer Laboratory Administrator — University of the Western Cape

• • Ansible for managing fleet of computers.
• • Managed Linux operating system installations.
• • Network configuration for lab.
• • Provide equipment for use by researchers.
• • Prepare lab for use by computer science students during networking practicals.
• • Fixed physical broken computers.
• • Built website for South African Sign Language university project.

Volunteering Roles

2017-2018 Volunteer Staff — BSides Cape Town

• Assisted with preparation for the event as well as operation during the event.

2017-2018 Volunteer Staff — DevOpsDays Cape Town

• Assisted with preparation for the event as well as operation during the event.

2018 Tutor — JEDI Workshop, Madagascar (Source.)

• Trained students in the following:
• • Navigating and using Jupyter notebooks.
• • Using Bash, navigating Linux and running scripts.
• • Using Docker and Singularity containers (creation and use).
• • Utilising Jupyter and software containers to develop reproducible scientific data processing pipelines in an astronomy focussed context.

2017 Tutor — Software Carpentry, Observatory

• Assisted in tutoring researchers in Bash, Python and Git at the Software Carpentry meeting during PyCon 2017.

2016 Hackathon Participant — H3ABioNet Cloud computing hackathon, University of Pretoria (Source.)

• Co-developed the Nextflow implementation of the H3Agwas pipeline for H3ABioNet. This involved Dockerising tools that the pipeline required and bringing them into use for Nextflow. GitHub Repository here.

2014 Tutor — Software Carpentry, University of Cape Town (Source.)

• Assisted in tutoring researchers in Bash, Python and Git at the Software Carpentry meeting at the University of Cape Town.

2012-2013 Volunteer Technical Assistant — University of the Western Cape

• Assisted with managing a computer laboratory for Computer Literacy. Duties included operating system installation, hardware repair and other administrative duties.

Education

Tertiary Education

2016-2019 M.Sc. Bioinformatics — South African National Bioinformatics Institute, University of the Western Cape

• Awarded Cum Laude.

2015 B.Sc. Hons Information Technology — University of Cape Town

• • Mini-thesis: de Beste, E., Stewart, G. and Gain, J. (2015). PyTeacher: Determining the Usefulness of Visual Learning for Computer Science.
• • Built a League of Legends statistics analysis platform for class on Visual Thinking and Visualisation. (Source.)
• • Awarded B.Sc. Hons in Information Technology Cum Laude.

2012-2014 B.Sc. Computer Science — University of the Western Cape

• Awarded B.Sc. Computer Science Cum Laude.

Secondary Education

2007-2011 National Senior Certificate — Fairbairn College

Academic Outputs

Journal Publications

2018 Ahmed, A.E., Mpangase, P.T., Panji, S., Baichoo, S., Souilmi, Y., Fadlelmola, F.M., Alghali, M., Aron, S., Bendou, H., de Beste, E. and Mbiyavanga, M. (2018). “Organizing and running bioinformatics hackathons within Africa: The H3ABioNet cloud computing experience”. AAS open research, 1. (Source.)

Thesis (M.Sc)

2019 de Beste, E. (2019). “Enabling the processing of bioinformatics workflows where data is located through the use of cloud and container technologies”. (Source.)

Posters

2017 de Beste, E., van Heusden, P., Christoffels, A., Bagula, A. (2017). “Moving Workfows to Data”. ISCB Africa ASBCB Conference on Bioinformatics. (Source.)

2019 de Beste, E., Christoffels, A., van Heusden, P. (2019). “Leveraging Open Source Technologies to Rapidly Deploy Infrastructure with Minimum Human Capacity”. Centre for High Performance Computing Annual Conference. (Source.)

Conferences

2013

• Centre for High Performance Computing Annual Conference — Cape Town, South Africa
• Participated in the local CHPC Student Cluster Competition. Achieved first place.

2014

• International Supercomputing Conference — Leipzig, Germany
• Participated in the international Student Cluster Challenge. Achieved first overall prize.
• Software Carpentry UCT — Cape Town, South Africa
• Tutored researchers and interested parties in scientific computing with BASH and Python.

2016

• Centre for High Performance Computing Annual Conference — East London, South Africa
• Mentored a team of students from the University of the Western Cape to compete in the 2016 local Student Cluster Competition. This team achieved first place.

2017

• Software Carpentry PyCon — Cape Town, South Africa
• Teaching assistant and tutor for scientific computing to researchers and interested parties.
• BSides Cape Town — Cape Town, South Africa
• Volunteered as staff for organising the event and doing duties on-floor such as coordination, audio-visual and more.
• ISCB Africa ASBCB Conference on Bioinformatics — Entebbe, Uganda
• Presented a poster on “Moving Workfows to Data”. Refer to above poster section.
• Global Emerging Pathogens Treatment Consortium (GET) workshop on Data Management Design and LIMS Usage — Freetown, Sierra Leone
• Presented work on my MSc. thesis, cloud and container technologies for bioinformatics.
• DevOpsDays Cape Town — Cape Town, South Africa
• Volunteered as staff for organising the event and doing duties on-floor such as coordination, audio-visual and more.

2018

• JEDI Workship and Conference — Madagascar
• Lead workshop for a group of students on reproducible science using software containers and pipeline/workflow language tools, specifically Singularity and the Common Workflow Language.
• DevOpsDays Cape Town — Cape Town, South Africa
• Volunteered as staff for organising the event and doing duties on-floor such as coordination, audio-visual and more.
• BSides Cape Town — Cape Town, South Africa
• Volunteered as staff for organising the event and doing duties on-floor such as coordination, audio-visual and more.

2019

• DevOpsDays Cape Town — Cape Town, South Africa
• Volunteered as staff for organising the event and doing duties on-floor such as coordination, audio-visual and more.
• Supercomputing Conference — Denver, Colorado, United States of America
• Awarded opportunity to attend through the STEM-TREK program in collaboration with the CHPC due to the work that my employer at the time was doing in collaboration with the CHPC’s HPC ecosystems project.
• Centre for High Performance Computing Annual Conference — Johannesburg, South Africa
• Presented a poster on “Leveraging Open Source Technologies to Rapidly Deploy Infrastructure with Minimum Human Capacity”. Refer to above poster section.

Achievements

2018 BSides Cape Town - Rite of Passage — Las Vegas, United States of America

• I was selected to attend DefCon (a cyber-security conference) 2018 in Las Vegas, as part of the BSides Cape Town “Rite of Passage” event. This was as a result of my volunteering work at said event. The program has me working on-the-floor at the BSides Las Vegas event. I managed recordings of the event, showing people to their conference venues and general management duties. After BSides Las Vegas, I was given access to DefCon where I attended numberous talks and demonstrations by elite hackers in the cyber-security sphere.

2014 First Place Overall Prize for the ISC’14 Student Cluster Challenge — International Supercomputing Conference, Leipzig, Germany

• The ISC Student Cluster Challenge is a multi-faceted competition that pits students from universities across the world together in order to design and implement high performance computing clusters on both hardware and software fronts. Students are tasked with building and optimising a networked compute cluster and running various scientific tools and benchmarks in order to achieve the most accurate or fastest results. My team and I finished first place in the International Supercomputing Conference Student Cluster Challenge in Germany.

Top 3 Academic Achiever in Third Year (undergraduate) — University of the Western Cape (Source.)

• Achieved third top student in the third year of the Computer Science bachelors degree at the University of the Western Cape.

2013 First Place for the CHPC Student Cluster Competition 2013 — Centre for High Performance Computing, Council for Scientific and Industrial Research

• Placed first in the Centre for High Performance Computing (CHPC) Student Cluster Competition. The CHPC run a competition that is a smaller scale version of the ISC Student Cluster Challenge, mentioned above. Students go through two phases of competition: The first is systems administration and Linux training and the second round is competing with a cluster that is designed by each team.

IBM Master the Mainframe Competition — University of the Western Cape

• IBM run an annual competition in which they introduce students to their “System z” platform. The goal is to educate students to leave them with hands-on and marketable skills in operating mainframe computing environments. I competed and completed the first phase of the competition.

2012 Golden Key Society — University of the Western Cape

• Invited to join the Golden Key International Honour Society. This is a program that aims to be made available to high academic achievers and provides them with experience and work opportunities, scholarships and more.

Top 5 Academic Achiever in First Year (undergraduate) — University of the Western Cape (Source.)

• Achieved placement in the top five best performers in the first year of the Computer Science bachelors degree at the University of the Western Cape.

Occupation

Cape Town, South Africa

The Mother City