The Democratization of HPC

Computational engineering and simulation tools, coupled with high-performance computing (HPC), offer an extraordinary opportunity for users to design better products, reduce time to market, streamline manufacturing, lower the cost of innovation and develop next-generation technologies that may otherwise be impossible.

Passive particle visualization of a contra-rotating, open-rotor simulation created using the Launch Ascent and Vehicle Aerodynamics (LAVA) code’s Cartesian higher-order accurate computational fluid dynamics solver. Red particles are seeded on the upstream blades, and blue particles are seeded on the aft blades. Solid colors are seeded on the tips, while faded colors are on the blade-trailing edges. (Image courtesy of Michael Barad and Cetin Kiris, NASA/Ames.)

How have these emerging technologies become more widely available?

The seismic shifts that have been occurring across the landscape of information technology are now making their way to the desk of simulation engineers. While supercomputing has been around since the 1960s, significant cost and ease-of-use barriers have prevented everyday scientists and engineers from accessing its power.

The rise of the Internet has spawned a whole new era in distributed computing that has driven down the cost of components and commoditized the business of HPC. Simulation software vendors are adapting to this changing landscape with new software licensing models and platforms that will enable small- and medium-size companies to tap into the power of HPC-driven simulation cost-effectively.

The Evolution of CFD Licensing

Before we get too far ahead of ourselves, let’s take a quick look back at the history of computational fluid dynamics (CFD) modeling, a tool of primary importance in the HPC simulation toolbag. Going back many years, CFD software licenses, whether with subscription or perpetual rights, came in a one-size-fits-all package; a seat, or session license, gave the user the right to run their simulations on whatever hardware resources they had available. The user owned the software and was responsible to make as much or as little use of it as they saw fit.

The rise of multiprocessor workstations earlier in this century, and later with the advent of multicore processors, motivated software vendors to reconsider that approach. Software vendors argued that licensing structures should reflect, at least to a degree, the value being provided by the CFD software as represented by overall simulation throughput. And thus the hybrid license model was born, typically consisting of a single session license with extended capabilities and costs as needed to cover the number of compute threads required. But even by 2010, most software vendors weren’t quite ready to tie the knot between cost and value as would be reflected in a true pay-for-usage model.

The Rise of the Cloud

The recent proliferation of cloud computing has sparked another licensing revolution, and today the pay-per-use movement is in full swing. All of the major software vendors now have such flexible licensing options, and many are facilitating the access of their software on cloud-based resources such as Amazon Web Services, Google Compute Engine and Microsoft Azure. Some software vendors, such as ANSYS, have gone as far as developing their own virtual-private cloud platforms. Wim Slagter, director of HPC and cloud marketing at ANSYS recently described the ANSYS Enterprise Cloud as follows.

The ANSYS Enterprise Cloud solution is a turnkey, end-to-end simulation platform—including remote graphics, HPC job orchestration and remote session management—for global deployment. It comes along with a web interface that enables users to monitor and make changes to their simulations virtually anywhere and at any time.

Meanwhile, third parties are also filling the need for containerized or platform approaches that allow users convenient and secure access to HPC resources for engineering simulation purposes. Vendors in this space most notably include Gompute, Rescale, Nimbix, Sabalcore and UberCloud, among others.

ANSYS Enterprise Cloud. (Image courtesy of ANSYS.)

The second piece of the puzzle, in addition to the availability of cloud hardware, is the cost-effective availability of the software needed to utilize the hardware resources. According to Slagter, ANSYS’ Elastic Licensing model “works by providing a pool of ‘elastic units,’ which you consume at a specified hourly rate that varies by the type of activity (pre- or post-processing, solving, etc.).” As mentioned previously, many leading software companies are also moving in this direction or already have similar offerings.

According to Addison Snell, CEO of research firm Intersect 360, about a third of all HPC is currently being executed in the cloud, primarily by large enterprise corporations. On the other hand, small engineering and manufacturing firms are the engines of technology innovation producing nearly 16 times more patents per employee than larger firms. These firms, which are not yet participating in HPC to a large degree, will benefit the most dramatically from the trends discussed here.

Planning Your HPC Strategy

A critical but often overlooked aspect of a cost-effective simulation strategy is metering your simulation usage in order to have accurate projections of future simulation needs. Depending upon your needs, you may wish to use a third-party software use-metering tool, or a Windows HPC–based tool, but for a small consulting firm, the most effective method could be just manually tracking software use with a spreadsheet. It only takes a few minutes for your simulation engineers to report their simulation usage. For each simulation performed, Resolved Analytics asks its engineers to log the start and end times, license number, number of processors used, backlogged simulations and any clarifying notes. Such tracking, in addition to forming the basis of a software/hardware resource strategy, can also help you discover inefficiencies in your simulation workflow.

Consider the graph below reflecting a hypothetical company’s CFD usage during a two-month period. The blue bars represent HPC tokens, or processors, being utilized for active simulations, and the red bars indicate the simulation backlog in queue. The hypothetical user owned three session licenses and 32 HPC tokens during this time period.

A company’s CFD usage can appear like this. Blue bars represent HPC tokens, or processors, being utilized for active simulations. Red bars are the simulation backlog in queue.

It is important to note that, at times, the user was either under- or oversubscribed. The average daily use during this time period is 27 HPC tokens across three session licenses. Having this data allows the user to analyze alternative future licensing and hardware/cloud strategies to minimize total CFD costs going forward. For example, one could target the baseline load, as shown in the figure above, or 24 HPC tokens across two sessions, for internal execution, and target all additional workload for the cloud. Cloud CPU costs are based upon a processor cost of $0.08/CPU hour and an on-demand (pay-per-use) CFD license cost of $20/hour (regular clock time). At Resolved Analytics, we assume those cloud projects can be efficiently parallelized across 16 threads.

Annual Cost	Baseline		Alternative
	#	Cost	#	Cost
Sessions Licenses	3	$37,500	2	$25,000
HPC Token Licenses	32	$32,000	24	$24,000
Local CPU Hours	9,855		8,760
Cloud CPU Hours	0		1,095	$87
On-Demand Cloud Clock Hours			68.42	$1,369
Total Cost of CFD		$69,500		$50,456

The analysis above demonstrates how a typical user could possibly save a meaningful percentage of total CFD costs, reduce the amount of internal computer hardware needed, extend workflow flexibility and reduce backlogs. On the other hand, the user would experience a small but meaningful increase in CFD administration time required in order to set up and process these simulations in the cloud. Specific benefits and value added should always be made on a case-by-case basis and based upon software and cloud provider quotations and internal personnel estimations.

The times they are a-changing, as a Nobel Laureate has sung, and it behooves simulation engineers these days begin to adjust their workflows to better take advantage of the new frontiers in HPC to not fall behind the competition.

ANSYS has sponsored ENGINEERING.com to write this article. It has provided no editorial input. All opinions are mine. —Stewart Bible

About the Author

Stewart Bible is a principal engineer at Resolved Analytics. His interest in computational fluid dynamics began with his undergraduate and graduate research at the University of Kentucky, where he encountered his first commercial CFD code, CFD2000, ironically enough, in the year 1999. His current interests are in the areas of multiphysics and uncertainty quantification, with particular emphasis on medical devices, renewable energy and air pollution control systems.