USA Banner

Official US Government Icon

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure Site Icon

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

U.S. Department of Transportation U.S. Department of Transportation Icon United States Department of Transportation United States Department of Transportation

Public Roads - Sept/Oct 2008

Date:
Sept/Oct 2008
Issue No:
Vol. 72 No. 2
Publication Number:
FHWA-HRT-08-006
Table of Contents

Using Supercomputers to Determine Bridge Loads

by Kornel Kerenyi, Tanju Sofu, and Junke Guo

FHWA and the Argonne National Laboratory are using multidimensional programs to study hydrodynamic forces on flooded bridge decks.

ker1
Computer modeling can enable engineers to design bridges that are sturdier against floodwaters. This section of I-80 in Iowa was inundated after heavy rains in June 2008, perhaps relieving pressure on the Cedar River bridges in the background.

Bridges are a vital component of the Nation's transportation network. Evaluating their stability and structural response after flooding is critical to highway safety. When a bridge crossing a waterway is partially or entirely submerged during a flood, the water can exert significant loading on the deck and threaten its integrity. Being able to estimate hydrodynamic loading accurately can help designers and transportation agencies build better, stronger bridges.

Accurately modeling turbulence and sediment transport is essential for estimating the impact of scour on bridges. Researchers typically address problems in science and engineering through two complementary approaches: experimental and analytical (or theoretical). In many applications, such as the fluid mechanics of streams and the impacts on bridges, the governing equations are nonlinear and, except in special circumstances, analytical solutions are not available. In addition, fluid mechanics applications often are multidimensional in nature and time-dependent, which further complicates attempts to understand and model real-life turbulence and scour conditions.

Pursuing a third approach, researchers at the Federal Highway Administration (FHWA) and the new Transportation Research and Analysis Computing Center (TRACC) in West Chicago, IL, a partnership of the U.S. Department of Transportation (USDOT) and the Argonne National Laboratory, are using supercomputers with multidimensional hydraulics programs to closely mimic real-life conditions. The researchers verify the computers' output using flume models at FHWA's Turner-Fairbank Highway Research Center (TFHRC) in McLean, VA, indicating that computers might play an even more valuable role in bridge hydraulics and safety.

"As computer technology improves and the ability to accurately model turbulence and sediment transport becomes more robust, transportation agencies will benefit in both the planning and design of bridges through improved hydraulic and scour estimation," says Kevin Flora, hydraulics branch chief for structure maintenance and investigation at the California Department of Transportation (Caltrans).

The Third Approach

Effective use of prototypic experiments is a key approach to understanding real-life phenomena. For many fluid dynamics applications, such as those associated with bridge hydraulics, full-scale tests are not possible. Researchers use smaller scale, and perhaps simplified, representations of the physical configuration, and they extrapolate the results to apply to actual conditions. Some uncertainty remains in this extrapolation, however, related to the use of simplified experiments to predict the behavior of complex physical systems.

Computational fluid dynamics (CFD) attempts to address these issues and complement the experimental and analytical approaches through numerical solutions. CFD is a branch of fluid mechanics that uses numerical methods and algorithms to analyze and solve problems involving fluid flows, such as water. Researchers use computers to perform the millions of calculations necessary to simulate the interaction of fluids with the complex surfaces involved in engineering. CFD enables scientists and engineers to perform numerical simulations in the laboratory and significantly reduce the amount of experimentation and overall cost. CFD is a highly interdisciplinary research area at the interface of physics, applied mathematics, and computer science.

CFD calculations are numerical solutions to the underlying equations that represent the flow of fluids. As a result, a computational grid — focusing many computers in a network on a single problem — can approximate the true geometry of a given physical condition. In the CFD approach, researchers can include actual observed boundary conditions that might be impossible to represent in a laboratory experiment and perform parametric studies on material properties and physical conditions that might be expensive or time-consuming to perform experimentally.

Although CFD can complement laboratory experiments and theoretical approaches, generating accurate numerical solutions requires establishing fine-mesh computational grids to represent the actual geometry of the problem. Also, in time-dependent cases, the accuracy and numerical stability of the solutions often require small time steps to capture the influence of temporal variations in the flow field.

Computing solutions on these grids can require many calculations, so the use of computers is essential. Development of powerful supercomputers, including massively parallel computers (which contain hundreds of processors sharing a common hardware platform), and adaptation of the CFD techniques for use with these supercomputers in recent years is enabling engineers and scientists to perform these complex CFD calculations.

"By comparing the computed results to some experimental information or to analytical solutions under simplified conditions, analysts can verify and validate the numerical approach," explains TRACC Director David P. Weber. "With verified and validated results, analysts can then feel much more confident in performing calculations on more complex representations of the true physical conditions. Thus, the three approaches of experiment, analysis, and computation are complementary."

Establishing TRACC

The Argonne National Laboratory is leading the initiative to establish a high-performance computing center, TRACC, to pursue transportation research and development (R&D) programs. Argonne analysts provide technical support to researchers at TFHRC on CFD simulations.

ker2
This photo shows an experimental model of a typical U.S. highway bridge deck partially inundated in a tilting flume at FHWA's J. Sterling Jones Hydraulics Research Laboratory.

The overall objective of this effort is to establish validated computational practices to address the transportation community's research needs in bridge hydraulics. Traditionally, the bridge hydraulics work relies on scaled experiments to provide measurements for flow field, which is the velocity and turbulence of a fluid as functions of position and time. Now, however, parallel computers and commercially available software provide an opportunity to shift the focus of these evaluations to the CFD domain. After being validated using the data from a limited set of experiments, high-fidelity CFD simulations can be used to expand the range and scope of parametric studies. The CFD simulations also can be used to predict the effects of scaling by studying differences between the reduced-scale experiments and full-scale bridges.

Most recently, reduced-scale experiments conducted at TFHRC's J. Sterling Jones Hydraulics Research Laboratory established the foundations of a CFD-based simulation methodology. Researchers at Argonne and TFHRC worked together to study CFD techniques for simulating open-channel flow around inundated bridges.

"The use of supercomputers and CFD code can potentially overcome the inherent problems of modeling sediment in the laboratory," says Flora. "Once physically based models of sediment transport and scour are developed, computer simulations will provide engineers with new insights into better design and mitigate for scour at structures."

Research Considerations

The Argonne-TFHRC research included analysis of lift forces (FL) (or simply lift), forces produced perpendicular to the flow of a fluid; drag forces (FD), forces exerted on objects in the path of fluids; and moments (M), the tendency to cause rotation about a point or axis, on inundated bridge decks under various flow configurations. The two research teams investigated the applicability of commercial CFD software for predicting flow field and evaluating lift, drag forces, and moments. They compared the results with laboratory data using an ultraprecise force balance (which is a mounting device to measure hydrodynamic forces on scale model bridge decks) to measure lift, drag force, and moments for various inundation ratios and flow rates.

ker3
This definition sketch shows a cross-section of the model six-girder bridge deck tested during the study. It defines the direction of the hydrodynamic forces and flow acting on the flooded bridge deck model. The sketch also shows the point of application of forces, that is, where the moments act. The three boxes below the bridge deck define the Froude number, Reynolds number, and the inundation ratio used in the design charts.
Source: FHWA

They also used Particle Image Velocimetry (PIV) to study flow fields around submerged model bridge decks. PIV is a noninvasive measurement technique to visualize flow distributions. The technique involves adding microscopically small, highly reflective particles to the flow and using a laser to illuminate a thin layer of the flow, so only the particles in that light sheet reflect the laser's light. Using cameras pointed with different angles toward the light sheet, researchers can capture images and an algorithm based on statistical probability to determine the speed and direction of the moving particles. PIV-related tasks included the following:

  • Assessing applicable phenomena for open-channel flow and identifying appropriate models — free surface, two-dimensional (2-D) versus three-dimensional (3-D) to include the effects of channel wall, appropriate flow profile at the inlet, and roughness at the bottom surface
  • Studying the sensitivity of the CFD solutions to the grid structure and mesh density (number of computational cells per unit area)
  • Identifying the most suitable turbulence model in terms of accuracy and computational requirements

The researchers also adopted numerical simulations using two turbulence models, high-Reynolds number k-(epsilon) model and large eddy simulation (LES), to resolve unsteady turbulent flow. Turbulent flow is an irregular condition in which the flow particles show a random variation with time and space coordinates. The k-(epsilon) model simulates transport of both the turbulent kinetic energy (k) and the turbulent energy dissipation rate (epsilon). The rationale for using LES is to simulate the larger and more easily resolvable scales of the motions while accepting that LES will not represent the smaller scales accurately.

The researchers examined the agreement between experimental data and the results of CFD simulations for a typical U.S. highway girder deck, corresponding to Froude numbers (dimensionless ratio) ranging from 0.12 to 0.40 characterizing the open-channel flow. That is, the Froude number is a ratio comparing inertial and gravitational forces. If the Froude number is less than 1, the flow is subcritical, and if it is greater than 1, the flow is supercritical. The TRACC researchers used STAR-CD (software for modeling fluid dynamics), while the TFHRC team used Fluent, Inc. CFD software in their simulations.

The researchers integrated forces and moments over the surface of the bridge deck along the flow and perpendicular directions, respectively. When calculating the lift over the bridge deck, however, they excluded the component of buoyancy, as they were interested only in hydrodynamic loading, not static loading.

Advances in CFD research provide the basis for modeling the multiphase nature of open-channel flow that consists of air and water separated by a free surface. In multiphase flows, each phase has individually defined physical properties and flow fields. In both the STAR-CD and Fluent approaches, the researchers had three multiphase flow models available. The first was the volume of fluid (VOF) model, which can model two or more nonmixable fluids by solving a single set of continuum equations and tracking the volume fraction of each of the component fluids throughout the domain. Applications of the VOF model include stratified flows, free-surface flows, filling, sloshing, the motion of large bubbles in a liquid, the motion of liquid after a dam break, the prediction of jet breakup (surface tension), and the steady or transient tracking of any liquid-gas interface.

In the VOF model, in addition to the conservation equation for mass and momentum, one has to introduce the volume fraction of phase in computational cell. In each control volume, the volume fraction of all phases sum to unity. The fields for all flow variables are shared by the phases, as they represent volume-averaged values. Thus, the variables and properties in any given cell are representative of one of the phases, or a mixture of phases, depending upon the volume fraction values.

Second was the mixture model, in which the flow consists of a continuous phase and one or more dispersed phases. Although it is computationally efficient, this Lagrangian approach is suitable only for dispersed flows and cannot be used to accurately resolve the stratified nature of free-surface flows. The third was the Eulerian model, which is a multiphase model in which the phases are treated as interpenetrating continua coexisting in the flow domain. The Eulerian model provides a general framework for both dispersed and stratified multiphase flows. However, the free surface calculated with this approach is usually less sharp in comparison with the VOF method, making the latter the preferred model. Therefore, the researchers are testing the VOF computational method with both STAR-CD and Fluent to capture the impact of free surface reasonably accurately and efficiently.

ker4
This photo shows a submerged sixgirder bridge deck model using PIV, a nonintrusive method to measure flow velocities, to visualize the flow field around the deck. A powerful laser light illuminates PIV tracer particles, and high-speed cameras track their displacements, which then are converted into velocities.

Experimental Setup At TFHRC

The TFHRC researchers conducted the laboratory experiments using a 12.8-meter (42-foot)-long, 0.4-meter (1.3-foot)-wide, and 0.5-meter (1.6-foot)-high Plexiglas® rectangular flume. They set the flume horizontally and controlled the depth of flow with an automatic adjustable tailgate located at the downstream end. The team used a 0.054-cubic-meter (2.0-cubic-feet)-per-second pumping system to supply the flow. They measured the water surface level using ultrasonic sensors at two cross sections along the flume. An electromagnetic flow meter measured the discharge. They used an Acoustic Doppler Velocimeter probe to measure the velocity distribution (speed of the water) of the flow.

The researchers constructed a six-girder bridge deck shape, which is typical of U.S. highways, using yellow Plexiglas and adopting a geometrical reduction scale of 1:40 based on the depths, maximum discharges (flow rate), and inundations (submergence level) available. This scaled bridge deck is optimal to produce values ranging from low to high flows, all at subcritical Froude numbers (less than 1) in the upstream flow.

To measure forces induced by the bridge deck model in the flow direction (drag) and simultaneously perpendicular to the flow direction (lift) using electric strain gauges, the research team constructed an ultraprecise force balance, which can capture even small forces. They mounted the bridge deck model between brackets attached to a measuring platform. To test different submergence ratios, they mounted the platform and model flexibly to allow vertical positioning of the bridge deck. The flexibility was important to perform testing at various submergence levels of the bridge deck.

They conducted the experiments for water approach velocities ranging from 0.25 to 0.50 meter (0.8 foot to 1.6 feet) per second. They kept the flow depth for all experiments constant at 0.25 meter (0.8 foot). They varied the Froude number within a range of 0.16 to 0.32 and the submergence of the bridge deck from slight submergence of the girders to complete overtopping of the bridge deck, inundation ratio, h* = 0.29 to 3.2.

The force balance experiments used the PIV technique to visualize and measure flow fields for the submerged bridge deck model. The PIV technique is an optical flow diagnostic based on the interaction of light refraction and scattering, using nonhomogeneous media. The fluid motion is made visible by tracking the locations of small tracer particles at two instances of time. The researchers then use the particle displacement as a function of time to infer the velocity flow field. They analyzed the data in a format that could be compared with the CFD modeling.

TRACC Computing Facility

TRACC is a general purpose advanced computing and visualization facility available to the transportation community for a broad spectrum of applications. Discussions among staff at USDOT's Research and Innovative Technology Administration and FHWA identified specific initial applications and technologies for assigning the highest priority for research and development (R&D) and user support.

The TRACC components include high-performance computing, visualization, and networking systems. To take advantage of Argonne's extensive experience in acquiring and operating similar facilities, TRACC acquired the system components and then set them up in dedicated facilities at the DuPage National Technology Park near the DuPage Airport in Illinois.

The TRACC computational cluster is a 512-core, customized LS-1 system from Linux Networx that comprises 128 computational nodes, each with two dual-core Advanced Micro Devices, Inc. OpteronTM 2216 central processing units and 4 gigabytes of random access memory; a DataDirectTM Networks storage system consisting of 240 terabytes of shared RAID (Redundant Array of Inexpensive (or Independent) Disks or (Drives)) storage that is expandable to 750 terabytes; a high-bandwidth, low-latency InfiniBand network for internode computations; and a high-bandwidth Gigabit Ethernet management network.

TRACC also provides scientific visualization capabilities through the National Center for Supercomputing Applications' Technology, Research, Education, and Commercialization Center at the same location. TRACC meets the needs for visualization of multidimensional data via a high-performance graphics cluster linked with a 15-panel liquid crystal display (LCD) tiled display and a portal optimized for visual simulation and high-speed broadband connectivity.

Test Results

The researchers conducted the CFD simulations using models of an 8.0-meter (26.2-foot)-long, 0.34-meter (1.1-foot)-wide, 0.5-meter (1.6-foot)-high rectangular channel. The study simulated both 2-D and 3-D models. The researchers placed the bridge deck 4.4 meters (14.4 feet) downstream of the inlet of the channel to obtain a fully developed flow upstream. They situated the outlet far away from the deck to avoid the influence of the outlet boundary on the flow field around it. The team conducted the simulations using Fluent and STAR-CD for approach velocities ranging from 0.25 meter (0.8 foot) to 0.50 meter (1.6 feet) per second. The Froude number varied within the range of 0.16 to 0.32, and the team ran the simulations for various inundation ratios. The researchers modeled the flow domain using either tetrahedral cells (in Fluent) or hexahedral cells (in STAR-CD), both with gradually refined mesh structure in the vicinity of the bridge deck. (The gradual refinement of mesh in the vicinity of the deck reduced computational time. If the researchers had refined the mesh in the complete domain, it would have increased the number of cells, which would require more computational time.)

ker5
The TRACC parallel computing system, shown here, provided researchers the visualization capabilities necessary for their experiments. The TRACC computational cluster consists of a 512-core, customized LS-1 system from Linux Networx.

The team presented the experimental and simulation results in dimensionless form for various Froude numbers (which characterize the open-channel flow) and the inundation ratio (which characterizes the flooding height). They presented the measured and simulated forces and moments in terms of dimensionless coefficients as drag, lift, and moment coefficients. They performed the Fluent simulations for 200 seconds using LES as well as the k-(epsilon) turbulence model. Due to fluctuating values of CD (drag force coefficient), CL (lift force coefficient), and CM (moment coefficient) with time, they averaged the final values from 50 to 200 seconds. They checked the velocity distribution along the depth of flow at various locations upstream of the bridge deck, and profiles clearly showed that fully developed flow occurs upstream of the bridge deck.

Defining the Terms

As customary, the measured and calculated drag, lift, and moments often are expressed in terms of dimensionless drag, lift, and moment coefficients. Depending on the value of upstream water level, hu, the drag coefficient, CD, is defined as

C subscript D equals F subscript D over 0.5 times p times V superscript 2 times s times L for h asterisk less than 1 where h asterisk equals h subscript u minus h subscript dC subscript D equals F subscript D over 0.5 times p times V superscript 2 times s times L times h asterisk superscript F for h asterisk less than 1

the lift coefficient CL is defined as

and the moment coefficient CM is defined as

C subscript m equals M over 0.5 times p times V superscript 2 times W superscript 2 times L superscript FC subscript equals F subscript L over 0.5 times p times V superscript 2 times W times L superscript F

Drag Force

FD

Lift Force

FL

Moment

M

Drag Force Coefficient

CD

Lift Force Coefficient

CL

Moment Coefficient

CM

Density of Fluid

r

Upstream Velocity

V

Gravitational Acceleration

G

Upstream Flow Depth

hu

Froude Number

Fr

Inundation Depth

hb

Inundation Ratio

h*

Deck Width

W

Deck Height

S

Deck Length

L

The experimental data can be compared with the Fluent and STAR-CD simulation results for the drag coefficient as a function of dimensionless inundation ratio, h*, and Froude number, Fr, which characterizes the flow rate. The simulated drag coefficient using Fluent increases as the inundation ratio increases from 1.0 to 1.5, and then tends to level off for the inundated bridge deck. In other words, increasing the water level over the submerged bridge causes CD to reach its maximum and then stay constant, similar to what the researchers observed in the experiments. Although the simulated data using STAR-CD are slightly above those observed from Fluent, they are generally compatible with each other. In practice, an empirical equation can be used to estimate CD, including the effects of the inundation ratio and Froude number.

ker6
This computer visualization shows the meshed geometry of the 3-D bridge deck for the CFD model in Fluent. The bridge deck is in yellow/brown. The portion of the flume wall behind the deck is in dark blue, and the portion below it is in light blue. Both portions are cross-hatched in black lines to indicate the triangular mesh placed in the flow domain, with the lines and triangles denser for the more detailed flow field investigation close to the bridge deck.

The researchers also looked at the variation of lift coefficient values, CL, versus inundation ratio using Fluent and STAR-CD simulation results along with the experimental data. Like the experiments, the component of buoyancy is excluded from the lift forces calculated with the CFD models. Both the simulated and experimental lift coefficients come out to be negative, indicating that the net hydrodynamic force is directed downward. The lift coefficient value decreases as the inundation ratio decreases (that is, it gets more negative) until h* = 1.0. The maximum (negative) value of CL is observed between h* = 0.5 and 1.0. Like the drag coefficient, when inundation is very large, the effect of free surface diminishes and CL tends to a constant. This means that when the bridge deck is immersed well below the water level, the net lift force on the bridge deck does not change.

ker7
The graph shows experimentally determined and computer simulated drag coefficients versus inundation ratio for the six-girder bridge deck model. One blue line roughly tracks with the Froude Number 0.22, Reynolds Number 20292 data points (hollow triangles), and the other roughly tracks with the Froude Number 0.32 and Reynolds Number 28965 data points (black triangles). The plot shows that the drag coefficient is constant for higher inundation ratios. Source: FHWA.

The researchers also compared the moment results of the drag and lift forces for the simulation data and experimental data. Again, the simulated results from Fluent and STAR-CD were close to the experimental results.

The research team conducted the PIV experiments in a special flume using transparent Plexiglas models. The PIV flow field analysis was necessary to calibrate the CFD models. The researchers observed strong agreement between computer simulation and PIV experiments; comparison of the contracted flow fields under the bridge deck especially showed excellent conformity.

ker8
The graph shows experimentally determined and computer simulated lift coefficients versus inundation ratio for the six-girder bridge deck model. The lift coefficient is negative for all tested inundation ratios. This observation corresponds to a down-pull force. Source: FHWA.

Significant Findings

The k-(epsilon) turbulence model and LES together with the VOF method simulates faithfully the flow past the bridge deck in an open channel. The predictions of drag, lift, and moment coefficients through the numerical modeling show a trend similar to the flume experimental results. Future bridge designers can use the numerical CFD model without much effort to obtain the coefficients on a bridge deck in an open channel for various flow conditions encountered in practice. The Argonne and FHWA researchers are undertaking additional simulations for other shapes of bridge decks to predict the drag, lift, and moments with the help of CFD.

ker9
The graph shows experimentally determined and computer simulated moment coefficients versus inundation ratio for the six-girder bridge deck model. The moment coefficients have a maximum when the girders of the bridge deck are inundated. Source: FHWA.

"The validation of the computer modeling gives confidence in the results from computer program analysis of complex hydraulics structures without using physical models," says Michael Fazio, deputy director of research and innovation at the Utah Department of Transportation. "This is a great advancement for hydraulics research."

FHWA's future strategic plan for hydraulics R&D proposes to move away gradually from physical experiments and use more CFD modeling to develop design guidance for practitioners. This successful collaboration between Argonne and FHWA's hydraulics R&D program is the first step toward that vision.

"The use of CFD software on high-speed computers will undoubtedly play an increasingly important role in the future for understanding scour potential at bridges," says Caltrans's Flora. "Rapid development of various bridge scenarios will be much easier through numeric simulation on supercomputers than through physical modeling in the lab."

ker10
ker11
Shown here are the velocity contour plots for the six-girder bridge deck using Fluent CFD code (top) and PIV technology (bottom). The red area below the bridge deck indicates higher flow velocities, green represents intermediate velocities, and blue indicates areas with negative velocities. The blue region between the girders shows negative velocities where the water is trapped under the bridge deck and the water is flowing backward. The higher velocities are mostly below the bridge deck due to the higher turbulence level, while the regions away from the deck show intermediate velocities due to lower turbulence levels. The negative velocities indicate that the water is flowing in a circular motion creating underpressure between the girders. The underpressure system creates turning moments that are dangerous to the bridge deck. Source: Argonne-TRACC, FHWA.

 


Kornel Kerenyi is a hydraulic research program manager in FHWA's Office of Infrastructure R&D. He coordinates FHWA's hydraulic and hydrology research activities with State and local agencies, academia, and various partners and customers, and he manages the Hydraulics Laboratory. He was previously a research engineer for GKY & Associates, Inc. and supervised the support staff in the data collection and analysis for this study. Kerenyi holds a doctorate in fluid mechanics and hydraulic steel structures from the Vienna University of Technology in Austria.

Tanju Sofu manages the Engineering Simulation and Safety Analysis section in the Engineering Analysis Department at Argonne. He specializes in large-scale computational physics and fluid dynamics simulations on high-performance computing platforms and has extensive experience with a wide range of engineering systems analyses involving multidimensional, multiscale, multiphysics phenomena. Sofu holds a doctorate from The University of Tennessee Knoxville.

Junke Guo is an assistant professor in the department of Civil Engineering at the Peter Kiewit Institute at the University of Nebraska-Omaha. He received his doctorate in fluid mechanics and hydraulics from Colorado State University. His research interests include CFD, turbulent boundary layer flows, open-channel turbulence and sediment transport, and environmental fluid mechanics. His current research emphasizes CFD applications in transportation-related flows.

For more information, contact Kornel Kerenyi at 202-493-3142, kornel.kerenyi@dot.gov; Tanju Sofu at 630-262-9673, tsofu@anl.gov; or Junke Guo at 402-554-3873, junkeguo@mail.unomaha.edu.