Skip to content
Tutorials Center
My learning

Topics

Lesson 32 of 39 ยท Parallel and Scientific Computing

do concurrent, Array Operations and Vectorization

Write fast, parallel-friendly Fortran: whole-array syntax, where and forall masks, do concurrent loops with locality, memory layout, and vectorization.

Fortran's biggest advantage for scientific code is that it lets you say what you want done to arrays and leaves the how to the compiler. Whole-array expressions, masks and do concurrent all tell the compiler something an ordinary loop cannot: these operations are independent, so reorder, vectorize, or run them in parallel as you see fit. This lesson shows how to give that information and how to avoid defeating it.

Whole-array expressions

Arithmetic operators and intrinsic functions work on entire arrays:

FORTRAN
real :: x(1000), y(1000), z(1000)

x = 1.0                  ! fill
y = [(0.001 * i, i = 1, 1000)]
z = 2.0 * x + y**2       ! every element
z = sin(y) + sqrt(abs(x))
print *, sum(z), maxval(z), minloc(z)

Operands must be conformable: the same shape, or scalars (which are broadcast). The right side is evaluated completely before the assignment, so a = a + a(n:1:-1) (add the reversed array) is well defined with no temporary array needed in your source.

Array sections select parts: x(2:n-1), x(::2) (every second element), a(:, j) (column j). Combined, they replace loops with one line. A centered difference over a grid:

FORTRAN
dudx(2:n-1) = (u(3:n) - u(1:n-2)) / (2.0 * dx)
Python Stand-in: array thinking without loops

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

where: masked assignment

where applies assignments only where a condition holds:

FORTRAN
where (temperature > 100.0)
  state = 3
elsewhere (temperature > 0.0)
  state = 2
elsewhere
  state = 1
end where

where (x > 0.0) y = log(x)       ! one-line form

It replaces an if inside a loop and tells the compiler the iterations are independent. Inside where you may only assign to arrays that conform to the mask.

Intrinsics that reduce and reshape

sum, product, maxval, minval, count, any, all, dot_product and matmul reduce or combine arrays; many take dim= and mask=:

FORTRAN
real :: a(3, 4)
print *, sum(a, dim=1)               ! column sums (4 values)
print *, sum(a, mask=a > 0.0)        ! sum of positives
print *, count(a > 1.0)              ! how many elements exceed 1

reshape, pack, spread, merge and cshift rearrange data without loops. merge(a, b, mask) is the elementwise ternary: y = merge(x, 0.0, x > 0.0).

do concurrent

do concurrent is a loop whose iterations you promise are independent:

FORTRAN
integer :: i
real :: a(n), b(n), c(n)

do concurrent (i = 1:n)
  c(i) = a(i) * b(i) + 1.0
end do

With that promise the compiler may run iterations in any order, in parallel on several cores, on a GPU, or vectorized. You are responsible for the promise being true: iteration i must not read what another iteration writes, and no two iterations may write the same element.

Restrictions: you cannot exit, return or stop out of it, and procedures called inside must be pure. See recursion and pure procedures.

Index masks, nested indices and locality

FORTRAN
do concurrent (i = 2:n-1, j = 2:m-1, mask(i, j) > 0)
  unew(i, j) = 0.25 * (u(i-1, j) + u(i+1, j) + u(i, j-1) + u(i, j+1))
end do

Several indices make one big iteration space, the optional mask filters it. Fortran 2018 adds locality specifiers that state how variables behave per iteration:

FORTRAN
do concurrent (i = 1:n) local(tmp) shared(a, b)
  tmp = a(i) * 2.0
  b(i) = tmp + 1.0
end do

local(tmp) gives each iteration a private tmp (without it, a shared scratch variable would be a data race), and shared names data used by all. local_init copies the initial value in, and reduce(+: total) (Fortran 2023) sums across iterations. Compilers such as gfortran can parallelize with -ftree-parallelize-loops=N, and nvfortran and ifx can offload to GPUs.

Three ways to say independent

Whole arrays a = b + c
where Masked assignment
do concurrent Loops with no dependencies

The compiler decides how to run them

1 / 2

A dependency is not independent

This loop looks innocent but has a loop-carried dependency: each element needs the previous one.

FORTRAN
do i = 2, n
  a(i) = a(i - 1) + b(i)      ! running sum: NOT independent
end do

It cannot be a do concurrent. Keep it as an ordinary sequential loop, or restructure the algorithm (a parallel prefix sum is a different, more involved method). Putting a dependent loop into do concurrent produces wrong answers, and nothing stops you. Compare:

Python Stand-in: independent and dependent loops

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Memory layout: column-major

Fortran stores a(i, j) with the first index varying fastest in memory (column-major). To walk memory in order, make the first index the inner loop:

FORTRAN
do j = 1, m
  do i = 1, n          ! inner loop over the first index: contiguous
    c(i, j) = a(i, j) + b(i, j)
  end do
end do

Swapping the loops makes each access jump n elements, and for large arrays it can be several times slower. This also explains why a(:, j) (a column) is contiguous and a(i, :) (a row) is strided. NumPy defaults to the opposite, row-major layout.

Avoiding temporaries and aliasing

When an assignment overlaps, as in a(2:n) = a(1:n-1), the compiler may need to copy. That is correct but can cost time. Passing arrays to procedures by assumed-shape (a(:)) never forces a copy unless the actual argument is a non-contiguous section. Declare contiguous when you know the data is contiguous. Unlike C, Fortran assumes dummy arguments do not alias each other, which is a big reason Fortran loops optimize so well. Violating that rule (passing the same array twice with one intent(out)) gives wrong results.

Common mistakes

  • Putting a loop-carried dependency inside do concurrent.
  • Missing local(...) for scratch variables.
  • Non-conformable arrays (different sizes in one expression).
  • Looping over the second index in the inner loop.
  • Calling an impure procedure from do concurrent.
  • Forgetting -O2 (or -O3 -march=native) and concluding that Fortran is slow.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What does a do concurrent loop promise the compiler?

    You promise independence; the compiler may reorder, vectorize or parallelize.
  2. 2. In Fortran, which array index varies fastest in memory?

    Fortran is column-major, so the first index is contiguous. Make it the innermost loop.
  3. 3. What does where (x > 0.0) y = log(x) do?

    The mask restricts the assignment to the elements where the condition holds.
  4. 4. What does local(tmp) do in do concurrent?

    Without it, a shared scratch variable would be a data race when iterations run in parallel.

Summary

  • Whole-array expressions, sections, where, merge and reductions replace most loops.
  • do concurrent promises independent iterations; add local/shared and keep called procedures pure.
  • Loop-carried dependencies are not independent.
  • Fortran is column-major: loop over the first index innermost.
  • Compile with optimization to benefit from all of this.

Next: OpenMP in Fortran