Minor build system and testing improvements and bug fixes

Minor build system and testing improvements and bug fixes

Bug fixes

• #455: A shell script stack implementation was failing for certain bash
  versions, most notably versions provided by default on CentOS 6
• #452: Coarray extension/convenience module was installed in the wrong place
  due to typo
• A typo was causing a CMake error when MPI was missing and has been fixed
• An issue where the build script was trying to query version information from
  dependencies with the wrong flag has been fixed
• #461: Some tests were failing if the build system detected < 3 cores
  available because the tests require more than 2 images and they were being
  run anyway, without oversubscription.
• #457: A problem with CMake caching was causing some system introspection
  relevant to windows systems to fail, and has now been resolved

Enhancements

• #267: CMake can now use properties (on test target executables) to specify
  the minimum number of images. This ensures ctest/make check/make test
  will oversubscribe these tests appropriately when not enough cores are
  avaialable.
• #413: Add a make dist target to help OpenCoarrays developers produce
  release assets: source tarballs, SHA 256 checksums and PGP signatures
  if GPG is setup locally

Installation

Please see the installation instructions for more details on how to build and install this version of OpenCoarrays

---

Bug fixes and better QA

Bug fixes

• #435 reported by @jbmaggard: Wrong type declaration for MPI variable
• Use standard Fortran for integration test
  coarray_distributed_transpose.F90; no calls to MPI
• Replace mpif90 with MPI standards endorsed mpifort in build script
• #381 Implement sane SOVERSIONing scheme and build static+dynamic lib by
  default:
  • 0 if GFortran < 6.0.0 OR non-GNU Fortran compiler
  • 1 if 6.0.0 <= GFortran < 7.0.0
  • 2 if GFortran >= 7.0.0
• #366 Build static and dynamic/shared libs, ensure parallel builds do not
  fail due to CMake Fortran .mod file idiosyncrasies
• Assert correct number of images in `async_comp_alloc_2
• #268 Configure caf and cafrun at build time in sane/canonical way
• #437 abstract MPI runtime flags needed for enabling failed image support
  and default to enabling them
• Fix Windows (WSL) install script, was failing due to versioning logic
• #447 Upstream changes to the GFortran array descriptor in trunk/8.x
  broke some OpenCoarrays functionality. Library and private descriptor
  header were updated to resolve this issue and wrapped in #ifdef guards
  to ensure backwards compatibility.

Enhancements

• #275 use generated caf script to compile tests and cafrun to run them
  • caf is well behaved enough that it can be passed to CMake as a Fortran
    compiler
  • Simple tests are compiled by directly calling caf with a CMake macro
  • One or two of the more complicated integration tests are compiled by
    setting CMAKE_Fortran_COMPILER to bin/caf
• Improved system introspection and support for
  • Intel MPI
  • Windows builds (other than WSL) (work in progress)
• Experimental OpenSHMEM based version added to git repo, but not available
  in releases

Installation

Please see the installation instructions for more details on how to build and install this version of OpenCoarrays

---

Bug fix release

Bug fixes

• #325 install.sh will help user download and install Xcode command line tools (CLT) on Mac OS which is needed to compile code on Macs if it is missing or excessively outdated
• #378 mpirun changed to mpiexec and mpif90 changed to mpifort, consistent with MPI standard recommendations
• #398 fix some erroneous internal library calls when exceptions are encountered
• #399 fix allocation of allocatable components of coarray derived types
• #402 increase portability of install script install.sh by removing dependency on tree command
• #404 fix issue where install.sh was ignoring user specified -m/--with-cmake CMake location
• #406 use secure https/encrypted sources for fetching and installing prerequisites to help mitigate the possibility of a man-in-the-middle (MITM) attack
• #408 switch to downloading the gzipped GCC archive rather than the bz2 because gzip is more common/portable and because bz2 compressed GCC archives seem to have disappeared for some recent releases on GCC mirrors
• #411 fix bug causing event_post to hang when going over the network (multiple nodes)
• #422 clarify runtime error messages for partially or un-implemented
• Corrected logic to control under which circumstances certain tests are run that may only work correctly under GFortran 6 or GFortran 7

Enhancements

• #410 add option --disable-bootstrap to install.sh to help speed builds of GCC when bootstrapping is not required because a recent GCC is performing the build
• #424 add option to install.sh to download the requested package from a user specified URL
• Prevent developer's advanced Makefiles and GASNet directory from being distributed with release tarballs

Installation

Please see the installation instructions for more details on how to build and install this version of OpenCoarrays

---

Experimental support for fault tolerance & asymmetric PGAS

Experimental failed-image detection

This feature is experimental and requires an MPI implementation with certain experimental, proposed MPIX functions and constants. These are present in MPICH 3.2 which is now the default, officially-supported MPI back end. Some/most/all of these features are available in OpenMPI through the ULFM project. If the build systems detects the required features are present it will default to enabling failed images support.

See the src/tests/unit/fail_images subdirectory for demonstrations of the new support for Fortran 2015 features related to fault-tolerance, including the following:

• The iso_fortran_envintrinsic module now contains a new stat_failed_image value that the compiler and runtime library assign to the stat argument of parallel synchronization and communication statements to signal that an image has ceased responding, a scenario considered increasingly likely as computing platforms approach exaflop scalability.
• A new failed_images() function returns an array containing the image numbers failed images.

Richer support for fault-tolerant execution necessitates the Fortran 2015 team feature. However, this release enables users to start experimenting with fault-tolerance in advance of anticipated team support.

Additional experimental support for derived-type coarrays with allocatable components:

This adds on an incomplete implementation in the 1.8.0 release for supporting derived type coarrays with allocatable components. Fortran requires that array coarrays have the same shape and bounds on each image. For intrinsic coarrays, this implies memory allocations that are invariant under image-number transformations. With coarrays of derived type, however, one can allocate data that are of varying size and shape across images:

type foo
   real, allocatable :: bar(:)
end type
type(foo) :: foobar[*]

which is a powerful enabler when used judiciously in problems that require such flexibility of distributed, non-uniform memory allocations. This feature requires GCC/GFortran 7.1 since compiler side interface changes were required to support this feature. This features is still considered experimental and is not yet fully implemented in all regards, so use we do not yet recommend using it in production.

Bug fixes

• #309 stop statements with numeric and string arguments were not handled correctly and are now fixed.
• #342 A maintainer flag was added to turn on tests intended only for OpenCoarrays developers. This can be turned on using OPENCOARRAYS_DEVELOPER=TRUE as an environment variable or by turning on the CAF_RUN_DEVELOPER_TESTS advanced CMake option.
• #354 sync (all|images) without stat= was not erroring out under certain error conditions. This is now resolved.
• #376 The CI build matrix was expanded for more complete test coverage using GCC 6 and 7 for compiling the library.
• #383 cafrun had a typo (missing space) with the -v flag. Thanks to @LaHaine for pointing this out.
• #384 install.sh does not work on HPC Linux. A new script was added to install OpenCoarrays on HPC Linux.
• #385 install.sh was not correctly reporting the path to the newly installed CMake under certain circumstances. This is now fixed.
• #388 Better build system robustness and diagnostics
• Excessive debug output has been reduced when building the Debug configuration
• Tests' oversubscription is now reduced

Installation

Please see the installation instructions for more details on how to build and install this version of OpenCoarrays

---

Small GCC compatibility patches release

Bug fixes

• #342: An "advanced" RUN_DEVELOPER_TESTS flag was added to the CMake build system so that users and package managers won't get failing tests for known regressions and partially implemented features. By default these developer tests are not run, and must be explicitly enabled through ccmake, cmake-gui or passing -DRUN_DEVELOPER_TESTS:BOOL=ON to cmake when called on the command line. Alternatively, OpenCoarrays developers can set the OPENCOARRAYS_DEVELOPER environment variable to ON in their dot files to always enable these tests. The default behavior, RUN_DEVELOPER_TESTS=OFF will skip tests corresponding to the convert-before-put regression (#292) and the async_alloc_comp corresponding to the yet-to-be-fully-implemented asynchronous allocation of allocatable components of coarray objects (#260)

Installation

Please see the installation instructions for more details on how to build and install this version of OpenCoarrays

---

More robust installation and MPI handling

Bug fixes

• #359: Under some circumstances the installation was getting confused
  about which MPI implementation to use, especially when mpiexec was
  not in PATH. A more robust solution was implemented, fixing this
  bug, and additional checks were added to expose any similar issues in
  the future.
• #317: co_reduce would segfault on some systems (Fedora 25, maybe
  others) because the MPI compiler wrappers were adding
  -wl,-z,noexecstack. This is also reported as GCC bug 71729. If
  the build system detects -z noexecstack is being passed to the
  liker, it will replace it with -z execstack, thereby fixing this
  bug.
• A typo in the top level CMakeLists.txt was fixed that could cause
  additional problems with MPI integration in some rare
  cases.
• #365: caf_single.c is maintained by and included in
  GFortran. Duplication in OpenCoarrays was an unneeded maintenance
  burden and caused some confusion for OpenCoarrays package
  maintainers.

Minimum allowable CMake was updated from 3.0 to 3.2

This was done because CMake 3.2 has

• Better support for Fortran
• Better capabilities for finding and integrating with MPI

Default MPICH installation was upgraded to 3.2

This was done in anticipation of supporting Fortran 2015 failed-image features that are currently available with the following:

• MPICH 3.2 or later,
• GCC 7.1 or later,
• OpenCoarrays vehre/failed-images branch.

The OpenCorrays installer patches the MPICH source to address a known MPICH issue. A new OpenCoarrays release will be produced after merging the failed-image support into the master branch by merging pull request #370 .

Miscellaneous

• Further enhancements to make the install system more robust in it's
  handling of MPI.
• Various typos and broken links in documentation were
  corrected. Thanks to @ocaisa for pointing some of these out.
• Other changes to README.md and documentation including fixing links,
  improved formatting and aesthetics, and SEO

Installation

Please see the installation instructions for more details on how to build and install this version of OpenCoarrays

---

Bug fixes, minor enhancements, and document cleanup

Bug fixes:

1. #172: Unexpected behavior with co_reduce
   • A bug when the value attribute was present for binary operator functions passed to co_reduce is now resolved
2. #340: Set saner/safer/smarter permissions for installed caf and cafrun scripts, now 0555
3. Replaced less portable #!/usr/bin/bash shebang with more portable #!/usr/bin/env bash shebang for caf and cafrun scripts.
4. #352: The caf compiler wrapper script did not accept the --wrapping flag, even though it was listed in the --help documentation.
5. #356: GCC's download-prerequisites script was not working on Mac OS due to different names of checksum programs (md5 and SHA). The opencoarrays install script will now patch this script so that it won't fail when checking the GCC checksums. A patch was also submitted to GCC to remedy the issue.

Enhancements

1. Update github repository name from opencoarrays to OpenCoarrays
2. co_reduce can now operate on character arrays
3. STATUS.md was deleted since it was difficult to maintain and frequently out of date. Check the issues page for the latest status information, as well as the status badges at the top of the README.md file
4. Issue and pull request templates added, but are currently too verbose and need further editing and refinement.

Installation

Please see the installation instructions for more details on how to build and install this version of OpenCoarrays

---

Windows installer & bugfixes

🌁 🐧 This release introduces support for building OpenCoarrays in the Windows Subsystem for Linux (WSL) Ubuntu bash shell via the new windows-install.sh installation script with the following limitations:

Windows Installation requirements

• WSL release 14936 or later, which provides Ubuntu 16.04.
• Ubuntu 16.04 requirements:
  • Windows Insider Preview,
  • "Fast" updates option, and
  • Run do-release-upgrade to update from the default Ubuntu 14.04 to 16.04.

The installation script uses apt-get to install GCC 5.4.0, CMake, and MPICH. Windows users who desire a newer version of GCC are welcome to submit a request via our Issues page and suggest a method for updating. Previously attempted upgrade methods are described here.

Installation

Please see the installation instructions for more details on how to build and install this version of OpenCoarrays

---

v1.8.2 Small installation enhancement and added regression tests

Enhancements

• Patch gcc's contributed download_prerequisites script to detect presence of wget and try alternates like curl if it is missing

Added Regression Tests

Added GCC 7 trunk regression tests. Thanks to @egiovan for reporting segfaults with event post when num_images > 2 and the type conversion before coarray puts regressions with GCC trunk.

Installation

Please see the installation instructions for more details on how to build and install this version of OpenCoarrays

---

Bug fixes & partial, experimental support for allocatable components

Enhancements

• experimental partial support for allocatable components. The
  register_alloc_vector.f90 test is known to currently fail by timing
  out after hanging. Please see issue #260 and #285-#289 for a discussion
  of what has yet to be implemented. For examples that are currently
  working, please see:
  • register_alloc_comp_1.f90
  • register_alloc_comp_2.f90
  • register_alloc_comp_3.f90
• #39 Use the more canonical CMake invocation of
  FC=gfortran CC=gcc cmake <path/to/src/dir> and rely on the more
  robust FindMPI CMake module, rather than relying on passing the MPI
  wrappers as the compiler to CMake. The previous method of passing the
  compiler wrappers as FC and CC still currently works, but is
  deprecated and no longer supported.
• #246 MPI and Fortran is handled more robustly: since Fortran .mod
  files have no standardized ABI, we fall back to using mpif.h
  instead of the mpi module if this will cause problems, and if that
  fails as well, then the auxiliary opencoarrays module is not built,
  since the core library only requires a functional C compiler and C
  MPI implementation which is also now checked at build time.
• More tests added and duplicate tests deleted
• #244 Disable building and running of tests that rely on assembly code
  for use by package maintainers using -DSKIP_ASSEMBLY_DEPS:BOOL=ON
• #245 Support for Building shared libraries with
  -DBUILD_SHARED_LIBS:BOOL=ON
• #204 Expose caf_init() and caf_finalize() via
  -DCAF_EXPOSE_INIT_FINALIZE:BOOL=ON so that e.g. Python can call
  coarray Fortran

Bug-fixes

• #254 bug fix for strided send-gets (image A gets a remote coarray
  object from image B and assigns it to memory on image C with strided
  access)
• #234 Library header file libcaf.h updated to include missing
  entities
• #234 Work around GCC's download_prerequisites script failing when a
  particular branch of GCC is requested to be built with install.sh
• #239 tests updated to ensure compliance with Fortran standard,
  code relying on compiler extensions replaced
• #265 Fix typo in OpenMPI flags specified for tests when they are
  oversubscribed

---

To view the issue related to the enhancement or bugfix on Github, please
point your browser to
https://github.com/sourceryinstitute/opencoarrays/issues and append
/<issue-number> to the end of the URL, replacing the text between
and including the angle braces with the actual issue number specified in
these release notes.