Forcallpy’s documentation

About

The common way to mix Fortran and Python languages is to build Fortran functions/subroutines and to generate wrapping modules with f2py. You have then just to import these (natively compiled) modules from your Python code, its automagic.

For a project, we needed to interact in the other way: the Fortran code containing the main program should be able to call some Python functions, providing data as parameters and retrieving results.

As Python not only can be extended with C dynamic libraries, but can also be embedded into C programs using its internal C API, and Fortran (especially since Fortran 2003) has language defined interfaces with C, we decided to embed Python within Fortran via some C glue code.

This is the job of this forcallpy library : embed a Python interpreter within your Fortran program, allow to call Python code and manage Fortran/Python data exchange. All this with secure type checking and less possible data duplication.

License

This library is providen under the CeCILL-B License, which is compatible with BSD + author citation licenses. You can find the license inside sources repository, or directly inline on cecill.info site.

Code

Even without Fortran, the C part of this library (file forcallpy_cwrapper.c) can be used as an example of Python embedding, with support of Python multithreading, C level use of Python well know numpy library and definition of an internal C module.

Some of these developments required further investigations in the WEB to achieve a working library, comments in the code can help to achieve a similar integration in another context.

Examples

Note : you can find these examples in demo/forcallpy_demo.f90 Fortran program, coming with demo/forcallpy_demomodule.py Python module.

The Fortran code is written using Fortran 2003 syntax and language tools. You should compile your program using at least that version of the language (still using extension .f90 for files).

For adaptation to different Fortran Integer and Real sizes, functions which transmit data are postfixed with data size indication for parameters and if necessary for return value. For examples we will use _i4r8 (Integer 4 bytes and Real 8 bytes).

Interpreter begin and halt

You will have to explicitly start and stop the Python interpreter from your Fortran code using subroutines pyinit() and pyterm(). The interpreter can effectively start only one time in your program, once stopped it cannot be restarted. The pyinit() subroutine can be called several times to change some general running options (typically verbosity, error processing…).

USE forcallpy
USE, INTRINSIC :: iso_c_binding, ONLY: c_ptr, c_null_ptr, c_loc
CALL pyinit()
!
!
! Here you can use the interpreter (here takes place following examples).
!
!
CALL pyterm()

As you can see, from Fortran 2003 iso C bindings, we import several names that may be used to define data which can be exchanged with Python code.

When starting the interpreter, the global namespace used in further calls to statements execution or expressions evaluations is automatically filled with names:

  • __builtins__ providing access to standard Python names,
  • np alias for numpy package which is imported,
  • forcallpy (internal C module) providing tools for Python code interaction with Fortran — typically to allocate and manipulate dynamic Fortran arrays on the Python side.

The Python Interpreter within your Fortran process normally uses environment variables to retrieve Python related ones. Particularly the PYTHONPATH environment variable which lists complement paths where Python must search when importing modules or packages.

Here are data definitions for following examples:

INTEGER :: coderr
INTEGER :: intres
INTEGER,DIMENSION(10),TARGET :: tabint;
DOUBLE PRECISION,DIMENSION(5),TARGET :: tabdbl;
TYPE(c_ptr),DIMENSION(3) :: tabptr;
DOUBLE PRECISION,DIMENSION(4),TARGET :: tabres;

tabint(1:10) = (/ 3, 4, 5, 6, 7, 8, 9, 10, 11, 12  /)
tabdbl(1:5) = (/ 10.1, 10.3, 10.5, 10.7, 10.9 /)
tabptr(1:3) = (/ c_null_ptr, c_null_ptr , c_null_ptr  /)
tabres(1:4) = (/ 0.0, 0.0, 0.0, 0.0 /)
tabptr(2) = c_loc(tabdbl)

Running statements

The library provides a pyrun() subroutine for that. It can execute multiple statements of Python code, import modules or packages, define variables / functions / classes, etc. The Python code is simply providen as first parameter (string) to the subroutine.

CALL pyrun_i4r8('for i in range(3): print("This is a Python loop", i, "directly in fortran source.")')
CALL pyrun_i4r8('import math')
CALL pyrun_i4r8('print("Pi value is", math.pi)')
CALL pyrun_i4r8('import sys'//NEW_LINE('a')//'sys.path.insert(0, ".")')

Modifications made to global environment via pyrun() (names imported or defined…) are memorized and made available for furthers calls to the Python interpreter.

When calling the Python interpreter within Fortran, you can provide some data via specific named parameters of Fortran functions/subroutines, which are made available as local variables in Python when evaluating statements and expressions (these variables names are excluded from memorization in the globals namespace between different calls, you must choose other names for persistance or use another Python namespace).

CALL pyrun_i4r8('print("result =",3 + 4 *  x)', x=3.6D0)

One statement

To just execute a single statement, like a subroutine call, you can use pycall() subroutine, it works like pyrun() but with limitations. It must be seen as a Fortran CALL statement to a Python callable (function, object, class…). Here again, Python code is first parameter (string).

The example is used to transmit arrays to the subroutine.

CALL pyrun_i4r8('import forcallpy_demomodule')
CALL pysub_i4r8("forcallpy_demomodule.a_subroutine(av,zv,p)", &
                                av=tabint,zv=tabdbl,p=tabptr)

Calling Python function

This is more general than simply calling a function, you can execute any Python expression which produces a result, including a call to some function or any callable (class, object method, callable object…). You have just to use pyfct() function which returns a result. It has a complement suffix to specify the attended type of the result (Integers _i4 or _i8, Reals _r4 or _r8, _bool for Fortran LOGICAL). Python code is still first parameter (string).

In this example, not only we provide in arrays av and zv, but also one inout array yw which is modified in the called function.

intres = pyfct_i4r8_i4("int(forcallpy_demomodule.a_function(a,b,c,x,av,zv,yw))", &
                                a=2,b=-4,c=7,x=3.5D+0,av=tabint,zv=tabdbl,yw=tabres)

Parameters

Functions/subroutines which execute some providen Python code (pyrun(), pysub(), pyfct()) support the following optional parameters to transmit data from Fortran to Python, and in some case back from Python to Fortran.

When evaluating code in the Python interpreter, same names as those explicitly providen are defined as local variables in the context of the evaluation, and can be directly used.

The use of identified names allows to directly know data types in Fortran and Python codes. Arrays are transmitted via numpy type ndarray objects, which are used as wrappers directly accessing Fortran arrays storage (there is no data copy). Arrays providen as in parameter are mapped to read-only ndarrays (numpy ensures that data are not modified). Arrays providen as inout parameter are mapped to read-write ndarrays. Names with single values may be re-affected within Python code, this will have no effect on the Fortran side (this is why they are noted read only).

Names at begin of the alphabet correspond to Integers, names at end of the alphabet to Reals. Two char names ending with a v are read-only one dimension arrays (or vectors), and those ending with a w are one dimension read-write arrays (or write vectors).

Names Fortran Python Direction
a to h Integer int in (read only)
s to z Real float in (read only)
av to cv Integer(:) ndarray in (read only)
xv to zv Real(:) ndarray in (read only)
aw to cw Integer(:) ndarray inout (read write)
xw to zw Real(:) ndarray inout (read write)
p c_ptr(:) ndarray inout (read write)

Depending on API functions suffix, Integer can be 4 bytes or 8 bytes int (i4 or i8), and Real 4 bytes or 8 bytes float (r4 or r8). Numpy’s ndarray are directly mapped to corresponding types sizes (no data copy). We dont support mixing of different Integer sizes or different Real sizes : all paramaters of a type have same size.

Note

currently only 4 bytes int and 8 bytes float (i4r8) is coded in the library, but adding other variations is easy.

These multiple definitions or subroutines and functions allows to have correct type checking when calling Python wrapper functions, and help construction of numpy’s ndarrays to correctly wrap Fortran arrays.

The p parameter is a placeholder array to exchange some memory addresses, by example to transmit unsupported data, or to allow direct modification of a Fortran POINTER by the Python code.

API reference

To be written…

Building

Notes about library construction.

It uses a C compiler (…gcc), a Fortran compiler (…gfortran) and a cPython installation with numpy (use Python libs and numpy C header).

Source directory contains a manualbuild.sh script, I initially used to build the library (kept here if you need to make your own build).

Build process switched to cmake, with some constraint (multiple languages, integration of version in produced library, variation in location of numpy headers… and long time to fix some weird results).

Download

The project repository is available on SourceSup (french academic code repository), just retrieve it:

git clone https://git.renater.fr/forcallpy.git path_to_source

Build

You need to have cmake installed for your platform. Create a build directory somewhere, and ask cmake to create build files. Then use your plaftorm build system to compile and link the library (example on Linux with GNU make toolchain):

[email protected]:~…/build$ cmake path_to_source
[email protected]:~…/build$ make

Build with anaconda

Note

Examples are given using Anaconda on Linux, they must be adapted to your system.

Currently, using Anaconda compilers on MacOS failed due to some Fortran 2003 option used in the source (C/Fortran integration) and not available in the compiler on that system. You may try to use macports (see receipt below).

Note: This has not yet been tested on Windows platform.

Install Anaconda compilers:

$ conda install -c anaconda gcc_linux-64
$ conda install -c anaconda gfortran_linux-64

Request CMake to use these compilers (else it will search for them in standard paths), it should use Anaconda Python installation found in path. Be careful with env variable names, their case must be exact (typically: CMAKE_Fortran_COMPILER):

[email protected]:~…/build$ cmake path_to_source -DCMAKE_C_COMPILER=x86_64-conda_cos6-linux-gnu-gcc -DCMAKE_Fortran_COMPILER=x86_64-conda_cos6-linux-gnu-gfortran

Build with MacPorts

You must install MacPorts for your MacOSX version (it will require installation of XCode). See installation guides on MacPorts site (don’t miss to install cli dev tools of XCode and to agree xcodebuild licence).

Once MacPorts is ready, you can install gcc7, python36, py36-numpy and also cmake (the standard MacOSX bundled CMake dont find Python libraries from MacPorts):

port search --name 'gcc*'
sudo port install gcc7
hash -r
port select --list gcc
sudo port select --set gcc mp-gcc7
port search --name 'python3*'
sudo port install python36
hash -r
sudo port select --set python3 python36
port search --name '*numpy*'
sudo port install py36-numpy
port search --name 'cmake*'
sudo port install cmake
hash -r

Then, you can go back to the standard previous Build procedure.

Test

To execute the demo test program (built with library), you must cd to demo source directory (to have Python demo scripts available) and execute the demo binary from this directory:

[email protected]:~…/build$ cd path_to_source/demo
[email protected]:path_to_source/demo$ ~…/build/forcallpy_demo