13.7.8. Macro “relauncherCodeFlowrateMpiStandalone.C”

13.7.8.1. Objective

The goal of this macro is to show how to handle a code run on several threads with another memory paradigm: when the TThreadedRun instance is relying on shared memory (leading to possible thread-safe problem, as discussed in TThreadedRun), the MPI implementation is based on the separation of the memory. The communication is made through messages. In order to this, the usual sequential runner will be removed and another runner will be called to do the job. The flowrate code is provided with Uranie and has been also used and discussed throughout these macros.

Warning

This macro is different from the one discussed previously in Macro “relauncherCodeFlowrateMPI.C” as here, one wants to handle MPI through standalone compilation (so C++ without using the ROOT interface)

13.7.8.2. Macro

#include "TAttribute.h"
#include "TCodeEval.h"
#include "TFlatScript.h"
#include "TDataServer.h"
#include "TFlatResult.h"
#include "TMpiRun.h"
#include "TLauncher2.h"

void usage(const char *scmd)
{
    printf("\n");
    printf("Usage: %s initType Type \n", scmd);
    printf("\n");
    printf("  Specific method to test standalone MPI distribution with reoptimizer dummy case.\n");
    printf("\t   initType     : The MPI initialisation chosen (explicit <-> 0 or implicit <-> 1) \n");    
    printf("\n");
}

int main(int argc, char **argv)
{    

    int initType=0; // explicit <-> 0 or implicit <-> 1
    if(argc>1)
    {
        for (int iarg=1; iarg<argc; iarg++)
        {
            if( string(argv[iarg]) == "initType") { initType=atof(argv[iarg+1]); iarg++;}
            else
            {
                cout<<"==============================================================="<<endl;
                cout<<"Don't know the following option ? "<<string(argv[iarg])<<endl;
                usage("mpirun -np N ./${EXECNAME}");
                cout<<"==============================================================="<<endl;
                return 0;
            }
        }
    }

    // variables
    URANIE::DataServer::TAttribute *rw = new URANIE::DataServer::TAttribute("rw");
    URANIE::DataServer::TAttribute *r = new URANIE::DataServer::TAttribute("r");
    URANIE::DataServer::TAttribute *tu = new URANIE::DataServer::TAttribute("tu");
    URANIE::DataServer::TAttribute *tl = new URANIE::DataServer::TAttribute("tl");
    URANIE::DataServer::TAttribute *hu = new URANIE::DataServer::TAttribute("hu");
    URANIE::DataServer::TAttribute *hl = new URANIE::DataServer::TAttribute("hl");
    URANIE::DataServer::TAttribute *l = new URANIE::DataServer::TAttribute("l");
    URANIE::DataServer::TAttribute *kw = new URANIE::DataServer::TAttribute("kw");

    // Create the output attribute
    URANIE::DataServer::TAttribute *yhat = new URANIE::DataServer::TAttribute("yhat");
    URANIE::DataServer::TAttribute *d = new URANIE::DataServer::TAttribute("d");

    // Set the reference input file and the key for each input attributes
    URANIE::Relauncher::TFlatScript fin("flowrate_input_with_values_rows.in");
    fin.setInputs(8, rw, r, tu, tl, hu, hl, l, kw);

    // The output file of the code
    URANIE::Relauncher::TFlatResult fout("_output_flowrate_withRow_.dat");
    fout.setOutputs(2, yhat, d);

    // Instanciation de mon code
    URANIE::Relauncher::TCodeEval mycode("flowrate -s -r");
    //mycode.setOldTmpDir();
    mycode.addInputFile(&fin);
    mycode.addOutputFile(&fout);

    // Create a runner
    URANIE::MpiRelauncher::TMpiRun *run=NULL;    
    if(initType==0)
    {
        MPI_Init(&argc, &argv);
        run = new URANIE::MpiRelauncher::TMpiRun(&mycode);
    }
    else if(initType==1)
        run = new URANIE::MpiRelauncher::TMpiRun(&mycode, &argc, &argv);

    run->startSlave();    
    if(run->onMaster())
    {
        // Create the TDS
        URANIE::DataServer::TDataServer tds("launchFlowrate", "launching flowrate with mpi");
        mycode.addAllInputs(&tds);
	tds.fileDataRead("flowrateUniformDesign.dat", kFALSE, kTRUE);

	URANIE::Relauncher::TLauncher2 lanceur(&tds, run);

        // resolution
        lanceur.solverLoop();

        std::stringstream outname; outname<<"_output_testFlowrateMPIStandalone_"<<initType<<"_.dat";
        tds.exportData( outname.str().c_str() );

        run->stopSlave();
    }

    delete run;
    
    return 0;
}

Here there are few differences the most obvious one being the proper C++ structure:

there are several includes that has to be included on top the macro in order to provided the needed headers;
the main function has to follow a classical form int main(int argc, char **argv) which allows to modify parameters on the fly once compiled (as for the initType integer here through a dedicated loop). The consequence of this signature will be discussed later on;
a usage function has been written in order to provide interactive guidelines on how to deal with this code.

Apart from this, from the variable definition to the assessor description, the structure is similar to the one discussed previously (see Macro). Once this point is reached there are two ways to construct the runner:

with the classical constructor. In this case, as one is not running a ROOT session, the constructor will not found the interactive command parameters and the MPI_Init will not automatically be called, thus meaning that one needs to call this method with the proper parameters as done below:

        MPI_Init(&argc, &argv);
        run = new URANIE::MpiRelauncher::TMpiRun(&mycode);

with the new constructor provided with v4.6.0 whose argument are the usual assessor and the interactive parameters:

        run = new URANIE::MpiRelauncher::TMpiRun(&mycode, &argc, &argv);

The final difference is the fact that this code has to be properly compiled, using MPI-complient compilor in order to be able to produce an executable that will be run later on. This can be done like this:

mpicc -o relaunStand relauncherCodeFlowrateMpiStandalone.C `echo $URANIECPPFLAG $URANIELDFLAG` -lstdc++

where mpicc is the c++ mpi-complient compilor, the argument after -o is the executable name, then one can see the input file while the rest is provinding all compiling and linking flag in order to reach the goal. Finally, instead of using the classical root -l command, one will use this line

mpirun -np N ./relaunStand initType XX

where the N part should be replaced by the number of requested threads (always strickly greater than 1) and XX should be replaced either by 0 or 1 depending on whether one wants to use the classical TMpiRun constructor. Once run, this macro also leads to the following plots (both initialisation are shown).

13.7.8.3. Graph

../../_images/relauncherCodeFlowrateMpiStandalone.png — Figure 13.51 Representation of the output as a function of the first input with a colZ option when using either the classical or dedicated constructor