Documentation / User's manual in C++ :

IV.3. External Code

IV.3. External Code
	Chapter IV. The Launcher module

IV.3. External Code

In Uranie, an external code is defined by a command line, and input and output attributes which are stored in ASCII files, named respectively input files and output files.

If, because of the way the function are defined, a TLauncherFunction is bound to have only double-precision attributes as input, dealing with a code allows more flexibility and it is now possible to handle both vectors and strings as inputs and outputs.

To launch an external code, it is necessary to define a TLauncher object. This object is used to launch the code defined in a TCode object and that use TInputFile input files and TOutputFile output files (as shown in Figure IV.2).

Figure IV.2. Inheritance diagram for the class TLauncher

This section is organised as follow: we first describe the definition of TInputFile and TOutputFile in Section IV.3.1, then the construction of a TCode in Section IV.3.2, and finally the construction of a TLauncher in Section IV.3.3. A schematic view of the code launching process is represented in Figure IV.3.

Figure IV.3. Schematic description of the launcher procedure when using an external code. Yellow boxes show instances of class, and green ones are precision about attributes. The design-of-experiments part can be replaced by an already-existing database.

Schematic description of the launcher procedure when using an external code. Yellow boxes show instances of class, and green ones are precision about attributes. The design-of-experiments part can be replaced by an already-existing database.

The paragraphs below describe the main part of this procedure, as it is displayed by Figure IV.3, remaining at the class level. It can indeed be decomposed as follow:

A design-of-experiments is produced (the corresponding part is highlighted in the red-dashed rectangle). On top of this, a file is "attached" to every input attribute, defining a way to spot the place where to find information when the launcher would have to run the code. This part is furthered discussed in Section IV.3.1.2
One or more TOutputFile are created and the output attributes are attached to them, specifying, as for previous step, how to extract the information once the launcher will be running the code. This part is also described in more details in Section IV.3.1.3.
A TCode is created by giving the input TDataServer along with the string used to launch the code in a terminal. The output files are then attached to the TCode object.
The TLauncher object is created by giving the input TDataServer and the previously defined TCode. During the creation, a working directory is made where input files are copied. Then the loop is performed, running the code as many times as there are points in the design-of-experiments. This loop can be decomposed as well in few key-steps:
1. An initialisation: a sub-directory is created for this specific computation, in which the input files are copied and modified to used the requested input values defined in the design-of-experiments and stored in the TDSNtupleD.
2. Running the external code itself
3. Parsing the output files to collect the desired values and send them back to the launcher which fills the ntuple of results. This ntuple is merged to the input one at the end of the process.

Precision on the described steps

One can ask to use more than one process in order to proceed with the computation. In this case, the main launcher calls the TLauncherMulti to distribute events one-by-one. The results are gathered and the output TDataServer is saved every 5 events (disregarding the number of processes requested). This is further discussed in Section IV.4.
The usual procedure is to write a file for every event processed (as most of the time the code to be run is slow) and back up the complete output file every 5 events (as said previously). In the unusual case where the code is fast, this step is considerably slowing down the process of running and collecting the outputs.
It is possible to skip this step, by calling the run method passing the following optional argument: "noIntermediateSaved". This has to be used with caution as the output file will be opened only once at the very end of the process, so if this step is not reach for one reason or another, everything will be lost and one would have to re-run from the beginning.

IV.3.1. Code input and output files

Uranie can manage different kinds of input and output files. We'll describe below these different IO file formats after a short introduction on the way these files are implemented.

IV.3.1.1. General discussion

All the input/output files that will be discussed in the following sections (meaning restrained to the Launcher module) are based on a class named TamponTexte, which comprehends a file as a group of field, apart one to another thanks to separators. It is absolutely crucial to keep this is mind.

With the introduction of the vectors and strings in version 3.10.0, more complex interactions with files were introduced: how to differ two iterations of a single vector and how to differentiate a double from a string. This depends highly on the nature of the input/output file under consideration, whether it is just a text file used as database (in this case it depends mostly on the way you've written the code that generates/parses it) or whether it corresponds to a stricter kind of file, for instance a piece of code (c++/python/zsh). In the latter case, strings and vectors are not written in the same way. To take this into account, a rule has been defined (commonly to both input and output files, both in the Launcher and Relauncher module). There is a method for any kind of file to define properties of vector and string objects:

setVectorProperties(string beg, string delim, string end): the first element is the string beginning of the vector (usually "[" for python, "(" for zsh/sh, nothing...), the second one is the delimiter between iterators (usually "," for c++/python, blank for zsh/sh...) and the last one is the end of the string (usually the opposite character of the beginning one).
setStringProperties(string beg, string end): the first and second elements are respectively the beginning and ending character used for string (oftenly """).

In order to illustrate how to deal with a complicated case (when the order provided by the database is not the one needed by the code, but also dealing with vectors and strings as inputs), one can have a look at the examples provided from Section XIV.4.29 to Section XIV.4.33. The output files, on the other hand, are more thoroughly (and explicitly) discussed in Section IV.3.1.3.

IV.3.1.2. Input files

When a code needs input parameters from one or more input files, Uranie must be able to duplicate and modify these files in order to set the input parameter under discussion to its requested value. Uranie can do this using different kinds of input files. Before going through the following sections, which will present the way input files are used with the flowrate problem presented in Section IV.1.2, a small discussion will introduced the different kinds of input file, some of their properties and the way they are usually declared.

The classes used to either do the substitution in the input file or to recreate a new input file to feed the code, were not much discussed up to now in this documentation as they were created on the fly, by the TCode itself at the initialisation. This is done by the mean of checking the list of EFileType for every attribute (one attribute can indeed have more than one file and type in case the code, or codes, need to have this information in different files, with different formats). An explicit declaration is however possibly needed for two reasons:

Letting the TCode object do, as described above, one can not choose the order to which the attribute are attached to the corresponding dumper object, and so, the order in which they will be written (which can be of uttermost importance for some format). Indeed the TCode object will follow the order in which the attribute have been added to the TDataServer by hand (but in the case the user just has to invert the corresponding attributes) or, more likely, through a database (e.g. using the fileDataRead method).
In the case of vector and strings handling, it might be necessary to specify the delimiter, beginning and ending sign (as already introduced above and in the last part of Section II.3.1.1).

The classes to handles inputs files can be split into two different types:

the one creating an input file from scratch: done with TInputFileRecreate class, it can use different implementations according to the type requested by the user, through the TAttribute::EFileType provided in the setFileKey method (discussed below). The possible values are: kNewKey, kNewTDS, kNewRow and kNewColumn. These flags are discussed below.
the ones which modify an existing file: this is done with specific classes, depending on the type of inputs used by the code and this is also precised by the user through the TAttribute::EFileType provided in the setFileKey method. The possible values are: kKey (creating implicitly a TInputFileKey object), kFlag (creating a TInputFileFlag object), and both kXMLAttribute and kXMLField (used in the TInputFileXML instance).

IV.3.1.2.1. Input files with "key=value" format

Let us consider the flowrate input file with "key = value" format, with the eight parameters (, , , , , , , ):

#
# 
# INPUT FILE with KEYS for the "FLOWREATE" code
# \date   2008-04-22 12:53:35
#


date = 123456 ;


#########################
##
##  exclude points
##
chu = 1050;
chl = 770;
cr  = 1100;
##
#########################

#########################
##
##  parameters : 8
##
Rw = 0.0500 ;
R = 33366.67 ;
Tu = 63070.0 ;
Tl = 116.00 ;
Hu = 1110.00 ;
Hl = 768.57;
L = 1200.0 ;
Kw = 11732.14 ;
##
#########################


#########################
##
## to simulate CPU time
##
## normal         1 :
## min     10000000 : 1.160u 0.000s 0:01.16 100.0%
## max    100000000 : 11.600u 0.010s 0:11.61 100.0%
##
nLoop = 1;
##
#########################

end = 6;

Please note that this file has been created out of Uranie and that the values are not managed by Uranie yet: it is just a model of input file for the code.

Now we write the beginning of the Uranie script that is going to define attributes and assign them a place in the input file:

{
// Define the DataServer
TDataServer *tds = new TDataServer("tdsFlowrate", "Doe for Flowrate");

// Add the study attributes ( min, max and nominal values)
tds->addAttribute( new TUniformDistribution("rw", 0.05, 0.15));           
tds->addAttribute( new TUniformDistribution("r", 100.0, 50000.0));
tds->addAttribute( new TUniformDistribution("tu", 63070.0, 115600.0));
tds->addAttribute( new TUniformDistribution("tl", 63.1, 116.0));
tds->addAttribute( new TUniformDistribution("hu", 990.0, 1110.0));
tds->addAttribute( new TUniformDistribution("hl", 700.0, 820.0));
tds->addAttribute( new TUniformDistribution("l", 1120.0, 1680.0));
tds->addAttribute( new TUniformDistribution("kw", 9855.0, 12045.0));

// The reference input file
TString sJDDReference = TString("flowrate_input_with_keys.in");            

// Set the reference input file and the key for each input attributes
tds->getAttribute("rw")->setFileKey(sJDDReference, "Rw");                  
tds->getAttribute("r")->setFileKey(sJDDReference, "R");
tds->getAttribute("tu")->setFileKey(sJDDReference, "Tu");
tds->getAttribute("tl")->setFileKey(sJDDReference, "Tl");
tds->getAttribute("hu")->setFileKey(sJDDReference, "Hu");
tds->getAttribute("hl")->setFileKey(sJDDReference, "Hl");
tds->getAttribute("l")->setFileKey(sJDDReference, "L");
tds->getAttribute("kw")->setFileKey(sJDDReference, "Kw");
...

Specification of a "key=value" input file for the input variables

	In this first part, definition of the attributes of the `TDataServer`
	Definition of the reference input file (the file copied before);
	At each of these lines, definition of the place the parameter will be set, thanks to the `setFileKey` method, that takes as parameters the input file name and the name of the variable to replace. For example, the "rw" parameter will be set in the file `flowrate_input_with_keys.in` where the `Rw` value stands.

After these definitions, if we create the code launcher and launch it (what will be presented in the following sections), then the input file would be modified by writing each sample value in a new input file. For example, the following input file is one of the generated ones:

#
#
# INPUT FILE with KEYS for the "FLOWREATE" code
# \date   2008-04-22 12:53:35
#


date = 123456 ;


#########################
##
##  exclude points
##
chu = 1050;
chl = 770;
cr  = 1100;
##
#########################

#########################
##
##  parameters : 8
##
Rw = 5.194485e-02 ;
R = 8.610663e+03 ;
Tu = 7.644908e+04 ;
Tl = 1.003881e+02 ;
Hu = 1.058603e+03 ;
Hl = 7.224698e+02;
L = 1.148747e+03 ;
Kw = 1.003128e+04 ;
##
#########################


#########################
##
## to simulate CPU time
##
## normal         1 :
## min     10000000 : 1.160u 0.000s 0:01.16 100.0%
## max    100000000 : 11.600u 0.010s 0:11.61 100.0%
##
nLoop = 1;
##
#########################

end = 6;

In this file, we will note the different values of the parameters, that have been modified compared to the "model" presented in the beginning of this section.

Tip

No matter the number of times the key appears in the input file, the value will only be replaced once!

IV.3.1.2.2. Input files with flag format

Let us consider the flowrate input file with flag format, with the eight parameters (, , , , , , , ). As seen in Section IV.1.2.3.1.2, it is necessary to write "flags" where the values will be written, as in the example below:

# 
# INPUT FILE with FLAG for the "FLOWREATE" code
# \date   2008-04-22 12:55:17
#

new Implicit_Steady_State sch {
    frottement_paroi { @Rw@ @R@ }
    tinit               0.0
    tmax                1000000.
    nb_pas_dt_max       1500
    dt_min              @Hu@
    dt_max              @Hl@
    facsec              1000000.
    kW                  @Kw@
    information_Tu      Champ_Uniforme 1        @Tu@
    information_Tl      Champ_Uniforme 1        @Tl@
    information_L {
        precision  @L@
    }	
    convergence {
        criterion relative_max_du_dt
        precision  @Rw@
    }

    stop_criterium {
        ch_abcsissa_hu 1050
        ch_ordinate_hl 770
        c_radius 1100
    }

    Solveur Newton3 {
        max_iter_matrice        1
        max_iter_implicite      1
        date                    5654321
        seuil_convg_implicite   1.e-6
        assemblage_implicite    10
        solveur_lineaire        BiCGS
            preconditionneur    ILU
            seuil_resol_implicite 1.e-5
    }
}

Please note that this file has not been created out of Uranie since the flags have been written manually: this file is just a model of input files for the code.

Now we write the beginning of the Uranie script that is going to define attributes and to assign them a place in the input file:

// Define the DataServer
TDataServer *tds = new TDataServer("tdsFlowrate", "Doe for Flowrate");

// Add the study attributes ( min, max and nominal values)
tds->addAttribute( new TUniformDistribution("rw", 0.05, 0.15));        
tds->addAttribute( new TUniformDistribution("r", 100.0, 50000.0));
tds->addAttribute( new TUniformDistribution("tu", 63070.0, 115600.0));
tds->addAttribute( new TUniformDistribution("tl", 63.1, 116.0));
tds->addAttribute( new TUniformDistribution("hu", 990.0, 1110.0));
tds->addAttribute( new TUniformDistribution("hl", 700.0, 820.0));
tds->addAttribute( new TUniformDistribution("l", 1120.0, 1680.0));
tds->addAttribute( new TUniformDistribution("kw", 9855.0, 12045.0));

// The reference input file
TString sJDDReference = TString("flowrate_input_with_flags.in");        

// Set the reference input file and the key for each input attributes
tds->getAttribute("rw")->setFileFlag(sJDDReference, "@Rw@");            
tds->getAttribute("r")->setFileFlag(sJDDReference, "@R@");
tds->getAttribute("tu")->setFileFlag(sJDDReference, "@Tu@");
tds->getAttribute("tl")->setFileFlag(sJDDReference, "@Tl@");
tds->getAttribute("hu")->setFileFlag(sJDDReference, "@Hu@");
tds->getAttribute("hl")->setFileFlag(sJDDReference, "@Hl@");
tds->getAttribute("l")->setFileFlag(sJDDReference, "@L@");
tds->getAttribute("kw")->setFileFlag(sJDDReference, "@Kw@");

Specification of a flag input file for the input variables

	In this first part, definition of the attributes of the `TDataServer`;
	Definition of the reference input file (the file copied before);
	For every attribe, the position to write the information in the file is specified thanks to the `setFileFlag` method. It takes as first parameter the input file name and then the flag where the value will be set. For example, the "rw" parameter will be set in the file `flowrate_input_with_keys.in` where the `@Rw@` flag stands.

After these definitions, if we create the code launcher and run it (what will be presented in the following sections), then the input file would be modified by writing each sample value in a new input file. For example, the following input file is one of the generated ones:

#
# INPUT FILE with FLAG for the "FLOWREATE" code
# \date   2008-04-22 12:55:17
#

new Implicit_Steady_State sch {
    frottement_paroi { 6.371966e-02 4.913216e+04 }
    tinit               0.0
    tmax                1000000.
    nb_pas_dt_max       1500
    dt_min              1.086881e+03
    dt_max              7.372423e+02
    facsec              1000000.
    kW                  1.190644e+04
    information_Tu      Champ_Uniforme 1        7.628909e+04
    information_Tl      Champ_Uniforme 1        7.386808e+01
    information_L {
        	    precision  1.402143e+03
	 	  }
    convergence {
        	    criterion relative_max_du_dt
            	    precision  1.e-6
		}

    stop_criterium {
    		    ch_abcsissa_hu 1050
		    ch_ordinate_hl 770
            	    c_radius 1100
		   }

    Solveur Newton3 {
    	             max_iter_matrice        1
            	     max_iter_implicite      1
  		     date                    5654321
           	     seuil_convg_implicite   1.e-6
            	     assemblage_implicite    10
            	     solveur_lineaire        BiCGS
            	     preconditionneur    ILU
            	     seuil_resol_implicite 1.e-5
		    }
}

In this file, we will note the different values of the parameters that have been modified compared to the "model" presented in the beginning of this section.

Tip

If a flag exists several times in a file, the value will be replaced at all the occurrences of the flag.

IV.3.1.2.3. Creation of input files without model files

In case the user does not want to create a model file, it is possible to fully create the input file. To do so, it is necessary to use the setFileKey method on an attribute and to specify the format of the file to be created. Uranie is able to generate input files with "key = value" format, "values in row" or "values in column".

For the "key = value" format, the setFileKey method must be called with a string that describes the input format (see the C++ reference page for the available formats) as third parameter (if the string is empty, the default format will be used), and the flag TAttributeFileKey::kNewKey as fourth parameter. As an example the definition below:

TString sIn = TString("test_input_with_keys.in");
tds->getAttribute("rw")->setFileKey(sIn,"Rw","%e",TAttributeFileKey::kNewKey);
tds->getAttribute("r")->setFileKey(sIn,"R","%e",TAttributeFileKey::kNewKey);
tds->getAttribute("tu")->setFileKey(sIn,"Tu","%e",TAttributeFileKey::kNewKey);
tds->getAttribute("tl")->setFileKey(sIn,"Tl","%e",TAttributeFileKey::kNewKey);
tds->getAttribute("hu")->setFileKey(sIn,"Hu","",TAttributeFileKey::kNewKey);
tds->getAttribute("hl")->setFileKey(sIn,"Hl","",TAttributeFileKey::kNewKey);
tds->getAttribute("l")->setFileKey(sIn,"L","",TAttributeFileKey::kNewKey);
tds->getAttribute("kw")->setFileKey(sIn,"Kw","",TAttributeFileKey::kNewKey);

will lead in the creation of files test_input_with_keys.in, as:

Rw = 1.136676e-01 ;
R = 2.705506e+04 ;
Tu = 9.259556e+04 ;
Tl = 6.400877e+01 ;
Hu = 1.087919e+03 ;
Hl = 7.600170e+02 ;
L = 1.537756e+03 ;
Kw = 1.060694e+04 ;

For the "value in row" format, the setFileKey method must be called with a string that describes the input format (see the C++ reference page for the available formats) as third parameter (if the string is empty, the default format will be used), and the flag TAttributeFileKey::kNewRow as fourth parameter. As an example the definition below:

TString sInR = TString("test_input_with_row.in");
tds->getAttribute("rw")->setFileKey(sInR,"Rw","%e",TAttributeFileKey::kNewRow);
tds->getAttribute("r")->setFileKey(sInR,"R","%e",TAttributeFileKey::kNewRow);
tds->getAttribute("tu")->setFileKey(sInR,"Tu","%e",TAttributeFileKey::kNewRow);
tds->getAttribute("tl")->setFileKey(sInR,"Tl","%e",TAttributeFileKey::kNewRow);
tds->getAttribute("hu")->setFileKey(sInR,"Hu","",TAttributeFileKey::kNewRow);
tds->getAttribute("hl")->setFileKey(sInR,"Hl","",TAttributeFileKey::kNewRow);
tds->getAttribute("l")->setFileKey(sInR,"L","",TAttributeFileKey::kNewRow);
tds->getAttribute("kw")->setFileKey(sInR,"Kw","",TAttributeFileKey::kNewRow);

will lead in the creation of files test_input_with_row.in, as:

1.136676e-01 2.705506e+04 9.259556e+04 6.400877e+01 1.087919e+03 7.600170e+02 1.537756e+03 1.060694e+04

Caution must be taken with vectors: before this version, the Row format and the DataServer format (for output files) were equivalent up to the headers. Now this is different, as vector are dumped on a single line for DataServer format (also called Salome-table) and with one value per line in Row files. A new flag as then be created as can be seen for instance when comparing Section XIV.4.30 and Section XIV.4.32. For the new DataServer format, the setFileKey method must be called with a string that describes the input format (see the C++ reference page for the available formats) as third parameter (if the string is empty, the default format will be used), and the flag TAttributeFileKey::kNewTDS as fourth parameter. As an example the definition below:
```
TString sInTds = TString("test_input_tds.in");
tds->getAttribute("rw")->setFileKey(sInTds,"Rw","%e",TAttributeFileKey::kNewTDS);
tds->getAttribute("r")->setFileKey(sInTds,"R","%e",TAttributeFileKey::kNewTDS);
tds->getAttribute("tu")->setFileKey(sInTds,"Tu","%e",TAttributeFileKey::kNewTDS);
tds->getAttribute("tl")->setFileKey(sInTds,"Tl","%e",TAttributeFileKey::kNewTDS);
tds->getAttribute("hu")->setFileKey(sInTds,"Hu","",TAttributeFileKey::kNewTDS);
tds->getAttribute("hl")->setFileKey(sInTds,"Hl","",TAttributeFileKey::kNewTDS);
tds->getAttribute("l")->setFileKey(sInTds,"L","",TAttributeFileKey::kNewTDS);
tds->getAttribute("kw")->setFileKey(sInTds,"Kw","",TAttributeFileKey::kNewTDS);
```
will lead in the creation of files test_input_with_tds.in, as:
```
#COLUMN_NAMES: rw|r|tu|tl|hu|hl|l|kw
#COLUMN_TYPES: D|D|D|D|D|D|D|D

1.136676e-01 2.705506e+04 9.259556e+04 6.400877e+01 1.087919e+03 7.600170e+02 1.537756e+03 1.060694e+04
	      
```
Once again, this file and the previous one looks very familiar, but the input file created with vectors, using respectively DataServer and Row format in Section XIV.4.30 and Section XIV.4.32 are not.

For the "values in column" format, the setFileKey method must be called with a string that describes the input format (see the C++ reference page for the available formats) as third parameter (if the string is empty, the default format will be used), and the flag TAttributeFileKey::kNewColumn as fourth parameter. As an example the definition below:

TString sInC = TString("test_input_with_column.in");
tds->getAttribute("rw")->setFileKey(sInC,"Rw","%e",TAttributeFileKey::kNewColumn);
tds->getAttribute("r")->setFileKey(sInC,"R","%e",TAttributeFileKey::kNewColumn);
tds->getAttribute("tu")->setFileKey(sInC,"Tu","%e",TAttributeFileKey::kNewColumn);
tds->getAttribute("tl")->setFileKey(sInC,"Tl","%e",TAttributeFileKey::kNewColumn);
tds->getAttribute("hu")->setFileKey(sInC,"Hu","",TAttributeFileKey::kNewColumn);
tds->getAttribute("hl")->setFileKey(sInC,"Hl","",TAttributeFileKey::kNewColumn);
tds->getAttribute("l")->setFileKey(sInC,"L","",TAttributeFileKey::kNewColumn);
tds->getAttribute("kw")->setFileKey(sInC,"Kw","",TAttributeFileKey::kNewColumn);

will lead in the creation of files test_input_with_column.in, as:

1.136676e-01
2.705506e+04
9.259556e+04
6.400877e+01
1.087919e+03
7.600170e+02
1.537756e+03
1.060694e+04

IV.3.1.2.4. Input files with XML format

Uranie is also able to generate data in XML input files. For example, let us consider the following input file that contains the eight parameters previously seen (, , , , , , , ):

<?xml version="1.0"?>
<problem>
<description name="flowrate" version="1.0" title="UseCase flowrate with XML input file" date="2008-04-22 12:55:17">
<tool name="uranie" version="0.3"/>
</description>

<steady_state name="sch">
<frottement_paroi rw="0.0500" r="33366.67"/>
<tinit>0.0</tinit>
<tmax>1000000</tmax>
<nb_pas_dt_max>1500</nb_pas_dt_max>
<parameter>
<tonode>mailleur</tonode><toport>dt_min</toport>
	    <value><double>1110.00</double></value>
</parameter>
<parameter>
<tonode>mailleur</tonode><toport>dt_max</toport>
<value><double>768.57</double></value>
</parameter>
<facsec>1000000.</facsec>
<kW value="11732.14"/>
	    <informations>
<parameter name="Tu">
<Champ_Uniforme>1</Champ_Uniforme>
<value><double>63070.0</double></value>
</parameter>
<parameter name="Tl">
<Champ_Uniforme>1</Champ_Uniforme>
<value><double>116.00</double></value>
</parameter>
<parameter name="L" precision="1200.0"/>
</informations>

<convergence>
<criterion>relative_max_du_dt</criterion>
<precision>1.e-6</precision>
</convergence>

<stop_criterium ch_abcsissa_hu="1050" ch_ordinate_hl="770" c_radius="1100" nLoop="1"/>

<solveur name="Newton3">
<max_iter_matrice>1</max_iter_matrice>
<max_iter_implicite>1</max_iter_implicite>
<date>5654321</date>
<seuil_convg_implicite>1.e-6</seuil_convg_implicite>
<assemblage_implicite>10</assemblage_implicite>
	    <solveur_lineaire name="BiCGS">
<preconditionneur>ILU</preconditionneur>
<seuil_resol_implicite>1.e-5</seuil_resol_implicite>
</solveur_lineaire>
</solveur>

</steady_state>

</problem>

The transformation of an XML file is described by using XSLT transformation directives. When these directives have been described in the script, Uranie generates an XSLT transformation file, that will be directly applied on the input XML file to transform it. This XSLT file will describe how to copy the reference XML file to an XML file whose values are changed. So the description of the location of the values to change is naturally written with XSLT directives. Below is a short summary of the main XSLT commands:

We can see above that there are two kinds of place a variable value may be changed: in an attribute or in a field. For Uranie to be able to modify the value, it is necessary to specify the kind of replacement to make, using the TAttributeFileKey::kXMLAttribute key for an attribute, or the TAttributeFileKey::kXMLField key for a field.

The example below demonstrates how to modify the attribute values in the file presented earlier:

TString sJDDReferenceXML = TString("flowrate_input_with_xml.xml");	  
tds->getAttribute("rw")->setFileKey(sJDDReferenceXML, "frottement_paroi/@rw", "%e", TAttributeFileKey::kXMLAttribute);
tds->getAttribute("r")->setFileKey(sJDDReferenceXML, "frottement_paroi/@r", "%e", TAttributeFileKey::kXMLAttribute);
tds->getAttribute("tu")->setFileKey(sJDDReferenceXML, "parameter[@name='Tu']/value/double", "%e", TAttributeFileKey::kXMLField);
tds->getAttribute("tl")->setFileKey(sJDDReferenceXML, "parameter[@name='Tl']/value/double", "%e", TAttributeFileKey::kXMLField);
tds->getAttribute("hu")->setFileKey(sJDDReferenceXML, "parameter[tonode='mailleur' and toport='dt_max']/value/double", "%e", TAttributeFileKey::kXMLField);
tds->getAttribute("hl")->setFileKey(sJDDReferenceXML, "parameter[tonode='mailleur' and toport='dt_min']/value/double", "%e", TAttributeFileKey::kXMLField);
tds->getAttribute("l")->setFileKey(sJDDReferenceXML, "parameter[name='L']/@precision", "%e", TAttributeFileKey::kXMLAttribute);
tds->getAttribute("kw")->setFileKey(sJDDReferenceXML, "kW/@value", "%e", TAttributeFileKey::kXMLAttribute);

Note that the key that describes the kind of replacement is different depending on whether the value is stored as an attribute (kXMLAttribute) or a field (kXMLField).

When Uranie is run, an XSLT file is created to describe the changes to make in the original XML file. Below is an example of such a generated XSLT file:

<?xml version="1.0" encoding="iso-8859-1"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">

<xsl:template match="frottement_paroi/@rw">
<xsl:attribute name="rw">
<xsl:value-of select="0.133648"/>
</xsl:attribute>
</xsl:template>

<xsl:template match="frottement_paroi/@r">
<xsl:attribute name="r">
<xsl:value-of select="6989.96"/>
</xsl:attribute>
</xsl:template>

<xsl:template match="parameter[@name='Tu']/value/double">
<double>
<xsl:value-of select="63442"/>
</double>
</xsl:template>

<xsl:template match="parameter[@name='Tl']/value/double">
<double>
<xsl:value-of select="77.2926"/>
</double>
</xsl:template>

<xsl:template match="parameter[tonode='mailleur' and toport='dt_max']/value/double">
<double>
<xsl:value-of select="1051.48"/>
</double>
</xsl:template>

<xsl:template match="parameter[tonode='mailleur' and toport='dt_min']/value/double">
<double>
<xsl:value-of select="750.107"/>
</double>
</xsl:template>

<xsl:template match="parameter[name='L']/@precision">
<xsl:attribute name="precision">
	  <xsl:value-of select="1543.31"/>
</xsl:attribute>
</xsl:template>

<xsl:template match="kW/@value">
<xsl:attribute name="value">
<xsl:value-of select="11745.4"/>
</xsl:attribute>
</xsl:template>

<!-- Copy all the rest of the file -->
<xsl:template match="node()|@*">
<xsl:copy>
<xsl:apply-templates select="node()|@*"/>
</xsl:copy>
</xsl:template>

</xsl:stylesheet>

When applied to the flowrate_input_with_xml.xml reference input file, this XSLT file produces the XML file Uranie will effectively use:

<?xml version="1.0"?>
<problem>
<description name="flowrate" version="1.0" title="UseCase flowrate with XML input file" date="2008-04-22 12:55:17">
<tool name="uranie" version="0.3"/>
</description>

<steady_state name="sch">
<frottement_paroi rw="0.133648" r="6989.96"/>
<tinit>0.0</tinit>
<tmax>1000000</tmax>
<nb_pas_dt_max>1500</nb_pas_dt_max>
<parameter>
<tonode>mailleur</tonode><toport>dt_min</toport>
<value><double>750.107</double></value>
</parameter>
<parameter>
<tonode>mailleur</tonode><toport>dt_max</toport>
<value><double>1051.48</double></value>
</parameter>
<facsec>1000000.</facsec>
<kW value="11745.4"/>
<informations>
<parameter name="Tu">
<Champ_Uniforme>1</Champ_Uniforme>
<value><double>63442</double></value>
</parameter>
<parameter name="Tl">
<Champ_Uniforme>1</Champ_Uniforme>
<value><double>77.2926</double></value>
</parameter>
<parameter name="L" precision="1200.0"/>
</informations>

<convergence>
<criterion>relative_max_du_dt</criterion>
<precision>1.e-6</precision>
</convergence>

<stop_criterium ch_abcsissa_hu="1050" ch_ordinate_hl="770" c_radius="1100" nLoop="1"/>

<solveur name="Newton3">
<max_iter_matrice>1</max_iter_matrice>
<max_iter_implicite>1</max_iter_implicite>
<date>5654321</date>
	    <seuil_convg_implicite>1.e-6</seuil_convg_implicite>
<assemblage_implicite>10</assemblage_implicite>
<solveur_lineaire name="BiCGS">
<preconditionneur>ILU</preconditionneur>
<seuil_resol_implicite>1.e-5</seuil_resol_implicite>
</solveur_lineaire>
</solveur>

</steady_state>

</problem>

Warning

Obviously, the use of the XML input needs an XSLT processor installed.

Tip

It is important to note that Uranie can manage inputs split between multiple input files, just by specifying the name of these different files, whatever the kind of input file under consideration.

Summary: Input files

setFileKey(TString sfile, TString skey = "", TString sformatToSubstitute = "%e", TAttributeFileKey::EFileType sFileType = TAttributeFileKey::kKey)
Defines:
- The input file sfile where to change the values of the attribute.
- The key skey to substitute in the input file, if different than the name of the attribute (which is the default value taken if an empty string is given). In case of a "flagged file" the key is the flag. In case of an XML file, the key is the XSLT path of the tag.
- The format to substitute sformatToSubstitute; the default value %e means that scientific format is used.
- The type of input files sFileType, whose default default value is kKey.
Note that the behaviour depends on the sFileType flag: the input file must exist for kKey, kFlag, kXMLAttribute or kXMLField, or will be created for kNewRow, kNewColumn, kNewKey or kNewTDS.
setFileFlag(TString sfile, TString sflag = "", TString sformatToSubstitute = "%e")
Equivalent to setFileKey(sfile, skey, sformatToSubstitute, TAttributeFileKey::kFlag), defines an input file with flags sflag.

IV.3.1.3. Output files

After the code has run, it has created output files we want to extract values from. We present here the different kinds of output files Uranie can manage, and how to retrieve information from them.

Uranie can retrieve information from files whose formats are "key = value", "values in row", "values in column", "dataserver value" and XML.

IV.3.1.3.1. Output files with "key=value" format

TOutputFileKey retrieves data from a file where values are stored in the format "key = value". An example of such a file is given below:

yhat = 6.757218e+01;
d = 4.092561e+03;

To get data from such a file, it is necessary to create a TOutputFileKey object, and add to it the attributes of the file:

TOutputFileKey *foutk = new TOutputFileKey("_output_flowrate_withKey_.dat");
foutk->addAttribute(new TAttribute("yhat"));
foutk->addAttribute(new TAttribute("d"));

When working with "key = value" files, the default name of the attribute to create (here yhat or d) is the name of the attribute in the output file. In order to create an attribute with a name different from the name in the file, it is necessary to create an attribute and set it to the output file:

TAttribute *attr = new TAttribute("y");
attr->setFileKey("_output_flowrate_withKey_.dat", "yhat");
foutk->addAttribute(attr);

It is also possible to retrieve only the values of the desired variables. In the previous example, it would have been possible to retrieve only the values of d just by deleting the line that treats yhat output.

IV.3.1.3.2. Output files with row format

TOutputFileRow retrieves data from a file where values are stored in rows. An example of such a file is given below:

#COLUMN_NAMES: yhat | d
		  
6.757218e+01  4.092561e+03

To get data from such a file, it is necessary to create a TOutputFileRow object, and add to it the attributes of the file:

TOutputFileRow *foutr = new TOutputFileRow("_output_flowrate_withRow_.dat");
foutr->addAttribute(new TAttribute("yhat"));
foutr->addAttribute(new TAttribute("d"));

When working with "values in row" files, the addition order of the attributes determines which value will be stored in each variable. In the previous example, the first value encountered in the file will be stored in the yhat attribute, and the second one in d. As a consequence, in order to get only the values from a given column, it is necessary to specify the column number (first column is number 1, not 0). When this number is not specified, the value used is the one following the previous value. For example, if we suppose the file test.dat contains 8 values stored in row, we can have:

foutr = new TOutputFileRow("test.dat");
foutr->addAttribute(new TAttribute("y2"), 2);  // get the 2nd column value
foutr->addAttribute(new TAttribute("y3"));     // get the 3rd column value
foutr->addAttribute(new TAttribute("y8"), 8);  // get the 8th column value
foutr->addAttribute(new TAttribute("y5"), 5);  // get the 5th column value
foutr->addAttribute(new TAttribute("y6"));     // get the 6th column value

Warning

If one is not considering vectors, Row and DataServer (Salome-table) format output are equivalent. Else, it differs a lot as it can be seen by comparing Section XIV.4.25 and Section XIV.4.27.

IV.3.1.3.3. Output files with column format

TOutputFileColumn retrieves data from a file where values are stored in columns. An example of such a file is given below:

2.618019e+01
3.602045e+03

To get data from such a file, it is necessary to create a TOutputFileColumns object, and add to it the attributes of the file:

TOutputFileColumn *foutc = new TOutputFileColumn("_output_flowrate_withColumn_.dat");
foutc->addAttribute(new TAttribute("yhat"));
foutc->addAttribute(new TAttribute("d"));

When working with "values in column" files, the addition order of the attributes determines which value will be stored in each variable. In the example, the first value encountered in the file will be stored in the yhat attribute, and the second one in d. As a consequence, in order to get only the values from a given row, it is necessary to specify the row number (first row is number 1, not 0). When this number is not specified, the value caught is supposed to have the next index that the previous value. For example, if we suppose the file test.dat contains 8 values stored in column, we can have:

foutc = new TOutputFileColumn("test.dat");
foutc->addAttribute(new TAttribute("y2"), 2);  // get the 2nd row value
foutc->addAttribute(new TAttribute("y3"));     // get the 3rd row value
foutc->addAttribute(new TAttribute("y8"), 8);  // get the 8th row value
foutc->addAttribute(new TAttribute("y5"), 5);  // get the 5th row value
foutc->addAttribute(new TAttribute("y6"));     // get the 6th row value

IV.3.1.3.4. Output files with DataServer format

Another output format is the format of the TDataServer: the first line describes the data and the data follows.

#COLUMN_NAMES: yhat | d
		  
6.757218e+01  4.092561e+03

To get data from such a file, it is necessary to create a TOutputFileDataServer object, and add to it the attributes to be read from the file:

TOutputFileDataServer *foutds = new TOutputFileDataServer("_output_flowrate_withRow_.dat");
foutds->addAttribute(new TAttribute("yhat"));
foutds->addAttribute(new TAttribute("d"));

Here the order in which the parameters are retrieved is not important: when looking for the "d" attribute for example, as the name "d" is the second name of the #COLUMN_NAMES: line, Uranie will retrieve the second value of the line (i.e. 3.602045e+03).

Warning

If one is not considering vectors, Row and DataServer (Salome-table) format output are equivalent. Else, it differs a lot as can be seen by comparing Section XIV.4.25 and Section XIV.4.27.

IV.3.1.3.5. Output files with XML format

The last output format is the XML format, where values can be retrieved from XML fields or attributes. Note this kind of output needs the LibXML2 library.

<?xml version="1.0"?>
<steady_state name="flowrate">
  <parameter>
    <tonode>mesher</tonode>
    <toport>dt_hl</toport>
    <value>
      <double>2.618019e+01</double>
    </value>
  </parameter>
  <distance value="3.602045e+03"/>
</steady_state>

To get data from such a file, it is necessary to create a TOutputFileXML object, and add to it the attributes to catch from the file. When adding attributes, it is necessary to pass the name of the attribute to create or the attribute itself to the addAttribute method, as well as its XSL path and the kind of XML object to get data between kXMLAttribute and kXMLField. For example to get values from the XML file generated by flowrate, use the following code:

TOutputFileXML *foutXML = new TOutputFileXML("_output_flowrate_withXML_.dat");
foutXML->addAttribute(TString("yhat"), "/steady_state[@name='flowrate']/parameter/value/double", URANIE::DataServer::TAttributeFileKey::kXMLField);
foutXML->addAttribute(new URANIE::DataServer::TAttribute("d"), "/steady_state[@name='flowrate']/distance/@value", URANIE::DataServer::TAttributeFileKey::kXMLAttribute);

As for the "key=value" and the DataServer formats, the order the parameters are provided is not important.

Summary: Output files

TOutputFileKey(TString str)
Create an object to read information from an ASCII file with the "key = value" format.
addAttribute(TAttribute * att, TString skey="")
Add the attribute att to the file and specify its key skey.
TOutputFileRow(TString str)
Create an object to read information from an ASCII file with the row format.
addAttribute(TAttribute * att, Int_t nrow=0)
Add the attribute att to the file and specify its row number nrow.
TOutputFileColumn(TString str)
Create an object to read information from an ASCII file with the column format.
addAttribute(TAttribute * att, Int_t nline=0)
Add the attribute att to the file and specify its line number nline.
TOutputFileDataServer(TString str)
Create an object to read information from an ASCII file with the DataServer formalism.
addAttribute(TAttribute * att, Int_t nline=0)
Add the attribute att to the file and specify its line number nline.
TOutputFileXML(TString str)
Create an object to read information from an XML file.
addAttribute(TAttribute * att, TString xmlPath, TAttributeFileKey::EFileType FileType)
Add the attribute att to the file, specify the XML path of the data xmlPath, and the type of XML data FileType, which can be kXMLAttribute or kXMLField.

IV.3.2. TCode definition

A TCode object is used to define an external code. In order to create such an object, the user must call the constructor whose arguments are a pointer on the useful TDataServer and a TString that contains the command to execute the external code. For example:

TCode *mycode = new TCode(tds, "flowrate -s -k");

This line will create the TCode object mycode, link it with the TDataServer, and assign the command line flowrate -s -k to it.

Some parameters can be set to the code: its working directory (Section IV.3.2.1), the definition of unmodified input files (Section IV.3.2.2), and the location of output files (Section IV.3.2.3).

IV.3.2.1. Working directory of the computations

By default, the computations are made in a new directory named URANIE in the current folder. To modify this behaviour, it is possible to specify the working directory by using the setWorkingDirectory method:

mycode->setWorkingDirectory(gSystem->pwd()+TString("/working_directory"));

With the previous line, the code will run in sub-directories of the newly created working_directory folder.

Tip

It is also possible to specify the working directory by using the setWorkingDirectory method on a TLauncher object (see Section IV.3.3). In case both methods are used, the code will use the path defined in the TLauncher, then the one defined in the TCode.

IV.3.2.2. Unmodified input files

While the variables with modifiable values are stored in the TDataServer (see Section IV.3.1), the code may need input files that contain parameters Uranie does not have to modify. To add such a file to the TCode object, just use the addInputFile method after having specified the directory to find it with the setReferenceDirectory method.

TCode *mycode2 = new TCode(tds, "flowrate -s -k");
mycode2->setReferenceDirectory( gSystem->pwd()+TString("/reference_directory") );
mycode2->addInputFile("flowrate_input_with_flags_1.in");
mycode2->addInputFile("flowrate_input_with_flags_2.in");

The previous lines will lead in the copy of the files $PWD/reference_directory/flowrate_input_with_flags_1.in and $PWD/reference_directory/flowrate_input_with_flags_2.in in the working directory of the code. Note that the reference directory definition should be made with an absolute path.

The method getReferenceDirectory will return this reference directory as a TString.

IV.3.2.3. Output file of the code

In order to specify the name of the output file of the code, it is necessary to use the addOutputFile(outfile) method, that defines the TOutputfile object outfile as an output file for the code:

foutk = new TOutputFileKey("_output_flowrate_withKey_.dat");
TCode *mycode3 = new TCode(tds, "flowrate -s -k");
mycode3->addOutputFile( foutk );

After having created the output file object fout, this code adds it as an output file to the mycode TCode object.

Summary: TCode object

TCode(TDataServer * tds, TString scmd)
Create a TCode object from the TDataServer tds and with the command line scmd.
setWorkingDirectory(TString str)
Set the directory for the code to run in. If not used, the code runs in the URANIE folder created in the current folder.
addInputFile(TString str)
Add the input file str to the TCode object.
addOutputFile(TOutputFile * ifile)
Add the (previously created) output file ifile. ifile can be an object of the following classes: TOutputFileRow, TTOutputFileColumn, TTOutputFileKey, TTOutputFileDataServer and TOutputFileXML

IV.3.3. Launcher definition

In order to run an external code, it is necessary to define an object that can launch this code. This object is a TLauncher. To create it, the constructor just needs a TDataServer and a TCode reference, as shown in the example below:

TDataServer *tds = new TDataServer("tdsFlowrate", "Doe for Flowrate");  
// (add data to the TDataServer)

TCode *mycode = new TCode(tds, "command");                              

TLauncher *mylauncher = new TLauncher(tds, mycode);

Creating a TLauncher

	Creation of a `TDataServer`. Then attributes and data can be appended (for further explanations and examples, please refer to the Chapter II).
	Creation of a `TCode`. See the Section IV.3.2 for a presentation of the construction of a `TCode` object.
	Construction of a `TLauncher`. As presented before, the constructor only needs the pointers on the `TDataServer` and on the `TCode`.

In order to manipulate the code, different methods can be used:

setWorkingDirectory: set the directory in which the code will be run and where the temporary folders needed by the code will be stored.
```
 mylauncher->setWorkingDirectory( gSystem->Getenv("HOME") + TString("/tmp/")); 
```
The code runs in $HOME/tmp/. Note that, as seen in Section IV.3.2.1, if this function is not called on the current TLauncher object, the working directory can be set on the TCode object. If none of these methods are used, it will be set to a directory named URANIE created in the reference directory.
The method getWorkingDirectory will return this directory as a TString.
setSave(i): activate the save mode and set the maximum number i of folders to be kept. A positive value for i stands for the explicit maximum number of saves, while no value or -1 value allow to keep all the patterns (all patterns are launched in a different working directory UranieLauncher_i). Default parameter is -1.
If setSave is not called, the temporary output folders of the code will be overwritten at each computing step, while they will be left unmodified if the method is called.
```
mylauncher->setSave();     // will save each launch
mylauncher->setSave(-1);   // will save each launch
mylauncher->setSave(10);   // will save 10 launches
```
The method getSave will return a boolean describing the state of the save flag, and the method unsetSave will deactivate the save mode.
Warning
In case of multi-process distributed computation, in order to prevent all the processes from trying and accessing the same data at the same time, a call to the setSave method is mandatory. As a consequence, if it is not manually set, Uranie will set it automatically.
setClean(kTrue/kFalse): set the clean flag to the value true or false. If it is set to kTrue, all the working directories are cleaned before launching. Then if the save flag is set to kTRUE, the working directories are cleaned a second time after the TLauncher run. If not specified, default parameter is kTrue, which means the working directory will be cleaned (i.e. temporary folders will be erased).
```
mylauncher->setClean();       // set the clean flag to "true"
mylauncher->setClean(kTRUE);  // set the clean flag to "true"
mylauncher->setClean(kFALSE); // set the clean flag to "false"
```
The method getClean will return the clean flag as a boolean.
setVarDraw(str,select,opt): activate and define the graphics during the launching. By default, there is no graphic during the launching process. To activate it, it is necessary to define the information of the graphics (the X, Y and Z attributes, the selection and the options of the graphic). The method's parameters are:
- str: the list of attributes to be drawn;
- select: the selection of patterns to visualise (default value is "");
- opt: the option of the graphics. Default value is "". To see all available values, please refer to the ROOT THistPainter reference page.
```
mylauncher->setVarDraw("hu:hl");                 // draw hu function of hl
mylauncher->setVarDraw("hu:hl","yhat>0");        // same for data where yhat>0
mylauncher->setVarDraw("hu:hl","yhat>0","SURF"); // same drawn as surface plot 
```
setDrawProgressBar(bool): activate and draw the progress bar that gives an idea of the remaining time before finishing the loop. The default is to have this bar drawn, so this method is usually only used when one decides not to displays the progress bar (if the output is redirected in an output file for instance).
```
mylauncher->setDrawProgressBar();             // draw the progress bar
mylauncher->setDrawProgressBar(false);        // do not draw the progress bar
bool draw = mylauncher->getDrawProgressBar(); // check if progress bar is drawn
```
After all the Launcher elements are set, it is then possible to launch it, by using the run method.
```
mylauncher->run();
```

We present below an example that uses the different methods described above:

TDataServer *tds = new TDataServer("tdsFlowrate", "DataBase flowrate");
// (add data to the TDataServer)

TCode *mycode = new TCode(tds, "command");

TLauncher *mylauncher = new TLauncher(tds, mycode);

mylauncher->setSave();
mylauncher->setClean();
mylauncher->setVarDraw("hu:hl","yhat>0","");
mylauncher->setWorkingDirectory(gSystem->Getenv("PWD") + TString("/tmpLanceurUranie/flowrate"));

mylauncher->run();

Summary: TLauncher object

TLauncher(TDataServer *tds, TCode *tc)
Create a launcher with the external code tc to apply on the TDataServer tds.
setWorkingDirectory(TString str)
Sets the working directory where to realise the computations to str.
setSave(Int_t nSave)
Activates the save mode and sets number of max saves to perform: a positive value stands for the explicit value of max saves, -1 allows to set max number of saves to the number of patterns.
setClean(Bool_t bbool=kTRUE)
Set the clean flag, which means that before launching, all the working directory are cleaned, and, if save is FALSE, after the launching all the working directory are also cleaned.
setVarDraw(TString str, TString select="", TString opt="")
Activates and defines the graphics during the launching. By default, there is no graphic during the launching process. To activate it, you must defines the information of the graphics: the X, Y and Z attributes, the select and the options of the graphic.
setDrawProgressBar(Bool_t bbool=kTRUE)
Activates and draws the progress bar that gives an idea of the remaining time before finishing the loop. The default is to have this bar drawn.
run()
Runs the code.


IV.2. Analytic function		IV.4. Distribution