<!-- Beg of adding_files chapter -->
<chapter id="adding_files">
<title>Adding New Resolutions or New Files to the build-namelist Database</title>
<para>
In the last chapter we gave the details on how to create new files for input into
CLM. These files could be either global resolutions, regional-grids or even a single
grid point. If you want to easily have these files available for continued use in your
development you will then want to include them in the build-namelist database so
that build-namelist can easily find them for you. You can deal with them, just by
editing your namelist by hand (or using a &usernlclm; namelist file), or by using
&CLMUSRDAT;. Another way to deal with them is to enter them into
the database for build-namelist, so that build-namelist can find them for you.
This keeps one central database for all your files, rather than having multiple locations
to keep track of files. If you have a LOT of files to keep track of it also might
be easier than keeping track by hand, especially if you have to periodically update
your files. If you just have a few quick experiments to try, for a short time period
you might be best off using the other methods mentioned above.
</para>
<para>
There are two parts to adding files to the build-namelist database. The first part
is adding new resolution names which is done in the
<filename>models/lnd/clm/bld/namelist_files/namelist_definition.xml</filename> file
(and in the 
<filename>models/lnd/clm/bld/config_files/config_definition.xml</filename> file
when adding supported single-point datasets).
The second part is actually adding the new filenames which is done in the 
<filename>models/lnd/clm/bld/namelist_files/namelist_defaults_clm.xml</filename> file
(<filename>models/lnd/clm/bld/namelist_files/namelist_defaults_clm_tools.xml</filename> 
file for CLM tools).
If you aren't adding any new resolutions, and you are just changing the files for existing
resolutions, you don't need to edit the namelist_definition file.
</para>

<sect1 id="managingyourdata">
<title>Managing Your Own Data-files</title>
<para>
If you are running on a supported machine (such as bluefire or jaguar) the standard
input datasets will already be available and you won't have to check them out of the
subversion inputdata server. However, you also will NOT be able to add your own datafiles
to these standard inputdata directories -- because most likely you won't have permissions
to do so. In order to add files to the XML database or to use &CLMUSRDAT; you need
to put data in the standard locations so that they can be found. The recommended 
way to do this is to use the <command>link_dirtree</command> tool in the &cesm; scripts.
Some information on <command>link_dirtree</command> is available in the 
<ulink url="&cesmwebmodelrel;cesm">&cesmrel; Scripts User's Guide</ulink>. We also have
some examples of it's use here and in other sections of this User's Guide.
</para>
<para>
Using <command>link_dirtree</command> is quite simple, you give the directory where
data exists and then the directory that you want to create where datasets will point
to the original source files. In the example below we use "$HOME/inputdata", but 
<envar>MYCSMDATA</envar> could be any directory you have access to where you want to
put your data.
<screen>
> cd scripts
# First make sure you have a inputdata location that you can write to 
# You only need to do this step once, so you won't need to do this in the future
# (except to bring in any updated files in the original $CSMDATA location).
> setenv MYCSMDATA $HOME/inputdata    # Set env var for the directory for input data
> ./link_dirtree $CSMDATA $MYCSMDATA
</screen>
Then when you create a case you will change <envar>DIN_LOC_ROOT_CSMDATA</envar> to
point to the location you linked to rather than the default location.
<screen>
> ./xmlchange -file env_run.xml -id DIN_LOC_ROOT_CSMDATA -val $MYCSMDATA
</screen>
</para>
<para>
In order to list the files that you have created you merely need to use the UNIX
command <command>find</command> to find the files that are NOT softlinks. So for
example executing the following command:
<screen>
> find $MYCSMDATA -type f -print
</screen>
for me gives the following list of &CLMUSRDAT; files that I have created.
<screen>
/blhome/erik/inputdata/atm/cam/chem/trop_mozart_aero/aero/aerosoldep_monthly_1849-2006_1x1pt_US-Ha1.nc
/blhome/erik/inputdata/atm/cam/chem/trop_mozart_aero/aero/aerosoldep_monthly_1849-2006_13x12pt_f19_alaskaUSA.nc
/blhome/erik/inputdata/atm/cam/chem/trop_mozart_aero/aero/aerosoldep_rcp8.5_monthly_1850-2100_13x12pt_f19_alaskaUSA.nc
/blhome/erik/inputdata/atm/cam/chem/trop_mozart_aero/aero/aerosoldep_rcp4.5_monthly_1850-2100_13x12pt_f19_alaskaUSA.nc
/blhome/erik/inputdata/atm/datm7/domain.clm/domain.lnd.1x1pt_US-Ha1_USGS.nc
/blhome/erik/inputdata/atm/datm7/domain.clm/domain.lnd.13x12pt_f19_alaskaUSA_gx1v6.nc
/blhome/erik/inputdata/lnd/clm2/griddata/fracdata_13x12pt_f19_alaskaUSA_gx1v6.nc
/blhome/erik/inputdata/lnd/clm2/griddata/fracdata_1x1pt_US-Ha1_USGS.nc
/blhome/erik/inputdata/lnd/clm2/griddata/topodata_13x12pt_f19_alaskaUSA.nc
/blhome/erik/inputdata/lnd/clm2/griddata/griddata_1x1pt_US-Ha1.nc
/blhome/erik/inputdata/lnd/clm2/griddata/griddata_13x12pt_f19_alaskaUSA.nc
/blhome/erik/inputdata/lnd/clm2/surfdata/surfdata_13x12pt_f19_alaskaUSA_simyr1850.nc
/blhome/erik/inputdata/lnd/clm2/surfdata/surfdata_1x1pt_US-Ha1_simyr2000.nc
/blhome/erik/inputdata/lnd/clm2/surfdata/surfdata.pftdyn_rcp4.5_13x12pt_f19_alaskaUSA_simyr1850-2100.nc
/blhome/erik/inputdata/lnd/clm2/surfdata/surfdata_1x1pt_US-Ha1_simyr1850.nc
/blhome/erik/inputdata/lnd/clm2/surfdata/surfdata_13x12pt_f19_alaskaUSA_simyr2000.nc
/blhome/erik/inputdata/lnd/clm2/surfdata/surfdata.pftdyn_1x1pt_US-Ha1_simyr1849-2006.nc
/blhome/erik/inputdata/lnd/clm2/surfdata/surfdata.pftdyn_13x12pt_f19_alaskaUSA_simyr1850-2100.nc
/blhome/erik/inputdata/lnd/clm2/surfdata/surfdata.pftdyn_rcp8.5_13x12pt_f19_alaskaUSA_simyr1850-2100.nc
/blhome/erik/inputdata/lnd/clm2/surfdata/surfdata.pftdyn_13x12pt_f19_alaskaUSA_simyr1849-2006.nc
/blhome/erik/inputdata/lnd/clm2/surfdata/surfdata.pftdyn_1x1pt_US-Ha1_simyr1850-2100.nc
</screen>
You can also use <command>find</command> to list files that have a particular pattern 
in the name as well (using the -name option with wildcards). Also you can always rerun the 
<command>link_dirtree</command> command if any new files are added that you need to be 
linked into your directory tree. Since, the files are soft-links -- it doesn't take up 
much space other than the files that you add there. This way all of the files are kept 
in one place, they are organized by usage according to &cesm; standards, and you can 
easily find your own files, and &clm; can find them as well.
</para>
</sect1>

<sect1 id="adding_resolutions">
<title>Adding Resolution Names</title>
<para>
If you are adding files for new resolutions which aren't covered in the
namelist_definition file -- you'll need to add them in. The list of valid resolutions
is in the id="res" entry in the
<filename>models/lnd/clm/bld/namelist_files/namelist_definition.xml</filename> file.
You need to choose a name for your new resolution and simply add it to the comma
delimited
list of valid_values for the id="res" entry. The convention for global Gaussian grids
is number_of_latitudes x number_of_longitudes. The convention for global finite
volume grids is latitude_grid_size x longitude_grid_size where latitude and longitude
is measured in degrees. For regional or single-point datasets the names have a grid size
number_of_latitudes x number_of_longitudes followed by an underscore and then a 
descriptive name such as a City name followed by an abbreviation for the Country in caps. 
The only hard requirement is that names be unique for different grid files.
Here's what the entry for resolutions looks like in the file:
<screen width="99">
&lt;entry id="res" type="char*30" category="default_settings"
       group="default_settings"  
       valid_values=
"128x256,64x128,48x96,32x64,8x16,94x192,0.23x0.31,0.47x0.63,
0.9x1.25,1.9x2.5,2.65x3.33,4x5,10x15,5x5_amazon,1x1_tropicAtl,
1x1_camdenNJ,1x1_vancouverCAN,1x1_mexicocityMEX,1x1_asphaltjungleNJ,
1x1_brazil,1x1_urbanc_alpha,0.5x0.5"&gt;
Horizontal resolutions
&lt;/entry&gt;
</screen>
As you can see you just add your new resolution names to the end of the valid_values
list.
</para>
<para>
When using &ptclm; and adding supported single-point resolutions, you'll also want to
add these resolutions to the
<filename>models/lnd/clm/bld/config_files/config_definition.xml</filename> under
the <envar>sitespf_pt</envar> name. The entry in that file looks like:
<screen width="99">
&lt;entry id="sitespf_pt" 
valid_values="none,1x1_brazil,1x1_tropicAtl,5x5_amazon,
1x1_camdenNJ,1x1_vancouverCAN,1x1_mexicocityMEX,1x1_asphaltjungleNJ,
1x1_urbanc_alpha,1x1_numaIA,1x1_smallvilleIA" 
value="none" category="physics"&gt;
Flag to turn on site specific special configuration flags for supported single
point resolutions.
Currently the only special settings are for MEXICOCITY and VANCOUVER, which make
changes to urban parameters.
&lt;/entry&gt;
</screen>
&ptclm; assumes that any supported single-point resolutions are valid settings for
<envar>sitespf_pt</envar>.
</para>
</sect1>

<sect1 id="adding_default_files">
<title>Adding or Changing Default Filenames</title>
<para>
To add or change the default filenames you edit the
<filename>models/lnd/clm/bld/namelist_files/namelist_defaults_clm.xml</filename>
and either change an existing filename or add a new one. Most entries in the
default namelist files, include different attributes that describe the different
properties that describe the differences in the datasets. Attributes include 
the: resolution, year to simulation, range of years to simulate for transient
datafiles, the land-mask, the representative concentration pathway (rcp) for future
scenarios, and the type of biogeochemistry (bgc) model used. For example the
<filename>fatmgrid</filename> for the 1.9x2.5 resolution is as follows:
<screen width="99">
&lt;fatmgrid hgrid="1.9x2.5" &gt;lnd/clm2/griddata/griddata_1.9x2.5_060404.nc
&lt;/fatmgrid&gt;
</screen>
Other <filename>fatmgrid</filename> files are distinguished from this one by 
their resolution (hgrid) attribute.
</para>
<para>
To add or change the default filenames for &clm; tools edit the 
<filename>models/lnd/clm/bld/namelist_files/namelist_defaults_clm_tools.xml</filename>
and either change an existing filename or add a new one. Editing this file is
similar to the <filename>namelist_defaults_clm.xml</filename> talked about above.
</para>
<sect2 id="required_files">
<title>What are the required files?</title>
<para>
Different types of simulations and different types of configurations for &clm; require
different lists of files. The Carbon Nitrogen (cn) Biogeochemistry model for example
requires <filename>stream_fldfilename_ndep</filename> files, which are NOT required by 
other bgc modes.  Transient simulations also require transient datasets, and the names 
of these datasets are sometimes different from the static versions (sometimes both are 
required as in the dynamic PFT cases).
</para>
<para>
In the following table we list the different files used by CLM, they are listed
in order of importance, dependencies, and customizing. So the required files 
are all near the top, and the files used only under different conditions are listed
later, and files with the fewest dependencies are near the top, as are the files
that are least likely to be customized.
</para>
<table id="table_required_files" tocentry="1" pgwide="1" frame="all">
<title>Required Files for Different Configurations and Simulation Types</title>
<tgroup cols="4" align="left" colsep="1" rowsep="1">
<colspec colnum="2" colname="config"></colspec>
<colspec colnum="5" colname="other"></colspec>
<spanspec spanname="notes_span" namest="config" nameend="other"></spanspec>
<thead>
<row>
   <entry morerows="1" valign="middle">Filename</entry>
   <entry><para>Config. type</para></entry>
   <entry><para>Simulation type</para></entry>
   <entry><para>Resol. Dependent?</para></entry>
   <entry><para>Other Dependencies?</para></entry>
</row>
<row>
   <entry spanname="notes_span">Notes
</entry>
</row>
</thead>
<tbody>
<row>
   <entry morerows="1" valign="middle">fpftcon</entry>
   <entry>ALL</entry>
   <entry>ALL</entry>
   <entry>No</entry>
   <entry>No</entry>
</row>
<row>
   <entry spanname="notes_span"><para>Not usually customized, as describes plant function
type properties. &ptclm; copies the file for you so that you can customize it if you
like, see <xref linkend="pftphyscopy"></xref>.</para></entry>
</row>
<row>
   <entry morerows="1" valign="middle">fsnowoptics</entry>
   <entry>ALL</entry>
   <entry>ALL</entry>
   <entry>No</entry>
   <entry>No</entry>
</row>
<row>
   <entry spanname="notes_span"><para>Not usually customized as describes global snow optical properties.</para></entry>
</row>
<row>
   <entry morerows="1" valign="middle">fsnowaging</entry>
   <entry>ALL</entry>
   <entry>ALL</entry>
   <entry>No</entry>
   <entry>No</entry>
</row>
<row>
   <entry spanname="notes_span"><para>Not usually customized as describes global snow aging properties.</para></entry>
</row>
<row>
   <entry morerows="1" valign="middle">fatmgrid</entry>
   <entry>ALL</entry>
   <entry>ALL</entry>
   <entry>Yes</entry>
   <entry>No</entry>
</row>
<row>
   <entry spanname="notes_span"><para>Creating, using <command>mkgriddata</command> 
usually gives you the amount of customization 
you need, as it just describes the grid and grid extents.</para></entry>
</row>
<row>
   <entry morerows="1" valign="middle">fatmlndfrc</entry>
   <entry>ALL</entry>
   <entry>ALL</entry>
   <entry>Yes</entry>
   <entry>land-mask</entry>
</row>
<row>
   <entry spanname="notes_span"><para>Describes the land-mask for points with active land, as well as the fraction
of each grid-cell covered by land. You might customize it to make sure the land-fraction
of your grid-cell matches the expected values for your site. But, usually you will just
use what mkgriddata gives you.</para></entry>
</row>
<row>
   <entry morerows="1" valign="middle">fsurdat</entry>
   <entry>ALL</entry>
   <entry>ALL</entry>
   <entry>Yes</entry>
   <entry>simulation-year</entry>
</row>
<row>
   <entry spanname="notes_span"><para>Describes percentages of different land-units, columns and 
vegetation types within each grid-cell. To customize for a specific point
or region you may want to use custom input datasets to <command>mksurfdata</command> when
creating the file. <command>mksurfdata</command> also allows you to customize the PFT,
and soil types to it see <xref linkend="mksurfdata_exp"></xref>. &ptclm; takes
advantage of this to create customized datasets as well, see the chapter on &ptclm;
at <xref linkend="PTCLMDOC"></xref>.</para></entry>
</row>
<row>
   <entry morerows="1" valign="middle">fpftdyn</entry>
   <entry>ALL</entry>
   <entry><para>transient land-use land-cover change</para></entry>
   <entry>Yes</entry>
   <entry><para>Simulation year range, and representative concentration pathway (rcp)</para></entry>
</row>
<row>
   <entry spanname="notes_span">See notes on fsurdat files.</entry>
</row>
<row>
   <entry morerows="1" valign="middle">frivinp_rtm</entry>
   <entry>RTM only</entry>
   <entry>ALL</entry>
   <entry>No</entry>
   <entry>No</entry>
</row>
<row>
   <entry spanname="notes_span"><para>We only provide a half-degree global river routing file. If you want
to model river flow for a smaller scale, or a basin regional scale, you would
need to create your own custom file to do that. Normally, we turn river-routing
OFF for regional or single point simulations.</para></entry>
</row>
<row>
   <entry morerows="1" valign="middle">flndtopo</entry>
   <entry>ALL</entry>
   <entry><para>fine-mesh simulations (specifying land resolution as a finer grid than
atmosphere resolution).</para></entry>
   <entry>Yes</entry>
   <entry>No</entry>
</row>
<row>
   <entry spanname="notes_span"><para>You may customize to give better surface heights for your site, or
input a higher resolution orography file when you create it using
<command>mkgriddata</command>.</para></entry>
</row>
<row>
   <entry morerows="1" valign="middle">fatmtopo</entry>
   <entry>ALL</entry>
   <entry><para>fine-mesh simulations (specifying land resolution as a finer grid than
atmosphere resolution).</para></entry>
   <entry>Yes</entry>
   <entry>No</entry>
</row>
<row>
   <entry spanname="notes_span"><para>You may customize to give better surface heights
    for your site, or
input a higher resolution orography file when you create it using
<command>mkgriddata</command>.</para></entry>
</row>
<row>
   <entry morerows="1" valign="middle">finidat</entry>
   <entry>ALL</entry>
   <entry><para>RUN_TYPE="startup", CLM_FORCE_COLDSTART="off"</para></entry>
   <entry>Yes</entry>
   <entry><para>mask, maxpft, bgc, simulation-year, start-date</para></entry>
</row>
<row>
   <entry spanname="notes_span"><para>Used for starting the model from a spun-up state.
Create these files by running the model
for multiple years and saving the restart file from the end of a spin-up 
simulation.</para>
</entry>
</row>
<row>
   <entry morerows="1" valign="middle">fglcmask</entry>
   <entry>glc_nec &gt; 0</entry>
   <entry><para>Used for simulations with the active glacier ice sheet model "cism"</para></entry>
   <entry>Yes</entry>
   <entry><para>glacier-grid</para></entry>
</row>
<row>
   <entry spanname="notes_span"><para>Needs to match the file used by "cism" and
be for the same glacier grid. Only customized as coupled with the glacier model.
</para>
</entry>
</row>
<row>
   <entry morerows="1" valign="middle">stream_fldfilename_ndep</entry>
   <entry>bgc=cn/cndv</entry>
   <entry>Yes</entry>
   <entry>No</entry>
   <entry><para>simulation-year</para></entry>
</row>
<row>
   <entry spanname="notes_span"><para>
You may customize this file to get the Nitrogen deposition characteristics 
of your site if available. This file will be interpolated while the model is
running from it's resolution to the resolution that &clm; is running at.
</para>
</entry>
</row>
</tbody>
</tgroup>
</table>
</sect2>

</sect1>

</chapter>
<!-- End of adding_files chapter -->
