This specifies all the steps necessary to create, set up, build, and run a case. The following assumes that $CCSMROOT is /user/ccsmroot. Create a new case named EXAMPLE_CASE in the ~/cesm1 directory. Use a an 1850 control compset at 1-degree resolution on bluefire. > cd /user/ccsmroot/scripts > ./create_newcase -case ~/cesm1/EXAMPLE_CASE \ -compset B_1850_CN \ -res 0.9x1.25_gx1v6 \ -mach bluefire Go to the $&CASEROOT; directory. Edit &env_mach_pes.xml; if a different pe-layout is desired first. Then set up and build the case. > cd ~/cesm1/EXAMPLE_CASE > ./cesm_setup > EXAMPLE_CASE.build Create a production test. Go to the test directory. Build the test first, then run the test and check the TestStatus (the first word should be PASS). > ./create_production_test > cd ../EXAMPLE_CASE_ERT > EXAMPLE_CASE_ERT.build > bsub < EXAMPLE_CASE_ERT.test > cat TestStatus Go back to the case directory, set the job to run 12 model months, use an editor to change the time limit in the run file to accommodate a 12-month run, and submit the job. > cd ../EXAMPLE_CASE > xmlchange -id STOP_OPTION -val nmonths > xmlchange -id STOP_N -val 12 > # use an editor to change EXAMPLE_CASE.run "#BSUB -W 1:30" to "#BSUB -W 6:00" > bsub < EXAMPLE_CASE.run Make sure the run succeeded. > grep "SUCCESSFUL TERMINATION" poe.stdout.* Set it to resubmit itself 10 times so that it will run a total of 11 years (including the initial year), and resubmit the case. (Note that a resubmit will automatically change the run to be a continuation run). > xmlchange -id RESUBMIT -val 10 > bsub < EXAMPLE_CASE.run The section setting the case initialization discussed starting a new case as a branch run or hybrid run by using data from a previous run. First you need to create a new case. Assume that $&CCSMROOT; is set to /user/ccsmroot and that $&EXEROOT; is /glade/scratch/$user/EXAMPLE_CASEp. Finally, assume that the branch or hybrid run is being carried out on NCAR's IBM system, bluefire. > cd /user/ccsmroot/scripts > create_newcase -case ~/cesm1/EXAMPLE_CASEp \ -compset B_2000 \ -res 0.9x1.25_gx1v6 \ -mach bluefire > cd ~/cesm1/EXAMPLE_CASEp For a branch run, modify &env_run.xml; to branch from EXAMPLE_CASE at year 0001-02-01. > xmlchange -id RUN_TYPE -val branch > xmlchange -id RUN_REFCASE -val EXAMPLE_CASE > xmlchange -id RUN_REFDATE -val 0001-02-01 For a hybrid run, modify &env_run.xml; to start up from EXAMPLE_CASE at year 0001-02-01. > xmlchange -id RUN_TYPE -val hybrid > xmlchange -id RUN_REFCASE -val EXAMPLE_CASE > xmlchange -id RUN_REFDATE -val 0001-02-01 For a branch run, &env_run.xml; for EXAMPLE_CASEp should be identical to EXAMPLE_CASE, except for the $RUN_TYPE setting. In addition, any modifications introduced into ~/cesm1/EXAMPLE_CASE/user_nl_cam, should be re-introduced into the user_nl_cam file in EXAMPLE_CASEp. Set up and build the case executable. > ./cesm_setup -case > ./EXAMPLE_CASEp.build Prestage the necessary restart/initial data in $RUNROOT (assumed to be /glade/scratch/$user/EXAMPLE_CASEp/run). Note that /glade/scratch/$user/EXAMPLE_CASEp/run was created during the build. Assume that the restart/initial data is on the NCAR HPSS. > cd /glade/scratch/$user/EXAMPLE_CASEbr/run > hsi > cget /CCSM/csm/EXAMPLE_CASE/rest/0001-02-01-00000/*" TODO: check this It is assumed that you already have a valid load-balanced scenario. Go back to the case directory, set the job to run 12 model months, use an editor to change the time limit in the run file to accommodate a 12-month run, then submit the job. > ~/cesm1/EXAMPLE_CASEp > xmlchange -id STOP_OPTION -val nmonths > xmlchange -id STOP_N -val 12 > # use an editor to change EXAMPLE_CASE.run "#BSUB -W 1:30" to "#BSUB -W 6:00" > bsub < EXAMPLE_CASEp.run Verify that the run succeeded. > grep "SUCCESSFUL TERMINATION" poe.stdout.* Change the run to a continuation run. Set it to resubmit itself 10 times so that it will run a total of 11 years (including the initial year), then resubmit the case. > xmlchange -id CONTINUE_RUN -val TRUE > xmlchange -id RESUMIT -val 10 > bsub < EXAMPLE_CASEp.run In general, the only supported calendar in the CESM time manager is the 365 day or no-leap calendar. This calendar has the standard 12 months, but it has 365 days every year and 28 days in every February. Monthly averages in CESM are truly computed over varying number of days depending on the month of the year. A gregorian calendar is available if the ESMF time manager is used. To use the ESMF time manager, the ESMF library has to be available locally and the ESMF library has to be turned on in CESM. See using ESMF in CESM for more information about setting up CESM with the ESMF library. This example modifies the PE layout for our original run, EXAMPLE_CASE. We now target the model to run on the jaguar supercomputer and modify our PE layout to use a common load balance configuration for CESM on large CRAY XT5 machines. Also see . In our original example, EXAMPLE_CASE, we used 128 pes with each component running sequentially over the entire set of processors. Original layout of EXAMPLE_CASE 128-pes/128-tasks layout Now we change the layout to use 1728 processors and run the ice, lnd, and cpl models concurrently on the same processors as the atm model while the ocean model will run on its own set of processors. The atm model will be run on 1664 pes using 832 MPI tasks each threaded 2 ways and starting on global MPI task 0. The ice model is run using 320 MPI tasks starting on global MPI task 0, but not threaded. The lnd model is run on 384 processors using 192 MPI tasks each threaded 2 ways starting at global MPI task 320 and the coupler is run on 320 processors using 320 MPI tasks starting at global MPI task 512. The ocn model uses 64 MPI tasks starting at global MPI task 832. New layout of EXAMPLE_CASE 1728-pes/896-tasks layout Since we will be modifying &env_mach_pes.xml; after cesm_setup was called, the following needs to be invoked: > ./cesm_setup -clean > xmlchange -id NTASKS_ATM -val 832 > xmlchange -id NTHRDS_ATM -val 2 > xmlchange -id ROOTPE_ATM -val 0 > xmlchange -id NTASKS_ICE -val 320 > xmlchange -id NTHRDS_ICE -val 1 > xmlchange -id ROOTPE_ICE -val 0 > xmlchange -id NTASKS_LND -val 192 > xmlchange -id NTHRDS_LND -val 2 > xmlchange -id ROOTPE_LND -val 320 > xmlchange -id NTASKS_ROF -val 192 > xmlchange -id NTHRDS_ROF -val 2 > xmlchange -id ROOTPE_ROF -val 320 > xmlchange -id NTASKS_CPL -val 320 > xmlchange -id NTHRDS_CPL -val 1 > xmlchange -id ROOTPE_CPL -val 512 > xmlchange -id NTASKS_OCN -val 64 > xmlchange -id NTHRDS_OCN -val 1 > xmlchange -id ROOTPE_OCN -val 832 > ./cesm_setup It is interesting to compare the timings from the 128- and 1728-processor runs. The timing output below shows that the original model run on 128 pes cost 851 pe-hours/simulated_year. Running on 1728 pes, the model cost more than 5 times as much, but it runs more than two and a half times faster. 128-processor case: Overall Metrics: Model Cost: 851.05 pe-hrs/simulated_year (scale= 1.00) Model Throughput: 3.61 simulated_years/day 1728-processor case: Overall Metrics: Model Cost: 4439.16 pe-hrs/simulated_year (scale= 1.00) Model Throughput: 9.34 simulated_years/day See understanding load balancing CESM for detailed information on understanding timing files. Like in CESM1.0, CESM1.1 cases are customized, built and run largely through setting what CESM1 calls "environment variables". These actually appear to the user as variables defined in xml files. Those files appear in the case directory once a case is created and are named something like env_*.xml. They are converted to actual environment variables via a csh script called ccsm_getenv. That script calls a perl script called xml2env that converts the xml files to shell files that are then sourced and removed. The ccsm_getenv and xml2env exist in the $CASEROOT/Tools directory. The environment variables are specified in xml files to support extra automated error checking and automatic generation of env variable documentation. If you want to have the cesm environment variables in your local shell environment, do the following > cd $CASEROOT > source ./Tools/ccsm_getenv You must run the ccsm_getenv from the CASEROOT directory exactly as shown above. There are multiple env_*.xml files including env_case.xml, env_mach_pes.xml, env_build.xml, and env_run.xml. To a large degree, the different env files exist so variables can be locked in different phases of the case setup, build, and run process. For more info on locking files, see . The important point is that env_case.xml variables cannot be changed after create_newcase is invoked. env_mach_pes cannot be changed after cesm_setup is invoked unless you plan to invoke the commands cesm_setup -clean, cesm_setup. env_build variables cannot be changed after the model is built unless you plan to clean and rebuild. env_run.xml variables can be changed anytime. The CESM1 scripting software checks that xml files are not changed when they shouldn't be. CESM recommends using the xmlchange tool to modify env variables. This will decrease the chance that typographical errors will creep into the xml files. Conversion of the xml files to environment variables can fail silently with certain xml format errors. To use xmlchange, do, for instance, > cd $CASEROOT > ./xmlchange -id STOP_OPTION -val nmonths > ./xmlchange -id STOP_N -val 6 which will change the variables STOP_OPTION and STOP_N in the file env_run.xml to the specified values. The xml files can be edited manually, but users should take care not to introduce any formatting errors that could lead to incomplete env settings. If there appear to be problems with the env variables (i.e. if the model doesn't seem to have consistent values compared to what's set in the xml files), then confirm that the env variables are being set properly. There are a couple of ways to do that. First, run the ccsm_getenv script as indicated above and review the output generated by the command "env". The env variables should match the xml settings. Another option is to edit the $CASEROOT/Tools/ccsm_getenv script and comment out the line "rm $i:r". That should leave the shell env files around, and they can then be reviewed. The latter approach should be undone as soon as possible to avoid problems running ccsm_getenv later. CESM recommends using the xmlchange tool to modify env variables. xmlchange supports error checking as part of the implementation. Also, using xmlchange will decrease the chance that typographical errors will creep into the xml files. Conversion of the xml files to environment variables can fail silently with certain xml format errors. To use xmlchange, do, for instance, > cd $CASEROOT > ./xmlchange -id STOP_OPTION -val nmonths > ./xmlchange -id STOP_N -val 6 which will change the variables STOP_OPTION and STOP_N in the file env_run.xml to the specified values. The xml files can be edited manually, but users should take care not to introduce any formatting errors that could lead to incomplete env settings. See also . It's possible that a formatting error has been introduced in the env xml files. This would lead to problems in setting the env variables. If there appear to be problems with the env variables (i.e. if the model doesn't seem to have consistent values compared to what's set in the xml files), then confirm that the env variables are being set properly. There are a couple of ways to do that. First, run the ccsm_getenv script via > cd $CASEROOT > source ./Tools/ccsm_getenv > env and review the output generated by the command "env". The env variables should match the xml settings. Another option is to edit the $CASEROOT/Tools/ccsm_getenv script and comment out the line "rm $i:r". That should leave the shell env files around, and they can then be reviewed. The latter approach should be undone as soon as possible to avoid problems running ccsm_getenv later. CESM supports use of either the CESM1 designed component interfaces which are based on MCT datatypes and are used by default in CESM or ESMF compliant component interfaces. In both cases, the driver and component models remain fundamentally the same. The ESMF interface implementation exists in CESM to support further development and testing of an ESMF driver or ESMF couplers and to allow CESM model components to interact with other coupled systems using ESMF coupling standards. The ESMF implementation in CESM1 has been tested with ESMF version TODO - ???. ESMF is NOT required or provided by CESM1. It must be downloaded and installed separately. It is safest to compile ESMF and CESM with identical compilers and mpi versions. It may be possible to use versions that are different but compatible; however, it is hard to predict which versions will be compatible and using different versions can result in problems that are difficult to track down. There are three possible modes of interaction between CESM and ESMF. No linking to an external ESMF library. CESM uses a native implementation of ESMF timekeeping interfaces (default). To run with the MCT interfaces and the native time manager, set the following env variables - cd to your case directory - edit env_build.xml - set COMP_INTERFACE to "MCT" - set USE_ESMF_LIB to "FALSE" Linking with an ESMF library to use the ESMF time manager but continued use of the native CESM component interfaces. To run with the native interfaces and ESMF time manager, set the following env variables - cd to your case directory - edit env_build.xml - set COMP_INTERFACE to "MCT" - set USE_ESMF_LIB to "TRUE" - set ESMF_LIBDIR to a valid installation directory of ESMF version 4.0.0rp2 Linking with an ESMF library in order to use ESMF component interfaces. In this mode ESMF timekeeping is also activated. To run with the ESMF interfaces and ESMF time manager, set the following env variables - cd to your case directory - edit env_build.xml - set COMP_INTERFACE to "ESMF" - set USE_ESMF_LIB to "TRUE" - set ESMF_LIBDIR to a valid installation directory of ESMF version 4.0.0rp2 The ESMF library can be activated in two ways in CCSM. The primary way is via the ESMF_LIBDIR env variable in the env_build.xml file described above. The secondary way is via a system environment variable called ESMFMKFILE. If this environment variable is set either through a system or module command, then the ESMF library will be picked up by the CCSM scripts, but the local CCSM variable, ESMF_LIBDIR, will always have precedence. To verify the correctness of the ESMF component interfaces in CESM, compute and compare CESM global integrals with identical runs differing only in the use of the MCT and ESMF interfaces. In both cases, the ESMF library should be active to guarantee identical time manager values. In both runs, the 'INFO_DBUG' parameter in env_run.xml should be set to 2 which activates the global integral diagnostics. A valid comparison would be a 10 day test from the same initial conditions. The global integrals produced in the cpl log file should be identical in both cases. This test can be set up manually as described above or a CME test can be carried out which is designed to test this exact capability. In CESM, there are several different $CASEROOT xml files. These include env_case.xml, env_mach_pes.xml, env_build.xml, and env_run.xml. These files are organized so variables can be locked during different phases of the case setup, build, and run. Locking variables is a feature of CESM that prevents users from changing variables after they have been resolved (used) in other parts of the scripts system. The variables in env_case are locked when create_newcase is called. The env_mach_pes variables are locked when &cesm_setup; is called. The env_build variables are locked when CESM is built, and the env_run variables are never locked and can be changed anytime. In addition, the Macros file is locked as part of the build step. The $CASEROOT/LockedFiles directory saves copies of the xml files to facilitate the locking feature. In summary: &env_case.xml; is locked upon invoking &create_newcase; and cannot be unlocked. To change settings in env_case, a new case has to be generated with create_newcase. &env_mach_pes.xml; are locked after running cesm_setup. After changing variable values in this file, you need to invoked cesm_setup -clean and then cesm_setup. Macros and &env_build.xml; are locked upon the successful completion of $CASE.build. Both Macros and &env_build.xml; can be unlocked by invoking $CASE.cleanbuild and then the model should be rebuilt. The following describes many of the files and directories in the $CASEROOT directory. Buildconf/ is the directory where the buildnml and buildexe component scripts reside and where the input_data_list files are generated by the buildnml scripts. CaseDocs/ is the directory where copies of the latest namelist/text input files from invoking preview_namelists are placed. These files should not be edited and exist only to help document the case setup and run. SourceMods/ contains directories for each component where case specific source code modifications can be included. The source files in these directories will always be used in preference to the source code in $CCSMROOT. This feature allows users to modify CESM source code on a case by case basis if that preferable over making modifications in the $CCSMROOT sandbox. LockedFiles/ is the directory that holds copies of the locked files. MachinesHist/ is a directory where previous case configurations are stored. In other words, when cesm_setup -clean is run, the current batch run script and env_mach_pes.xml file are copied into this directory, so when &cesm_setup; is subsequently run, there is an opportunity to review and compare previous setups. Macros is the Makefile Macros file for the current configuration. The Makefile is located in the Tools directory and is identical on all machines. The Macros file is a machine and compiler dependent file. This file is locked during the build step. README.case provides a summary of the commands used to generate this case. $CASE.build is the script that is run interactively to build the CESM model. $CASE.clean_build is the script that cleans the CESM build. $CASE.l_archive is the script that is submitted to the batch queue to archive CESM data to the long-term archive disk, like an hpss or mass storage system. $CASE.run is the script that is submitted to the batch queue to run a CESM job. This script could also be run interactively if resources allow. check_input_data is a tool that checks for missing input datasets and provides a capability for exporting them to local disk. cesm_setup is the script that is run to generate the $CASE.run script for the target env_mach_pes.xml file and if they have not already been created, the user_nl_xxx files for the target components. create_production_test is a tool that generates an exact restart test in a separate directory based on the current case. env_*.xml files contain variables used to set up, build, and run CESM. logs/ is the directory that contains a copy of the component log files from successful case runs. timing/ is the directory that contains timing output from each successful case run. xmlchange is a utility that supports changing xml variables in the $CASEROOT xml files. $CASEROOT/Tools/ a directory containing many scripts that are used to set up the CESM model as well as run it. Some of particular note are Makefile is the Makefile that will be used for the build. cesm_buildexe.csh is invoked by $CASEROOT/$CASE.build to generate the model executable. This script calls the component buildexe scripts in Buildconf. cesm_buildnml.csh is invoked by $CASEROOT/$CASE.build to generate the component namelists in $RUNROOT. This script calls the component buildnml scripts in Buildconf. ccsm_check_lockedfiles checks that any files in the $CASEROOT/LockedFiles/ directory match those in the $CASEROOT directory. This helps protect users from overwriting variables that should not be changed. ccsm_getenv converts the xml variables in $CASEROOT to csh environmental variables. getTiming.csh generates the timing information. getTiming.pl generates timing information and is used by getTiming.csh. mkDepends generates Makefile dependencies in a form suitable for inclusion into a Makefile. perf_summary.pl generates timing information. st_archive.sh is the short-term archive script. It moves model output out of run directory to the short-term archive directory. Associated with DOUT_S and DOUT_S_ROOT env variables in env_run.xml. taskmaker.pl derives pe counts and task and thread geometry info based on env var values set in the env_mach_pes file. xml2env converts env_*xml files to shell environment variable files that are then sourced for inclusion in the model environment. Used by the ccsm_getenv script. The parallel IO (PIO) library is included with CESM1 and is automatically built as part of the CESM build. Several CESM1 components use the PIO library to read and/or write data. The PIO library is a set of interfaces that support serial netcdf, parallel netcdf, or binary IO transparently. The implementation allows users to easily modify the pio setup on the fly to change the method (serial netcdf, parallel netcdf, or binary data) as well as various parameters associated with PIO to optimize IO performance. CESM1 prefers that data be written in CF compliant netcdf format to a single file that is independent of all parallel decomposition information. Historically, data was written by gathering global arrays on a root processor and then writing the data from the root processor to an external file using serial netcdf. The reverse process (read and scatter) was done for reading data. This method is relatively robust but is not memory scalable, performance scalable, or performance flexible. PIO works as follows. The PIO library is initialized and information is provided about the method (serial netcdf, parallel netcdf, or binary data), and the number of desired IO processors and their layout. The IO parameters define the set of processors that are involved in the IO. This can be as few as one and as many as all processors. The data, data name and data decomposition are also provided to PIO. Data is written through the PIO interface in the model specific decomposition. Inside PIO, the data is rearranged into a "stride 1" decomposition on the IO processors and the data is then written serially using netcdf or in parallel using pnetcdf. There are several benefits associated with using PIO. First, even with serial netcdf, the memory use can be significantly decreased because the global arrays are decomposed across the IO processors and written in chunks serially. This is critical as CESM1 runs at higher resolutions where global arrays need to be minimized due to memory availability. Second, pnetcdf can be turned on transparently potentially improving the IO performance. Third, PIO parameters such as the number of IO tasks and their layout can be tuned to reduce memory and optimize performance on a machine by machine basis. Fourth, the standard global gather and write or read and global scatter can be recovered by setting the number of io tasks to 1 and using serial netcdf. CESM1 uses the serial netcdf implementation of PIO and pnetcdf is turned off in PIO by default. Several components provide namelist inputs that allow use of pnetcdf in PIO. To use pnetcdf, a pnetcdf library (like netcdf) must be available on the local machine and PIO pnetcdf support must be turned on when PIO is built. This is done as follows Locate the local copy of pnetcdf. It must be version 1.1.1 or library Set LIB_PNETCDF in the Macros file to the directory of the pnetcdf library (ie. /contrib/pnetcdf1.1.1/lib). Add PNETCDF_PIO to the pio CONFIG_ARGS variable in the Macros file, and set it to the directory of the top level of a standard pnetcdf installation (ie /contrib/pnetcdf1.1.1). Run the clean_build script if the model has already been built. Run the build script to rebuilt pio and the full CESM1 system. Change component IO namelist settings to pnetcdf and set appropriate IO tasks and layout. The PNETCDF_PIO variable tells pio to build with pnetcdf support turned on. The LIB_PNETCDF variable tells the CESM Makefile to link in the pnetcdf library at the link step of the CESM1 build. There is an ongoing effort between CESM, pio developers, pnetcdf developers and hardware vendors to understand and improve the IO performance in the various library layers. To learn more about pio, see http://code.google.com/p/parallelio. See In this example, we further modify our EXAMPLE_CASEp code to set various CAM output fields. The variables that we set are listed below. See CAM Namelist Variables for a complete list of CAM namelist variables. avgflag_pertape Sets the averaging flag for all variables on a particular history file series. Default is to use default averaging flags for each variable. Average (A), Instantaneous (I), Maximum (X), and Minimum (M). nhtfrq Array of write frequencies for each history files series. When NHTFRQ(1) = 0, the file will be a monthly average. Only the first file series may be a monthly average. When NHTFRQ(i) > 0, frequency is input as number of timesteps. When NHTFRQ(i) < 0, frequency is input as number of hours. mfilt Array of number of time samples to write to each history file series (a time sample is the history output from a given timestep). ndens Array specifying output format for each history file series. Valid values are 1 or 2. '1' implies output real values are 8-byte and '2' implies output real values are 4-byte. Default: 2,2,2,2,2,2 fincl1 = 'field1', 'field2', ... List of fields to add to the primary history file. fincl[2..6] = 'field1', 'field2', ... List of fields to add to the auxiliary history file. fexcl1 = 'field1', 'field2', ... List of field names to exclude from the default primary history file (default fields on the Master Field List). fexcl[2..6] = 'field1', 'field2',... List of the field names to exclude from the auxiliary history files. Edit user_nl_cam and add the following lines at the end of the file: avgflag_pertape = 'A','I' nhtfrq = 0 ,-6 mfilt = 1 , 30 ndens = 2 , 2 fincl1 = 'FSN200','FSN200C','FLN200', 'FLN200C','QFLX','PRECTMX:X','TREFMXAV:X','TREFMNAV:M', 'TSMN:M','TSMX:X' fincl2 = 'T','Z3','U','V','PSL','PS','TS','PHIS' avgflag_pertape specifies how the output data will be averaged. In the first output file, b40.2000p.cam2.h0.yyyy-mm.nc, data will be averaged monthly. In the second output file, b40.2000p.cam2.h1.yyyy-mm-dd.nc, data will be instantaneous. nhtfrq sets the frequency of data writes, so b40.2000p.cam2.h0.yyyy-mm.nc will be written as a monthly average, while b40.2000p.cam2.h1.yyyy-mm-dd.nc will contain time slices that are written every 6 hours. mfilt sets the model to write one time sample in b40.2000p.cam2.h0.yyyy-mm.nc and 30 time samples in b40.2000p.cam2.h1.yyyy-mm-dd.nc. ndens sets both files to have 32-bit netCDF format output files. fincl1 sets the output fields for b40.2000p.cam2.h0.yyyy-mm.nc. A complete list of the CAM output fields appears here. In this example, we've asked for more variables than will fit on a Fortran line. As you can see, it is all right to split variable lists across lines. Also in this example, we've asked for maximum values of TREFMXAV and TSM, and minimum values of TREFMNAV and TSMN. fincl2 sets the output fields for b40.2000p.cam2.h1.yyyy-mm-dd.nc, much the same as fincl1 sets output fields for b40.2000p.cam2.h0.yyyy-mm.nc, only in this case, we are asking for instantaneous values rather than averaged values, and choosing different output fields. To set the greenhouse gas forcings, you must first understand the namelist variables associated with them. See CAM Namelist Variables for a complete list of CAM namelist variables. scenario_ghg Controls treatment of prescribed co2, ch4, n2o, cfc11, cfc12 volume mixing ratios. May be set to 'FIXED' or 'RAMPED' or 'RAMP_CO2_ONLY'. FIXED => volume mixing ratios are fixed and have either default or namelist input values. RAMPED => volume mixing ratios are time interpolated from the dataset specified by bndtvghg. RAMP_CO2_ONLY => only co2 mixing ratios are ramped at a rate determined by the variables ramp_co2_annual_rate, ramp_co2_cap, and ramp_co2_start_ymd. Default: FIXED bndtvghg Full pathname of time-variant boundary dataset for greenhouse gas surface values. rampyear_ghg If scenario_ghg is set to "RAMPED" then the greenhouse gas surface values are interpolated between the annual average values read from the file specified by bndtvghg. In that case, the value of this variable (> 0) fixes the year of the lower bounding value (i.e., the value for calendar day 1.0) used in the interpolation. For example, if rampyear_ghg = 1950, then the GHG surface values will be the result of interpolating between the values for 1950 and 1951 from the dataset. Default: 0 Edit user_nl_cam and add the following lines at the end of the file. The following assumes that "my_inputdata_path" is identical to $DIN_LOC_ROOT. scenario_ghg = 'RAMPED' bndtvghg = 'my_inputdata_path/atm/cam/ggas/ghg_hist_1765-2005_c091218.nc' rampyear_ghg = 2000 If you want to change the frequency of output for CAM or CLM (i.e. generate output every 6 model hours instead of once a model day) in the middle of a run, or if you want to change the fields that are output, in the middle of a run, you need to stop the run, rebuild and rerun it with the same casename and branch from the same casename. See the steps below for doing a branch run while retaining the casename. Rebuilding the case and restarting it where you left off, are necessary because CAM and CLM only read namelist variables once, at the beginning of a run. This is not the case for POP and CICE, they read the namelist input on every restart, and therefore for POP and CICE, you can change output fields and frequency by modifying the appropriate namelist variables and then doing a restart. The following example shows case B40.20th.1deg which runs from 1850 to 2005, and will generate high frequency output for years 1950 through 2005. CAM will output data every six hours instead of once a day. Also starting at year 1950 additional fields will be output by the model. The first step is to create case b40.20th.1deg and run the case for years 1850 through 1949 with your initial settings for output. Next move your entire case directory, $CASEDIR, somewhere else, because you need to rebuild and rerun the case using the same name. > cd $CASEDIR > mv b40.20th.1deg b40.20th.1deg.1850-1949 Now move your run directory, $RUNDIR, somewhere else as well. > cd $RUNDIR > mv b40.20th.1deg b40.20th.1deg.1850-1949 Next create a new case in your case directory with the same name, b40.20th.1deg. > cd $CASEDIR/scripts > create_newcase -mach bluefire -compset B_1850-2000_CN -res f09_g16 -case b40.20th.1deg cd $RUNDIR Next invoke the following commands > cd $CASEROOT > xmlchange -id RUN_TYPE -val 'branch' > xmlchange -id RUN_REFCASE -val 'b40.20th.1deg' > xmlchange -id RUN_REFDATE -val '1948-01-01' > xmlchange -id CAM_NML_USE_CASE -val '1850-2005_cam4' > xmlchange -id BRNCH_RETAIN_CASENAME -val 'TRUE' > xmlchange -id GET_REFCASE -val 'FALSE' Next set up the case and edit the coupler and CAM namelists. Set up the case. > ./cesm_setup -case Edit user_nl_cpl. Add the following to the end of the file. brnch_retain_casename = .true. Edit user_nl_cam. Check that bndtvghg = '$DIN_LOC_ROOT' and add the following to the end of the file doisccp = .true. isccpdata = '/fis/cgd/cseg/csm/inputdata/atm/cam/rad/isccp.tautab_invtau.nc' mfilt = 1,365,30,120,240 nhtfrq = 0,-24,-24,-6,-3 fincl2 = 'TREFHTMN','TREFHTMX','TREFHT','PRECC','PRECL','PSL' fincl3 = 'CLDICE','CLDLIQ','CLDTOT','CLOUD','CMFMC','CMFMCDZM','FISCCP1', 'FLDS','FLDSC','FLNS','FLUT','FLUTC','FSDS','FSDSC','FSNS', 'FSNSC','FSNTOA','FSNTOAC','LHFLX','OMEGA','OMEGA500', 'PRECSC','PRECSL','PS','Q','QREFHT','RELHUM','RHREFHT','SHFLX', 'SOLIN','T','TGCLDIWP','TGCLDLWP','U','V','Z3' fincl4 = 'PS:I','PSL:I','Q:I','T:I','U:I','V:I','Z3:I' fincl5 = 'CLDTOT','FLDS','FLDSC','FLNS','FLNSC','FSDS','FSDSC','FSNS', 'LHFLX','PRECC','PRECL','PRECSC','PRECSL','SHFLX', 'PS:I','QREFHT:I','TREFHT:I','TS:I' / Now build and run the case. > b40.20th.1deg.build > bsub < b40.20th.1deg.run The following was contributed by Art Mirin and outlines the procedure you would use if you want to run an F-configuration case forced by monthly averages of SST and ice coverage from a B-configuration case. As an example, the following uses an f09_g16 CESM B-configuration simulation using CAM5 physics and with cosp enabled. The procedure to create the SST/ICE file is as follows: Save monthly averaged 'aice' information from cice code (this is the default). Save monthly averaged SST information from pop2. To do this, copy $CCSMROOT/models/ocn/pop2/input_templates/gx1v6_tavg_contents, to $CASEROOT/SourceMods/src.pop2 and change the 2 in front of SST to 1 for monthly frequency. Extract (using ncrcat) SST from monthly pop2 history files and form a single netcdf file containing just SST; change SST to SST_cpl. > ncrcat -v SST case.pop.h.*.nc temp.nc > ncrename -v SST,SST_cpl temp.nc sst_cpl.nc Extract aice from monthly cice history files and form a single netcdf file containing aice; change aice to ice_cov; divide values by 100 (to convert from percent to fraction). > ncrcat -v aice case.cice.h.*.nc temp.nc > ncrename -v aice,ice_cov temp.nc temp2.nc > ncap2 -s 'ice_cov=ice_cov/100.' temp2.nc ice_cov.nc Modify fill values in the sst_cpl file (which are over land points) to have value -1.8 and remove fill and missing value designators; change coordinate lengths and names: to accomplish this, first run ncdump, then replace _ with -1.8 in SST_cpl, then remove lines with _FillValue and missing_value. (Note: although it might be possible to merely change the fill value to -1.8, this is conforming to other SST/ICE files, which have SST_cpl explicitly set to -1.8 over land.) To change coordinate lengths and names, replace nlon by lon, nlat by lat, TLONG by lon, TLAT by lat. The last step is to run ncgen. Note: when using ncdump followed by ncgen, precision will be lost; however, one can specify -d 9,17 to maximize precision - as in the following example: > ncdump -d 9,17 old.nc > old > ncgen -o new.nc new Modify fill values in the ice_cov file (which are over land points) to have value 1 and remove fill and missing value designators; change coordinate lengths and names; patch longitude and latitude to replace missing values: to accomplish this, first run ncdump, then replace _ with 1 in ice_cov, then remove lines with _FillValue and missing_value. To change coordinate lengths and names, replace ni by lon, nj by lat, TLON by lon, TLAT by lat. To patch longitude and latitude arrays, replace values of those arrays with those in sst_cpl file. The last step is to run ncgen. (Note: the replacement of longitude and latitude missing values by actual values should not be necessary but is safer.) Combine (using ncks) the two netcdf files. > ncks -v ice_cov ice_cov.nc sst_cpl.nc Rename the file to ssticetemp.nc. The time variable will refer to the number of days at the end of each month, counting from year 0, whereas the actual simulation began at year 1 (CESM default); however, we want time values to be in the middle of each month, referenced to the first year of the simulation (first time value equals 15.5); extract (using ncks) time variable from existing amip sst file (for correct number of months - 132 in this example) into working netcdf file. > ncks -d time,0,131 -v time amipsst.nc ssticetemp.nc Add date variable: ncdump date variable from existing amip sst file; modify first year to be year 0 instead of 1949 (do not including leading zeroes or it will interpret as octal) and use correct number of months; ncgen to new netcdf file; extract date (using ncks) and place in working netcdf file. > ncks -v date datefile.nc ssticetemp.nc Add datesec variable: extract (using ncks) datesec (correct number of months) from existing amip sst file and place in working netcdf file. > ncks -d time,0,131 -v datesec amipsst.nc ssticetemp.nc At this point, you have an SST/ICE file in the correct format. However, due to CAM's linear interpolation between mid-month values, you need to apply a procedure to assure that the computed monthly means are consistent with the input data. To do this, you can invoke the bcgen code in models/atm/cam/tools/icesst and following the following steps: Rename SST_cpl to SST, and ice_cov to ICEFRAC in the current SST/ICE file: > ncrename -v SST_cpl,SST -v ice_cov,ICEFRAC ssticetemp.nc In driver.f90, sufficiently expand the lengths of variables prev_history and history (16384 should be sufficient); also comment out the test that the climate year be between 1982 and 2001 (lines 152-158). In bcgen.f90 and setup_outfile.f90, change the dimensions of xlon and ???TODO xlat to (nlon,nlat); this is to accommodate use of non-cartesian ocean grid. In setup_outfile.f90, modify the 4th and 5th ???TODO arguments in the calls to wrap_nf_def_var for lon and lat to be 2 and dimids; this is to accommodate use of non-cartesian ocean grid. Adjust Makefile to have proper path for LIB_NETCDF and INC_NETCDF. Modify namelist accordingly. Make bcgen and execute per instructions. The resulting sstice_ts.nc file is the desired ICE/SST file. Place new SST/ICE file in desired location. In the $CASEROOT for the F compset you create, modify env_run.xml to have : SSTICE_DATA_FILENAME point to the complete path of your SST/ICE file. SSTICE_GRID_FILENAME correspond to full path of (in this case) gx1v6 grid file. SSTICE_YEAR_START set to 0, and SSTICE_YEAR_END to one less than the total number of years; set SSTICE_YEAR_ALIGN to 1 (since CESM starts counting at year 1). The pop and cice models both have similar decompositions and strategies for specifying the decomposition. Both models support decomposition of the horizontal grid into two-dimensional blocks, and these blocks are then allocated to individual processors inside each component. The decomposition must be specified when the models are built. There are four environment variables in env_build.xml for each model that specify the decomposition used. These variables are POP or CICE followed by _BLCKX, _BLCKY, _MXBLCKS, and _DECOMP. BLCKX and BLCKY specify the size of the local block in grid cells in the "x" and "y" direction. MXBLCKS specifies the maximum number of blocks that might be on any given processor, and DECOMP specifies the strategy for laying out the blocks on processors. The values for these environment variables are set automatically by the scripts in the $CASEROOT/Buildconf directory whenever the model is built or run is run. The scripts that generate the decompositions are CASEROOT/Buildconf/generate_pop_decomp.pl and $CASEROOT/Buildconf/generate_cice_decomp.pl. Those tools leverage decompositions stored in xml files, $CCSMROOT/models/ocn/pop2/bld/pop_decomp.xml and $CCSMROOT/models/ice/cice/bld/cice_decomp.xml, respectively. These utilities set the decomposition for a given resolution and total processor count. The decomposition used can have a significant effect on the model performance, and the decompositions specified by the tools above generally provide optimum or near optimum values for the given resolution and processor count. More information about cice and pop decompositions can be found in each of those user guides. The decompositions can be specified manually by setting the environment variable POP_AUTO_DECOMP or CICE_AUTO_DECOMP to false in env_build.xml (which turns off use of the scripts above) and then setting the four BLCKX, BLCKY, MXBLCKS, and DECOMP environment variables in env_build.xml. In general, relatively square and evenly divided Cartesian decompositions work well for pop at low to moderate resolution. Cice performs best with "tall and narrow" blocks because of the load imbalance for most global grids between the low and high latitudes. At high resolutions, more than one block per processor can result in land block elimination and non-Cartesian decompositions sometimes perform better. Testing of several decompositions is always recommended for performance and validation before a long run is started. The recommended method for initializing the CESM active ocean model (POP2) in a CESM startup case is to use the default settings; these initialize the ocean model from a state of rest. Occasionally, however, researchers are interested in initializing the ocean model from a "spun up" ocean condition that was generated from an existing CESM run. To accommodate their request, a nonstandard method of initializing the model was developed. It is called the startup_spunup option. The startup_spunup initialization is a research option that is designed for use by expert users only. Because of the complex interactions between the ocean-model parameterizations used to generate the spun-up case, $CASE_SP, and those used in the new startup case, it is impossible to provide a single recommended spun-up ocean initial condition for all cases. Instead, researchers must carefully select an existing solution whose case conditions closely match those in the new case. A mismatch of options between the spun-up case and the new case can result in scientifically invalid solutions. When a startup_spunup case is necessary, use this procedure: Create a new case, $CASE. By default, $CASE will be a "startup run." > create_newcase -case ~/cesm1/EXAMPLE_CASEocn \ -mach bluefire \ -compset B20TR \ -res 0.9x1.25_gx1v6 > cd ~/cesm1/EXAMPLE_CASEocn > ./cesm_setup The ocean restart filename is of the form ${CASE_SP}.pop.r.$date, where $date is the model date of your spun-up dataset. If the ocean restart files were written in binary format, a companion ascii-formatted restart "header" file will also exist. The companion header file will have the same name as the restart file, except that it will have the suffix ".hdr" appended at the end of the filename. You must copy both the binary restart file and the header file to your data directory. The spun-up ocean restart and restart header files must be available to your new case. Copy them directly into $RUNDIR. It is critically important to copy both the binary restart file and its companion header file to the $RUNDIR. > cp ${CASE_SP}.pop.r.$date $RUNDIR > cp ${CASE_SP}.pop.r.${date}.hdr $RUNDIR Redefine the ocean-model initial-condition dataset by editing user_nl_pop2 and add the following lines at the end of the file (enter the resolved string for ${CASE_SP}). set init_ts_suboption = 'spunup' init_ts_file = '${CASE_SP}.pop.r.$date Note that the model will automatically look for the ${CASE_SP}.pop.r.${date}.hdr file in $RUNDIR. Build and run as usual. Additional implementation details are provided in the the CESM coupler user guide about sequencing, parallel IO, performance, grids, threading, budgets, and other items. TODO: Fill this in Several compsets are hardwired in the CESM1 release. "create_newcase -l" provides a current listing of supported "out-of-the-box" compsets. To create a customized compset, > cd $CCSMROOT/scripts Now copy sample_compset_file.xml to another file, e.g. my_compset.xml. > cp sample_compset_file.xml my_compset.xml Edit the file, my_compset.xml, to create your own compset configuration. In particular, the NAME, SHORTNAME, DESC, and COMP_ variables must be specified. Other CESM env variables can optionally be added here. See scripts/ccsm_utils/Case.template/config_compsets.xml for other variables that might be related to compset configuration. Next run create_newcase with the optional -compset_file argument. Assume below that the compset name that is given in the my_compset.xml file is MYCS. > create_newcase -case mycase -res f19_g16 -compset MYCS -mach mymach -compset_file my_compset.xml The case mycase should have been generated and the configuration should be consistent with settings from the my_compset.xml file. In CESM1.1, the directory containing the model executable is cleanly separated from the directory where the model is run. As a result, it is now straightforward to run multiple cases where the env_build.xml for each case is identical all using a pre-built executable. As an example, this type of flexibility greatly simplifies carrying out UQ analysis. The following outlines the steps involved to do this. Create the executable that all runs will use. Call this case RefExe. The following would be a sample create_newcase command: > cd $CCSMROOT/scripts > create_newcase -case RefExe -compset B1850CN -res ne30_g16 -mach hopper > cd RefExe > ./cesm_setup > ./RefExe.build Verify that the model has build successfully. For reference below - the $EXEROOT for the RefExe case will be /scratch/scratchdirs/$CCSMUSER/RefExe/bld. All subsequent calls to create_newcase that will use the RefExe executable must have identical arguments for -compset, -res and -mach. Lets say that you want to run 2 separate cases, RefExe_Case1 and RefExe_Case2 that both use the executable RefExe. You would then do the following: > cd $CCSMROOT/scripts > create_newcase -case RefExe_Case1 -compset B1850CN -res ne30_g16 -mach hopper > cd RefExe_Case1 > ./cesm_setup > xmlchange -id EXEROOT -val /scratch/scratchdirs/$CCSMUSER/RefExe/bld. > xmlchange -id BUILD_COMPLETE -val TRUE > qsub RefExe_Case1.run > cd $CCSMROOT/scripts > create_newcase -case RefExe_Case2 -compset B1850CN -res ne30_g16 -mach hopper > cd RefExe_Case1 > ./cesm_setup > xmlchange -id EXEROOT -val /scratch/scratchdirs/$CCSMUSER/RefExe/bld. > xmlchange -id BUILD_COMPLETE -val TRUE > qsub RefExe_Case2.run Note that by setting BUILD_COMPLETE in &env_build.xml; to TRUE, the scripts assumes that the model has already been built for the case. Normally, the $CASE.build script fills this in when the build is successful. However, since you will not invoke the build for RefExe_Case1 and RefExe_Case2, you must then manually tell the scripts where the build is - and that it has been successful. This option is FOR EXPERTS ONLY and should only be used by those users that are completely familiar with the CESM scripts. Ensemble Kalman filter data assimilation (DA) can now be conducted within the software framework of CESM. This form of DA uses the multi-instance capability of CESM in which CESM advances an ensemble of model states of one or more CESM components forward to the same forecast time, when observations are available. Then the ensemble of forecast model states is passed to the Data Assimilation Research Testbed (DART), where each state is adjusted toward the observations which are available at that time. For details of this process see an introduction in BAMS (2009) and/or the DART home page. DART then passes the ensemble of adjusted model states back to CESM to be used as initial conditions for the next forecast. The references above describe the many uses of ensemble data assimilation, which include: generation of analyses (blends of model forecast and observations which are a better description of the physical system than either by itself), model development testing against actual observations (as opposed to other analyses), sensitivity analysis between model variables of interest in a particular synoptic situation, variability studies using the ensemble of equally valid model states, observation system simulation experiments (OSSEs). This use case outlines assimilation for a CAM (F comp set) build only. Assimilation is possible with the ocean component (B comp sets), and experimental assimilations with the land component (I comp sets) have been conducted. Additional use case descriptions will be added to cover those and any future evolution of the CESM+DART software. This use case assumes that the user is familiar with setting up and using CESM, and is willing to learn how to set up and use DART in the CESM context. There is no simple example which users can grab and run, because understanding what is being run is crucial to success and there are many choices to be made. The major steps of assimilating observations into CAM follow. Download DART Started. DART relieves researchers of the need to develop data assimilation capabilities, but familiarity with data assimilation and the DART facility is required in order to use it productively. This can be gained through the DART tutorial. Build the DART executables for a simple model to check that DART has been installed correctly. Build the DART executables for CAM, following a similar procedure to 2. The script .../DART/models/cam/CESM_setup.csh builds a CAM which combines the user's desired features and DART's required features. The characteristics of the CAM and assimilation set in CESM_setup.csh are: locations of the build, run, and archive directories, features of the $CASE to be built, locations of input files, including the initial ensemble of CAM (and CLM and CICE) states date and timing characteristics of the assimilation, machine and resource characteristics. - Copy CESM_setup.csh to the directory where the user wants to build CAM. - Edit that CESM_setup.csh to set most of the assimilation parameters. - Run CESM_setup.csh. Set the rest of the assimilation parameters: - cd to $CASE - Edit input.nml to set other characteristics of the assimilation. For details see the online help pages or the html in the user's $DART/filter/filter.html#GettingStarted. - Edit assimilate.csh to set the location of the observations to be assimilated. Sets of real observations are available for use, or synthetic observations can be created using the user's model. Submit the job using $CASE.submit in the $CASEROOT directory. Output from the assimilation is handled by the CESM archiver(s), which has been modified to handle DART output. Output appears in a new short-term archive directory .../archive/.../dart/hist. The 3 files created at each assimilation time are Prior_Diag.nc: the ensemble mean, spread, members (optionally), and 'inflation' fields from before the assimilation (at the end of the forecast). Posterior_Diag.nc: same as Prior, but from after the assimilation. obs_seq.final: the actual observations assimilated and the ensemble members estimates of those observations. The obs_seq.final files are usually processed by the obs_diag program in DART (.../DART/diagnostics/threed_sphere/obs_diag.f90), and the resulting NetCDF files are usually processed with Matlab scripts included in DART (or similar). Little knowledge of Matlab is needed to use them. The Prior and Posterior files can be examined with any NetCDF viewing tool. The following provides a very general overview of what you need to do to add a new atm, lnd, ocn, ice, glc or rof component to CESM. You must support init, run, and final top level interfaces. You must "send" the component grid and decomposition at initialization. You must pack and unpack coupling fields to/from interface datatypes. You must integrate forward a fixed amount of time and confirm that your model is in sync with the driver clock. You must provide/use "expected" scalar information as needed. - provide present/prognostic flags at initialization - provide "nextsw_cday" if atm component, use nextsw_cday if surface model - use mpicom - use stop and restart information - use inst_name, inst_index, inst_suffix (for cesm1.1+) - use other infodata information as needed (ie. starttype, case_name, configuration settings like aqua_planet, orbital settings) Use I/O unit manager in CESM You must meet filename conventions for history, restart, and log files There are some component to component variations in the interfaces and in the provide/use of scalar data, so it's best to follow another component of the same flavor. The top level interface will be a fortran file called ***_comp_mct.F90 where *** is atm, ocn, ice, or lnd. That file exists in all components. Below is a generic summary of what is going on using the atm component as an example. Other components are very close. You must support init, run, and final top level interfaces. The interfaces must follow the naming convention and argument types exactly. These interfaces must be in a file called atm_comp_mct.F90 and the module must be called atm_comp_mod. The driver will access the component model only through the init, run, and final interfaces. module atm_comp_mct public :: atm_init_mct public :: atm_run_mct public :: atm_final_mct subroutine atm_init_mct( EClock, cdata_a, x2a_a, a2x_a, NLFilename ) type(ESMF_Clock),intent(in) :: EClock type(seq_cdata), intent(inout) :: cdata_a type(mct_aVect), intent(inout) :: x2a_a type(mct_aVect), intent(inout) :: a2x_a character(len=*), optional, intent(IN) :: NLFilename ! Namelist filename subroutine atm_run_mct( EClock, cdata_a, x2a_a, a2x_a) type(ESMF_Clock) ,intent(in) :: EClock type(seq_cdata) ,intent(inout) :: cdata_a type(mct_aVect) ,intent(inout) :: x2a_a type(mct_aVect) ,intent(inout) :: a2x_a subroutine atm_final_mct( ) You must "send" the component grid and decomp at initialization The cdata datatype contains data for a grid and decomp. The decomp is an mct gsmap and the grid is an general grid. To access these data type from the init method, do the following. type(mct_gsMap), pointer :: gsMap_atm type(mct_gGrid), pointer :: dom_a call seq_cdata_setptrs(cdata_a, gsMap=gsMap_atm, dom=dom_a) ! call an mct_gsmap_init method and specify the global index ! of each local gridcell ! call an mct_gGrid_init method and fill the lon/lat/area/mask/frac ! arrays You must pack and unpack coupling fields to/from interface datatypes The fields coupling datatypes are mct attribute vectors. x2a contains the coupler->atm fields. a2x contains the atm->coupler fields. This datatype must be initialized by the component in the init method. To do that use the fields list provided by seq_flds_mod and initialize the gsmap first. lsize below is the local number of gridcells on the processor. the mct_aVect_init calls below allocate arrays in the attribute vector to store the appropriate number of fields of appropriate local size. use seq_flds_mod lsize = mct_gsMap_lsize(gsMap_atm, mpicom_atm) call mct_aVect_init(a2x_a, rList=seq_flds_a2x_fields, lsize=lsize) call mct_aVect_zero(a2x_a) call mct_aVect_init(x2a_a, rList=seq_flds_x2a_fields, lsize=lsize) call mct_aVect_zero(x2a_a) To pack the data, it's easiest just to write directly into the arrays inside the attribute vector in the following manner, integer :: index_a2x_Sa_pslv ! sea level atm pressure index_a2x_Sa_pslv = mct_avect_indexra(a2x_a,'Sa_pslv') do i=is,ie a2x_a%rAttr(index_a2x_Sa_pslv ,i) = psl(i) a2x_a%rAttr(index_a2x_Sa_z ,i) = zbot(i) a2x_a%rAttr(index_a2x_Sa_u ,i) = ubot(i) a2x_a%rAttr(index_a2x_Sa_v ,i) = vbot(i) enddo To unpack, basically do the same thing in the opposite direction. integer :: index_x2a_Sx_t ! surface temperature index_x2a_Sx_t = mct_avect_indexra(x2a_a,'Sx_t') do i=is,ie ts(i) = x2a_a%rAttr(index_x2a_Sx_t ,i) enddo The attribute vectors store only the "local" data and you basically just need to copy data from the model datatype to the coupling MCT (or ESMF) datatype. You must integrate forward a fixed amount of time and confirm that your model is in sync with the driver clock. You can access clock information from the EClock passed through the coupling interfaces and the EClockGetData method. The approach to sync the model and driver clock is very model specific. In some cases, a model may just get the current time from the EClock and use it. That probably only happens for models that don't advance in time (like maybe a data model). In other cases, model clocks may be initialized based on the EClock data at initialization and then the model time and driver time are regularly compared for consistency. Another approach is to use the "dt" provided by the EClock to advance a model "dt" seconds. Some examples are provided. type(ESMF_Clock) ,intent(in) :: EClock ! EClock initialization data call seq_timemgr_EClockGetData(EClock, & start_ymd=start_ymd, start_tod=start_tod, & ref_ymd=ref_ymd, ref_tod=ref_tod, & stop_ymd=stop_ymd, stop_tod=stop_tod, & calendar=calendar ) ! EClock dt call seq_timemgr_EClockGetData(Eclock,dtime=atm_cpl_dt) ! EClock current time call seq_timemgr_EClockGetData(EClock,curr_ymd=ymd_sync,curr_tod=tod_sync, & curr_yr=yr_sync,curr_mon=mon_sync,curr_day=day_sync) ! Check synchronization ymd=model_ymd tod=model_tod if (.not. seq_timemgr_EClockDateInSync( EClock, ymd, tod)) then write(*,*) "Clocks not in sync" call shr_sys_abort() endif You must provide/use "expected" scalar information as needed. Each component varies a bit wrt what's provided and used. Provide present/prognostic flags at initialization. The logical flag present means the component provides data. The logical flag prognostic means the component uses data. Stub models set both to false. Data models generally set present to true and prognostic to false, except in cases where a data model needs some coupling data for some internal computations (e.g. DOCN-SOM). Active models generally set both flags to true. call seq_infodata_PutData(infodata, atm_present=.true.) call seq_infodata_PutData(infodata, atm_prognostic=.true.) Provide "nextsw_cday" if you are adding an atm component. Use nextsw_cday if you are adding a surface model. The real variable nextsw_cday is the time of the next atm radiation calculation if it occurs at the next coupling period. If there is no radiation calculation on the next timestep, this should be set to -1. This allows surface albedos and the atm radiation calculation to stay synced up. The surface models use the nextsw_day field and compute albedos based on that time. call seq_infodata_PutData( infodata, nextsw_cday=nextsw_cday ) Use the mpi communicator, mpicom, which is provided by the driver to the component. This must be used for all internal model communication. The data is provided in the cdata datatype. call seq_cdata_setptrs(cdata_a, mpicom=mpicom_atm) Use stop and restart information provided by the driver. The driver will tell the component if this is the last coupling period and/or if a restart is required at the end of this coupling period. The component should listen to both these flags. stop_now = seq_timemgr_StopAlarmIsOn(EClock) restart_now = seq_timemgr_RestartAlarmIsOn(EClock) Use inst_name, inst_index, inst_suffix needed for for the multiple instance capability. As a starting point, the following provides standard code that can be added: integer(IN) :: COMPID ! mct comp id integer :: inst_index ! number of current instance (ie. 1) character(len=16) :: inst_name ! fullname of current instance (ie. "lnd_0001") character(len=16) :: inst_suffix ! char string associated with instance call seq_cdata_setptrs(cdata, ID=COMPID) inst_name = seq_comm_name(COMPID) inst_index = seq_comm_inst(COMPID) inst_suffix = seq_comm_suffix(COMPID) Use other infodata information as needed (ie. starttype, case_name, configuration settings like aqua_planet, orbital settings) Use the I/O unit manager in CESM. To avoid conflicts in I/O unit numbers between components, models should call the shr_file_getUnit and shr_file_freeUnit methods to acquire and release available unit numbers. nunit = shr_file_getUnit() open(nunit,file='xyz') read(nunit,*) xyz close(nunit) call shr_file_freeUnit(nunit) Meet filename conventions for history, restart, and log files. There are specific filename conventions for CESM. In particular, all history files should be CF compliant netcdf. This format is also recommended for restart files. The format is something like $CASE.atm.ha.2001-01.nc ! history $CASE.atm.r.2001-01-00000.nc ! restart The log files are set by a shared method. In particular, models should do something like !--- open log file --- if (my_task == master_task) then logUnit = shr_file_getUnit() call shr_file_setIO('atm_modelio.nml',logUnit) else logUnit = 6 endif That logunit value should then be used by all processors in the model to write "stdout" messages. shr_file_setIO associates the logUnit number with a unique log filename for the case. unit 6 is used on non-root processors and information written from those processors goes to a stdout file which is machine dependent.