================================================================================ This file describes changes made to the PTCLM tool ================================================================================ ================================================================================ Originator: erik Date: Jan/20/2020 Tag: PTCLM2_20200118 One-line: Update to python3 add izumi support Testing: Standard PTCLM test script on cheyenne Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values Update to python3 and add support for izumi for testing. Add code of conduct file. Allow hostname check in buildtoools for cheyenne to have 2-digit numbers. Fixes #10 - Add izumi Fixes #9 -- Update to python3 Fixes #8 -- compiler versions on cheyenne Fixes #7 -- change last xmlchange_cmnds files to shell_commands ================================================================================ Originator: erik Date: Jun/11/2018 Tag: PTCLM2_180611 One-line: Fix a few bugs in latest PTCLMmkdata Testing: Standard PTCLM test script on cheyenne Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values Fix so properly finds historical time-series files from 1850-2015 rather than ending in 2005. Setup to handle CESM checkout as well as CTSM checkout better. Better matching for cheyenne nodes. More work on options for different machines. Update README file. Fixes: #3, #4, #5 ================================================================================ Originator: erik Date: Feb/14/2018 Tag: PTCLM2_180214 One-line: Get working with the new CTSM checkout on the move to git Testing: Standard PTCLM test script on cheyenne Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values CESM_DIR env variable gets turned into CTSM_DIR and CIME_DIR. Set defaults according to a the new CTSM checkout in git. Update the testing files, and make sure everything is working in a CTSM checkout with the standard testing. ================================================================================ Originator: erik Date: Dec/16/2017 Tag: PTCLM2_171216c One-line: Add back in missing compare directories: std_US-Ha1 std_US-Var std_Us-Ne1 trans_US-Ha1 Testing: Standard PTCLM test script on yellowstone/hobart/cheyenne Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values Missing compare directories: std_US-Ha1 std_US-Var std_Us-Ne1 trans_US-Ha1 A test/compdirs/* ================================================================================ Originator: erik Date: Dec/16/2017 Tag: PTCLM2_171216b One-line: Add back in missing compare directory US-ARM Testing: Standard PTCLM test script on yellowstone/hobart/cheyenne Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values Missing compare files: A test/compdirs/std_US-ARM/README.PTCLM A test/compdirs/std_US-ARM/run.log A test/compdirs/std_US-ARM/shell_commands A test/compdirs/std_US-ARM/user_nl_clm ================================================================================ Originator: erik Date: Dec/16/2017 Tag: PTCLM2_171216 One-line: Update compare files one more time to make generic Testing: Standard PTCLM test script on yellowstone Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values M test/PTCLMtestlist.py ----- Add replacement of absolute path of CESMDIR and also ncl path. This should make it generic for different machines. Update the compare files M test/compdirs/*/* M test/listings/help M test/listings/list M test/listings/sitelist ================================================================================ Originator: erik Date: Dec/15/2017 Tag: PTCLM2_171215 One-line: Remove subversion keywords and make comparisons generic Testing: Standard PTCLM test script on yellowstone Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values Remove use of Subversion keywords, and substitute referring to the latest tagnmae in the ChangeLog file. Also for testing replace strings that are very user or machine specific with generic strings, so that comparisons can be done correctly. These are for the: CESMDIR, version, inputdata directory, or current date. M PTCLMmkdata --- Remove svn keyword expansion for --version with latest tagname in ChangeLog file M PTCLMsublist_prog.py - Remove svn keyword expansion for --version with latest tagname in ChangeLog file M test/PTCLMtesting_prog.py - Remove svn keyword expansion for --version with latest tagname in ChangeLog file M test/PTCLMtestlist.py ----- Add method ReplaceIDInfoInFile to replace specific identifying information in a file with generic strings. Call it when appropriate for comparisons. Add test for it. And correct default cesmdir. Update the compare files so they have generic strings for user/machine/version/date identifying information. M test/compdirs/*/* M test/listings/help M test/listings/list M test/listings/sitelist ================================================================================ Originator: erik Date: Oct/24/2017 Tag: PTCLM2_171024b One-line: Add needed ending single quote for CLM_BLDNML_OPTS setting in shell_commands Testing: Standard PTCLM test script on cheyenne Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values A test/compdirs/crop_file_creation_US-Ne1/shell_commands - Add in missing file M PTCLMmkdata --- Add needed ending single quote in CLM_BLDNML_OPTS setting M test/compdirs/*/shell_commands - Update with above fix ================================================================================ Originator: erik Date: Oct/24/2017 Tag: PTCLM2_171024 One-line: Add --crop/--no-crop option (--no-crop is default) add US-Ne1 to test crop Testing: Standard PTCLM test script on cheyenne Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values M PTCLMmkdata - Add note to MPILIB, add new option --crop/--no-crop that will be passed to CLM_BLDNML_OPTS and mksurfdata.pl change CLM_BLDNML_OPTS so it uses --append M test/PTCLMtestlist.xml Add US-Ne1 site to test crop M PTCLM_sitedata/PTCLMDATA_sitedata.txt M PTCLM_sitedata/PTCLMDATA_soildata.txt M PTCLM_sitedata/PTCLMDATA_pftdata.txt A test/compdirs/crop_file_creation_US-Ne1/README.PTCLM A test/compdirs/crop_file_creation_US-Ne1/user_nl_clm A test/compdirs/crop_file_creation_US-Ne1/run.log M test/compdirs/copyfiles.csh ================================================================================ Originator: erik Date: Oct/16/2017 Tag: PTCLM2_171016d One-line: Update default walltime for cheyenne Testing: Standard PTCLM test script on cheyenne Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values M PTCLMsublist_prog.py --- Update default walltime to cheyenne syntax for an hour and a half M README -- Fix bug 1928 (documentation on compset to use) ================================================================================ Originator: erik Date: Oct/16/2017 Tag: PTCLM2_171016 One-line: PTCLM update to clm4_5_16_r259 with cime5.4.0-alpha.03 Get batch working on cheyenne, and make cheyenne default add --wall option to PCLMsublist Testing: Standard PTCLM test script on cheyenne Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values Add --wall option to PTCLMsublist and get setup and working for cheyenne. Remove hopper and janus as they are decommissioned. Correct the default CESM root directory for batch submission. Update the compare listings for testing M PTCLMsublist_prog.py - Add --wall option, make default cheyenne, correct cesmdir default M batchque.py ---------- Add cheyenne settings, remote hopper and janus, cheyenne has to cd to run directory in job script, rather than set in batch M PTCLMsublist --------- Update list of machines setup for M README --------------- Update to work on cheyenne and demonstrate batch submission Update compare listings M test/listings/help M test/compdirs/*/run.log M test/compdirs/*/user_nl_clm M test/compdirs/*/shell_commands M test/compdirs/*/README.PTCLM ================================================================================ Originator: erik Date: Jul/06/2017 Tag: PTCLM2_170706 One-line: Get PTCLM working again in clm4_5_14_r244 with cime5.3.0-alpha.21 and on cheyenne and hobart Testing: Standard PTCLM test script on cheyenne Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values Test for cheyenne compute node hostnames as well Remove hopper, make some changes to get closer to working on yongi Add module load for hobart Update README file, add a README file/script that runs on cheyenne Correct path to modules for cheyenne Add needed space for three more tests, now all, but one test passes as expected Update compare files, and make sure have a space after mksurfdata options M test/README -- Add notes about loading ncl module M buildtools --- Remove hopper, get working on hobart, and cheyenne M test/compdirs/*/* --------- Update compare directories M test/listings/* ----------- Update compare listings A test/README.run_cheyenne -- Add README/script to run on cheyenne ================================================================================ Originator: erik Date: Mar/02/2017 Tag: PTCLM2_170302 One-line: Get PTCLM working again in clm4_5_14_r226 with cime5.2.0-alpha23 Testing: Standard PTCLM test script on yellowstone and test_driver.sh in clm4_5_14_r226 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values 34 tests still fail in standard PTCLM test script Remove shared level directory. Remove options no longer needed phys and rcp option sent to mksurfdata_map. Get build of tools working with latest cime. Remove tests that don't apply anymore. Update ompare files. --- Remove tests that aren't options anymore D test/compdirs/rcp26_US-Dk2/user_nl_clm D test/compdirs/rcp26_US-Dk2/run.log D test/compdirs/rcp26_US-Dk2/shell_commands D test/compdirs/rcp26_US-Dk2/README.PTCLM D test/compdirs/rcp26_US-Dk2 D test/compdirs/rcp6_US-IB1/user_nl_clm D test/compdirs/rcp6_US-IB1/run.log D test/compdirs/rcp6_US-IB1/shell_commands D test/compdirs/rcp6_US-IB1/README.PTCLM D test/compdirs/rcp6_US-IB1 D test/compdirs/rcp45_US-Dk3/run.log D test/compdirs/rcp45_US-Dk3/shell_commands D test/compdirs/rcp45_US-Dk3/README.PTCLM D test/compdirs/rcp45_US-Dk3/user_nl_clm D test/compdirs/rcp45_US-Dk3 D test/compdirs/rcp85_US-Me4/user_nl_clm D test/compdirs/rcp85_US-Me4/run.log D test/compdirs/rcp85_US-Me4/shell_commands D test/compdirs/rcp85_US-Me4/README.PTCLM D test/compdirs/rcp85_US-Me4 M KnownBugs M README M test/PTCLMtestlist.xml M test/PTCLMtesting_prog.py M buildtools M PTCLMmkdata M mydatafiles/1x1pt_US-UMB/user_nl_clm M mydatafiles/1x1pt_US-UMB/README.PTCLM M PTCLMsublist_prog.py ------ Update compare files M test/listings/help M test/listings/list M test/listings/sitelist M test/compdirs/std_BR-Sa3/user_nl_clm M test/compdirs/std_BR-Sa3/run.log M test/compdirs/std_BR-Sa3/shell_commands M test/compdirs/std_BR-Sa3/README.PTCLM M test/compdirs/std_quiet_US-MMS/run.log M test/compdirs/std_quiet_US-MMS/shell_commands M test/compdirs/std_quiet_US-MMS/README.PTCLM M test/compdirs/std_quiet_US-MMS/user_nl_clm M test/compdirs/notowyrs_US-WCr/user_nl_clm M test/compdirs/notowyrs_US-WCr/run.log M test/compdirs/notowyrs_US-WCr/shell_commands M test/compdirs/notowyrs_US-WCr/README.PTCLM M test/compdirs/std_2000_US-MOz/shell_commands M test/compdirs/std_2000_US-MOz/README.PTCLM M test/compdirs/std_2000_US-MOz/user_nl_clm M test/compdirs/std_2000_US-MOz/run.log M test/compdirs/noopt_US-CHATS/README.PTCLM M test/compdirs/noopt_US-CHATS/user_nl_clm M test/compdirs/noopt_US-CHATS/run.log M test/compdirs/noopt_US-CHATS/shell_commands M test/compdirs/std_ES-ES1/README.PTCLM M test/compdirs/std_ES-ES1/user_nl_clm M test/compdirs/std_ES-ES1/run.log M test/compdirs/std_ES-ES1/shell_commands M test/compdirs/trans_clm45_US-Ha1/shell_commands M test/compdirs/trans_clm45_US-Ha1/README.PTCLM M test/compdirs/trans_clm45_US-Ha1/user_nl_clm M test/compdirs/trans_clm45_US-Ha1/run.log M test/compdirs/std_LTER-Sev/shell_commands M test/compdirs/std_LTER-Sev/README.PTCLM M test/compdirs/std_LTER-Sev/user_nl_clm M test/compdirs/std_LTER-Sev/run.log M test/compdirs/std_clm45_US-Ha1/shell_commands M test/compdirs/std_clm45_US-Ha1/README.PTCLM M test/compdirs/std_clm45_US-Ha1/user_nl_clm M test/compdirs/std_clm45_US-Ha1/run.log M test/compdirs/std_DE-Tha/shell_commands M test/compdirs/std_DE-Tha/README.PTCLM M test/compdirs/std_DE-Tha/user_nl_clm M test/compdirs/std_DE-Tha/run.log M test/compdirs/soigrd_CA-Qfo/user_nl_clm M test/compdirs/soigrd_CA-Qfo/run.log M test/compdirs/soigrd_CA-Qfo/shell_commands M test/compdirs/soigrd_CA-Qfo/README.PTCLM M test/compdirs/std_IT-Cpz/user_nl_clm M test/compdirs/std_IT-Cpz/run.log M test/compdirs/std_IT-Cpz/shell_commands M test/compdirs/std_IT-Cpz/README.PTCLM M test/compdirs/soigrd_CA-Ojp/user_nl_clm M test/compdirs/soigrd_CA-Ojp/run.log M test/compdirs/soigrd_CA-Ojp/shell_commands M test/compdirs/soigrd_CA-Ojp/README.PTCLM M test/compdirs/std_FL-Hyy/shell_commands M test/compdirs/std_FL-Hyy/README.PTCLM M test/compdirs/std_FL-Hyy/user_nl_clm M test/compdirs/std_FL-Hyy/run.log M test/compdirs/std_RF-Bra/run.log M test/compdirs/std_RF-Bra/shell_commands M test/compdirs/std_RF-Bra/README.PTCLM M test/compdirs/std_RF-Bra/user_nl_clm M test/compdirs/std_clm45_US-Var/shell_commands M test/compdirs/std_clm45_US-Var/README.PTCLM M test/compdirs/std_clm45_US-Var/user_nl_clm M test/compdirs/std_clm45_US-Var/run.log M test/compdirs/std_1850_US-Me2/run.log M test/compdirs/std_1850_US-Me2/shell_commands M test/compdirs/std_1850_US-Me2/README.PTCLM M test/compdirs/std_1850_US-Me2/user_nl_clm M test/compdirs/stdurbpt_pftgrd_CA-Oas/user_nl_clm M test/compdirs/stdurbpt_pftgrd_CA-Oas/run.log M test/compdirs/stdurbpt_pftgrd_CA-Oas/shell_commands M test/compdirs/stdurbpt_pftgrd_CA-Oas/README.PTCLM M test/compdirs/std_IT-Col/shell_commands M test/compdirs/std_IT-Col/README.PTCLM M test/compdirs/std_IT-Col/user_nl_clm M test/compdirs/std_IT-Col/run.log M test/compdirs/std_FL-Kaa/run.log M test/compdirs/std_FL-Kaa/shell_commands M test/compdirs/std_FL-Kaa/README.PTCLM M test/compdirs/std_FL-Kaa/user_nl_clm M test/compdirs/std_soigrd_BR-Sa1/user_nl_clm M test/compdirs/std_soigrd_BR-Sa1/run.log M test/compdirs/std_soigrd_BR-Sa1/shell_commands M test/compdirs/std_soigrd_BR-Sa1/README.PTCLM M test/compdirs/pftgrd_CA-Obs/user_nl_clm M test/compdirs/pftgrd_CA-Obs/run.log M test/compdirs/pftgrd_CA-Obs/shell_commands M test/compdirs/pftgrd_CA-Obs/README.PTCLM M test/compdirs/trans_20th_US-Ne3/user_nl_clm M test/compdirs/trans_20th_US-Ne3/run.log M test/compdirs/trans_20th_US-Ne3/shell_commands M test/compdirs/trans_20th_US-Ne3/README.PTCLM M test/compdirs/notowyrs_CA-Man/user_nl_clm M test/compdirs/notowyrs_CA-Man/run.log M test/compdirs/notowyrs_CA-Man/shell_commands M test/compdirs/notowyrs_CA-Man/README.PTCLM M test/compdirs/std_BE-Vie/user_nl_clm M test/compdirs/std_BE-Vie/run.log M test/compdirs/std_BE-Vie/shell_commands M test/compdirs/std_BE-Vie/README.PTCLM M test/compdirs/std_LBA-Cax/shell_commands M test/compdirs/std_LBA-Cax/README.PTCLM M test/compdirs/std_LBA-Cax/user_nl_clm M test/compdirs/std_LBA-Cax/run.log M test/compdirs/std_US-UMB/user_nl_clm M test/compdirs/std_US-UMB/run.log M test/compdirs/std_US-UMB/shell_commands M test/compdirs/std_US-UMB/README.PTCLM M test/compdirs/std_clm45_US-ARM/shell_commands M test/compdirs/std_clm45_US-ARM/README.PTCLM M test/compdirs/std_clm45_US-ARM/user_nl_clm M test/compdirs/std_clm45_US-ARM/run.log M test/compdirs/crop_US-IB1/user_nl_clm M test/compdirs/crop_US-IB1/run.log M test/compdirs/crop_US-IB1/shell_commands M test/compdirs/crop_US-IB1/README.PTCLM M test/compdirs/std_quiet_US-Bo1/user_nl_clm M test/compdirs/std_quiet_US-Bo1/run.log M test/compdirs/std_quiet_US-Bo1/shell_commands M test/compdirs/std_quiet_US-Bo1/README.PTCLM M test/compdirs/global_US-UMB/run.log M test/compdirs/global_US-UMB/shell_commands M test/compdirs/global_US-UMB/README.PTCLM M test/compdirs/global_US-UMB/user_nl_clm M test/compdirs/cyc_TS-Ts1/user_nl_clm M test/compdirs/cyc_TS-Ts1/run.log M test/compdirs/cyc_TS-Ts1/shell_commands M test/compdirs/cyc_TS-Ts1/README.PTCLM M test/compdirs/notowyrs_CA-Let/user_nl_clm M test/compdirs/notowyrs_CA-Let/run.log M test/compdirs/notowyrs_CA-Let/shell_commands M test/compdirs/notowyrs_CA-Let/README.PTCLM M test/compdirs/cyc_TS-Ts2/user_nl_clm M test/compdirs/cyc_TS-Ts2/run.log M test/compdirs/cyc_TS-Ts2/shell_commands M test/compdirs/cyc_TS-Ts2/README.PTCLM M test/compdirs/cyc_TS-Ts3/README.PTCLM M test/compdirs/cyc_TS-Ts3/user_nl_clm M test/compdirs/cyc_TS-Ts3/run.log M test/compdirs/cyc_TS-Ts3/shell_commands M test/compdirs/pftgrd_CA-Ca1/user_nl_clm M test/compdirs/pftgrd_CA-Ca1/run.log M test/compdirs/pftgrd_CA-Ca1/shell_commands M test/compdirs/pftgrd_CA-Ca1/README.PTCLM M test/compdirs/std_quiet_US-Ho1/run.log M test/compdirs/std_quiet_US-Ho1/shell_commands M test/compdirs/std_quiet_US-Ho1/README.PTCLM M test/compdirs/std_quiet_US-Ho1/user_nl_clm ================================================================================ Originator: erik Date: Aug/18/2016 Tag: PTCLM2_160818 One-line: Update compare tests, and allow clm5_0 and make it the default Testing: Standard PTCLM test script on yellowstone and test_driver.sh in clm4_5_11_r189 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values M PTCLMmkdata ------------- Change so can work for clm5_0 and make clm5_0 the default M test/compdirs/*/run.log M test/listings/help M test/listings/list M test/listings/sitelist ================================================================================ Originator: sacks Date: Jan/27/2016 Tag: PTCLM2_160127a One-line: Fix typos for building on geyser and pronghorn Testing: NONE (but will run test_driver.sh from CLM) Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values M buildtools --- Fix typos for geyser and pronghorn ================================================================================ Originator: sacks Date: Jan/27/2016 Tag: PTCLM2_160127 One-line: Build on pronghorn same as for yellowstone Testing: NONE (but will run test_driver.sh from CLM) Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values M buildtools --- Extend conditional to apply to pronghorn ================================================================================ Originator: erik Date: Oct/09/2015 Tag: PTCLM2_151009 One-line: Testing: Standard PTCLM test script on yellowstone, and test_driver.sh in clm4_5_3_r136 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values M buildtools --- Add softlink to Macros.make so can build gen_domain with cime4.0.1 M PTCLMmkdata -- Add a wildcard for PFT number in landuse timeseries file print out wildcard when file isn't found. ================================================================================ Originator: erik Date: Aug/26/2015 Tag: PTCLM2_150826 One-line: Allow for pft??_ in surface dataset filename Testing: Standard PTCLM test script on yellowstone, and test_driver.sh in clm4_5_1_r119 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values In clm4_5_1_r108, surface datasets change standard filenames to include the PFT number. So PTCLMmkdata would no longer work for clm4_5. Add a wildcard for the pft number, so both clm4_0 and clm4_5 will now work. Move goldbach to hobart. M buildtools M PTCLMmkdata M test/compdirs/*/* ================================================================================ Originator: erik Date: Apr/14/2015 Tag: PTCLM2_150414 One-line: Send in cimeroot to buildtools Testing: Standard PTCLM test script on yellowstone, and test_driver.sh in clm4_5_1_r105 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values M buildtools ================================================================================ Originator: erik Date: Apr/13/2015 Tag: PTCLM2_150413 One-line: Change output filenames of xmlchange_cmnds to shell_commands Testing: Standard PTCLM test script on yellowstone, and test_driver.sh in clm4_5_1_r105 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values R test/compdirs/*/xmlchange_cmnds A test/compdirs/*/shell_commands M test/compdirs/*/* M test/PTCLMtestlist.py M PTCLMmkdata ================================================================================ Originator: erik Date: Apr/10/2015 Tag: PTCLM2_150410 One-line: Update to new cime directory structure Testing: Standard PTCLM test script on yellowstone, and test_driver.sh in clm4_5_1_r105 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values Change the relative paths to the new cime directory structure where clm source and tools are under components, and CESM scripts are under cime/scripts. M PTCLMmkdata M mydatafiles/1x1pt_US-UMB/xmlchange_cmnds M mydatafiles/1x1pt_US-UMB/README.PTCLM M mydatafiles/1x1pt_US-UMB/user_nl_clm M KnownBugs M README M test/PTCLMtesting_prog.py M test/listings/help M test/listings/list M test/listings/sitelist M test/compdirs/* M test/PTCLMtestlist.py M buildtools ================================================================================ Originator: erik Date: Jan/26/2015 Tag: PTCLM2_150126 One-line: Update test comparison and listings Testing: Standard PTCLM test script on yellowstone, and test_driver.sh in clm4_5_1_r104 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values ------- Update test comparison and listings M test/compdirs/* M test/listings/sitelist M test/listings/help M test/listings/list ================================================================================ Originator: erik Date: Aug/16/2014 Tag: PTCLM2_140816 One-line: Add geyser/caldera to yellowstone Testing: Standard PTCLM test script on yellowstone, and test_driver.sh in clm4_5_1_r080 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values M buildtools -- Allow yellowstone to use geyser and/or caldera ------- Update test comparison and listings M test/compdirs/* M test/listings/sitelist M test/listings/help M test/listings/list ================================================================================ Originator: sacks Date: May/21/2014 Tag: PTCLM2_140521 One-line: rename fpftdyn to flanduse_timeseries, and do some other related variable renames Testing: Standard PTCLM test script on yellowstone, and test_driver.sh in dynlu_crops_n03_addclm50bld_n06_clm4_5_72 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values This is meant to go with an upcoming CLM tag, currently slated for clm4_5_74 A + PTCLM_sitedata/cnvrt_trnsyrs2_landuse_timeseries_txtfile.pl D PTCLM_sitedata/cnvrt_trnsyrs2_pftdyntxtfile.pl M PTCLMmkdata M README M test/compdirs/global_US-UMB/user_nl_clm M test/compdirs/rcp26_US-Dk2/user_nl_clm M test/compdirs/rcp45_US-Dk3/user_nl_clm M test/compdirs/rcp6_US-IB1/user_nl_clm M test/compdirs/rcp85_US-Me4/user_nl_clm M test/compdirs/trans_20th_US-Ne3/user_nl_clm M test/compdirs/trans_clm45_US-Ha1/user_nl_clm ================================================================================ Originator: erik Date: Apr/23/2014 Tag: PTCLM2_140423 One-line: Fix documentation, set DIN_LOC_ROOT_CLMFORC, correct call to mkmapdata.sh allow release_tags in version, add hopper/edison to machine list. Testing: Standard PTCLM test script on yellowstone, and test_driver.sh in clm4_5_71 Known Bugs: RF-Bra will work even without --donot_use_tower_yrs since it doesn't check for valid values A PTCLMsublist --------- Script to submit a list of sites to the batch queue A PTCLMsublist_prog.py - Python code to handle the submision list A batchque.py ---------- Python code to handle batch command submission for single processor M PTCLMmkdata -- Set DIN_LOC_ROOT_CLMFORC, Correct call to mkmapdata.sh Also allow release_tags in version find. M README ------- Correct compset used. M buildtools --- Add edison/hopper to list of machines will work for change frankfurt to goldbach, update netcdf version for yellowstone M test/compdirs/* -------- Update the comparison files M test/listings/sitelist - Update sitelist M test/listings/help ----- Update help M test/listings/list ----- Update list M test/PTCLMtestlist.py ----- Change result codes to be more consistent with other testing mechanisms M test/PTCLMtesting_prog.py - Change result codes, and set sdate/map_gdate so sdate won't change with the day testing was run. M test/PTCLMtestlist.xml ---- Add new tests for new sites ------> Add new sites for Rosie, RF-Bra,LBA-Cax,LTER-Sev ------> And Jinyun US-Bol M PTCLM_sitedata/PTCLMDATA_pftdata.txt M PTCLM_sitedata/PTCLMDATA_sitedata.txt M PTCLM_sitedata/PTCLMDATA_soildata.txt ------> Add new comparison files for testing A test/compdirs/std_RF-Bra A test/compdirs/std_RF-Bra/user_nl_clm A test/compdirs/std_RF-Bra/run.log A test/compdirs/std_RF-Bra/xmlchange_cmnds A test/compdirs/std_RF-Bra/README.PTCLM A test/compdirs/std_LBA-Cax A test/compdirs/std_LBA-Cax/user_nl_clm A test/compdirs/std_LBA-Cax/run.log A test/compdirs/std_LBA-Cax/xmlchange_cmnds A test/compdirs/std_LBA-Cax/README.PTCLM A test/compdirs/std_LTER-Sev A test/compdirs/std_LTER-Sev/user_nl_clm A test/compdirs/std_LTER-Sev/run.log A test/compdirs/std_LTER-Sev/xmlchange_cmnds A test/compdirs/std_LTER-Sev/README.PTCLM ================================================================================ Originator: erik Date: Feb/04/2014 Tag: PTCLM2_140204 One-line: Add testing directory, some reworking, and bug fixes, remove -mach Remove --mach, --sitegroupname options add --map_gdate options. Testing: Ran PTCLM tests in test directory, and test_driver.sh tests on yellowstone. Add test directory and XML based test list A test A test/compdirs A test/compdirs/* ----------- Compare directory files A test/PTCLMtestlist.py ----- Class for reading the test list A test/run_PTCLM_tests ------ Main test script A test/PTCLMtestlist.xml ---- XML test list A test/PTCLMtesting_prog.py - Main class for test script A test/listings ------------- List option compare directory A test/listings/sitelist ---- Site list compare A test/listings/help -------- Help compare A test/listings/list -------- List compare A test/README M buildtools ---- Remove *.o files for gen_domain and gmake clean everything else M PTCLMmkdata --- STARTDATE and ALIGNYR only set if using tower years. Add more metadata to filenames. More work on debug option so works well. Also run mksurfdata.pl in debug mode with --debug and --allownofile so will test it's usage. Remove -mach option, add --map_gdate option so can use old maps easier. Add function to get filenames of created files, and touch the file in debug mode. Fix some of the use-case logic. Fix directory logic, and allow being able to run from a different directory. Exercise more commands in system function for debug mode. Remove -m option M mydatafiles/1x1pt_US-UMB/user_nl_clm M mydatafiles/1x1pt_US-UMB/xmlchange_cmnds M mydatafiles/1x1pt_US-UMB/README.PTCLM M README --- Remove -m option Add some test sites for testing: TS-Ts1, TS-Ts2, TS-Ts3 M PTCLM_sitedata/PTCLMDATA_sitedata.txt ----- Add test sites, change align year as first year to use when cycle_forcing used This is usually the first year, unless it's a leap year and it's the year after that. If the site has less than 3 years it's "1". M PTCLM_sitedata/PTCLMDATA_soildata.txt M PTCLM_sitedata/PTCLMDATA_pftdata.txt ================================================================================ Originator: erik Date: Nov/22/2013 Tag: PTCLM2_131122c One-line: Add some example files Testing: limited setup for US-UMB case on yellowstone A mydatafiles/1x1pt_US-UMB/user_nl_clm A mydatafiles/1x1pt_US-UMB/xmlchange_cmnds A mydatafiles/1x1pt_US-UMB/README.PTCLM ================================================================================ Originator: erik Date: Nov/22/2013 Tag: PTCLM2_131122b One-line: Rename a bunch of files, add US-UMB sample data Testing: limited setup for US-UMB case on yellowstone Rename files to new names: A + PTCLMmkdata A + PTCLM_sitedata/PTCLMDATA_pftdata.txt A + PTCLM_sitedata/PTCLMDATA_sitedata.txt A + PTCLM_sitedata/PTCLMDATA_soildata.txt A mydatafiles/1x1pt_US-UMB ---- sample directory for US-UMB site D PTCLM.py D PTCLM_sitedata/EXAMPLE_sitedata.txt D PTCLM_sitedata/EXAMPLE_soildata.txt D PTCLM_sitedata/EXAMPLE_pftdata.txt >>>>> Remove other site data files as we already have a lot of sample data in the PTCLMDATA file set. D PTCLM_sitedata/LBA_pftdata.txt D PTCLM_sitedata/LBA_sitedata.txt D PTCLM_sitedata/LBA_soildata.txt D PTCLM_sitedata/Fluxnet-Canada_sitedata.txt D PTCLM_sitedata/Fluxnet-Canada_soildata.txt D PTCLM_sitedata/Fluxnet-Canada_pftdata.txt D PTCLM_sitedata/AmeriFlux_sitedata.txt D PTCLM_sitedata/AmeriFlux_soildata.txt D PTCLM_sitedata/AmeriFlux_pftdata.txt D PTCLM_sitedata/CarboEurope_pftdata.txt D PTCLM_sitedata/CarboEurope_sitedata.txt D PTCLM_sitedata/CarboEurope_soildata.txt >>>>>> Update documentation M KnownBugs M README ================================================================================ Originator: erik Date: Nov/22/2013 Tag: PTCLM2_131122 One-line: Add more documentation, remove some case settings not needed, make work a bit cleaner Testing: limited setup for US-UMB case on yellowstone >>>> Add README file for the sample mydatafiles directory A https://svn-ccsm-models.cgd.ucar.edu/PTCLM/trunk_tags/PTCLM2_131119/mydatafiles/README M https://svn-ccsm-models.cgd.ucar.edu/PTCLM/trunk_tags/PTCLM2_131119/PTCLM.py -- remove some options not needed. Fix directory, and setup user_nl_clm and xmlchange_cmnds files. M https://svn-ccsm-models.cgd.ucar.edu/PTCLM/trunk_tags/PTCLM2_131119/README -- update doc... ================================================================================ Originator: erik Date: Nov/19/2013 Tag: PTCLM2_131119 One-line: Remove create_newcase call from PTCLM, just create datasets Testing: limited setup for US-UMB case on yellowstone Requires: scripts4_131119 Change PTCLM to JUST be a tool that creates datasets rather than also creates a case for you. Running create_newcase is now a seperate step that you use the -user_mods_dir option with. There are some issues with this version. But, it has basic functionality. The xmlchange_xcmnds file that is created has many fields multiple times. The dataset location is also screwy. Change name for CHATS datasite to US-CHATS so name is more similar to the other FluxNet sites. CHATS is an experiment that NCAR ran. A buildtools -- new script to build all the tools needed for PTCLM M PTCLM.py ---- Change so that only creates datasets, does NOT run create_newcase M README ------ Update documentation M PTCLM_sitedata/EXAMPLE_sitedata.txt -- Add campaign name to end M PTCLM_sitedata/EXAMPLE_soildata.txt -- Change CHATS to US-CHATS M PTCLM_sitedata/EXAMPLE_pftdata.txt --- Change CHATS to US-CHATS ================================================================================ Originator: erik Date: Sep/29/2013 Tag: PTCLM1_130929 cesm_baseline: scripts4_130929 One-line: default compset: I1PTCLM45, files go under site subdir, add script to rename maps Testing: limited ran mexicocity, US-UMB, US-Ha1, and BE-Vie cases M PTCLM.py -------------------- Make default compset: I1PTCLM45, make files in resolution subdiretory (under mydatadir) M README ---------------------- Update documentation A mydatafiles/renamemapfiles -- Script to rename mapping files creation date, so don't have to recreate mapping files again. Remove directory NOT needed anymore... D usr_files/mkgriddata.TEMPLATE D usr_files ================================================================================ Originator: erik Date: Sep/23/2013 Tag: PTCLM1_130923 cesm_baseline: scripts4_130916 One-line: Add --cycle_forcing, and --mydatadir options, remove useQIAN, rename --QIAN_tower_yrs to --use_tower_yrs Testing: limited ran a couple cases M PTCLM.py -------- Changes from Keith Oleson to cycle_forcing, check what model version is being used (clm4_0 or clm4_5) set time-step. D testcases.csh --- Remove as not working well on yellowstone ================================================================================ Originator: erik Date: Sep/20/2013 Tag: PTCLM1_130920 cesm_baseline: scripts4_130920 One-line: Make datafiles in mydatafiles directory Testing: Ran basic US-UMB, US-NR1 and 1x1_mexicocityMEX cases M PTCLM.py ---------------- File changes from Keith Oleson M PTCLM_sitedata/Fluxnet-Canada_sitedata.txt M PTCLM_sitedata/EXAMPLE_sitedata.txt M PTCLM_sitedata/Fluxnet-Canada_soildata.txt M PTCLM_sitedata/AmeriFlux_sitedata.txt M PTCLM_sitedata/EXAMPLE_soildata.txt M PTCLM_sitedata/AmeriFlux_soildata.txt M PTCLM_sitedata/Fluxnet-Canada_pftdata.txt M PTCLM_sitedata/EXAMPLE_pftdata.txt M PTCLM_sitedata/AmeriFlux_pftdata.txt ================================================================================ Originator: erik Date: Sep/10/2013 Tag: PTCLM1_130910 cesm_baseline: scripts4_130910 One-line: Get rid of pft-phys file copy, fix mapping file names, correct domain file Testing: Limited, basic test setup M PTCLM.py --------- Remove pft-phys file copy, fix mapping file name, correct domain file path M testcases.csh ---- change compsets testing, unzip log file to check status ================================================================================ Originator: erik Date: Jul/24/2013 Tag: PTCLM1_130724 cesm_baseline: scripts4_130724 One-line: Add some more error checking, more changes to get things working Testing: Limited, basic test setup M PTCLM.py ------- error checking, correct mapfile name, get gen_domain working, set ATM/LND_DOMAIN_FILE ================================================================================ Originator: erik Date: May/29/2013 Tag: PTCLM1_130529 cesm_baseline: scripts4_130529 One-line: Get PTCLM working with the new CLM tools Testing: Limited, test mexicocity, and 1x1_US-UMB basic case setup M PTCLM.py ------- Remove spinup options, get working with new tools M testcases.csh -- Remove spinup tests, get working again ================================================================================ Originator: erik Date: Feb/16/2013 Tag: PTCLM1_130216 cesm_baseline: scripts4_130130 One-line: Get supported compsets working and some progress with tools Testing: Limited -- but surported res work on ys M PTCLM.py M testcases.csh ================================================================================ Originator: erik Date: Jan/30/2013 Tag: PTCLM1_130130 cesm_baseline: scripts4_130130 One-line: Initial changes to get PTCLM working with new scripts and tools Testing: Limited testing Initial add of yellowstone to testcases and switch frankfurt for edinburgh. Remove generic stuff for userdefined. Remove ndep/aer grid. M PTCLM.py M testcases.csh D usr_files/mkdatadomain.TEMPLATE ================================================================================ Originator: erik Date: Jan/11/2013 Tag: PTCLM1_130111 cesm_baseline: scripts4_120125 One-line: Add some new sitedata from Keith Oleson Testing: none A PTCLM_sitedata/LBA_pftdata.txt M PTCLM_sitedata/Fluxnet-Canada_sitedata.txt M PTCLM_sitedata/EXAMPLE_sitedata.txt A PTCLM_sitedata/CarboEurope_pftdata.txt M PTCLM_sitedata/AmeriFlux_sitedata.txt M PTCLM_sitedata/Fluxnet-Canada_soildata.txt M PTCLM_sitedata/EXAMPLE_soildata.txt M PTCLM_sitedata/AmeriFlux_soildata.txt A PTCLM_sitedata/LBA_sitedata.txt A PTCLM_sitedata/LBA_soildata.txt A PTCLM_sitedata/CarboEurope_sitedata.txt A PTCLM_sitedata/CarboEurope_soildata.txt M PTCLM_sitedata/Fluxnet-Canada_pftdata.txt M PTCLM_sitedata/EXAMPLE_pftdata.txt M PTCLM_sitedata/AmeriFlux_pftdata.txt ================================================================================ Originator: mvertens Date: Jan/25/2012 Tag: PTCLM1_120125 cesm_baseline: scripts4_120125 One-line: Remove USE_MPISERIAL replace with MPILIB ================================================================================ Originator: erik Date: Nov/29/2011 Tag: PTCLM1_111129 cesm_baseline: clm4_0_39 / scripts4_111129 One-line: Use CLM_USRDAT for resolution or supported single-point res name so will work with latest scripts version Bugs-fixed: Known Bugs: (See KnownBugs file for details) 1379 (All of the spinup tests fail for PTCLM) 1364 (Problem running US-UMB case on Macintosh) 1251 (Test case aborts on Macintosh with seq-fault in US-Ha1 I_QIAN case) Testing: Limited testing bluefire testing: All PASS M PTCLM.py -- Set res in create_newcase to either supported 1pt name or to CLM_USRDAT. ================================================================================ Originator: erik Date: Nov/14/2011 Tag: PTCLM1_111114 cesm_baseline: clm4_0_38 / scripts4_111108 One-line: Update path for NetCDF4 Bugs-fixed: Known Bugs: (See KnownBugs file for details) 1379 (All of the spinup tests fail for PTCLM) 1364 (Problem running US-UMB case on Macintosh) 1251 (Test case aborts on Macintosh with seq-fault in US-Ha1 I_QIAN case) Testing: Limited testing bluefire testing: All PASS Update path for NetCDF for bluefire, jaguar, lynx, and mirage. M testcases.csh ================================================================================ Originator: erik Date: Sep/02/2011 Tag: PTCLM1_110902 cesm_baseline: clm4_0_35 / scripts4_110902 One-line: Change name of mksurfdata to mksurfdata_map, fix soil texture of US-UMB Bugs-fixed: 1392 (US-UMB site has some incorrect data) Known Bugs: (See KnownBugs file for details) 1379 (All of the spinup tests fail for PTCLM) 1364 (Problem running US-UMB case on Macintosh) 1251 (Test case aborts on Macintosh with seq-fault in US-Ha1 I_QIAN case) Testing: Limited testing bluefire testing: All PASS Get basic script working with non2D grid version of clm. mksurfdata directory was changed to mksurfdata_map, and tools directories now have a "src" subdirectory to build the code in them. Also fix soil texture for US-UMB site (bug 1392). M PTCLM.py M testcases.csh M PTCLM_sitedata/EXAMPLE_soildata.txt M PTCLM_sitedata/AmeriFlux_soildata.txt ================================================================================ Originator: erik Date: Jul/26/2011 Tag: PTCLM1_110726 cesm_baseline: clm4_0_33 / scripts4_110724 One-line: Test if should use skip_rundb option or not, fix transient problem Bugs-fixed: 1368 (PTCLM for US-UMB spins up with zero GPP) 1361 (Problem with transient compsets for PTCLM) Known Bugs: (See KnownBugs file for details) 1392 (US-UMB site has some incorrect data) 1379 (All of the spinup tests fail for PTCLM) 1364 (Problem running US-UMB case on Macintosh) 1251 (Test case aborts on Macintosh with seq-fault in US-Ha1 I_QIAN case) Testing: All tests pass up to the spinup tests where they all fail bluefire testing: PTCLM.*_US-UMB_ICN_exit_spinup.PTCLM jaguar testing: PTCLM._US_US-UMB_ICN_exit_spinup.PTCLM edinburgh testing: PTCLM.*_US-UMB_ICN_exit_spinup.PTCLM yong/intel testing: PTCLM.*_US-UMB_ICN_exit_spinup.PTCLM M PTCLM.py ------ Add check for finidat file, check if should use -skip_rundb option, fix transient issue M testcases.csh - Change order of arguments for spinup tests, so more likely to die on an error with finidat file M KnownBugs ----- Update list of bugs ================================================================================ Originator: erik Date: May/04/2011 Tag: PTCLM1_110504 cesm_baseline: clm4_0_28 / scripts4_110428a One-line: Move PTCLM to own external, and fix cnvrt script so can take year-range allow it to use older versions of Python Bugs-fixed: 1279 (Latest version of PTCLM requires python2.5) 1248 (PTCLM can only run until 2005) Known Bugs: (See KnownBugs file for details) 1392 (US-UMB site has some incorrect data) 1251 (Test case aborts on Macintosh with seq-fault in US-Ha1 I_QIAN case) Testing: bluefire testing: All PASS jaguar testing: All PASS edinburgh testing: All PASS except 4 PTCLM.8696_US-UMB_I_1850.PTCLM FAIL 0 5 PTCLM.8696_US-UMB_I20TR.PTCLM FAIL 0 6 PTCLM.8696_US-UMB_I20TRCN.PTCLM FAIL 0 7 PTCLM.8696_US-UMB_ICN.PTCLM FAIL 0 8 PTCLM.8696_US-UMB_I1850CN.PTCLM FAIL 0 9 PTCLM.8696_US-UMB_IRCP85CN.PTCLM FAIL 0 10 PTCLM.8696_US-UMB_I.PTCLM FAIL 0 11 PTCLM.8696_US-UMB_I_QIAN.PTCLM FAIL 0 12 PTCLM.8696_US-UMB_I.PTCLM FAIL 0 14 PTCLM.8696_US-UMB_ICN_exit_spinup.PTCLM FAIL 0 15 PTCLM.8696_US-UMB_ICN_final_spinup.PTCLM FAIL 0 yong/intel testing: All PASS Send a year range to the convert script and have years that start at the beginning of the year range and go to the end of it. Add a DEBUG option to testcases.csh so can just run tests quickly to debug test script. M PTCLM.py M PTCLM_sitedata/cnvrt_trnsyrs2_pftdyntxtfile.pl M testcases.csh M KnownBugs M PTCLM_sitedata/EXAMPLE_sitedata.txt - Remove all but US-UMB M PTCLM_sitedata/EXAMPLE_soildata.txt - Remove all but US-UMB M PTCLM_sitedata/EXAMPLE_pftdata.txt -- Remove all but US-UMB ================================================================================ Originator: erik Date: Feb/03/2011 Tag: clm4_0_23 / scripts4_110111 One-line: Fix --list and --cesm_root options, add --scratchroot, fix bugs update use-cases and compset names, add CESM_ROOT/CLM_SOFF to testcases.csh Bugs-fixed: 1256 (fix PTCLM testcases.csh test script on jaguar to use netcdf/3.6.2) 1254 (pft-physiology copy doesn't have .nc extension/can't handle new files) 1250 (add ability to set scratchroot) 1224 (Fix -aerdepgrid/ndepgrid options in PTCLM.py) Known Bugs: (See KnownBugs file for details) 1392 (US-UMB site has some incorrect data) 1251 (Test case aborts on Macintosh with seq-fault in US-Ha1 I_QIAN case) 1248 (PTCLM can only run until 2005) Testing: bluefire testing: All PASS edinburgh testing: All Fail (Python is too old 2.4 when needs 2.5) yong/intel testing: All PASS except... myPTCLMtests_US-Ha1_I_1850.PTCLM FAIL 0 myPTCLMtests_US-Ha1_I20TR.PTCLM FAIL 0 M PTCLM.py ------- Fix bugs, get --list and --cesm_root options working, add --scratchroot, update CLM use-cases and how queried add ability to add directory to caseprefix M testcases.csh -- Add ability to use env vars CESM_ROOT and CLM_SOFF set scratchroot for generic machines. Put case directories under the PTCLM directory, so can delete it easier M KnownBugs ================================================================================ Originator: erik Date: Jan/12/2011 Tag: clm4_0_21 / scripts4_110108 One-line: Create ChangeLog file for PTCLM Testing: bluefire testing: All PASS edinburgh testing: All PASS up to ... myPTCLMtests_US-Ha1_I_1850.PTCLM FAIL 0 yong/intel testing: All PASS up to... myPTCLMtests_US-Ha1_I_QIAN.PTCLM PASS ================================================================================ Originator: erik Date: Dec/06/2010 Tag: scripts4_101206 One-line: Add some documentation files M ccsm_utils/Tools/lnd/clm/PTCLM/testcases.csh Add USER_CC setting to yong M ccsm_utils/Tools/lnd/clm/PTCLM/README ------- Add note about aerdepgrid/ndepgrid A ccsm_utils/Tools/lnd/clm/PTCLM/KnownBugs ---- Add file with list of bugs ================================================================================ Originator: erik Date: Dec/02/2010 Tag: scripts4_101202 One-line: Updates for script changes >>>>>>>>>>>> Get PTCLM working with changes >>>>>>>>>>>> PTCLM updates from mpiserial branch >>>>>>>>>>>> Add PTCLM tests for yong (Mac OS-X laptop) M ccsm_utils/Tools/lnd/clm/PTCLM/PTCLM.py M ccsm_utils/Tools/lnd/clm/PTCLM/testcases.csh M ccsm_utils/Tools/lnd/clm/PTCLM/README ================================================================================ Originator: erik Date: Aug/30/2010 Tag: scripts4_100830 One-line: Bring PTCLM branch to scripts trunk ================================================================================