Minutes from 4 June 2008

2008/06/04 CSEP meeting

Attendees: Yan, Phil, Jeremy, Masha

1. Introducing Western Pacific testing region into the CSEP:
   • Yan confirmed that we converged in generated forecasts:
     
     1. CSEP and UCLA forecast discrepancies are caused by different input catalogs
     2. Column #8 should be ignored when comparing forecasts since columns #4-8 are for focal mechanisms forecasts that we don't use in CSEP for now
        • NW Pacific forecast rates are under 1% of differences
        • SW Pacific forecast rates are under 1% of differences if generated with Yan's input catalog
     3. Yan recommended to monitor generated forecasts:
        • If there are greater than 1% differences, generate forecast with Yan's input catalog to see if differences become less than 1%
   • Yan suggested new evaluation tests for the models:
     
     1. Jeremy will look into it
   • Jeremy suggested to push CSEP generated input catalog to UCLA - to avoid input catalog discrepancies if we want to compare CSEP generated forecasts against UCLA ones:
     
     1. Yan would consider it after monitoring CSEP generated forecasts for 2-3 months
     2. Phil suggested to develop interface for CSEP data sets:
        • Use publish/subscribe model
   • Software development that is associated with new testing regions:
     
     1. Done:
        
        1. CSEP populates master ForecastML templates in XML format with Yan's forecast rates

        2. Jeremy generated reference data for unit test that calculates bin area in km**2, unit test trunk/src/generic/test/unitTest/GeoUtilsTest.py passes

        3. Observation catalog for evaluation tests:
           • No catalog uncertainties
           • Filter by depth and magnitude
           • Filter by geographical area
             
             1. Jeremy defined testing regions
             2. There is no collection area - the entire globe is used

        4. Set up 2 testing regions on csep-cert - SW and NW Pacific:

           • One-day and one-year forecasts models within each region
           • Crontab-driven Dispatchers generate forecasts, and run N and L evaluation tests

           • N and L tests svg plots are published to intensity

     2. Code freeze is postponed until Monday, June 9, tasks to be finished:
        

        1. Automate Jeremy's Triple_S model

        2. GMT scripts to generate forecasts maps - Danijel
        3. Invoke R evaluation test once Jeremy's model is automated
        4. Add unit and acceptance tests for newly added functionality - after the code freeze

        5. Set up CruiseControl on new server csep-cert-new

        6. Web interface to support new testing regions - after the code freeze
   • Danijel and Masha responded to Anna Maria's e-mail:
     
     1. Asked to split testing region into SW and NW Pacific regions
     2. Sent an example of CMT input catalog in one-line-per-event format
     3. Sent ForecastML testing regions templates
2. Phil suggested for upcoming CSEP Version 8.7 release:
   • One page user guide on how to install a model in CSEP:
     
     1. Contact John Yu to get an account on motion.usc.edu
     2. ..
   • List of accomplishments
   • Distribution of testing regions to interested modelers