Minutes from 3 November 2009

2009/11/03 CSEP Meeting

Attendees: John Y., Masha L.

1. csep-op:
   1. V9.10.0 release:
      • Re-processing of three-month models due to ANSS catalog fixes:
        1. Matlab format of three-month forecast with start date of 2008-07-01 caused scipy exception:
           1. scipy V0.7.0 is installed on csep-op
           2. scipy V0.7.1 does not have problem loading the same forecast file - tested on csep-devel
           3. John installed scipy V0.7.1 on csep-cert to validate current CSEP installation
              1. Nightly build and Dispatchers succeeded
           4. Will upgrade csep-op to scipy V0.7.1 on CSEP V9.10.1 release date - November 2, 2009
      • FIXED: Trac #181 Re-store ability to re-process one-day forecasts with existing observation catalogs
      • Evaluation of Global and NW Pacific one-day forecasts failed for 2009-09-17 and 2009-09-18 test dates:
        1. FIXED: #187 Negative centroid time offset relative to 00:00 seconds in CMT raw catalog results in invalid time
           • Evaluation of Global and NW Pacific one-year failed for 2009-09-28
        2. FIXED: #188 OneDayModelPostProcess for CMT catalog might include events from the day following the test date if any are available
           • Masha wrote a script to identify which test dates are affected by the problem - total of about 100 dates for Global and Western Pacific regions
   2. Intermediate release V9.10.1 with fixes for described above Trac tickets
      • Due to much longer processing time on csep-op (new NFS server) release is re-scheduled for Wednesday or Thursday, November 4 or 5, 2009, when running Dispatchers complete
2. Software development:
   1. Working on Trac #177 XML format of forecast should still be passed through master XML template
      • If use warning messages, it won't address a problem with existing TripleS forecasts. Would need to install fixed version of TripleS model
      • Can automatically reinterpret XML forecasts to use [-90;90] for latitude and [-180;180] range for longitude
        • Generic implementation to affect all testing regions of all testing centers
        • Masha sent email to Jeremy asking if it's a correct generic assumption and should proceed with implementation
3. From 2009/10/06 meeting: Fabian presented slides for new implementation of CSEP results web viewer
   • Proposed to use Django based on Model-View-Controller paradigm
   • Split results viewer into:
     • web service for CSEP data products (maps, XML results datasets)
       • Use Database to store CSEP results by Dispatcher
         • John pointed out that it might be redundant to split already available XML results for injection to the database, then reconstruct XML from the database
         • John pointed out that would need to inject already existing data into the database
         • Masha is worried about making it error-prone due to the reprocessing
         • Very minimal work on operational CSEP software if only need to change to publish XML results vs. SVG plots
         • Fabian could work on implementation
         • Danijel suggested to create tests summaries by Dispatcher
           1. Masha suggested to implement visualization of the summary on web server site by CSEP Toolkit modules
     • interactive web application to present data products
       • Danijel asked about authentication for Japanese viewer
         • John suggested to use Apache's one
     • Danijel asked if it shold be integrated into Drupal?
       • John suggested to try a simple example to see if it works
     • James asked why not to use Plone
       • John said that we used to use Plone, but migrated away from it, but may be should investigate more
     • John asked how to discover what is available:
       1. Danijel suggested to use URL queries to discover
       2. Masha suggested that Dispatcher could update already existing configuration files for forecasts groups with current forecasts filenames and publish it as well
       3. John suggested to use allmodels.xml that current web viewer has
4. Hardware:
   1. John replaces NFS server on csep-op on Thursday, 2009-10-22
      • Runtime for nightly Dispatcher went from 2hours 15 minutes to 4,5-5 hours
      • Masha reported that tests with simulations are affected (L, M, S and R-tests) - writing random seed values for each simulation to the file
      • John will try direct connection of NFS server to csep-op server
      • Will discuss possible solutions with Phil
   2. csep-cert
      • CSEPWebTest.py is not part of nightly build - some issues connecting to the web server
        1. John is looking at the problem