Minutes from 9 February 2010

2010/02/09 CSEP Meeting

Attendees: Phil M., John Y., Masha L.

csep-op:
1. From 2010/01/26 metting: Performance issue due to new NFS server:
  - John wants to try separating testing regions to different disks
    1. Masha said it's OK since all testing regions are independent from each other
2. USC Matlab license R2009b got picked up by nightly processing on csep-op (since 2010/01/26)
  - Dispatcher for Global one-year forecasts groups is still running
  - Danijel suggested not to reprocess test dates when older R2007a is restored on the server
  - Nightly builds show problem only with STEP forecast
3. V10.1.0 is running on csep-op as of 2009/01/08:
  - Re-processing: Trac #207 Inconsistent number of events for 2009-10-03, 2009-11-03, 2009-12-03 observation catalogs of RELM-Mainshock group
From 2010/01/26 meeting: Jeremy had a discussion with Peter Shebalin:
- Should run alarm-based evaluation tests for EAST (alarm-based) model against EEPAS and PPE David Rhoades models (rate-based)
  - Masha responded that EAST model has a single magnitude bin vs. rate-based models having multiple magnitude bins
  - Jeremy said that alarm-based tests won't work then
  - Jeremy will get back to Masha regarding alarm-based tests for alarm-based model vs. rate-based model
From 2009/12/15 meeting: Phil suggested to use iRODS as data management for CSEP:
- John will look into it some time early January
- John suggested to use VirtualMachine to distribute CSEP software
  1. Will need to have different images for different testing centers due to the installed forecasts models
Software development:
1. Matt promised Java version of STEP to be available for the installation within SCEC Testing Center by March, 2010
  - Matt will get back to Masha if he wants to continue testing of fixed Matlab version
2. From 2010/02/01 meeting: priorities for known tasks:
  - Web service
    1. Phil likes web service since it's easy to write clients
    2. Phil asked about data formats for returned data products by web service:
      - Attachments
      - If XML, then need to define format and have end-user parse the document
      - Masha's responded that CSEP is using already defined ForecastML format for forecast data, and once CSEP switches to QuakePy, will use QuakeML format for catalogs
      - John suggested to use zip archive to return multiple products to the end user
  - Get rid of Matlab within testing center:
    1. Use QuakePy to prepare catalogs
    2. Use original Fortran code for declustering
3. Trac tickets:
  - From 2010/02/02 meeting: Max will submit ETAS model for testing some time in March-April 2010
  - Working on: #185 Generate summary tables per each evaluation test
    1. #209 Add per event and sum of true log-likelihoods to L, S and M RELM evaluation tests results
    2. "RELM report"-like plot will be generated only for N-test (forecasted number of events vs. observed interval of events)
      - Masha is waiting for Danijel's or Jeremy's response to clarify plotting of forecasted interval
4. From 2010/02/02 meeting: High resolution Global testing region:
  1. Danijel reported that forecast template and format are being defined
  2. Yan sent out an announcement about "FULL EARTH HIGH-RESOLUTION (0.1x0.1 degree) EARTHQUAKE FORECASTS"
    - Using PDE catalog
    - Will integrate within CSEP when testing region is established
From 2009/12/08 meeting: CSEP policies
1. Should discuss if SCEC Testing Center results should be open to the public
  - Danijel will get back to the group
2. Danijel will work on more content for the main CSEP web pages
From 2009/10/06 meeting: Fabian presented slides for new implementation of CSEP results web viewer
- Proposed to use Django based on Model-View-Controller paradigm
- Split results viewer into:
  - web service for CSEP data products (maps, XML results datasets)
    - Use Database to store CSEP results by Dispatcher
      - John pointed out that it might be redundant to split already available XML results for injection to the database, then reconstruct XML from the database
      - John pointed out that would need to inject already existing data into the database
      - Masha is worried about making it error-prone due to the reprocessing
      - Very minimal work on operational CSEP software if only need to change to publish XML results vs. SVG plots
      - Fabian could work on implementation
      - Danijel suggested to create tests summaries by Dispatcher
        
        Masha suggested to implement visualization of the summary on web server site by CSEP Toolkit modules
  - interactive web application to present data products
    - Danijel asked about authentication for Japanese viewer
      - John suggested to use Apache's one
  - Danijel asked if it shold be integrated into Drupal?
    - John suggested to try a simple example to see if it works
  - James asked why not to use Plone
    - John said that we used to use Plone, but migrated away from it, but may be should investigate more
  - John asked how to discover what is available:
    1. Danijel suggested to use URL queries to discover
    2. Masha suggested that Dispatcher could update already existing configuration files for forecasts groups with current forecasts filenames and publish it as well
    3. John suggested to use allmodels.xml that current web viewer has
From 2010/01/12 meeting: John and Masha discussed necessary changes to support web application to view CSEP results:
- CSEP will publish already existing configuration file for each forecast group to the web server
- Web service will use file system discover method to identify:
  1. avalable testing regions
  2. available forecasts groups within each testing region
- Configuration file for each forecast group should include:
  1. Base name of currently tested forecast file
  2. Flag if forecast start date is part of the forecast filename
- John asked if CSEP Dispatchers should publish any XML format files to the web server at all:
  1. May run into problem of displaying results for not finished evaluation tests
    - May introduce a "lock" to indicate if evaluation is in progress
  2. Makes sense not to duplicate results - especially the ones that nobody is going to look at
Hardware:
1. csep-cert
  - CSEPWebTest.py is not part of nightly build - some issues connecting to the web server
    1. John is looking at the problem

Minutes from 9 February 2010

Testing Centers

Testing Regions

CSEP Tests